# Advanced Functions

Course: [Python](https://www.devtutorials.ai/learn/python) · Section: Advanced Python
Source: https://www.devtutorials.ai/learn/python/advanced-python/advanced-functions

> Lambdas, closures and nonlocal, iterators and StopIteration, generators and yield, and decorators with functools.wraps.

A function you have written so far has a name, a body, and a `def` above it.

But a function in Python is also a **value**. You can put one in a variable, pass it to another function, build one inside another one, and return it. Once you see that, five things that look like separate topics turn out to be one idea wearing different hats.

Here is that idea in action — a function stored in a variable and handed to another function:

```python runnable title="A function is a value like any other"
def shout(text):
    return text.upper()

loud = shout
print(loud("asha"))

def apply_twice(func, text):
    return func(func(text))

print(apply_twice(shout, "asha"))
print(type(shout))
```

Functions are objects. They can be stored, passed, built at run time and returned — and that one fact is what lambdas, closures, generators and decorators are all built on.

## What You Will Learn

In this lesson, you will learn:

- How to write a function with no name, and when that is honestly better
- What a closure is, and why a function can remember a value after the function that made it has finished
- The difference between something you can loop over and the thing that does the looping
- How a generator produces values one at a time instead of building a list
- How a decorator wraps a function without changing a line inside it

By the end of this lesson, you will be able to write lambdas where they are the right tool, build functions that remember values through closures, explain iterables against iterators, write generators that produce values one at a time, and wrap functions with decorators.

## Anonymous Functions

### What Is a Lambda Function?

A **lambda** is a function written as a single expression, with no name and no `def`. It exists for the cases where naming a function would be more ceremony than the function is worth — a one-line rule handed to `sorted()` and never used again.

```python runnable title="The same function, written twice"
def square(n):
    return n * n

square_lambda = lambda n: n * n

print(square(9))
print(square_lambda(9))
print(type(square_lambda))
```

Both are functions. The only difference is how they were written.

### Lambda Syntax

```text
lambda parameters: a single expression
```

Three rules, and they are the whole language of lambdas:

1. No `def` and no name.
2. The body is **one expression**. No `if` statement, no loop, no `return`.
3. The value of that expression is returned automatically.

```python runnable title="Parameters work the same way"
total = lambda a, b: a + b
greet = lambda name, greeting="Hello": f"{greeting}, {name}"
pass_fail = lambda mark: "pass" if mark >= 35 else "fail"

print(total(78, 85))
print(greet("Asha"))
print(greet("Ravi", "Welcome"))
print(pass_fail(78), pass_fail(20))
```

That last one uses a conditional *expression*, which is allowed, because it produces a value. An `if` *statement* is not.

### Lambda with `map()`

Lambdas become most useful when they are passed directly into another function. `map()` is one of the common places where that happens.

`map()` applies a function to every item and hands back the results one at a time.

```python runnable title="map applies a rule to every mark"
marks = [78, 85, 60]

moderated = map(lambda mark: mark + 5, marks)
print(moderated)
print(list(moderated))

print(list(map(str, marks)))
print([mark + 5 for mark in marks])
```

`map()` gives back a lazy object, not a list — printing it shows that plainly. Wrap it in `list()` to see the values. The last line is the comprehension that does the same job, and for most readers it is easier to follow.

### Lambda with `filter()`

Where `map()` changes items, `filter()` decides which items stay and which are removed.

```python runnable title="filter keeps what passes"
marks = [78, 20, 85, 30, 60]

print(list(filter(lambda mark: mark >= 35, marks)))
print(list(filter(lambda mark: mark < 35, marks)))
print([mark for mark in marks if mark >= 35])
```

### Lambda with `sorted()`

This is where a lambda genuinely earns its place. `sorted()` takes a `key` — a function that says what to sort *by*.

```python runnable title="Sorting records by a part of them"
rows = [("Asha", 78), ("Ravi", 85), ("Meera", 60)]

print(sorted(rows))
print(sorted(rows, key=lambda row: row[1]))
print(sorted(rows, key=lambda row: row[1], reverse=True))

marks = {"Asha": 78, "Ravi": 85, "Meera": 60}
print(sorted(marks, key=marks.get, reverse=True))
```

Without `key`, sorting a list of tuples goes by the first item — here, the name. With `key=lambda row: row[1]` it goes by the mark. Note the last line: `marks.get` is already a function, so no lambda is needed at all.

### Lambda against Normal Functions

Side by side, the differences are few:

| | `lambda` | `def` |
|---|---|---|
| Has a name | No — `__name__` is `<lambda>` | Yes |
| Body | One expression | Any number of statements |
| Docstring | Cannot have one | Can, and should |
| Shows in a traceback as | `<lambda>` | Its own name |
| Best for | A rule passed straight to `sorted`, `map` or `filter` | Anything you will call by name, test, or read twice |

```python runnable title="What a lambda is called"
square = lambda n: n * n

def cube(n):
    return n ** 3

print(square.__name__)
print(cube.__name__)
```

Assigning a lambda to a name, as `square = lambda n: n * n`, gets you the worst of both: a function that needs a name anyway, but shows as `<lambda>` in every error message. If it deserves a name, give it a `def`.

## Closures

The lambda captured nothing — it only used its own parameters. A function defined inside another one can also use the outer one's variables, and that is the next idea.

### What Is a Closure?

A **closure** is a function that remembers a value from the function that made it, even after that outer function has finished.

```python runnable title="A function that remembers"
def make_adder(bonus):
    def add(mark):
        return mark + bonus
    return add

add_five = make_adder(5)
add_ten = make_adder(10)

print(add_five(78))
print(add_ten(78))
```

`make_adder` finished running before `add_five` was ever called. The `5` should be gone — and it is not. `add` kept it.

### Nested Functions

A `def` inside a `def` is an ordinary function that happens to be built when the outer one runs. It is a new object every time.

```python runnable title="A new function each time"
def make_adder(bonus):
    def add(mark):
        return mark + bonus
    return add

a = make_adder(5)
b = make_adder(5)

print(a(78), b(78))
print(a is b)
```

Same behaviour, two different objects — because the inner `def` ran twice.

### Capturing Variables from the Outer Scope

The captured values are not hidden. Python stores them on the function itself.

```python runnable title="Where the remembered value lives"
def make_adder(bonus):
    def add(mark):
        return mark + bonus
    return add

add_five = make_adder(5)

print(add_five.__code__.co_freevars)
print(add_five.__closure__[0].cell_contents)
```

`co_freevars` names what was captured; the closure cell holds the value.

### The `nonlocal` Keyword

Reading an outer variable works by itself. **Changing** one does not — Python assumes any name you assign to is local, and then complains that you read it before it had a value.

```python runnable title="The mistake, and what Python says"
def counter():
    count = 0
    def tick():
        count += 1
        return count
    return tick

tick = counter()

try:
    print(tick())
except UnboundLocalError as error:
    print("UnboundLocalError:", error)
```

`nonlocal` is the fix. It says: this name belongs to the enclosing function, not to me.

```python runnable title="nonlocal makes the change stick"
def counter():
    count = 0
    def tick():
        nonlocal count
        count += 1
        return count
    return tick

attendance = counter()
print(attendance(), attendance(), attendance())

other = counter()
print(other())
```

Note the last two lines: a second counter starts again at 1, because it has its own `count`.

### A Closure Example

Here is the pattern in use — one rule, two settings:

```python runnable title="A grader built once, used many times"
def grader(pass_mark):
    def grade(mark):
        return "pass" if mark >= pass_mark else "fail"
    return grade

school = grader(35)
scholarship = grader(80)

for mark in [78, 30, 85]:
    print(mark, school(mark), scholarship(mark))
```

One rule, two settings, no `if` repeated anywhere.

### Closures against Global Variables

A global variable would also survive. The difference is who can reach it.

| | Closure | Global |
|---|---|---|
| Who can change it | Only the inner function | Any line in the file |
| How many copies | One per call of the outer function | Exactly one, ever |
| Changed with | `nonlocal` | `global` |
| Found by reading | The enclosing function | The whole file |

When two parts of a program each need their own running total, a closure gives each one its own. A global gives them one shared total and a bug that appears only when both are used.

## Iterators

### What Is an Iterator?

There are two different things here, and keeping them apart makes the rest easy.

- An **iterable** is something you can loop over: a list, a string, a dictionary.
- An **iterator** is the thing that does the walking. It remembers where it got to, and it gives one item at a time.

A `for` loop quietly makes an iterator from your iterable and calls it until it runs out.

### `iter()`

Most of the time Python creates iterators for you, but `iter()` lets you create one yourself and see how it works.

```python runnable title="Getting the walker"
marks = [78, 85, 60]

walker = iter(marks)
print(type(marks))
print(type(walker))
```

### `next()`

Once you have an iterator, `next()` moves it forward one step at a time. When there are none left it raises `StopIteration` — which is exactly how a `for` loop knows to stop.

```python runnable title="Walking by hand"
marks = [78, 85]
walker = iter(marks)

print(next(walker))
print(next(walker))

try:
    next(walker)
except StopIteration:
    print("StopIteration — nothing left")

print(next(iter([]), "nothing there"))
```

The second argument to `next()` is a default to return instead of raising.

### Writing Your Own Iterator

An object is an iterator if it has `__iter__` (which returns itself) and `__next__` (which returns the next item or raises `StopIteration`).

```python runnable title="A countdown you can loop over"
class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        self.current -= 1
        return self.current + 1

print(list(Countdown(3)))

for number in Countdown(3):
    print(number)
```

An iterator is used up. Once it has raised `StopIteration`, looping over it again gives nothing at all — it does not start over. A list can be looped over again and again; the iterator made from it cannot.

```python runnable title="Used up, not reset"
marks = [78, 85]
walker = iter(marks)

print(list(walker))
print(list(walker))
print(list(marks))
```

## Generators

### What Is a Generator?

A **generator** is an iterator written as a function. You do not write `__iter__` or `__next__`, and you do not raise `StopIteration` — Python does all of it for you. All you do is use `yield` instead of `return`.

### `yield`

`yield` hands back one value and **pauses the function**, keeping every local variable exactly where it was. The next request carries on from that line.

```python runnable title="Watch where it pauses"
def names_of(students):
    for student in students:
        print("  (working on", student + ")")
        yield student.upper()

walker = names_of(["asha", "ravi"])
print("nothing has run yet")

print(next(walker))
print(next(walker))

try:
    next(walker)
except StopIteration:
    print("StopIteration")
```

Read the output in order. Calling `names_of` ran *nothing*. The work for each name happened only when it was asked for.

### Generator Functions

Any function with a `yield` in it is a generator function. Calling it gives back a generator; it does not run the body.

```python runnable title="A generator function"
def countdown(start):
    while start > 0:
        yield start
        start -= 1

print(type(countdown(3)))
print(list(countdown(3)))

for number in countdown(3):
    print(number)
```

### Generator Expressions

A generator expression looks like a list comprehension with round brackets instead of square ones. It produces values one at a time instead of building the whole list.

```python runnable title="Brackets change everything"
squares_list = [n * n for n in range(5)]
squares_gen = (n * n for n in range(5))

print(squares_list)
print(squares_gen)
print(list(squares_gen))

print(sum(n * n for n in range(5)))
```

When a generator expression is the only argument, the brackets can be dropped — that is the last line.

### Why It Matters

Here is the memory difference between the two, side by side:

```python runnable title="A list builds everything; a generator does not"
import sys

as_list = [n * n for n in range(100000)]
as_generator = (n * n for n in range(100000))

print("list:", sys.getsizeof(as_list), "bytes")
print("generator:", sys.getsizeof(as_generator), "bytes")
print(sum(as_generator))
```

The list holds a hundred thousand numbers at once. The generator holds its place in the count and nothing else.

Use a list when you need the values more than once, or need their length, or need to reach item 500 directly. Use a generator when you walk through once — reading a large file line by line is the everyday case.

## Decorators

### What Are Decorators?

A **decorator** is a function that takes a function and gives back a new one that does something extra. It is how you add timing, logging or a permission check to a function without opening it.

The `@` line is shorthand for one assignment, and seeing that written out is the fastest way to understand the whole topic.

```python runnable title="The @ line, written out longhand"
def announce(func):
    def wrapper(marks):
        print("  starting")
        result = func(marks)
        print("  finished")
        return result
    return wrapper

def average(marks):
    return sum(marks) / len(marks)

average = announce(average)

print(average([78, 85, 60]))
```

### Creating Decorators

The `@` does exactly what that assignment did.

```python runnable title="The same thing with @"
def announce(func):
    def wrapper(marks):
        print("  starting")
        result = func(marks)
        print("  finished")
        return result
    return wrapper

@announce
def average(marks):
    return sum(marks) / len(marks)

print(average([78, 85, 60]))
```

### Wrapping Functions

A wrapper that only accepts one argument can only decorate functions that take one argument. `*args` and `**kwargs` make it work for any function at all.

```python runnable title="A wrapper that fits any function"
def announce(func):
    def wrapper(*args, **kwargs):
        print("  calling", func.__name__)
        return func(*args, **kwargs)
    return wrapper

@announce
def average(marks):
    return sum(marks) / len(marks)

@announce
def report(name, mark, subject="Maths"):
    return f"{name} scored {mark} in {subject}"

print(average([78, 85]))
print(report("Asha", 78))
print(report("Ravi", 85, subject="Science"))
```

There is one casualty, and it catches everybody.

```python runnable title="The decorated function has lost its name"
def announce(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@announce
def average(marks):
    """Average of the marks."""
    return sum(marks) / len(marks)

print(average.__name__)
print(average.__doc__)
```

The name is `wrapper` and the docstring is gone, because `average` now *is* the wrapper. `functools.wraps` copies them across.

```python runnable title="functools.wraps keeps the identity"
import functools

def announce(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@announce
def average(marks):
    """Average of the marks."""
    return sum(marks) / len(marks)

print(average.__name__)
print(average.__doc__)
print(average([78, 85, 60]))
```

Always put `@functools.wraps(func)` on your wrapper. Without it, every decorated function in your program reports itself as `wrapper` in tracebacks, in `help()` and in logs — and finding the real one is then your problem.

### Passing Arguments to a Decorator

A decorator that takes its own settings needs one more layer: a function that returns the decorator.

```python runnable title="A decorator with a setting"
import functools

def repeat(times):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for _ in range(times):
                func(*args, **kwargs)
        return wrapper
    return decorator

@repeat(3)
def roll_call(name):
    print("Present:", name)

roll_call("Asha")
```

Read `@repeat(3)` as: call `repeat(3)`, which gives back a decorator, and apply that.

### Multiple Decorators

Several `@` lines stack. The one **closest to the `def` is applied first**, and the outermost one runs its own code first.

```python runnable title="Order matters"
def shout(func):
    def wrapper(*args):
        return func(*args).upper()
    return wrapper

def exclaim(func):
    def wrapper(*args):
        return func(*args) + "!"
    return wrapper

@shout
@exclaim
def label(name):
    return name

@exclaim
@shout
def other(name):
    return name

print(label("asha"))
print(other("asha"))
```

Both printed `ASHA!` here, because upper-casing and adding `!` happen not to clash. With a decorator that trims text, the order would change the answer — so read the stack from the bottom up.

### Common Uses

Here is the shape in real use — timing a function without touching a line inside it:

```python runnable title="Timing, with nothing changed inside the function"
import functools
import time

def timed(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        started = time.perf_counter()
        result = func(*args, **kwargs)
        print(f"  {func.__name__} took {(time.perf_counter() - started) * 1000:.2f} ms")
        return result
    return wrapper

@timed
def total_marks(count):
    return sum(range(count))

print(total_marks(50000))
```

The everyday jobs for a decorator are all of this shape: timing, logging a call, caching a result, checking a permission before the function runs, and retrying after a failure. In every case the function itself stays unchanged and unaware.

## Common mistakes

Before moving on, here are some of the mistakes beginners make with these topics.

Each row is one mistake. The middle column is what you will actually see on screen.

<div className="tscroll">

| Mistake | What you see | Why it happens | Fix |
|---|---|---|---|
| `lambda m: return m * 2` | `SyntaxError` | A lambda body is an expression, not a statement | Drop `return`: `lambda m: m * 2` |
| Printing `map(...)` | `<map object at 0x...>` | `map` and `filter` are lazy | Wrap in `list()` |
| Reusing a `map` or `filter` result twice | The second use is empty | It is an iterator, and it is used up | Build a list once |
| Changing an outer variable without `nonlocal` | `UnboundLocalError` | Assigning makes the name local | Add `nonlocal count` |
| `nonlocal` used at the top level | `SyntaxError: no binding for nonlocal` | There is no enclosing function | Use `global`, or restructure |
| `lambda: i` inside a loop | Every one gives the last `i` | The lambda looks `i` up when called | `lambda i=i: i` |
| Looping over an iterator twice | Nothing the second time | An iterator does not reset | Loop over the list, not the iterator |
| `return` where `yield` was meant | A value, not a generator | One `yield` is what makes it a generator | Use `yield` |
| A wrapper without `*args` | `TypeError: wrapper() takes 1 positional argument` | The wrapper's shape must fit any function | `def wrapper(*args, **kwargs)` |
| A decorator without `functools.wraps` | `__name__` is `wrapper` | The original's identity was not copied | `@functools.wraps(func)` |
| Forgetting `return wrapper` | The decorated function becomes `None` | The decorator returned nothing | Return the wrapper |

</div>

## What to Remember

- A function is a value: it can be stored, passed, built and returned.
- A **lambda** is a one-expression function with no name. Use it where it is read, not as a named definition.
- `map()` and `filter()` are lazy and used up once. A comprehension is usually clearer.
- `sorted(..., key=...)` is where a lambda earns its place.
- A **closure** is an inner function that remembers a value from the function that made it. Each call of the outer function makes a new one.
- Reading an outer variable is free; changing one needs `nonlocal`.
- An **iterable** can be looped over; an **iterator** does the walking and is used up once.
- `iter()` makes one, `next()` advances it, `StopIteration` ends it.
- A **generator** is an iterator written with `yield`. Calling the function runs nothing.
- A generator expression is a comprehension in round brackets — one value at a time, not a whole list.
- A **decorator** returns a replacement function. `@thing` is just `name = thing(name)`.
- Give every wrapper `*args, **kwargs` and `@functools.wraps(func)`.

Next: **Class Relationships**