Locked learning resources

Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Locked learning resources

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Decorators

00:00 In the previous lesson, I showed you how to use closures. In this lesson, I’m going to talk about a special use of closures called decorators. You’ve seen closures used to contain data and update data.

00:13 Everything in Python is an object, including the functions, which means they can be treated like data. In the top area here, I have a closure that takes a function as its data. “Dude, I heard you like functions!

00:25 How about an inner function that takes a reference to a function?” Like the turtles, it’s functions all the way down. Why would you do this craziness? Well, this mechanism allows you to create preconditions and postconditions for a function. That would be code that is automatically run before and/or after the wrapped function. Two common uses for this are logging—for example, logging a function was called and when it was returned—and timing, timing how long a function took. In the top area here, I’ve written a closure that takes a function. It records the start time, runs the function passed in, then checks the amount of time that has passed.

01:04 Don’t let this hurt your brain. This really isn’t any different from the closures you’ve seen before. The only change is that the data being passed in is a function reference instead of an integer. Line 4 defines the containing function that takes a reference to a function called func().

01:22 Line 5 is the inner function. If you haven’t seen the *args and **kwargs mechanism before, they’re a way of accepting any number of arguments and keyword arguments to a function.

01:34 Line 6 grabs the current time and line 7 is the meat. This is where the data function contained in the closure is run. Whatever arguments are passed to the closure are used with the data function.

01:46 Lines 8 and 9 figure out how much time has passed and then print out the result. Line 11 is our standard “Return the inner function so this is a closure” line.

01:57 Let’s see this in action. First, I’ll define a function in the REPL that I will time.

02:08 This function doesn’t do much but keep the processor busy by counting from 0 to num. This takes me back. My very first programming language was BASIC on a cartridge.

02:17 My first computer didn’t have a hard drive. Yes, I’m that old. The version of BASIC I was using didn’t have timers in the library, so if you wanted to wait some amount of time, you’d spin like this in a loop. You had to base your counter on your processor speed to turn loop iterations into seconds.

02:34 When someone tells you about the good old days, they’re either lying to you or misremembering. Tangents aside, let me call this function. Looping 10000 doesn’t take very long on a modern machine.

02:48 Now let’s use the fn_timer() closure to wrap the count() function. First, I’ll import it.

02:57 Now I’m going to create a closure on the count() function.

03:02 The timed_count closure is now ready to go. Calling timed_count(), passing in 10000,

03:11 results in the closure being called, the time recorded, countthe enclosed captured variable data function—is run, and the execution time calculated and printed out.

03:22 This pattern of closures on functions to be executed is powerful and so common that Python provides some additional syntax to make it easier to write. This is called a decorator.

03:34 What I just demonstrated was the hard way to do things. The easier way is to use the decorator syntax. Remember, a decorator is simply a closure that takes a data function and executes it. Python uses the @ symbol to denote the use of a decorator.

03:50 In this example, @decorator is the closure function that is wrapping decorated_func(). This saves you the step of creating and executing the actual closure.

04:00 You just sprinkle the decorator over top and you’re good to go. Somewhere else in this code, you would have had to have written a closure-generating function called decorator(), which returns a closure, but if you have, the @ symbol makes it easier to wrap anything.

04:14 There’s less code to write. All right, I’m back in the REPL. In the top area, I have the exact same code as before. Nothing has changed in fn_timer() (function timer). In the bottom area, I’m going to define the count() function like I did before but this time, instead of using fn_timer() as a closure generator, I’ll use it as a decorator. Let me just import the decorator,

04:39 use the @ symbol to wrap the function, and then define the function.

04:53 And that’s it! This is the same as before, but this time you don’t have to track another variable for the function closure. You just use count() directly, and it is already wrapped. Let me call it.

05:06 And there you go! One side effect to note here. Because the decorator’s replacing a closure that wraps a data function, the calling information associated with count() will no longer be correct.

05:20 Here’s the function name associated with count. Notice that it is actually the wrapping function rather than 'count' itself. This can also mess with your documentation.

05:33 The docstring associated with count is no longer what you’d expect. It’s the docstring associated with time_wrapper, which in this case, doesn’t have one. All of this can be fixed with functools.

05:44 Let me show you that. What’s the solution to your function name and docs being wrong because you used a decorator? Well, if you guessed another decorator, have a cookie!

05:56 The functools library has a decorator inside of it called @wraps that fixes the problem I showed you before. All you have to do to use it is decorate your function closure.

06:06 This time, I’ve switched it up and written a little logging decorator. It’s a bad logging decorator as it only uses print(), but it’s good enough for our purposes. On line 5, I use the @wraps decorator to wrap the closure.

06:20 You’ll notice that I’ve named the closure wrapper(). This is a common pattern. As it’s an inner function, nobody really cares what it’s called.

06:27 The name of the decorator has to be meaningful to other programmers, but the closure inside can just be generic. With the @wraps decorator in place on the closure wrapper(), the name and the docs problem goes away. Let me import it,

06:44 define a function that’s decorated,

06:53 call the function, and from the output, you can see it did its job. Now let’s examine what the function thinks its name is.

07:06 Hm, that’s better! And here’s the docstring. functools to the rescue! Oh, were you here for an argument? That’s down the hall. Next up, decorators that take parameters.

Avatar image for Bastian

Bastian on Sept. 2, 2021

What would you recommend in terms of typing hints for a decorator?

Avatar image for Bartosz Zaczyński

Bartosz Zaczyński RP Team on Sept. 3, 2021

@Bastian Decorators are usually defined as functions, so regular type hinting rules apply:

import functools
import time
from typing import Any, Callable

def measure_time(function: Callable) -> Callable:
    @functools.wraps(function)
    def wrapper(*args, **kwargs) -> Any:
        t1 = time.perf_counter()
        result = function(*args, **kwargs)
        t2 = time.perf_counter()
        print(f"{function.__name__}() executed in {t2 - t1:.2f} seconds")
        return result
    return wrapper

@measure_time
def add(a, b):
    return a + b

On the other hand, if you decide not to use the @-syntax but rather decorate functions manually, then you could try using the Callable hint to declare a decorator:

def decorate(function: Callable, decorator: Callable) -> Callable:
    return decorator(function)

decorated_add = decorate(add, measure_time)

Is that what you were asking about?

Avatar image for Geir Arne Hjelle

Geir Arne Hjelle RP Team on Sept. 3, 2021

Hi @Bastian,

as Bartosz says, decorators are functions so you will typically use Callable to type hint them. However, this is not completely satisfactory because there are several typical decorator patterns that are simply not possible to annotate so that they can be correctly type checked. Unless you have very specific decorators, it’s hard to preserve the type information of the original function that is decorated.

To continue on Bartosz’ example from above, if you annotate add() as follows:

@measure_time
def add(a: int, b: int) -> int:
    return a + b

Even though you have annotated both input types and return types, those hints are lost because the function has been replaced by wrapper() returned by the @measure_time decorator.

It’s possible to use a type variable to preserve the return value:

import functools
from typing import Any, TypeVar

R = TypeVar("R")

def measure_time(function: Callable[..., R]) -> Callable[..., R]:
    @functools.wraps(function)
    def wrapper(*args: Any, **kwargs: Any) -> R:
        # Same as above
    return wrapper

Here, the type R will be matched, so that for instance for add() as above, R will represent int and the decorated version of add() will also have int as its return type.

In Callable[..., R], the three dots are valid code and represent any specification of input parameter types. For a general decorator like this, that is really the best we can do. However, there is no validation that there is a match between the different instances of ... or the use of Any in the wrapper() signature.

Type variables don’t really work for this, because they would force you to specify the number of input parameters (one type variable per parameter), and there is currently not any other mechanism for preserving an undetermined number of types.

This has been recognized and in Python 3.10, which will be officially released in October, a new feature called Parameter Specification will be included, which will solve exactly this use case. You can read all the details, including examples of what is currently not possible and how that will be fixed, in the enhancement proposal: PEP 612

Avatar image for Bastian

Bastian on Sept. 3, 2021

Great! Thanks to both of you for the detailed answers!

Small off-topic note: the code in the videao @03:26 reports seconds as milliseconds.

Avatar image for Christopher Trudeau

Christopher Trudeau RP Team on Jan. 16, 2022

Thanks @mo. Good catch. Code’s been there for almost a year and you’re the first to catch that. I’ll get it patched. …ct

Become a Member to join the conversation.