More Precise Types

In this lesson, you’ll learn about more precise types in Python 3.8. Python’s typing system is quite mature at this point. However, in Python 3.8, some new features have been added to typing to allow more precise typing:

• Literal types
• Typed dictionaries
• Final objects
• Protocols

Python supports optional type hints, typically as annotations on your code:

>>> def double(number: float) -> float:
...     return 2 * number

In this example, you say that number should be a float and double() should return a float as well. However, Python treats these annotations as hints. They are not enforced at runtime:

>>> double(3.14)
6.28

>>> double("I'm not a float")
"I'm not a floatI'm not a float"

double() happily accepts "I'm not a float" as an argument, even though that’s not a float.

Type hints allow static type checkers to do type checking of your Python code, without actually running your scripts. This is reminiscent of compilers catching type errors in other languages like Java, and Rust. Additionally, type hints act as documentation of your code, making it easier to read, as well as improving auto-complete in your IDE.

$ python -m pip install mypy

Try out Mypy on the code example from before. Create a new file named float_check.py:

# float_check.py

def double(number: float) -> float:
    return 2 * number

double(3.14)

double("I'm not a float")

Now run Mypy on this code:

$ mypy float_check.py
float_check.py:8: error: Argument 1 to "double" has incompatible
                        type "str"; expected "float"
Found 1 error in 1 file (checked 1 source file)

Based on the type hints, Mypy is able to tell you that you are using the wrong type on line 8. Change the argument for the second call to double() to a float:

# float_check.py

def double(number: float) -> float:
    return 2 * number

double(3.14)

double(2.4)

Now run Mypy on this code again:

$ mypy float_check.py
Success: no issues found in 1 source file

In some sense, Mypy is the reference implementation of a type checker for Python, and is being developed at Dropbox under the lead of Jukka Lehtasalo. Python’s creator, Guido van Rossum, is part of the Mypy team.

You can find more information about type hints in Python in the original PEP 484, as well as in Python Type Checking (Guide), and the video course Python Type Checking.

There are four new PEPs about type checking that have been accepted and included in Python 3.8. You’ll see short examples from each of these.

PEP 586 introduced the Literal type. Literal is a bit special in that it represents one or several specific values. One use case of Literal is to be able to precisely add types, when string arguments are used to describe specific behavior. Consider the following example:

# draw_line.py

def draw_line(direction: str) -> None:
    if direction == "horizontal":
        ...  # Draw horizontal line

    elif direction == "vertical":
        ...  # Draw vertical line

    else:
        raise ValueError(f"invalid direction {direction!r}")

draw_line("up")

The program will pass the static type checker, even though "up" is an invalid direction. The type checker only checks that "up" is a string. In this case, it would be more precise to say that direction must be either the literal string "horizontal" or the literal string "vertical". Using Literal, you can do exactly that:

# draw_line.py

from typing import Literal

def draw_line(direction: Literal["horizontal", "vertical"]) -> None:
    if direction == "horizontal":
        ...  # Draw horizontal line

    elif direction == "vertical":
        ...  # Draw vertical line

    else:
        raise ValueError(f"invalid direction {direction!r}")

draw_line("up")

By exposing the allowed values of direction to the type checker, you can now be warned about the error:

$ mypy draw_line.py
draw_line.py:15: error:
    Argument 1 to "draw_line" has incompatible type "Literal['up']";
    expected "Union[Literal['horizontal'], Literal['vertical']]"
Found 1 error in 1 file (checked 1 source file)

The basic syntax is Literal[<literal>]. For instance, Literal[38] represents the literal value 38. You can express one of several literal values using Union:

Union[Literal["horizontal"], Literal["vertical"]]

Since this is a fairly common use case, you can (and probably should) use the simpler notation Literal["horizontal", "vertical"] instead. You already used the latter when adding types to draw_line().

If you look carefully at the output from Mypy above, you can see that it translated the simpler notation to the Union notation internally.

There are cases where the type of the return value of a function depends on the input arguments. One example is open(), which may return a text string or a byte array depending on the value of mode. This can be handled through overloading.

The following example shows the skeleton of a calculator that can return the answer either as regular numbers (38) or as roman numerals (XXXVIII):

# calculator.py

from typing import Union

ARABIC_TO_ROMAN = [(1000, "M"), (900, "CM"), (500, "D"), (400, "CD"),
                   (100, "C"), (90, "XC"), (50, "L"), (40, "XL"),
                   (10, "X"), (9, "IX"), (5, "V"), (4, "IV"), (1, "I")]

def _convert_to_roman_numeral(number: int) -> str:
    """Convert number to a roman numeral string"""
    result = list()
    for arabic, roman in ARABIC_TO_ROMAN:
        count, number = divmod(number, arabic)
        result.append(roman * count)
    return "".join(result)

def add(num_1: int, num_2: int, to_roman: bool = True) -> Union[str, int]:
    """Add two numbers"""
    result = num_1 + num_2

    if to_roman:
        return _convert_to_roman_numeral(result)
    else:
        return result

The code has the correct type hints: the result of add() will be either str or int. However, often this code will be called with a literal True or False as the value of to_roman, in which case you would like the type checker to infer exactly whether str or int is returned. This can be done using Literal together with @overload:

# calculator.py

from typing import Literal, overload, Union

ARABIC_TO_ROMAN = [(1000, "M"), (900, "CM"), (500, "D"), (400, "CD"),
                   (100, "C"), (90, "XC"), (50, "L"), (40, "XL"),
                   (10, "X"), (9, "IX"), (5, "V"), (4, "IV"), (1, "I")]

def _convert_to_roman_numeral(number: int) -> str:
    """Convert number to a roman numeral string"""
    result = list()
    for arabic, roman in ARABIC_TO_ROMAN:
        count, number = divmod(number, arabic)
        result.append(roman * count)
    return "".join(result)

@overload
def add(num_1: int, num_2: int, to_roman: Literal[True]) -> str: ...
@overload
def add(num_1: int, num_2: int, to_roman: Literal[False]) -> int: ...

def add(num_1: int, num_2: int, to_roman: bool = True) -> Union[str, int]:
    """Add two numbers"""
    result = num_1 + num_2

    if to_roman:
        return _convert_to_roman_numeral(result)
    else:
        return result

The added @overload signatures will help your type checker infer str or int depending on the literal values of to_roman. Note that the ellipses (...) are a literal part of the code. They stand in for the function body in the overloaded signatures.

As a complement to Literal, PEP 591 introduces Final. This qualifier specifies that a variable or attribute should not be reassigned, redefined, or overridden. The following is a typing error:

# final_id.py

from typing import Final

ID: Final = 1

...

ID += 1

Mypy will highlight the line ID += 1, and note that you Cannot assign to final name "ID". This gives you a way to ensure that constants in your code never change their value.

Additionally, there is also a @final decorator that can be applied to classes and methods. Classes decorated with @final can’t be subclassed, while @final methods can’t be overridden by subclasses:

# final_class.py

from typing import final

@final
class Base:
    ...

class Sub(Base):
    ...

Mypy will flag this example with the error message Cannot inherit from final class "Base". To learn more about Final and @final, see PEP 591.

The third PEP allowing for more specific type hints is PEP 589, which introduces TypedDict. This can be used to specify types for keys and values in a dictionary using a notation that is similar to the typed NamedTuple.

Traditionally, dictionaries have been annotated using Dict. The issue is that this only allowed one type for the keys and one type for the values, often leading to annotations like Dict[str, Any]. As an example, consider a dictionary that registers information about Python versions:

py38 = {"version": "3.8", "release_year": 2019}

The value corresponding to version is a string, while release_year is an integer. This can’t be precisely represented using Dict. With the new TypedDict, you can do the following:

# typed_dict.py

from typing import TypedDict

class PythonVersion(TypedDict):
    version: str
    release_year: int

py38 = PythonVersion(version="3.8", release_year=2019)

The type checker will then be able to infer that py38["version"] has type str, while py38["release_year"] is an int. At runtime, a TypedDict is a regular dict, and type hints are ignored as usual:

>>> from typed_dict import *
>>> py38
{'version': '3.8', 'release_year': 2019}
>>> type(py38)
<class 'dict'>

Mypy will let you know if any of your values has the wrong type, or if you use a key that has not been declared. See PEP 589 for more examples.

Mypy has supported Protocols for a while already. However, the official acceptance only happened in May 2019.

Protocols are a way of formalizing Python’s support for duck typing:

When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck. (Source)

Duck typing allows you to, for example, read .name on any object that has a .name attribute, without really caring about the type of the object. It may seem counter-intuitive for the typing system to support this. Through structural subtyping, it’s still possible to make sense of duck typing.

You can for instance define a protocol called Named that can identify all objects with a .name attribute:

# protocol.py

from typing import Protocol

class Named(Protocol):
    name: str

def greet(obj: Named) -> None:
    print(f"Hi {obj.name}")

class Dog:
    ...

x = Dog()

greet(x)

After creating the Named protocol, and defining the function greet() that takes an argument of obj, the type hint is specifying that obj follows the Named protocol. Run the code through Mypy to see what it finds:

$ mypy protocol.py
protocol.py:16: error: Argument 1 to "greet" has incompatible type "Dog"; expected "Named"
Found 1 error in 1 file (checked 1 source file)

The Dog class did not have an attribute of .name, and therefore did not meet the check for the Named protocol. Add a .name attribute to the class, with a default string:

# protocol.py

from typing import Protocol

class Named(Protocol):
    name: str

def greet(obj: Named) -> None:
    print(f"Hi {obj.name}")

class Dog:
    name = 'Good Dog'

x = Dog()

greet(x)

Run protocol.py through Mypy again:

$ mypy protocol.py
Success: no issues found in 1 source file

As you have confirmed, greet() takes any object, as long as it defines a .name attribute. See PEP 544 and the Mypy documentation for more information about protocols.