Annotations

Annotations#

Python has an option to annotate types, which is useful during relatively serious development.

Check resources:

  • Python type checking article on real python.

  • typing package description - contains typical patterns for annotation using.

  • PEP 484 that introduce type hints.

  • PEP 526 that introduce variable annotations.

import sys
sys.path.append("/tmp")

Usecases#

In general, you can define types of parameters and return values for the function and specify the type for some variable/attribute. This doesn’t affect the behavior of the interpreter in any way, but different code analisys tools make use of it.

The following cell creates python file that defines function that expect to take float and int and return bool. The function is wrapped with typeguard.typechecked decorator, which throws raise an error every time it is called with wrong types.

%%writefile /tmp/typed_fun.py
from typeguard import typechecked

@typechecked
def some_function(arg1: float, arg2: int) -> bool:
    return arg1 > arg2

Overwriting /tmp/typed_fun.py

The following code shows the exception you get when you try to pass parameters of the wrong type to the function.

from typed_fun import some_function
try: some_function(5.5, 3.3) 
except Exception as e: print(e)

argument "arg2" (float) is not an instance of int

The next cell generates a file that contains a variable that shoudl be of the datatype float datatype, but assigns str to the variable.

%%writefile /tmp/typed_var.py
pi: float = "test"

Overwriting /tmp/typed_var.py

mypy static analisator throws corresponding error to that file.

!python3 -m mypy /tmp/typed_var.py

/tmp/typed_var.py:1: error: Incompatible types in assignment (expression has type "str", variable has type "float")  [assignment]
Found 1 error in 1 file (checked 1 source file)

Annotation cases#

This section provided an overview of the various syntax and typing attributes that can be used to define types.

Category

Syntax

Description

Example

Basic types

int, str, bool, float, None

Built-in types

x: int = 5

Union

typing.Union[A, B] / A | B (3.10+)

One of several types

x: int | None or x: Union[int, None]

Optional

typing.Optional[T]

Shorthand for Union[T, None]

x: Optional[str] = None

List

typing.List[T] / list[T] (3.9+)

A list of items

names: list[str]

Tuple

typing.Tuple[<types>] / tuple

A fixed-size tuple

point: Tuple[int, int]

Dict

typing.Dict[K, V] / dict[K, V] (3.9+)

Dictionary with keys and values

user: dict[str, int]

Set

typing.Set[T] / set[T]

A set of items

tags: set[str]

Callable

typing.Callable[[Args], Return]

A function signature

f: Callable[[int, str], bool]

Literal

typing.Literal[val1, val2]

Restrict to specific literal values

mode: Literal["r", "w"]

Any

typing.Any

Skip type checking

data: Any

None

None

Variable takes None value

def void() -> None

NoReturn

typing.NoReturn

Function never returns

def fail() -> NoReturn:

Annotated

typing.Annotated[T, metadata]

Attach metadata to a type

x: Annotated[int, "positive"]

NewType

typing.NewType('Name', BaseType)

Create a distinct type based on an existing one

UserId = NewType('UserId', int)

TypeVar

typing.TypeVar('T')

Generic type variable

def f(x: T) -> T:

Generic

typing.Generic[T]

Base class for generic classes

class Box(Generic[T]):

Self

typing.Self (3.11+)

Return type of method returning its own class

def clone(self) -> Self:

ClassVar

typing.ClassVar[T]

Type of class-level variables

version: ClassVar[str] = "1.0"

Final

typing.Final

Marks values as constants

MAX_SIZE: Final[int] = 100

TypedDict

class Movie(typing.TypedDict):

Dict with specific key types

title: str; year: int

Protocol

class P(typing.Protocol):

Structural subtyping (duck typing)

def read(self) -> bytes:

Union with ellipsis

tuple[int, ...]

Homogeneous tuple of any length

values: Tuple[int, ...]

Predefinition

val: 'SomeType'

If the type isn’t defined yet use literal as annotation

T = list[tuple['T', 'T']]

Check more accurate description with examples on the conrresponding page.

Type aliases#

Note that you can save your annotations as regular python object - your programmes will be neater.

Below is an example that defines a function to create triangles as a tuple of three two-dimensional points.

from random import random

def get_triangle() -> tuple[
        tuple[float, float],
        tuple[float, float],
        tuple[float, float]
    ]:
    return ( 
        (random(), random()),
        (random(), random()),
        (random(), random())
    )

To make things simplier, you can define an annotation for the point and use it to define an annotation for the triangle.

point = tuple[float, float]
def get_triangle() -> tuple[point, point, point]:
    return ( 
        (random(), random()),
        (random(), random()),
        (random(), random())
    )

__annotations__ attribute#

The __annotations__ attribute allows you to retrieve annotated types for a Python object.

For function it returns a dictionary with keys corresponding to the names of the arguments and values corresponding to the types of the arguments. The return type of the function can be accessed using the return key.

def some_function(arg1: float, arg2: bool) -> int | None:
    if arg1 > 10 and arg2:
        return 20

some_function.__annotations__

{'arg1': float, 'arg2': bool, 'return': int}

You can simply access it from any Python namespace to get annotations of the waribles in that namespace.

new_variable : float = 10
test_variable : float = 10

__annotations__

{'new_variable': float, 'test_variable': float}

Overloads#

Overloads in Python allow you to define multiple function signatures with different argument and return type annotations, while sharing a single implementation.

Check the Overloads page of the official python documentation.

The following cells defines a foo function with overloads: it is annotated to return an int when given an int input, and a str when given a str input.

%%bash
rm -fr /tmp/test_module
mkdir /tmp/test_module

%%writefile /tmp/test_module/foo.py
from typing import overload

@overload
def foo(x: int) -> int:
    ...
@overload
def foo(x: str) -> str:
    ...
def foo(x: int | str) -> int | str: 
    return x

Writing /tmp/test_module/foo.py

The following cell contains a simple script that uses foo and lints it with mypy.

%%writefile /tmp/test_module/s.py
from foo import foo


a: int = foo(42)
b: str = foo("data")

Overwriting /tmp/test_module/s.py

!python3 -m mypy /tmp/test_module/s.py

Success: no issues found in 1 source file

In case of an int implementation, the function is assigned an int variable and a str implementation is assigned to the str variable.

The following cell performs the same trick, but assigns a function with str value to anint variable.

%%writefile /tmp/test_module/s.py
from foo import foo

a: int = foo("data")

Overwriting /tmp/test_module/s.py

!python3 -m mypy /tmp/test_module/s.py

/tmp/test_module/s.py:3: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 1 error in 1 file (checked 1 source file)

As the str input, there is just overload that returns str - we got the error.

Exceptions#

Practice shows that it’s difficult to write fully type-annotated code in python, especially if you’re a data scientist. It’s common for packages to violate the rules of specific linting systems. However, this is not a reason to avoid linting your code; there are tools for such cases:

  • Most linters understand comment # type: ingore as a reason to ignore linting mistakes in some code.

  • If package doesn’t provide annotations or makes it in the definitely wrong way you can use typing.cast to specify the type for the linting systems.

Consider a really typical example: a package is actually annotated, yet there are still issues with the linters you’re using.

The following cell creates a module containing a funciton for which for which the kwargs are not annotated.

%%bash
rm -rf /tmp/typing_exceptions
mkdir /tmp/typing_exceptions

%%writefile /tmp/typing_exceptions/function.py
from typing import overload

@overload
def function(a: int, **kwargs) -> int: ...
@overload
def function(a: str, **kwargs) -> str: ...
def function(a: int | str, **kwargs) -> int | str:
    return a

Writing /tmp/typing_exceptions/function.py

The following code represents a module that uses the “wrong” function from the linter’s perspective.

%%writefile /tmp/typing_exceptions/main.py
import function
function.function(10)

Writing /tmp/typing_exceptions/main.py

With the “strict” configuration, the pyright returns an error.

%%bash
cd /tmp/typing_exceptions
cat << EOF > pyrightconfig.json
{
    "typeCheckingMode": "strict"
}
EOF

python3 -m pyright main.py & true

/tmp/typing_exceptions/main.py
  /tmp/typing_exceptions/main.py:2:1 - error: Type of "function" is partially unknown
    Type of "function" is "Overload[(a: int, **kwargs: Unknown) -> int, (a: str, **kwargs: Unknown) -> str]" (reportUnknownMemberType)
1 error, 0 warnings, 0 informations

In a real-world project, there are no other solutions besides using the # type: ignore instruction of linter.

%%writefile /tmp/typing_exceptions/main.py
import function
function.function(10) # type: ignore

Overwriting /tmp/typing_exceptions/main.py

Now pyright doesn’t have any issues with the use of funciton.

%%bash
cd /tmp/typing_exceptions
python3 -m pyright main.py & true

0 errors, 0 warnings, 0 informations
Contents