Intro

Intro#

There is a set of packages that are preinstalled with the Python build or so common that they are used in almost any Python project, either directly or indirectly. We will call these packages the standard library, and they are considered in this section.

Logging#

Logging is a built-in Python library for organizing logs. Its purpose is to create different Logger objects, each of which can be used in a specific part of the program, allowing you to maintain control over your program’s output.

For more details check corresponding page.

The following example demonstrates how to create a logger, show_logger, and attach different handlers to it. Handlers define the destination of the output, and each handler has a unique formatter—an object that defines the format of the records produced by the corresponding handler.

import logging

show_logger = logging.getLogger("show logger")

handler1 = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
handler1.setFormatter(formatter)

handler2 = logging.StreamHandler()
formatter = logging.Formatter('%(message)s|%(asctime)s|%(levelname)s')
handler2.setFormatter(formatter)

show_logger.addHandler(handler1)
show_logger.addHandler(handler2)

show_logger.critical("This is my message")

2024-08-30 11:58:58,516 - CRITICAL - This is my message
This is my message|2024-08-30 11:58:58,516|CRITICAL

In the end, we receive messages formatted according to the logger’s settings.

Functools#

Functools is a module in Python that provides functions that return other functions that have specific properties.

Function	Description	Example
`cache(user_function)` (3.9+)	Decorator that caches results indefinitely (like `lru_cache` with `maxsize=None`).	`@cache` on `fib(n)` stores computed results.
`cached_property(func)`	Turns a method into a read-only property whose value is computed once and cached.	`@cached_property def area(self): return self.w*self.h`
`cmp_to_key(func)`	Converts an old-style cmp function to a key function for sorting.	`sorted(items, key=cmp_to_key(my_cmp))`
`lru_cache(maxsize=128, typed=False)`	Memoizes function results with an LRU eviction policy.	`@lru_cache(maxsize=None)` caches without limit.
`partial(func, /, args, *keywords)`	Fixes some args of a function and returns a new callable.	`partial(pow, exp=2)(3) → 9`
`partialmethod(func, /, args, *keywords)`	Like `partial`, but for methods in classes.	`partialmethod(setattr, name='x')`
`reduce(function, iterable[, initializer])`	Cumulatively applies a function to items, reducing to one value.	`reduce(add, [1,2,3], 0) → 6`
`singledispatch(func)`	Creates a single-dispatch generic function.	`@singledispatch def func(x): ...`
`singledispatchmethod(func)`	Like `singledispatch`, but for methods.	`@singledispatchmethod def m(self, arg): ...`
`total_ordering(cls)`	Fills in missing rich comparison methods given `__eq__` and one ordering method.	`@total_ordering` on a class with `__eq__` and `__lt__`.
`update_wrapper(wrapper, wrapped, ...)`	Updates a wrapper function to look like the wrapped function.	`update_wrapper(wrapper, original_func)`
`wraps(wrapped, ...)`	Decorator that applies `update_wrapper` to a wrapper function.	`@wraps(func) def wrapper(a, *k): ...`

Check corresponding documentation.

Partial#

With functools.partial you can create a function that wraps another function in order to define default values for the arguments of the wrapped function. This is really useful when you need to pass the same set of arguments many times for example in unit tests.

The following cell defines a function with three parameters and demostrates how to call it.

from functools import partial

def some_fun(a, b, c):
    print(f"{a}, {b}, {c}")

some_fun(1, 2, 3)

1, 2, 3

Now created default_some_fun function, which is the same as some_fun but uses a=3 and b=10 as default values.

default_some_fun = partial(some_fun, a=3, c=10)

The following cell shows the default_some_fun call with b set to “hello”

default_some_fun(b="hello")

3, hello, 10

Obviously it is just like some_fun(3, "hello", 10).

Note: It is possible to redefine values specified in parital. The following code calls default_some_fun with new value for a parameter.

default_some_fun(a="three", b="hello")

three, hello, 10

As a result, instead of 3 specified by partical, we get three specified in the call to the result.

Note: With partial you can redefine any callable you like not only functions, for example you can specify a class that will always have the same argument of the __init__. The following cell creates a special object that will behave just like a dict but will have a default hello: 10 key-value pair.

value = partial(dict, hello=10)
value(), value(new=3)

({'hello': 10}, {'hello': 10, 'new': 3})

Itertools#

Functions that create iterators for effecient looping. The following table shows some of the most widely used functions:

Function	Description	Example
`count(start=0, step=1)`	Infinite count starting at `start`, incremented by `step`.	`count(10, 2) → 10, 12, 14, …`
`cycle(iterable)`	Repeats elements of the iterable indefinitely.	`cycle('AB') → A, B, A, B, …`
`repeat(object, times=None)`	Repeats the object, optionally `times` times.	`repeat(5, 3) → 5, 5, 5`
`accumulate(iterable, func=add)`	Running totals or accumulated results with a binary func.	`accumulate([1,2,3]) → 1, 3, 6`
`chain(*iterables)`	Concatenates iterables.	`chain('AB', 'CD') → A, B, C, D`
`chain.from_iterable(iterables)`	Flattens iterable of iterables.	`chain.from_iterable(['AB', 'CD']) → A, B, C, D`
`compress(data, selectors)`	Filters data by selectors.	`compress('ABC', [1,0,1]) → A, C`
`dropwhile(predicate, iterable)`	Skips while predicate is true, returns rest.	`dropwhile(x<5, range(10)) → 5…9`
`filterfalse(predicate, iterable)`	Returns items where predicate is false.	`filterfalse(is_even, [1,2,3]) → 1, 3`
`groupby(iterable, key=None)`	Groups consecutive equal elements.	`groupby('AAAB') → ('A', ['A','A','A']), ('B',['B'])`
`islice(iterable, start, stop, step=1)`	Iterator slice.	`islice(range(10), 2, 8, 2) → 2, 4, 6`
`pairwise(iterable)`	Consecutive overlapping pairs.	`pairwise([1,2,3]) → (1,2), (2,3)`
`starmap(func, iterable)`	Maps func with arguments unpacked from tuples.	`starmap(pow, [(2,3), (3,2)]) → 8, 9`
`takewhile(predicate, iterable)`	Takes while predicate is true.	`takewhile(x<5, range(10)) → 0…4`
`tee(iterable, n=2)`	Returns n independent iterators.	`tee([1,2], 2) → two iters of 1,2`
`zip_longest(*iterables, fillvalue=None)`	Zips to the longest iterable, filling missing values.	`zip_longest('AB', 'C', fillvalue='-') → (A,C), (B,-)`
`product(*iterables, repeat=1)`	Cartesian product.	`product([1,2], repeat=2) → (1,1),(1,2),(2,1),(2,2)`
`permutations(iterable, r=None)`	r-length permutations.	`permutations('AB') → ('A','B'), ('B','A')`
`combinations(iterable, r)`	r-length combinations, no repeats.	`combinations('ABC', 2) → ('A','B'), ('A','C'), ('B','C')`
`combinations_with_replacement(iterable, r)`	r-length combinations, with repeats.	`combinations_with_replacement('AB', 2) → ('A','A'), ('A','B'), ('B','B')`

Check more details in the itertools page of official documentation.

Textwrap#

The textwrap package is for text formatting. It contains the following functions:

Function	Description	Key Parameters (examples)
`wrap()`	Wraps a single paragraph into a list of lines.	`width=70`, `expand_tabs`, `replace_whitespace`, `drop_whitespace`
`fill()`	Wraps text and returns it as a single string with newlines.	Same as `wrap()`
`shorten()`	Truncates text to fit within a given width, ending with placeholder.	`width`, `placeholder="..."`
`dedent()`	Removes common leading whitespace from every line.	(no extra parameters)
`indent()`	Adds a prefix to selected lines of text.	`prefix`, `predicate` (function deciding which lines get prefix)

The TextWrapper class is used for more efficient processing.

For more details, check textwrap page of the official documentation.

Tracemalloc#

The tracemalloc package allows you to monitor memory in python. For more information, check tracemalloc - Trace memory allocations page of official documentation.

Check more detainls in the Tracemalloc page.

To eliminate calls associated with the jupyter notebook, separate script supposed to be reated.

%%writefile /tmp/my_file.py
import tracemalloc

# Start monitoring
tracemalloc.start()

value = 10

# Taking information from the calls up to the present moment
snapshot = tracemalloc.take_snapshot()
print(snapshot.statistics('lineno'))

Overwriting /tmp/my_file.py

The script’s run is in the following cell.

!python3 /tmp/my_file.py

[]

The snapshot.statistisc('lineno') returns a list of different memory allocations. In the example under consideration, there is just one line that corresponds to the value = 10 call.