common.runtime.collector

Collector Objects

class Collector(ABC)

[view_source]

update

@abstractmethod
def update(name: str,
           inc: int = 1,
           total: int = None,
           message: str = None,
           label: str = None) -> None

[view_source]

Creates or updates a counter

This function updates a counter name with a value inc. If counter does not exist, it is created with optional total value of total. Depending on implementation label may be used to create nested counters and message to display additional information associated with a counter.

Arguments:

• name str - An unique name of a counter, displayable.
• inc int, optional - Increase amount. Defaults to 1.
• total int, optional - Maximum value of a counter. Defaults to None which means unbound counter.
• message str, optional - Additional message attached to a counter. Defaults to None.
• label str, optional - Creates nested counter for counter name. Defaults to None.

__call__

def __call__(step: str) -> TCollector

[view_source]

Syntactic sugar for nicer context managers

NullCollector Objects

class NullCollector(Collector)

[view_source]

A default counter that does not count anything.

DictCollector Objects

class DictCollector(Collector)

[view_source]

A collector that just counts

LogCollector Objects

class LogCollector(Collector)

[view_source]

A Collector that shows progress by writing to a Python logger or a console

__init__

def __init__(log_period: float = 1.0,
             logger: Union[logging.Logger, TextIO] = sys.stdout,
             log_level: int = logging.INFO,
             dump_system_stats: bool = True) -> None

[view_source]

Collector writing to a logger every log_period seconds. The logger can be a Python logger instance, text stream, or None that will attach dlt logger

Arguments:

• log_period float, optional - Time period in seconds between log updates. Defaults to 1.0.
• logger logging.Logger | TextIO, optional - Logger or text stream to write log messages to. Defaults to stdio.
• log_level str, optional - Log level for the logger. Defaults to INFO level
• dump_system_stats bool, optional - Log memory and cpu usage. Defaults to True

TqdmCollector Objects

class TqdmCollector(Collector)

[view_source]

A Collector that shows progress with tqdm progress bars

__init__

def __init__(single_bar: bool = False, **tqdm_kwargs: Any) -> None

[view_source]

A Collector that uses tqdm to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to tqdm in kwargs

AliveCollector Objects

class AliveCollector(Collector)

[view_source]

A Collector that shows progress with alive-progress progress bars

__init__

def __init__(single_bar: bool = True, **alive_kwargs: Any) -> None

[view_source]

Collector that uses alive_progress to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to alive_progress in kwargs

EnlightenCollector Objects

class EnlightenCollector(Collector)

[view_source]

A Collector that shows progress with enlighten progress and status bars that also allow for logging.

__init__

def __init__(single_bar: bool = False, **enlighten_kwargs: Any) -> None

[view_source]

Collector that uses Enlighten to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to Enlighten in kwargs