plugins.h_tqdm.ProgressBar¶

Provides a progress bar for Apache Hamilton execution. Must have tqdm installed to use it:

pip install apache-hamilton[tqdm] (use quotes if using zsh)

class hamilton.plugins.h_tqdm.ProgressBar(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)[source]¶

An adapter that uses tqdm to show progress bars for the graph execution.

Note: you need to have tqdm installed for this to work. If you don’t have it installed, you can install it with pip install tqdm (or pip install apache-hamilton[tqdm] – use quotes if you’re using zsh).

from hamilton.plugins import h_tqdm

dr = (
    driver.Builder()
    .with_config({})
    .with_modules(some_modules)
    .with_adapters(h_tqdm.ProgressBar(desc="DAG-NAME"))
    .build()
)
# and then when you call .execute() or .materialize() you'll get a progress bar!
__init__(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)[source]¶

Create a new Progress Bar adapter.

Parameters:

  • desc – The description to show in the progress bar. E.g. DAG Name is a good choice.

  • kwargs – Additional kwargs to pass to TQDM. See TQDM docs for more info.

  • node_name_target_width – the target width for the node name so that the progress bar is consistent. If this is None, it will take the longest, until it hits max_node_name_width.

post_graph_execute(*, run_id: str, graph: FunctionGraph, success: bool, error: Exception | None, results: dict[str, Any] | None)¶

Just delegates to the interface method, passing in the right data.

post_node_execute(*, run_id: str, node_: Node, kwargs: dict[str, Any], success: bool, error: Exception | None, result: Any | None, task_id: str | None = None)¶

Wraps the after_execution method, providing a bridge to an external-facing API. Do not override this!

pre_graph_execute(*, run_id: str, graph: FunctionGraph, final_vars: list[str], inputs: dict[str, Any], overrides: dict[str, Any])¶

Implementation of the pre_graph_execute hook. This just converts the inputs to the format the user-facing hook is expecting – performing a walk of the DAG to pass in the set of nodes to execute. Delegates to the interface method.

pre_node_execute(*, run_id: str, node_: Node, kwargs: dict[str, Any], task_id: str | None = None)¶

Wraps the before_execution method, providing a bridge to an external-facing API. Do not override this!

run_after_graph_execution(*, success: bool = True, **future_kwargs)[source]¶

This is run after graph execution. This allows you to do anything you want after the graph executes, knowing the results of the execution/any errors.

Parameters:

  • graph – Graph that is being executed

  • results – Results of the graph execution

  • error – Error that occurred, None if no error occurred

  • success – Whether the graph executed successfully

  • run_id – Run ID (unique in process scope) of the current run. Use this to track state.

  • future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_after_node_execution(**future_kwargs)[source]¶

Hook that is executed post node execution.

Parameters:

  • node_name – Name of the node in question

  • node_tags – Tags of the node

  • node_kwargs – Keyword arguments passed to the node

  • node_return_type – Return type of the node

  • result – Output of the node, None if an error occurred

  • error – Error that occurred, None if no error occurred

  • success – Whether the node executed successfully

  • task_id – The ID of the task, none if not in a task-based environment

  • run_id – Run ID (unique in process scope) of the current run. Use this to track state.

  • future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_before_graph_execution(*, graph: HamiltonGraph, final_vars: list[str], inputs: dict[str, Any], overrides: dict[str, Any], execution_path: Collection[str], **future_kwargs: Any)[source]¶

This is run prior to graph execution. This allows you to do anything you want before the graph executes, knowing the basic information that was passed in.

Parameters:

  • graph – Graph that is being executed

  • final_vars – Output variables of the graph

  • inputs – Input variables passed to the graph

  • overrides – Overrides passed to the graph

  • execution_path – Collection of nodes that will be executed – these are just the nodes (not input nodes) that will be run during the course of execution.

  • run_id – Run ID (unique in process scope) of the current run. Use this to track state.

  • future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_before_node_execution(*, node_name: str, node_tags: dict[str, Any], node_kwargs: dict[str, Any], node_return_type: type, task_id: str | None, **future_kwargs: Any)[source]¶

Hook that is executed prior to node execution.

Parameters:

  • node_name – Name of the node.

  • node_tags – Tags of the node

  • node_kwargs – Keyword arguments to pass to the node

  • node_return_type – Return type of the node

  • task_id – The ID of the task, none if not in a task-based environment

  • run_id – Run ID (unique in process scope) of the current run. Use this to track state.

  • node_input_types – the input types to the node and what it is expecting

  • future_kwargs – Additional keyword arguments – this is kept for backwards compatibility