Dynamic Loader (runtime.loader)

Dynamic component loader for pipeline components.

pyspark_pipeline_framework.runtime.loader.load_component_class(class_path)

Dynamically load a PipelineComponent subclass by its fully-qualified path.

Parameters:

class_path (str) – Dotted path such as "my_package.transforms.MyTransform".

Returns:

The loaded class (not an instance).

Raises:

ComponentInstantiationError – If the path is malformed, the module cannot be imported, the attribute does not exist, or the attribute is not a valid PipelineComponent subclass.

Return type:

type[PipelineComponent]

pyspark_pipeline_framework.runtime.loader.instantiate_component(config)

Create a component instance from a ComponentConfig.

Uses from_config(config.config) if the class provides it, otherwise falls back to cls(**config.config).

Parameters:

config (ComponentConfig) – Component configuration containing class_path and config dict.

Returns:

An instantiated PipelineComponent.

Raises:

ComponentInstantiationError – If loading or instantiation fails.

Return type:

PipelineComponent

pyspark_pipeline_framework.runtime.loader.validate_component_class(class_path)

Validate a component class and return any warnings.

Parameters:

class_path (str) – Fully-qualified path to the component class.

Returns:

A list of warning messages. An empty list means the class is valid with no concerns.

Raises:

ComponentInstantiationError – If the class cannot be loaded at all.

Return type:

list[str]

pyspark_pipeline_framework.runtime.loader.list_available_components(package)

List all PipelineComponent subclasses in a given package.

Parameters:

package (str) – Dotted package path to scan (e.g. "my_app.transforms").

Returns:

A sorted list of fully-qualified class paths.

Raises:

ComponentInstantiationError – If the package cannot be imported.

Return type:

list[str]