Secrets (core.secrets)

Base

Secrets management abstractions and models.

class pyspark_pipeline_framework.core.secrets.base.SecretResolutionStatus(*values)

Bases: str, Enum

Outcome of a secret resolution attempt.

SUCCESS = 'success'

NOT_FOUND = 'not_found'

ERROR = 'error'

class pyspark_pipeline_framework.core.secrets.base.SecretsReference(provider, key)

Bases: object

Reference to a secret in a specific provider.

Parameters:

• provider (str) – Provider name (e.g. "env", "aws", "vault").

• key (str) – Secret key or path within the provider.

provider: str

key: str

class pyspark_pipeline_framework.core.secrets.base.SecretResolutionResult(reference, status, value=None, error=None)

Bases: object

Result of resolving a secret reference.

The value field is masked in __repr__ to prevent accidental leakage in logs or tracebacks.

Parameters:

• reference (SecretsReference) – The original reference that was resolved.

• status (SecretResolutionStatus) – Outcome of the resolution.

• value (str | None) – The secret value (only set on success).

• error (str | None) – Error description (only set on failure).

reference: SecretsReference

status: SecretResolutionStatus

value: str | None = None

error: str | None = None

class pyspark_pipeline_framework.core.secrets.base.SecretsProvider

Bases: ABC

Base class for secrets providers.

Subclasses implement resolve() to fetch secrets from a specific backend (environment variables, AWS Secrets Manager, etc.).

abstract property provider_name: str

Unique name for this provider (e.g. "env", "aws").

abstractmethod resolve(reference)

Resolve a single secret reference.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

resolve_all(references)

Resolve multiple secret references.

Parameters:

references (list[SecretsReference])

Return type:

list[SecretResolutionResult]

Providers

Built-in secrets provider implementations.

class pyspark_pipeline_framework.core.secrets.providers.EnvSecretsProvider

Bases: SecretsProvider

Resolve secrets from environment variables.

No external dependencies required.

property provider_name: str

Unique name for this provider (e.g. "env", "aws").

resolve(reference)

Resolve a single secret reference.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

class pyspark_pipeline_framework.core.secrets.providers.AwsSecretsProvider(region_name=None)

Bases: SecretsProvider

Resolve secrets from AWS Secrets Manager.

Requires boto3 to be installed. The client is created lazily on the first call to resolve().

Parameters:

region_name (str | None) – AWS region. Defaults to boto3’s default region.

property provider_name: str

Unique name for this provider (e.g. "env", "aws").

resolve(reference)

Resolve a single secret reference.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

class pyspark_pipeline_framework.core.secrets.providers.VaultSecretsProvider(url, token=None, mount_point='secret')

Bases: SecretsProvider

Resolve secrets from HashiCorp Vault (KV v2 engine).

Requires hvac to be installed. The client is created lazily on the first call to resolve().

Key format: "path/to/secret" returns the "value" field, or "path/to/secret:field" returns a specific field.

Parameters:

• url (str) – Vault server URL.

• token (str | None) – Vault token. Defaults to VAULT_TOKEN environment variable.

• mount_point (str) – KV v2 mount point. Defaults to "secret".

property provider_name: str

Unique name for this provider (e.g. "env", "aws").

resolve(reference)

Resolve a single secret reference.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

Resolver

Secrets resolver and caching layer.

class pyspark_pipeline_framework.core.secrets.resolver.SecretsResolver

Bases: object

Composite resolver that routes requests to registered providers.

Each SecretsReference is dispatched to the provider whose provider_name matches SecretsReference.provider.

register(provider)

Register a secrets provider.

Parameters:

provider (SecretsProvider)

Return type:

None

resolve(reference)

Resolve a secret using the appropriate provider.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

resolve_all(references)

Resolve multiple secret references.

Parameters:

references (list[SecretsReference])

Return type:

list[SecretResolutionResult]

class pyspark_pipeline_framework.core.secrets.resolver.SecretsCache(resolver, ttl_seconds=300, clock=None)

Bases: object

Thread-safe caching wrapper for secret resolution.

Caches all resolution results (success, not-found, and error) with a configurable TTL. Use clear() to manually invalidate.

Parameters:

• resolver (SecretsResolver) – The underlying resolver to delegate to on cache miss.

• ttl_seconds (int) – Cache entry lifetime in seconds. Defaults to 300.

• clock (Callable[[], float] | None) – Injectable monotonic clock for testing. Defaults to time.monotonic.

resolve(reference)

Resolve a secret, returning a cached result if available.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

resolve_all(references)

Resolve multiple secret references with caching.

Parameters:

references (list[SecretsReference])

Return type:

list[SecretResolutionResult]

clear()

Clear all cached entries.

Return type:

None

Audit

Audit-aware wrapper for secrets resolution.

class pyspark_pipeline_framework.core.secrets.audit.SecretsAuditLogger(resolver, sink, actor='secrets_resolver')

Bases: object

Decorator that emits audit events for secret access.

Wraps a SecretsResolver or SecretsCache and emits an AuditEvent with AuditAction.SECRET_ACCESSED for every resolve() call. The secret value is never included in the audit trail.

Parameters:

• resolver (SecretsResolver | SecretsCache) – The underlying resolver or cache to delegate to.

• sink (AuditSink) – Audit sink that receives the events.

• actor (str) – Actor name recorded in audit events. Defaults to "secrets_resolver".

resolve(reference)

Resolve a secret and emit an audit event.

Parameters:

reference (SecretsReference)

Return type:

SecretResolutionResult

resolve_all(references)

Resolve multiple secrets, emitting an audit event for each.

Parameters:

references (list[SecretsReference])

Return type:

list[SecretResolutionResult]