Runtime and assets

Environment paths, local asset manifests, readiness inspection, and bounded subprocess execution.

On this page

Runtime helpers make machine-specific state explicit. They resolve shared roots, inspect locally staged assets, redact credentials for manifests, and turn timeout-prone subprocesses into structured results. They do not download protected data or silently replace missing assets.

RequiredEnvReport

This report separates present and missing variable names. Values are deliberately absent so it can be included in logs and preflight responses.

class RequiredEnvReport(missing: tuple[str, ], present: tuple[str, ])
clsworldfoundry.runtime.env.RequiredEnvReportfrom worldfoundry.runtime.env import RequiredEnvReport
source

Overview

Report of required vs present environment variables / tools for a workload.

Attributes

missingtuple[str, ...]
Required environment variable names that were not set.
presenttuple[str, ...]
Required environment variable names that were set.

Methods

propok -> boolsource

Overview

Return whether all required variables were present.

Parameters

self

Returns: bool

methto_dict() -> dict[str, Any]source

Overview

Serialize the report for manifests and JSON preflight output.

Returns: dict[str, Any]

WorldFoundryEnv

WorldFoundryEnv provides one object-oriented view over the path-resolution and preflight functions in worldfoundry.runtime.env.

from worldfoundry.runtime.env import WorldFoundryEnv

env = WorldFoundryEnv.from_os()
print(env.resolve_model_dir())
print(env.resolve_data_dir())
print(env.resolve_artifact_dir())

report = env.check_required(("HF_TOKEN", "WORLDFOUNDRY_DATA_DIR"))
if not report.ok:
    print("missing:", report.missing)

The check reports presence only. Use redact_for_manifest() rather than copying os.environ into a run record.

class WorldFoundryEnv(values: EnvMapping)
clsworldfoundry.runtime.env.WorldFoundryEnvfrom worldfoundry.runtime.env import WorldFoundryEnv
source

Overview

Helpers for reading and validating WorldFoundry-related environment settings.

Attributes

valuesEnvMapping
Environment mapping used for path and metadata resolution.

Methods

cmethfrom_os() -> 'WorldFoundryEnv'source

Overview

Build an environment view from the current process environment.

Returns: 'WorldFoundryEnv'

methresolve_cache_dir() -> Pathsource

Overview

Resolve the root cache directory for downloads and transient assets.

Returns: Path

methresolve_data_dir() -> Pathsource

Overview

Resolve the shared local dataset directory.

Returns: Path

methresolve_model_dir() -> Pathsource

Overview

Resolve the local checkpoint and model asset directory.

Returns: Path

methresolve_ckpt_dir() -> Pathsource

Overview

Resolve the local checkpoint directory.

Returns: Path

methresolve_artifact_dir() -> Pathsource

Overview

Resolve the generated artifact output directory.

Returns: Path

methresolve_hf_cache_dir() -> Pathsource

Overview

Resolve the Hugging Face dataset/model cache directory.

Returns: Path

methredact_for_manifest(keys: Sequence[str] | None = None) -> dict[str, Any]source

Overview

Return manifest-safe environment values with secrets reduced to presence.

Parameters

keysSequence[str] | None
Optional variable names to include instead of the standard public set.default: None

Returns: dict[str, Any]

methcheck_required(names: Sequence[str]) -> RequiredEnvReportsource

Overview

Check that required variables are set without exposing their values.

Parameters

namesSequence[str]
Environment variable names required by a runtime.

Returns: RequiredEnvReport

methcapture_runtime(include_torch: bool = True,include_nvidia_smi: bool = True) -> dict[str, Any]source

Overview

Capture local Python, CUDA, GPU, and Torch metadata for preflight logs.

Parameters

include_torchbool
Whether to import Torch and record CUDA availability.default: True
include_nvidia_smibool
Whether to call `nvidia-smi` when available.default: True

Returns: dict[str, Any]

LocalAsset

A local asset combines a stable manifest identity with its resolved path and current ready state. Readiness means the path exists; it does not prove that a checkpoint is complete, a dataset license was accepted, or a runtime can load the contents.

class LocalAsset(benchmark_id: str | None,asset_id: str,kind: str,path: Path | None,canonical_path: Path | None,status: str,ready: bool,metadata: Mapping[str, Any])
clsworldfoundry.runtime.assets.LocalAssetfrom worldfoundry.runtime.assets import LocalAsset
source

Overview

Descriptor for a staged local asset (logical name → path) used by runners and tests.

Attributes

benchmark_idstr | None
Optional benchmark or integration id that owns the asset.
asset_idstr
Stable asset id inside the benchmark group.
kindstr
Asset kind, such as dataset, checkpoint, repo, manifest, or artifact.
pathPath | None
Resolved local path recorded by the manifest.
canonical_pathPath | None
Preferred path under the WorldFoundry root layout.
statusstr
Current path status computed at load time.
readybool
Whether the resolved path exists locally.
metadataMapping[str, Any]
Extra manifest fields preserved for consumers.

Methods

cmethfrom_manifest_item(item: Mapping[str, Any],benchmark_id: str | None = None,env: EnvMapping | None = None) -> 'LocalAsset'source

Overview

Build a local asset view from a manifest item.

Parameters

itemMapping[str, Any]
Manifest asset mapping.
benchmark_idstr | None
Optional parent benchmark id.default: None
envEnvMapping | None
Optional environment mapping used for path token expansion.default: None

Returns: 'LocalAsset'

methto_dict() -> dict[str, Any]source

Overview

Serialize the resolved asset status for logs and diagnostics.

Returns: dict[str, Any]

expand_worldfoundry_path

This helper expands supported $WORLDFOUNDRY_* tokens using the same root conventions as bootstrap and preflight commands. Relative results are anchored to the repository root.

def expand_worldfoundry_path(value: str | Path, env: EnvMapping | None = None) -> Path
funcworldfoundry.runtime.assets.expand_worldfoundry_pathfrom worldfoundry.runtime.assets import expand_worldfoundry_path
source

Overview

Expand WorldFoundry path tokens and ~ into concrete filesystem paths.

Parameters

valuestr | Path
Path string or `Path with optional $VAR or ${VAR}` tokens.
envEnvMapping | None
Optional environment mapping; defaults to `os.environ`.default: None

Returns: Path

load_local_assets

The loader reads the selected local asset manifest and returns resolved LocalAsset objects. This makes a small readiness script possible without invoking any GPU runtime.

from worldfoundry.runtime.assets import load_local_assets

assets = load_local_assets()
missing = [asset for asset in assets if not asset.ready]

for asset in missing[:10]:
    print(asset.asset_id, asset.path or asset.canonical_path)
def load_local_assets(path: str | Path | None = None,env: EnvMapping | None = None) -> tuple[LocalAsset, ...]
funcworldfoundry.runtime.assets.load_local_assetsfrom worldfoundry.runtime.assets import load_local_assets
source

Overview

Load the local-assets mapping used to stage checkpoints, clips, and fixtures.

Parameters

pathstr | Path | None
Optional explicit manifest path.default: None
envEnvMapping | None
Optional environment mapping used for path expansion.default: None

Returns: tuple[LocalAsset, ...]

run_bounded_command

Official evaluators and simulators sometimes hang inside native code. This helper starts a separate process group, enforces a hard timeout, captures stdout and stderr, and returns timeout state in a dictionary rather than losing it in an uncaught traceback.

from worldfoundry.runtime.jobs import run_bounded_command

completed = run_bounded_command(
    ["python", "-c", "print('preflight ok')"],
    timeout=30,
)
assert completed["returncode"] == 0

Do not pass untrusted user-controlled commands through this function. It bounds process lifetime; it is not a security sandbox.

def run_bounded_command(command: Sequence[str],cwd: str | Path | None = None,env: Mapping[str, str] | None = None,timeout: int,kill_timeout: int = 5) -> dict[str, Any]
funcworldfoundry.runtime.jobs.run_bounded_commandfrom worldfoundry.runtime.jobs import run_bounded_command
source

Overview

Run a subprocess with time/memory bounds and captured output for evaluation jobs.

Source docstring

Run a command with a hard timeout and always return captured output.

This helper is intended for official benchmark subprocesses. Some simulator or CUDA-backed scripts can ignore ordinary timeout handling while stuck in native code, so timeout failures are converted into structured results that callers can write into scorecards instead of surfacing a traceback.

Parameters

commandSequence[str]
cwdstr | Path | None
default: None
envMapping[str, str] | None
default: None
timeoutint
kill_timeoutint
default: 5

Returns: dict[str, Any]

For root-directory conventions and setup commands, see the environment reference and local assets guide.