Quickstart

Configure the environment, prepare assets, use the TUI or CLI, then run inference and evaluation.

Last updated: Jul 22, 2026

On this page

Workflow Prerequisites 1. Environment 2. Assets 3. TUI 4. CLI 5. Inference 6. Evaluation Where Details Live

This guide walks through the shortest complete path from a clone to a reviewable WorldFoundry run:

Environment
Assets
TUI
CLI inference
Artifact inspection
Evaluation

Most runs follow the same shape even when the model or benchmark changes: make the Python environment importable, discover stable IDs, prepare only the required checkpoints and benchmark assets, generate artifacts, inspect them, and only then score them.

Tip

You do not need a GPU or model weights to browse catalogs and inspect readiness. Complete environment and discovery checks, read the reported blockers, and stop before asset download if you are only evaluating coverage.

Workflow

Create the conda environment and source WorldFoundry paths.

Prepare only the model checkpoints, benchmark data, and metric assets needed for the run.

Use the TUI when you want an interactive command builder, or use the CLI commands below when scripting.

Run inference first. Run benchmark scoring only after generated artifacts match the benchmark layout.

Prerequisites

OS: Linux recommended for GPU inference and most benchmark runners
Python: 3.10+ through the WorldFoundry unified conda environment
Optional: CUDA-capable GPU, Hugging Face token for gated checkpoints

Note

Catalog browsing and readiness inspection work without weights. The expensive path begins only when you choose a concrete model run and download assets.

1. Environment

The bootstrap step creates the default WorldFoundry runtime and writes a local shell file with the paths used by the CLI, Studio, and evaluation runners. Always source that file before running commands in a new shell.

DefaultShared pathsVerify

Start from the repository root:

git clone https://github.com/OpenEnvision/WorldFoundry.git
cd WorldFoundry

bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

On shared machines, keep datasets, checkpoints, and outputs outside git:

bash scripts/setup/bootstrap_worldfoundry.sh \
  --home /path/to/worldfoundry-home \
  --data-root /path/to/worldfoundry-data \
  --model-root /path/to/worldfoundry-models \
  --artifact-root /path/to/worldfoundry-artifacts

Verify an existing machine before a run:

bash scripts/setup/bootstrap_worldfoundry.sh --verify-only
worldfoundry-eval --help

Detailed environment mapping by model and benchmark lives in Environment. Use that page when a model needs an extra conda profile or a benchmark evaluator depends on a metric package outside the unified runtime.

2. Assets

WorldFoundry keeps executable code in tree. Checkpoints, datasets, metric weights, reference files, and generated outputs are staged locally, usually through Hugging Face.

Important

Model assets and benchmark assets are separate. A model checkpoint lets you generate candidate artifacts; benchmark assets define what to generate, how outputs should be named, and which files or metric weights are needed to compute scores.

ModelBenchmark

bash scripts/inference/prepare_model_infer.sh <model-id> --verify-env-only
bash scripts/inference/prepare_model_infer.sh <model-id> --download
worldfoundry-eval zoo model-download --model-id <model-id> --check-local --json

Start by asking WorldFoundry to print the asset plan:

python scripts/setup/prepare_benchmark_assets.py \
  --benchmark-id <benchmark-id> \
  --json

python scripts/setup/prepare_benchmark_assets.py \
  --benchmark-id <benchmark-id> \
  --write-env "${WORLDFOUNDRY_HOME:-${HOME}/.cache/worldfoundry}/<benchmark-id>.env" \
  --create-dirs

source "${WORLDFOUNDRY_HOME:-${HOME}/.cache/worldfoundry}/<benchmark-id>.env"

Use HF_TOKEN only for gated assets whose terms you have accepted:

export HF_TOKEN=<your-huggingface-token>

Full cache and path conventions are in Local assets.

3. TUI

Use the TUI when you do not want to remember ids and flags. It reads the same manifests as the CLI, shows available choices, and can print the final command before anything expensive runs.

python -m pip install -e ".[tui]"
worldfoundry-eval tui

Generate a command without entering the interactive session:

worldfoundry-eval tui \
  --model-id <model-id> \
  --benchmark-id <benchmark-id> \
  --print-command

Note

The TUI does not bypass missing checkpoint, dataset, license, or environment requirements. If it prints a command for a benchmark whose assets are not staged yet, run the benchmark asset preparation step first.

4. CLI

Use the CLI for scripts, batch jobs, and reproducible notes. Start with discovery commands instead of guessing ids:

worldfoundry-eval zoo models --json
worldfoundry-eval zoo benchmarks --json
worldfoundry-eval zoo model-download --model-id <model-id> --check-local --json

Run model inference from the command line:

python -m worldfoundry.studio.workspace_job infer \
  --model-id <model-id> \
  --prompt "a cinematic scene, high quality" \
  --output-dir tmp/worldfoundry_infer/<model-id> \
  --device cuda

For image-to-video or other conditioned models:

python -m worldfoundry.studio.workspace_job infer \
  --model-id <model-id> \
  --input-path /path/to/input.png \
  --prompt "camera moves forward through the scene" \
  --frames 81 \
  --steps 30 \
  --seed 42 \
  --output-dir tmp/worldfoundry_infer/<model-id>

The full command map is in CLI.

5. Inference

Inference produces artifacts. It does not produce benchmark evidence by itself.

Important

A generated video can look correct in Studio and still fail a benchmark run if filenames, prompt coverage, split, or metadata do not match what the benchmark runner expects.

The practical loop is: generate a small batch, inspect the outputs, confirm the directory layout, and then scale to the full prompt set required by the target benchmark.

For repeated visual review, start the workspace:

bash scripts/workspace/run_workspace.sh

Open http://127.0.0.1:7870/ and inspect outputs before scoring:

find tmp/worldfoundry_infer/<model-id> -maxdepth 2 -type f | head

More details: Run inference and Studio.

6. Evaluation

A benchmark run needs three concrete inputs. If any one is missing, the runner should stop with a blocked scorecard rather than silently producing a leaderboard-looking number.

Input	Meaning
Benchmark assets	Prompt/task metadata, reference files, official result files, judge credentials, simulator assets, or metric checkpoints.
Candidate artifacts	Videos, frames, rollouts, traces, or official-shaped result dumps produced by the model.
Runner mode	`official-validation` imports existing official-shaped results; `official-run` computes scores from generated artifacts.

Official validationOfficial runOne-shot run

If the official benchmark already produced a result file, import it:

worldfoundry-eval zoo benchmark-run \
  --benchmark-id <benchmark-id> \
  --mode official-validation \
  --official-results-path /path/to/official/results-or-report \
  --generated-artifact-dir /path/to/generated/artifacts \
  --output-dir tmp/<benchmark-id>/official-validation \
  --json

If generated artifacts are ready and the in-tree runner can compute metrics:

worldfoundry-eval zoo benchmark-run \
  --benchmark-id <benchmark-id> \
  --mode official-run \
  --generated-artifact-dir /path/to/generated/artifacts \
  --output-dir tmp/<benchmark-id>/official-run \
  --json

If model-zoo and benchmark-zoo support the one-cell path:

worldfoundry-eval run \
  --benchmark <benchmark-id> \
  --model <model-id> \
  --mode official-run \
  --output-dir tmp/<benchmark-id>/<model-id> \
  --json

Tip

Open scorecard.json first. Public leaderboard claims require complete assets, complete generated artifacts, and benchmark-specific eligibility flags.

Quickstart

Environment

Local assets

TUI

CLI

Benchmark Hub