Architecture

Purpose

The architecture docs explain how the repository works at the subsystem level. They are not a file-by-file source index. Use them to understand the concepts, choose the right workflow, and decide which layer owns a change. The pages are written as narrative prose: they explain responsibilities and data flow first, then reference key paths only when needed. For end-to-end run flow and evaluation internals, start with Workflow and Evaluation Core.

Who should read this: maintainers adding models or benchmarks, reviewers tracing a run from CLI to scorecard, and anyone debugging why a catalog entry says "supported" but a run still fails. When to start here: after Quickstart if you need the mental model behind the commands—not before your first successful run. Pick a subpage from the cards below once you know which layer you are changing.

Core Idea

WorldFoundry separates model execution from benchmark scoring. A model produces normalized artifacts; a benchmark evaluates those artifacts; the runner records evidence about what actually happened. This separation lets the same framework handle video generation, 3D/4D assets, world-action rollouts, hosted APIs, and embodied policies without rewriting every benchmark for every model family.

For example, a video model run and a VLA policy run both end with GenerationResult rows and a scorecard.json, even though inference happens in completely different runtimes. Benchmarks never import model checkpoints directly—they consume the normalized artifact contract that runners write to disk.

Main Layers

When you change behavior, ask which layer should own it. If the fix belongs in model runtime, it should not leak scoring logic into a runner. If it belongs in reporting, it should not assume a particular checkpoint layout. Layers communicate through serializable contracts (GenerationRequest, GenerationResult, BenchmarkSpec, scorecard fields), not through shared globals or machine-local paths.

• CLI and orchestration owns user commands, run modes, suites, resume behavior, and output locations. It should not own model-specific inference details.
• Model runtime owns pipelines, operators, local runtimes, hosted API wrappers, and normalized generation results. It should not own benchmark scoring logic.
• Benchmark runtime owns benchmark manifests, datasets, evaluators, metrics, and result normalization. It should not own model loading or checkpoint-specific setup.
• Catalog and readiness owns model/benchmark metadata, source provenance, assets, environment requirements, blockers, and support status. It should not describe runtime behavior that is not backed by code.
• Reporting owns manifests, summaries, reports, scorecards, and leaderboard eligibility decisions. It should not hide missing assets or skipped work.
• Studio owns interactive preview and inspection using the same catalog and runtime contracts. It should not contain core logic that only works inside the UI.

Design Principles

These principles resolve day-to-day tradeoffs. They explain why official validation exists separately from official benchmark runs, why catalog metadata can describe intent before code lands, and why every run writes manifests and ledgers even when metrics are simple artifact checks.

• One public runner contract — Every model path should eventually produce the same normalized result shape.
• Operator plus pipeline — Runnable methods need both a pipeline and an operator so inputs are explicit and repeatable.
• Evidence over claims — Catalog entries can describe intent, but support status must match real runnable evidence.
• External code is provenance first — Official repos and checkpoints are tracked, but machine-local paths are not runtime dependencies.
• Official validation is not benchmark parity — Official-validation import proves result normalization against real files; benchmark readiness needs real data, metrics, and scorecards.

Common Reading Paths

Most tasks touch two layers. Adding a model means both Model Runtime and Evaluation Core. Adding a benchmark means Benchmarks and Data plus Workflow.

• Run an evaluation — Start with Quickstart, then Evaluation Guide and CLI Reference.
• Choose a model — Start with Supported Models, then worldfoundry-eval zoo model-show, zoo model-download --check-local, and per-profile preflight runtime --profile <benchmark-id>.
• Add a model — Start with Add a Model, then Model Runtime.
• Add a benchmark — Start with Add a Benchmark, then Benchmarks and Data.
• Understand output artifacts — Start with Evaluation Core, then inspect report.md, scorecard.json, and metrics summaries from a real run.
• Check support status — Start with Supported Models, then zoo model-show / zoo benchmark-show.