Benchmark Hub

Open benchmark and model inventory for embodied AI, video generation, and world models.

Last updated: Jul 11, 2026

On this page

About At a glance Benchmark list Embodied AI — 22 Video Generation — 27 World Models — 9 How to run one benchmark Import official result files Score generated artifacts Benchmarks Embodied AI Video Generation World Models

About

Index of benchmarks and models tracked by WorldFoundry. Integration status, blockers, metrics, and evidence flags come from machine-readable manifests under worldfoundry/data/benchmarks/ and worldfoundry/data/models/.

This page does not publish leaderboard rankings.

At a glance

3 families — Embodied AI · Video Generation · World Models
58 benchmarks — 22 embodied / 27 video / 9 world models
60 models in core catalogs (VLA/VA/WAM, open video, world models)
0 leaderboard-valid cells claimed here → Leaderboard Eligibility

worldfoundry-eval zoo benchmarks --json   # refresh counts

Benchmark list

Embodied AI — 22

Language-conditioned agents, manipulation, sim rollouts, action traces, offline policy eval. All rows currently leaderboard_valid=false. VLA/VA/WAM catalog: 15 models.

Video Generation — 27

T2V / I2V quality, alignment, dynamics, preference judges, safety, physics, and camera control. Open-video catalog: 19 models.

World Models — 9

Long-horizon / interactive worlds (action, camera, state, memory). World-model catalog: 26 models.

How to run one benchmark

Use the benchmark-specific page first. The exact prompt file, generated-artifact layout, checkpoint names, judge model, and official result format are benchmark-dependent.

Import official result files

worldfoundry-eval zoo benchmark-run \
  --benchmark-id <benchmark-id> \
  --mode official-validation \
  --official-results-path <official-result-file-or-dir> \
  --generated-artifact-dir <generated-artifact-dir> \
  --output-dir tmp/<benchmark-id>/official-validation \
  --json

Score generated artifacts

Use this only when the benchmark page says the in-tree metric runtime can recompute scores from generated artifacts and the required metric checkpoints are staged.

worldfoundry-eval run \
  --benchmark <benchmark-id> \
  --model <model-id> \
  --mode official-run \
  --output-dir tmp/<benchmark-id>/<model-id> \
  --json

If generated artifacts already exist:

worldfoundry-eval zoo benchmark-run \
  --benchmark-id <benchmark-id> \
  --mode official-run \
  --generated-artifact-dir <generated-artifact-dir> \
  --output-dir tmp/<benchmark-id>/official-run \
  --json

Use the per-benchmark page to match the expected artifact layout before scoring: prompt index videos, case id videos, prompt id videos, sampled frame directories, reports, or structured rollout traces.

Benchmark Hub

Related