3D & 4D · open generative 3D and 4D world models

Lyra

NVIDIA Spatial Intelligence Lab

WorldFoundry exposes in-tree Lyra-1 and Lyra-2 pipelines backed by vendored runtime adapters.

Runner verified独立环境lyra

构建运行命令

选择仓库中已记录的 variant，然后复制对应的准备、检查或运行命令。

worldfoundry.pipeline

Variant

任务image-or-video-to-world-video

环境lyra

设备CUDA 12.4

worldfoundry-eval evaluate \
  --mode model \
  --model-id lyra-1 \
  --model-runner worldfoundry:pipeline \
  --model-manifest-dir worldfoundry/data/models/catalog \
  --requests-path tmp/requests.jsonl \
  --output-dir tmp/model_eval/lyra-1 \
  --metric artifact_count \
  --json

兼容性与版本

由 Manifest 生成

集成状态: integrated
Runner 证据: verified
环境: lyra
Python: 3.10
CUDA: CUDA 12.4
PyTorch: torch==2.6.0
源码 revision: —
Checkpoint revision: —

Runtime profile: lyra
Pipeline binding: lyra
Runner: worldfoundry.pipeline
Pipeline target: worldfoundry.pipelines.lyra.pipeline_lyra2:LyraPipeline
Backend stage: in_tree_runtime
Runtime 状态: in_tree_runtime_checkpoint_and_gpu_readiness_pending
Driver 状态: compatible
环境类型: 独立环境

安装环境

环境解析器会读取已记录 profile，并选择此处显示的统一或独立环境。

bash scripts/setup/model_env_install.sh --model lyra-1

依赖版本约束 37

attrs==25.1.0
better-profanity==0.7.0
boto3==1.35.99
decord==0.6.0
diffusers==0.32.2
einops==0.8.1
huggingface-hub==0.29.2
hydra-core==1.3.2
imageio[pyav,ffmpeg]==2.37.0
iopath==0.1.10
ipdb==0.13.13
loguru==0.7.2
mediapy==1.2.2
megatron-core==0.10.0
nltk==3.9.1
numpy==1.26.4
nvidia-ml-py==12.535.133
omegaconf==2.3.0
opencv-python==4.10.0.84
pandas==2.2.3
peft==0.14.0
pillow==11.1.0
protobuf==4.25.3
pynvml==12.0.0
pyyaml==6.0.2
retinaface-py==0.0.2
safetensors==0.5.3
scikit-image==0.25.2
sentencepiece==0.2.0
setuptools==76.0.0
termcolor==2.5.0
torch==2.6.0
torchvision==0.21.0
tqdm==4.66.5
transformers==4.49.0
warp-lang==1.7.2
openexr==3.3.5

Conda 依赖 12

python=3.10
pip=25.0
cmake
ninja
gcc=12.4.0
gxx=12.4.0
cuda=12.4
cuda-nvcc=12.4
cuda-toolkit=12.4
ffmpeg
setuptools
wheel

Checkpoint 与资产

分配算力前先做本地检查；gated、private 与 license 字段直接来自 checkpoint manifest。

worldfoundry-eval zoo model-download --model-id lyra --check-local --json

depth-anything/DA3NESTED-GIANT-LARGE-1.1

Revision: —
License: —
Gated: —
Private: —

运行与输出

生成的命令通过共享 evaluation 边界运行，并持久保存结果 manifest 与 artifact。

worldfoundry-eval evaluate \
  --mode model \
  --model-id lyra-1 \
  --model-runner worldfoundry:pipeline \
  --model-manifest-dir worldfoundry/data/models/catalog \
  --requests-path tmp/requests.jsonl \
  --output-dir tmp/model_eval/lyra-1 \
  --metric artifact_count \
  --json

输入契约

字段	已记录契约
`prompt`	Required
`image`	Required
`video`	Required
`actions`	navigation_actions, camera_trajectory

Artifact 契约

Artifact 类型	文件名 / 路径
`generated_world`	`—`
`generated_3d_asset`	`—`
`generated_world`	`lyra.mp4`

证据与来源

Catalog 集成、原生 demo parity 与 runner parity 是三条独立记录。

集成状态: integrated
Runner 证据: verified
原生 Demo 证据: 未记录
验证 Imports: torch, torchvision, diffusers, transformers, megatron.core, transformer_engine, apex, warp

配方溯源

worldfoundry/data/models/catalog/three_d_four_d/lyra.yamlProjecthttps://research.nvidia.com/labs/sil/projects/lyra2/Paperhttps://openreview.net/forum?id=tIVCfVnIHoGitHubhttps://github.com/nv-tlabs/lyra

构建运行命令

输入契约

Artifact 契约

相关配方