# 模型 (/zh/docs/guides/supported-models) ## 使用本页 [#使用本页] 当你需要选择模型、确认本机能不能跑，或者理解某个支持状态是什么意思时，先看这一页。下面的 catalog 名称是模型分组，不是单独命令；日常使用优先从 CLI 开始： | 目标 | 命令 | | ----------------------------- | ---------------------------------------------------------------------------------- | | 查看已知模型 id 和 aliases | `worldfoundry-eval zoo models --json` | | 查看单个模型 manifest | `worldfoundry-eval zoo model-show --model-id --include-manifest --json` | | 检查本地 checkpoint、凭据和资产 blocker | `worldfoundry-eval zoo model-download --model-id --check-local --json` | | 列出 runtime runner | `worldfoundry-eval models runtime-runners --json` | ## 跑推理 [#跑推理] 准备 `requests.jsonl` 后，用 pipeline runner 生成 artifact： ```bash worldfoundry-eval evaluate \ --mode model \ --model-id \ --model-runner worldfoundry:pipeline \ --model-manifest-dir worldfoundry/data/models/catalog \ --requests-path tmp/requests.jsonl \ --output-dir tmp/model_eval \ --metric artifact_count \ --required-artifact video \ --json ``` 完整推理流程（环境、checkpoint、Studio）见 [推理指南](/zh/docs/guides/inference)。 ## 输出文件 [#输出文件] | 文件 | 用途 | | ---------------------- | ----------------------------------------- | | `run_manifest.json` | run id、model、benchmark metadata、时间戳、输出路径 | | `results.jsonl` | 每样本状态、artifact、耗时 | | `metrics/summary.json` | 聚合指标 | | `scorecard.json` | leaderboard eligibility 与 blocking reason | ## 跑失败时 [#跑失败时] 1. `zoo model-show --include-manifest` — integration status 与 blocker 2. `zoo model-download --check-local` — checkpoint、credential 和 asset readiness 3. 对照 benchmark 页面确认 dataset、API key 和生成结果目录要求 4. 对照下方 model registry，确认该 model id 是否已有 runner 证据 ## 选择模型 [#选择模型] 投入 GPU 时间前，按顺序确认这些问题： | 问题 | 检查方式 | | ------------------------- | ------------------------------------------------------------------------------------------------------------------- | | WorldFoundry 是否认识这个模型？ | 它出现在 `worldfoundry-eval zoo models --json` 中，并且有稳定的 `model_id`。 | | 这个 repo 能否路由到该模型？ | Manifest 能解析到 runner 和 pipeline binding。`runtime_ported` 表示 route 已存在，但真实 checkpoint/API/simulator parity 可能还没完成。 | | 你的环境能否运行它？ | `zoo model-download --check-local`、`models assets` 和 benchmark 页面会说明需要的 checkpoints、API keys、datasets 或 simulators。 | | 它是否对你的任务 benchmark-ready？ | 需要看该模型是否有对应 benchmark 或 artifact 类型的证据。视频、3D、world-action 和 VLA 任务需要不同证据。 | ## 怎么读 catalog 行 [#怎么读-catalog-行] | 列 | 含义 | | --------------- | -------------------------------------------------------- | | Family | 公开项目或 provider 家族；不同 variant 仍可能有独立 manifest。 | | Capability | 模型在 WorldFoundry 语义下能生成或预测什么。 | | Runtime | `local`、`official repo`、`subprocess`、`hosted API` 或混合形态。 | | Evidence target | 升级公开声明前最有用的下一步证据。 | ## 当前覆盖摘要 [#当前覆盖摘要] WorldFoundry 按 manifest、runner 和 benchmark evidence 三层跟踪模型支持状态。先用这个摘要找到合适的模型分组，再用 `zoo model-show` 查看具体 model id。 | Catalog directory | 条目数 | 覆盖内容 | 代表 id | | -------------------------------------------------- | --: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | `worldfoundry/data/models/catalog/video/` | 48 | 视频、音频与多模态生成模型 catalog（原 `video_open` 与 `video_pipeline_runner` 合并）。含开源 provenance 记录与带 pipeline binding 的 runtime 集成。 | `wan2.1`、`hunyuanvideo`、`animatediff`、`zeroscope`、`step-video-t2v`、`sana`、`vmem`、`krea-realtime-video`。 | | `worldfoundry/data/models/catalog/world_models/` | 69 | 交互式世界模型、simulator、导航与 camera-control 模型 catalog（原 `world_models` 与 `world_pipeline_runner` 合并）。 | `matrix-game-2`、`lingbot-world`、`hunyuanworld-voyager`、`diamond`、`dino-wm`、`leworldmodel`、`starwm`、`irasim`。 | | `worldfoundry/data/models/catalog/three_d_four_d/` | 47 | 3D/4D generation、reconstruction、point-cloud、geometry、scene-representation 和 perception-prior 记录。状态分布：25 integrated、10 `runtime_ported`、10 blocked、2 planned；多数通过 pipeline binding 解析。 | `flashworld`、`fantasyworld`、`dust3r`、`monst3r`、`splatt3r`、`pixelsplat`、`vggt-omega`、`dvlt`。 | | `worldfoundry/data/models/catalog/vla_va_wam/` | 42 | VLA、VA、WAM、robot-policy 和 embodied-action 记录。所有条目都有 runner target 和 binding；多数是 `runtime_ported`，完整 policy/runtime 声明仍受 checkpoint 与 simulator 验证约束。 | `openvla`、`openpi`、`pi0-fast`、`smolvla`、`xvla`、`eo1`、`tdmpc`、`vqbet`、`wall-oss`、`act`。 | ## 模型来源索引 [#模型来源索引] 这个索引结合 `worldfoundry/data/models/catalog` 的 manifest 与本轮确认到的官方论文、项目页和源码链接。如果没有单独发布 paper、project page 或 code repository，则使用最接近的官方 model card、source page 或 project URL。 ### 开源视频模型 [#开源视频模型] 上游视频生成模型族，以及仓内 runtime / metadata 记录。支持层级不一：有的已有本地生成验证路径，有的是 checkpoint-gated wrapper、hosted/API 形态，或仅 provenance。跑之前先看 catalog readiness 和 `zoo model-show`。
Wan2.1 ( `wan2.1` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2503.20314](https://arxiv.org/abs/2503.20314) * **项目主页**: [https://wan.video/](https://wan.video/) * **GitHub**: [https://github.com/Wan-Video/Wan2.1](https://github.com/Wan-Video/Wan2.1)
Wan2.2 ( `wan2.2` ) * **能力**: text-to-video, image-to-video, text-image-to-video * **论文**: [https://arxiv.org/abs/2503.20314](https://arxiv.org/abs/2503.20314) * **项目主页**: [https://wan.video/](https://wan.video/) * **GitHub**: [https://github.com/Wan-Video/Wan2.2](https://github.com/Wan-Video/Wan2.2)
HunyuanVideo ( `hunyuanvideo` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2412.03603](https://arxiv.org/abs/2412.03603) * **项目主页**: [https://aivideo.hunyuan.tencent.com/](https://aivideo.hunyuan.tencent.com/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HunyuanVideo](https://github.com/Tencent-Hunyuan/HunyuanVideo)
LTX-Video ( `ltx-video` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2501.00103](https://arxiv.org/abs/2501.00103) * **项目主页**: [https://ltx.video/](https://ltx.video/) * **GitHub**: [https://github.com/Lightricks/LTX-Video](https://github.com/Lightricks/LTX-Video)
LTX-2.x ( `ltx-2.x` ) * **能力**: text-to-video, image-to-video, video-to-video, audio-video-generation * **论文**: [https://arxiv.org/abs/2601.03233](https://arxiv.org/abs/2601.03233) * **项目主页**: [https://ltx.video/](https://ltx.video/) * **GitHub**: [https://github.com/Lightricks/LTX-Video](https://github.com/Lightricks/LTX-Video)
CogVideoX ( `cogvideox` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2408.06072](https://arxiv.org/abs/2408.06072) * **项目主页**: [https://yzy-thu.github.io/CogVideoX-demo/](https://yzy-thu.github.io/CogVideoX-demo/) * **GitHub**: [https://github.com/THUDM/CogVideo](https://github.com/THUDM/CogVideo)
Open-Sora ( `open-sora` ) * **能力**: text-to-video * **论文**: [https://arxiv.org/abs/2412.20404](https://arxiv.org/abs/2412.20404) * **项目主页**: [https://hpcaitech.github.io/Open-Sora/](https://hpcaitech.github.io/Open-Sora/) * **GitHub**: [https://github.com/hpcaitech/Open-Sora](https://github.com/hpcaitech/Open-Sora)
Open-Sora-Plan ( `open-sora-plan` ) * **能力**: text-to-video * **论文**: [https://arxiv.org/abs/2412.00131](https://arxiv.org/abs/2412.00131) * **项目主页**: [https://github.com/PKU-YuanGroup/Open-Sora-Plan](https://github.com/PKU-YuanGroup/Open-Sora-Plan) * **GitHub**: [https://github.com/PKU-YuanGroup/Open-Sora-Plan](https://github.com/PKU-YuanGroup/Open-Sora-Plan)
Mochi-1 ( `mochi-1` ) * **能力**: text-to-video * **论文**: [https://huggingface.co/genmo/mochi-1-preview](https://huggingface.co/genmo/mochi-1-preview) * **项目主页**: [https://www.genmo.ai/play](https://www.genmo.ai/play) * **GitHub**: [https://github.com/genmoai/mochi](https://github.com/genmoai/mochi)
Allegro ( `allegro` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2410.15458](https://arxiv.org/abs/2410.15458) * **项目主页**: [https://rhymes.ai/allegro\_gallery](https://rhymes.ai/allegro_gallery) * **GitHub**: [https://github.com/rhymes-ai/Allegro](https://github.com/rhymes-ai/Allegro)
EasyAnimate ( `easyanimate` ) * **能力**: text-to-video, image-to-video, video-to-video * **论文**: [https://arxiv.org/abs/2405.18991](https://arxiv.org/abs/2405.18991) * **项目主页**: [https://easyanimate.github.io/](https://easyanimate.github.io/) * **GitHub**: [https://github.com/aigc-apps/EasyAnimate](https://github.com/aigc-apps/EasyAnimate)
VideoCrafter ( `videocrafter` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2401.09047](https://arxiv.org/abs/2401.09047) * **项目主页**: [https://ailab-cvc.github.io/videocrafter2/](https://ailab-cvc.github.io/videocrafter2/) * **GitHub**: [https://github.com/VideoCrafter/VideoCrafter](https://github.com/VideoCrafter/VideoCrafter)
DynamiCrafter ( `dynamicrafter` ) * **能力**: image-to-video * **论文**: [https://arxiv.org/abs/2310.12190](https://arxiv.org/abs/2310.12190) * **项目主页**: [https://doubiiu.github.io/projects/DynamiCrafter/](https://doubiiu.github.io/projects/DynamiCrafter/) * **GitHub**: [https://github.com/Doubiiu/DynamiCrafter](https://github.com/Doubiiu/DynamiCrafter)
I2VGen-XL ( `i2vgen-xl` ) * **能力**: image-to-video, text-to-video * **论文**: [https://arxiv.org/abs/2311.04145](https://arxiv.org/abs/2311.04145) * **项目主页**: [https://i2vgen-xl.github.io/](https://i2vgen-xl.github.io/) * **GitHub**: [https://github.com/ali-vilab/i2vgen-xl](https://github.com/ali-vilab/i2vgen-xl)
ModelScopeT2V ( `modelscope-t2v` ) * **能力**: text-to-video * **论文**: [https://arxiv.org/abs/2308.06571](https://arxiv.org/abs/2308.06571) * **项目主页**: [https://modelscope.cn/models/damo/text-to-video-synthesis/summary](https://modelscope.cn/models/damo/text-to-video-synthesis/summary) * **GitHub**: [https://github.com/modelscope/modelscope](https://github.com/modelscope/modelscope)
HunyuanVideo 1.5 ( `hunyuanvideo-1.5` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2511.18870](https://arxiv.org/abs/2511.18870) * **项目主页**: [https://hunyuan.tencent.com/video/zh?tabIndex=0](https://hunyuan.tencent.com/video/zh?tabIndex=0) * **GitHub**: [https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5)
SkyReels-V2 ( `skyreels-v2` ) * **能力**: text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2504.13074](https://arxiv.org/abs/2504.13074) * **项目主页**: [https://github.com/SkyworkAI/SkyReels-V2](https://github.com/SkyworkAI/SkyReels-V2) * **GitHub**: [https://github.com/SkyworkAI/SkyReels-V2](https://github.com/SkyworkAI/SkyReels-V2)
MAGI-1 ( `magi-1` ) * **能力**: image-to-video, video-generation * **论文**: [https://arxiv.org/abs/2505.13211](https://arxiv.org/abs/2505.13211) * **项目主页**: [https://magi.sand.ai/](https://magi.sand.ai/) * **GitHub**: [https://github.com/SandAI-org/MAGI-1](https://github.com/SandAI-org/MAGI-1)
FramePack ( `framepack` ) * **能力**: image-to-video, long-video * **论文**: [https://arxiv.org/abs/2504.12626](https://arxiv.org/abs/2504.12626) * **项目主页**: [https://lllyasviel.github.io/frame\_pack\_gitpage/](https://lllyasviel.github.io/frame_pack_gitpage/) * **GitHub**: [https://github.com/lllyasviel/FramePack](https://github.com/lllyasviel/FramePack)
Step-Video-T2V ( `step-video-t2v` ) * **能力**: text-to-video, video-generation * **论文**: [https://arxiv.org/abs/2502.10248](https://arxiv.org/abs/2502.10248) * **项目主页**: [https://yuewen.cn/videos](https://yuewen.cn/videos) * **GitHub**: [https://github.com/stepfun-ai/Step-Video-T2V](https://github.com/stepfun-ai/Step-Video-T2V)
Open-MAGVIT2 ( `open-magvit2` ) * **能力**: video-tokenization, autoregressive-generation * **论文**: [https://arxiv.org/abs/2409.04410](https://arxiv.org/abs/2409.04410) * **项目主页**: [https://huggingface.co/TencentARC/Open-MAGVIT2](https://huggingface.co/TencentARC/Open-MAGVIT2) * **GitHub**: [https://github.com/TencentARC/Open-MAGVIT2](https://github.com/TencentARC/Open-MAGVIT2)
Show-o ( `show-o` ) * **能力**: multimodal-generation, text-to-image, video-generation * **论文**: [https://arxiv.org/abs/2408.12528](https://arxiv.org/abs/2408.12528) * **项目主页**: [https://showlab.github.io/Show-o/](https://showlab.github.io/Show-o/) * **GitHub**: [https://github.com/showlab/Show-o](https://github.com/showlab/Show-o)
AnimateDiff ( `animatediff` ) * **能力**: text-to-video, classic-unet-diffusion * **论文**: [https://arxiv.org/abs/2307.04725](https://arxiv.org/abs/2307.04725) * **项目主页**: [https://animatediff.github.io/](https://animatediff.github.io/) * **GitHub**: [https://github.com/guoyww/AnimateDiff](https://github.com/guoyww/AnimateDiff)
ZeroScope ( `zeroscope` ) * **能力**: text-to-video, classic-unet-diffusion * **论文**: [https://huggingface.co/cerspense/zeroscope\_v2\_576w](https://huggingface.co/cerspense/zeroscope_v2_576w) * **项目主页**: [https://huggingface.co/cerspense/zeroscope\_v2\_576w](https://huggingface.co/cerspense/zeroscope_v2_576w) * **GitHub**: [https://huggingface.co/cerspense/zeroscope\_v2\_576w](https://huggingface.co/cerspense/zeroscope_v2_576w)
### 世界模型 [#世界模型] 交互式、相机控制、导航、驾驶与长时程仿真系统，也包括 provenance / metadata-only 条目。能力标签是契约提示：action-controlled 模型应保留 action traces，camera-controlled 模型应暴露 camera metadata，长时程系统需要能保持序列顺序的 artifact 命名。
Cosmos Predict 2.5 ( `cosmos-predict-2.5` ) * **能力**: text-to-video, image-to-video, video-to-video, world-generation * **论文**: [https://arxiv.org/abs/2511.00062](https://arxiv.org/abs/2511.00062) * **项目主页**: [https://research.nvidia.com/labs/cosmos-lab/cosmos-predict2.5/](https://research.nvidia.com/labs/cosmos-lab/cosmos-predict2.5/) * **GitHub**: [https://github.com/nvidia-cosmos/cosmos-predict2.5](https://github.com/nvidia-cosmos/cosmos-predict2.5)
Cosmos Transfer 2.5 ( `cosmos-transfer-2.5` ) * **能力**: controlled-video, world-generation, spatial-control * **论文**: [https://arxiv.org/abs/2511.00062](https://arxiv.org/abs/2511.00062) * **项目主页**: [https://research.nvidia.com/labs/cosmos-lab/cosmos-transfer2.5/](https://research.nvidia.com/labs/cosmos-lab/cosmos-transfer2.5/) * **GitHub**: [https://github.com/nvidia-cosmos/cosmos-transfer2.5](https://github.com/nvidia-cosmos/cosmos-transfer2.5)
DreamDojo ( `dreamdojo` ) * **能力**: interactive-world-model, robot-world-model, action-conditioned-video2world, long-horizon * **论文**: [https://arxiv.org/abs/2602.06949](https://arxiv.org/abs/2602.06949) * **项目主页**: [https://dreamdojo-world.github.io/](https://dreamdojo-world.github.io/) * **GitHub**: [https://github.com/NVIDIA/DreamDojo](https://github.com/NVIDIA/DreamDojo)
Matrix-Game 1.0 ( `matrix-game-1` ) * **能力**: interactive-world-model, game-video, action-control * **论文**: [https://arxiv.org/abs/2506.18701](https://arxiv.org/abs/2506.18701) * **项目主页**: [https://matrix-game-homepage.github.io/](https://matrix-game-homepage.github.io/) * **GitHub**: [https://github.com/SkyworkAI/Matrix-Game](https://github.com/SkyworkAI/Matrix-Game)
Matrix-Game 2.0 ( `matrix-game-2` ) * **能力**: interactive-world-model, streaming, game-video, action-control * **论文**: [https://arxiv.org/abs/2508.13009](https://arxiv.org/abs/2508.13009) * **项目主页**: [https://matrix-game-v2.github.io/](https://matrix-game-v2.github.io/) * **GitHub**: [https://github.com/SkyworkAI/Matrix-Game](https://github.com/SkyworkAI/Matrix-Game)
Matrix-Game 3.0 ( `matrix-game-3` ) * **能力**: interactive-world-model, streaming, long-horizon, game-video * **论文**: [https://arxiv.org/abs/2604.08995](https://arxiv.org/abs/2604.08995) * **项目主页**: [https://matrix-game-v3.github.io/](https://matrix-game-v3.github.io/) * **GitHub**: [https://github.com/SkyworkAI/Matrix-Game](https://github.com/SkyworkAI/Matrix-Game)
LingBot-World ( `lingbot-world` ) * **能力**: interactive-world-model, camera-control, world-video * **论文**: [https://arxiv.org/abs/2601.20540](https://arxiv.org/abs/2601.20540) * **项目主页**: [https://technology.robbyant.com/lingbot-world](https://technology.robbyant.com/lingbot-world) * **GitHub**: [https://github.com/robbyant/lingbot-world](https://github.com/robbyant/lingbot-world)
HunyuanWorld 1.0 ( `hunyuanworld-1` ) * **能力**: text-to-3d-world, image-to-3d-world, panorama, mesh-export * **论文**: [https://arxiv.org/abs/2507.21809](https://arxiv.org/abs/2507.21809) * **项目主页**: [https://3d-models.hunyuan.tencent.com/world/](https://3d-models.hunyuan.tencent.com/world/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0](https://github.com/Tencent-Hunyuan/HunyuanWorld-1.0)
HunyuanWorld-Voyager ( `hunyuanworld-voyager` ) * **能力**: image-to-video, 3d-reconstruction, world-generation, long-range * **论文**: [https://arxiv.org/abs/2506.04225](https://arxiv.org/abs/2506.04225) * **项目主页**: [https://3d-models.hunyuan.tencent.com/voyager/](https://3d-models.hunyuan.tencent.com/voyager/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HunyuanWorld-Voyager](https://github.com/Tencent-Hunyuan/HunyuanWorld-Voyager)
HY-WorldPlay ( `hy-worldplay` ) * **能力**: interactive-world-model, streaming, real-time, camera-action-control * **论文**: [https://arxiv.org/abs/2512.14614](https://arxiv.org/abs/2512.14614) * **项目主页**: [https://3d-models.hunyuan.tencent.com/world/](https://3d-models.hunyuan.tencent.com/world/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HY-WorldPlay](https://github.com/Tencent-Hunyuan/HY-WorldPlay)
HunyuanWorld-Mirror ( `hunyuanworld-mirror` ) * **能力**: 3d-reconstruction, world-mirror, image-to-3d, video-to-3d * **论文**: [https://3d-models.hunyuan.tencent.com/world/worldMirror1\_0/HYWorld\_Mirror\_Tech\_Report.pdf](https://3d-models.hunyuan.tencent.com/world/worldMirror1_0/HYWorld_Mirror_Tech_Report.pdf) * **项目主页**: [https://3d-models.hunyuan.tencent.com/world/](https://3d-models.hunyuan.tencent.com/world/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror](https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror)
HY-World 2.0 ( `hy-world-2.0` ) * **能力**: text-to-3d-world, image-to-3d-world, video-to-3d-world, simulation * **论文**: [https://arxiv.org/abs/2604.14268](https://arxiv.org/abs/2604.14268) * **项目主页**: [https://3d-models.hunyuan.tencent.com/world/](https://3d-models.hunyuan.tencent.com/world/) * **GitHub**: [https://github.com/Tencent-Hunyuan/HY-World-2.0](https://github.com/Tencent-Hunyuan/HY-World-2.0)
InSpatio-WorldFM ( `worldfm` ) * **能力**: 3d-world-model, spatial-intelligence, frame-generation * **论文**: [https://arxiv.org/abs/2603.11911](https://arxiv.org/abs/2603.11911) * **项目主页**: [https://inspatio.github.io/worldfm/](https://inspatio.github.io/worldfm/) * **GitHub**: [https://github.com/inspatio/worldfm](https://github.com/inspatio/worldfm)
Infinite-World ( `infinite-world` ) * **能力**: interactive-world-model, long-horizon, image-to-video, memory * **论文**: [https://arxiv.org/abs/2602.02393](https://arxiv.org/abs/2602.02393) * **项目主页**: [https://huggingface.co/MeiGen-AI/Infinite-World](https://huggingface.co/MeiGen-AI/Infinite-World) * **GitHub**: [https://github.com/MeiGen-AI/Infinite-World](https://github.com/MeiGen-AI/Infinite-World)
WorldCam ( `worldcam` ) * **能力**: interactive-world-model, camera-pose-control, game-video, long-horizon * **论文**: [https://arxiv.org/abs/2603.16871](https://arxiv.org/abs/2603.16871) * **项目主页**: [https://cvlab-kaist.github.io/WorldCam/](https://cvlab-kaist.github.io/WorldCam/) * **GitHub**: [https://github.com/cvlab-kaist/WorldCam](https://github.com/cvlab-kaist/WorldCam)
CameraCtrl ( `cameractrl` ) * **能力**: camera-control, text-to-video, video-diffusion-plugin * **论文**: [https://arxiv.org/abs/2404.02101](https://arxiv.org/abs/2404.02101) * **项目主页**: [https://hehao13.github.io/projects-CameraCtrl/](https://hehao13.github.io/projects-CameraCtrl/) * **GitHub**: [https://github.com/hehao13/CameraCtrl](https://github.com/hehao13/CameraCtrl)
MotionCtrl ( `motionctrl` ) * **能力**: camera-control, object-motion-control, text-to-video, image-to-video * **论文**: [https://arxiv.org/abs/2312.03641](https://arxiv.org/abs/2312.03641) * **项目主页**: [https://wzhouxiff.github.io/projects/MotionCtrl/](https://wzhouxiff.github.io/projects/MotionCtrl/) * **GitHub**: [https://github.com/TencentARC/MotionCtrl](https://github.com/TencentARC/MotionCtrl)
ViewCrafter ( `viewcrafter` ) * **能力**: novel-view-synthesis, camera-control, sparse-view, 3d-reconstruction * **论文**: [https://arxiv.org/abs/2409.02048](https://arxiv.org/abs/2409.02048) * **项目主页**: [https://drexubery.github.io/ViewCrafter/](https://drexubery.github.io/ViewCrafter/) * **GitHub**: [https://github.com/Drexubery/ViewCrafter](https://github.com/Drexubery/ViewCrafter)
IRASim ( `irasim` ) * **能力**: robotics-world-model, video-generation, action-conditioned-generation * **论文**: [https://arxiv.org/abs/2406.14540](https://arxiv.org/abs/2406.14540) * **项目主页**: [https://gen-irasim.github.io/](https://gen-irasim.github.io/) * **GitHub**: [https://github.com/bytedance/IRASim](https://github.com/bytedance/IRASim)
Pandora ( `pandora` ) * **能力**: diffusion-world-model, world-generation, text-to-video * **论文**: [https://arxiv.org/abs/2406.09455](https://arxiv.org/abs/2406.09455) * **项目主页**: [https://world-model.maitrix.org/](https://world-model.maitrix.org/) * **GitHub**: [https://github.com/maitrix-org/Pandora](https://github.com/maitrix-org/Pandora)
### 3D 和 4D 模型 [#3d-和-4d-模型] 3D/4D 生成、重建、点云、几何与 perception-prior 模型，也包括 provenance / metadata-only 条目。需要可度量的 3D 输出时，优先选带显式 geometry、camera 或 asset metadata 的条目；部分行在 WorldFoundry 可跑之前仍需 port plan。
MVDiffusion ( `mvdiffusion` ) * **能力**: correspondence-aware multiview image generation and scene texturing, 3d * **论文**: [https://arxiv.org/abs/2307.01097](https://arxiv.org/abs/2307.01097) * **项目主页**: [https://mvdiffusion.github.io/](https://mvdiffusion.github.io/) * **GitHub**: [https://github.com/Tangshitao/MVDiffusion](https://github.com/Tangshitao/MVDiffusion)
4D-GS ( `4d-gs` ) * **能力**: real-time dynamic scene rendering, 4d * **论文**: [https://arxiv.org/abs/2310.08528](https://arxiv.org/abs/2310.08528) * **项目主页**: [https://guanjunwu.github.io/4dgs/](https://guanjunwu.github.io/4dgs/) * **GitHub**: [https://github.com/hustvl/4DGaussians](https://github.com/hustvl/4DGaussians)
MonST3R ( `monst3r` ) * **能力**: geometry estimation in the presence of motion, 4d * **论文**: [https://monst3r-project.github.io/files/monst3r\_paper.pdf](https://monst3r-project.github.io/files/monst3r_paper.pdf) * **项目主页**: [https://monst3r-project.github.io/](https://monst3r-project.github.io/) * **GitHub**: [https://github.com/Junyi42/monst3r](https://github.com/Junyi42/monst3r)
Shape of Motion ( `shape-of-motion` ) * **能力**: single-video 4D reconstruction, 4d * **论文**: [https://arxiv.org/abs/2407.13764](https://arxiv.org/abs/2407.13764) * **项目主页**: [https://shape-of-motion.github.io/](https://shape-of-motion.github.io/) * **GitHub**: [https://github.com/vye16/shape-of-motion](https://github.com/vye16/shape-of-motion)
FlashWorld ( `flashworld` ) * **能力**: fast 3D scene generation from image or text, 3d * **论文**: [https://arxiv.org/abs/2510.13678](https://arxiv.org/abs/2510.13678) * **项目主页**: [https://imlixinyang.github.io/FlashWorld-Project-Page/](https://imlixinyang.github.io/FlashWorld-Project-Page/) * **GitHub**: [https://github.com/imlixinyang/FlashWorld](https://github.com/imlixinyang/FlashWorld)
FantasyWorld ( `fantasyworld` ) * **能力**: geometry-consistent world modeling, 3d * **论文**: [https://arxiv.org/abs/2509.21657](https://arxiv.org/abs/2509.21657) * **项目主页**: [https://fantasy-amap.github.io/fantasy-world/](https://fantasy-amap.github.io/fantasy-world/) * **GitHub**: [https://github.com/Fantasy-AMAP/fantasy-world](https://github.com/Fantasy-AMAP/fantasy-world)
WonderWorld ( `wonderworld` ) * **能力**: interactive 3D scene generation from a single image, 3d * **论文**: [https://arxiv.org/abs/2406.09394](https://arxiv.org/abs/2406.09394) * **项目主页**: [https://kovenyu.com/wonderworld/](https://kovenyu.com/wonderworld/) * **GitHub**: [https://github.com/KovenYu/WonderWorld](https://github.com/KovenYu/WonderWorld)
NeoVerse ( `neoverse` ) * **能力**: 4D world modeling from in-the-wild monocular videos, 4d * **论文**: [https://arxiv.org/abs/2601.00393](https://arxiv.org/abs/2601.00393) * **项目主页**: [https://neoverse-4d.github.io/](https://neoverse-4d.github.io/) * **GitHub**: [https://github.com/IamCreateAI/NeoVerse](https://github.com/IamCreateAI/NeoVerse)
ReCamMaster ( `recammaster` ) * **能力**: camera-controlled video rerendering, 4d * **论文**: [https://arxiv.org/abs/2503.11647](https://arxiv.org/abs/2503.11647) * **项目主页**: [https://jianhongbai.github.io/ReCamMaster/](https://jianhongbai.github.io/ReCamMaster/) * **GitHub**: [https://github.com/KlingAIResearch/ReCamMaster](https://github.com/KlingAIResearch/ReCamMaster)
Lyra ( `lyra` ) * **能力**: open generative 3D and 4D world models, 3d\_4d * **论文**: [https://openreview.net/forum?id=tIVCfVnIHo](https://openreview.net/forum?id=tIVCfVnIHo) * **项目主页**: [https://research.nvidia.com/labs/sil/projects/lyra2/](https://research.nvidia.com/labs/sil/projects/lyra2/) * **GitHub**: [https://github.com/nv-tlabs/lyra](https://github.com/nv-tlabs/lyra)
Stable Virtual Camera ( `stable-virtual-camera` ) * **能力**: generative novel view synthesis with camera control, 3d * **论文**: [https://arxiv.org/abs/2503.14489](https://arxiv.org/abs/2503.14489) * **项目主页**: [https://stable-virtual-camera.github.io/](https://stable-virtual-camera.github.io/) * **GitHub**: [https://github.com/Stability-AI/stable-virtual-camera](https://github.com/Stability-AI/stable-virtual-camera)
Splatt3R ( `splatt3r` ) * **能力**: 3d-reconstruction, gaussian-splatting * **论文**: [https://arxiv.org/abs/2408.13912](https://arxiv.org/abs/2408.13912) * **项目主页**: [https://splatt3r.active.vision/](https://splatt3r.active.vision/) * **GitHub**: [https://github.com/btsmart/splatt3r](https://github.com/btsmart/splatt3r)
pixelSplat ( `pixelsplat` ) * **能力**: 3d-reconstruction, gaussian-splatting, novel-view-synthesis * **论文**: [https://arxiv.org/abs/2312.12337](https://arxiv.org/abs/2312.12337) * **项目主页**: [https://dcharatan.github.io/pixelsplat/](https://dcharatan.github.io/pixelsplat/) * **GitHub**: [https://github.com/dcharatan/pixelsplat](https://github.com/dcharatan/pixelsplat)
### VLA、VA 和 WAM 模型 [#vlava-和-wam-模型] 这类模型评测的是行为，而不只是图像质量。一次可用 run 需要 observation、语言或任务指令、action output、simulator/dataset context，以及能映射到归一化指标的 success signal。 `openvla` 是这一组 checkpoint-verified embodied-action baseline。MolmoAct2 也有仓内 WorldFoundry pipeline/runtime surface，用于 DROID 和 bimanual YAM 的 action-trace contract；checkpoint-backed GPU parity 仍待验证。原来的 generic LeRobot policy-zoo surface 已取消；`lerobot/src/lerobot/policies` 里的 policy 现在按具体算法追踪各自上游来源，例如 ACT、Diffusion Policy、TD-MPC、VQ-BeT、pi0/pi0.5/pi0-FAST、SmolVLA、GR00T、X-VLA、WALL-OSS、EO-1、MolmoAct2、Multi-task DiT、Gaussian Actor 和 RTC。多数 VLA/VA/WAM 行当前是 `runtime_ported`：它们已有 WorldFoundry route contract，但 task support 仍受 checkpoint、simulator 和真实推理证据约束。
OpenVLA ( `openvla` ) * **能力**: vla, vla.action\_prediction, vla.policy\_rollout * **论文**: [https://arxiv.org/abs/2406.09246](https://arxiv.org/abs/2406.09246) * **项目主页**: [https://openvla.github.io/](https://openvla.github.io/) * **GitHub**: [https://github.com/openvla/openvla](https://github.com/openvla/openvla)
OpenPI ( `openpi` ) * **能力**: vla, vla.policy\_rollout, robot\_policy * **论文**: [https://arxiv.org/abs/2410.24164](https://arxiv.org/abs/2410.24164) * **项目主页**: [https://www.physicalintelligence.company/blog/pi0](https://www.physicalintelligence.company/blog/pi0) * **GitHub**: [https://github.com/Physical-Intelligence/openpi](https://github.com/Physical-Intelligence/openpi)
MolmoAct2 ( `molmoact2` ) * **能力**: vla, vla.action\_prediction, vla.policy\_rollout, robot\_policy * **论文**: [https://arxiv.org/abs/2605.02881](https://arxiv.org/abs/2605.02881) * **项目主页**: [https://allenai.org/blog/molmoact2](https://allenai.org/blog/molmoact2) * **GitHub**: [https://github.com/allenai/molmoact2](https://github.com/allenai/molmoact2)
TD-MPC ( `tdmpc` ) * **能力**: model\_predictive\_control, robot\_policy, policy\_rollout * **论文**: [https://www.nicklashansen.com/td-mpc/](https://www.nicklashansen.com/td-mpc/) * **项目主页**: [https://www.nicklashansen.com/td-mpc/](https://www.nicklashansen.com/td-mpc/) * **GitHub**: [https://github.com/nicklashansen/tdmpc](https://github.com/nicklashansen/tdmpc)
VQ-BeT ( `vqbet` ) * **能力**: behavior\_generation, imitation\_learning, robot\_policy * **论文**: [https://arxiv.org/abs/2403.03181](https://arxiv.org/abs/2403.03181) * **项目主页**: [https://sjlee.cc/vq-bet/](https://sjlee.cc/vq-bet/) * **GitHub**: [https://github.com/jayLEE0301/vq\_bet\_official](https://github.com/jayLEE0301/vq_bet_official)
SmolVLA ( `smolvla` ) * **能力**: vla, vla.policy\_rollout, robot\_policy * **论文**: [https://arxiv.org/abs/2506.01844](https://arxiv.org/abs/2506.01844) * **项目主页**: [https://huggingface.co/lerobot/smolvla\_base](https://huggingface.co/lerobot/smolvla_base) * **GitHub**: [https://github.com/huggingface/lerobot](https://github.com/huggingface/lerobot)
X-VLA ( `xvla` ) * **能力**: vla, cross\_embodiment\_vla, vla.policy\_rollout * **论文**: [https://huggingface.co/papers/2510.10274](https://huggingface.co/papers/2510.10274) * **项目主页**: [https://huggingface.co/collections/2toINF/x-vla](https://huggingface.co/collections/2toINF/x-vla) * **GitHub**: [https://github.com/2toinf/X-VLA](https://github.com/2toinf/X-VLA)
EO-1 ( `eo1` ) * **能力**: vla, embodied\_vla, vla.policy\_rollout * **论文**: [https://arxiv.org/abs/2508.21112](https://arxiv.org/abs/2508.21112) * **项目主页**: [https://eo-robotics.ai/eo-1](https://eo-robotics.ai/eo-1) * **GitHub**: [https://github.com/EO-Robotics/EO](https://github.com/EO-Robotics/EO)
Being-H0.5 ( `being-h05` ) * **能力**: vla, vla.policy\_rollout, robot\_policy, cross\_embodiment\_policy * **论文**: [https://arxiv.org/abs/2601.12993](https://arxiv.org/abs/2601.12993) * **项目主页**: [https://research.beingbeyond.com/being-h05](https://research.beingbeyond.com/being-h05) * **GitHub**: [https://github.com/BeingBeyond/Being-H](https://github.com/BeingBeyond/Being-H)
NVIDIA DreamZero ( `dreamzero` ) * **能力**: wam, world\_action\_model, wam.world\_action\_modeling, robot\_policy, video\_prediction * **论文**: [https://arxiv.org/abs/2602.15922](https://arxiv.org/abs/2602.15922) * **项目主页**: [https://dreamzero0.github.io/](https://dreamzero0.github.io/) * **GitHub**: [https://github.com/dreamzero0/dreamzero](https://github.com/dreamzero0/dreamzero)
NVIDIA GR00T ( `gr00t` ) * **能力**: vla, vla.policy\_rollout, humanoid\_policy * **论文**: [https://arxiv.org/abs/2503.14734](https://arxiv.org/abs/2503.14734) * **项目主页**: [https://developer.nvidia.com/isaac/gr00t](https://developer.nvidia.com/isaac/gr00t) * **GitHub**: [https://github.com/NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T)
StarVLA ( `starvla` ) * **能力**: vla, wam, vla.policy\_rollout, wam.world\_action\_modeling * **论文**: [https://arxiv.org/abs/2604.05014](https://arxiv.org/abs/2604.05014) * **项目主页**: [https://starvla.github.io/](https://starvla.github.io/) * **GitHub**: [https://github.com/starVLA/starVLA](https://github.com/starVLA/starVLA)
LingBot-VA ( `lingbot-va` ) * **能力**: vla, va, vam, wam, vla.policy\_rollout, vam.video\_action\_modeling, wam.world\_action\_modeling, robot\_policy * **论文**: [https://arxiv.org/abs/2601.21998](https://arxiv.org/abs/2601.21998) * **项目主页**: [https://technology.robbyant.com/lingbot-va](https://technology.robbyant.com/lingbot-va) * **GitHub**: [https://github.com/Robbyant/lingbot-va](https://github.com/Robbyant/lingbot-va)
Latent Action Pretraining ( `lapa` ) * **能力**: va, vam, latent\_action, video\_action\_modeling * **论文**: [https://arxiv.org/abs/2410.11758](https://arxiv.org/abs/2410.11758) * **项目主页**: [https://latentactionpretraining.github.io/](https://latentactionpretraining.github.io/) * **GitHub**: [https://github.com/LatentActionPretraining/LAPA](https://github.com/LatentActionPretraining/LAPA)
Octo ( `octo` ) * **能力**: vla, vla.policy\_rollout, robot\_policy, generalist\_policy * **论文**: [https://octo-models.github.io/paper.pdf](https://octo-models.github.io/paper.pdf) * **项目主页**: [https://octo-models.github.io/](https://octo-models.github.io/) * **GitHub**: [https://github.com/octo-models/octo](https://github.com/octo-models/octo)
RT-1 ( `rt-1` ) * **能力**: vla, vla.policy\_rollout, robot\_policy, robotics\_transformer * **论文**: [https://arxiv.org/abs/2212.06817](https://arxiv.org/abs/2212.06817) * **项目主页**: [https://robotics-transformer1.github.io/](https://robotics-transformer1.github.io/) * **GitHub**: [https://github.com/google-research/robotics\_transformer](https://github.com/google-research/robotics_transformer)
Diffusion Policy ( `diffusion-policy` ) * **能力**: visuomotor\_policy, robot\_policy, action\_diffusion, policy\_rollout * **论文**: [https://diffusion-policy.cs.columbia.edu/#paper](https://diffusion-policy.cs.columbia.edu/#paper) * **项目主页**: [https://diffusion-policy.cs.columbia.edu/](https://diffusion-policy.cs.columbia.edu/) * **GitHub**: [https://github.com/real-stanford/diffusion\_policy](https://github.com/real-stanford/diffusion_policy)
Action Chunking Transformer ( `act` ) * **能力**: action\_chunking\_policy, robot\_policy, imitation\_policy, policy\_rollout * **论文**: [https://arxiv.org/abs/2304.13705](https://arxiv.org/abs/2304.13705) * **项目主页**: [https://tonyzhaozh.github.io/aloha/](https://tonyzhaozh.github.io/aloha/) * **GitHub**: [https://github.com/tonyzhaozh/act](https://github.com/tonyzhaozh/act)
RoboFlamingo ( `roboflamingo` ) * **能力**: vla, vla.policy\_rollout, vlm\_policy, robot\_policy * **论文**: [https://arxiv.org/abs/2311.01378](https://arxiv.org/abs/2311.01378) * **项目主页**: [https://github.com/RoboFlamingo/RoboFlamingo](https://github.com/RoboFlamingo/RoboFlamingo) * **GitHub**: [https://github.com/RoboFlamingo/RoboFlamingo](https://github.com/RoboFlamingo/RoboFlamingo)