Embodied · vla · vla.action_prediction · vla.policy_rollout · robot_policy

MolmoAct2

allenai

WorldFoundry in-tree MolmoAct2 runtime lives under worldfoundry/synthesis/action_generation/molmoact2 and no longer depends on the external molmoact2 checkout at runtime.

Runner verified统一环境molmoact2

构建运行命令

选择仓库中已记录的 variant，然后复制对应的准备、检查或运行命令。

worldfoundry.pipeline

Variant

任务vla.policy_rollout

环境worldfoundry-unified-cu128

设备CUDA 12.8

worldfoundry-eval evaluate \
  --mode model \
  --model-id molmoact2-droid \
  --model-runner worldfoundry:pipeline \
  --model-manifest-dir worldfoundry/data/models/catalog \
  --requests-path tmp/requests.jsonl \
  --output-dir tmp/model_eval/molmoact2-droid \
  --metric artifact_count \
  --json

兼容性与版本

由 Manifest 生成

集成状态: in tree checkpoint runtime static verified
Runner 证据: verified
环境: worldfoundry-unified-cu128
Python: 3.11
CUDA: CUDA 12.8
PyTorch: torch
源码 revision: —
Checkpoint revision: —

Runtime profile: molmoact2
Pipeline binding: molmoact2
Runner: worldfoundry.pipeline
Pipeline target: worldfoundry.pipelines.component_pipelines:MolmoAct2Pipeline
Backend stage: in_tree_runtime
Runtime 状态: in_tree_molmoact2_hf_predict_action_gpu_validated
Driver 状态: compatible
环境类型: 统一环境

安装环境

环境解析器会读取已记录 profile，并选择此处显示的统一或独立环境。

bash scripts/setup/model_env_install.sh --model molmoact2-droid

依赖版本约束 10

torch
torchvision
transformers>=4.57,<4.58
accelerate>=1.0
safetensors>=0.4
pillow>=10
numpy>=1.26,<3
einops>=0.7
sentencepiece>=0.2
protobuf>=4.25

Conda 依赖 2

python
pip

Checkpoint 与资产

分配算力前先做本地检查；gated、private 与 license 字段直接来自 checkpoint manifest。

worldfoundry-eval zoo model-download --model-id molmoact2 --check-local --json

allenai/MolmoAct2-DROID

Revision: —
License: apache-2.0
Gated: —
Private: —

allenai/MolmoAct2-BimanualYAM

Revision: —
License: apache-2.0
Gated: —
Private: —

allenai/MolmoAct2-SO100_101

Revision: —
License: apache-2.0
Gated: —
Private: —

allenai/MolmoAct2-LIBERO

Revision: —
License: apache-2.0
Gated: —
Private: —

allenai/MolmoAct2-Think-LIBERO

Revision: —
License: apache-2.0
Gated: —
Private: —

allenai/MolmoAct2

Revision: —
License: —
Gated: —
Private: —

allenai/MolmoAct2-Think

Revision: —
License: —
Gated: —
Private: —

运行与输出

生成的命令通过共享 evaluation 边界运行，并持久保存结果 manifest 与 artifact。

worldfoundry-eval evaluate \
  --mode model \
  --model-id molmoact2-droid \
  --model-runner worldfoundry:pipeline \
  --model-manifest-dir worldfoundry/data/models/catalog \
  --requests-path tmp/requests.jsonl \
  --output-dir tmp/model_eval/molmoact2-droid \
  --metric artifact_count \
  --json

输入契约

字段	已记录契约
`actions`	action_chunk, robot_action
`camera_keys`	external_cam, external_cam_2, wrist_cam
`default_embodiment`	droid
`image`	Required
`instruction`	Required
`norm_tag`	franka_droid
`prompt`	Required
`state`	Required
`state_dim`	8
`video`	Optional

Artifact 契约

Artifact 类型	文件名 / 路径
`action_trace`	`—`
`action_trace`	`molmoact2_action_trace.json`

证据与来源

Catalog 集成、原生 demo parity 与 runner parity 是三条独立记录。

集成状态: in tree checkpoint runtime static verified
Runner 证据: verified
原生 Demo 证据: 未记录
验证 Imports: numpy, PIL, torch, torchvision, transformers

配方溯源

worldfoundry/data/models/catalog/vla_va_wam/molmoact2.yamlGitHubhttps://github.com/allenai/molmoact2Hugging Facehttps://huggingface.co/allenai/MolmoAct2Hugging Facehttps://huggingface.co/allenai/MolmoAct2-ThinkHugging Facehttps://huggingface.co/allenai/MolmoAct2-DROIDHugging Facehttps://huggingface.co/allenai/MolmoAct2-BimanualYAMHugging Facehttps://huggingface.co/allenai/MolmoAct2-SO100_101Hugging Facehttps://huggingface.co/allenai/MolmoAct2-LIBEROHugging Facehttps://huggingface.co/allenai/MolmoAct2-Think-LIBERO

构建运行命令

输入契约

Artifact 契约

相关配方