Studio

Introduction

WorldFoundry Studio is a thin visual entrypoint around repo pipelines. The model is fixed at launch time; use a new process for a different model family.

Studio exposes multiple frontends:

What it shows

By default, Studio hides most advanced panels. Use --show-aux-panels for history, notes, catalog, 3DGS import, and low-level runtime controls.

Workspace Mode

For multi-model validation and release QA, run the workspace app instead of a single-model Studio process:

source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

bash scripts/workspace/run_workspace.sh \
  --host 127.0.0.1 \
  --port 7870 \
  --max-jobs 8

Use explicit roots when your checkpoints or benchmark data live outside the default cache:

bash scripts/workspace/run_workspace.sh \
  --ckpt-dir /path/to/checkpoints \
  --data-dir /path/to/benchmark-data

The launcher is open-source friendly and does not depend on machine-specific paths. It resolves assets in this order: command-line option, already exported environment variable, repo-sibling ckpt/ or data/ directory when present, env-file value from tmp/worldfoundry_unified_env.sh, then ~/.cache/worldfoundry/....

Start the Workspace from the unified model runtime environment, not from a lightweight UI-only environment. For CUDA 12.8 hosts this is usually worldfoundry-unified-cu128 after source tmp/worldfoundry_unified_env.sh. If LOAD or START fails with a missing import such as cv2, restart 7870 from that unified environment before debugging model code.

Do not copy or relocate a prepared conda environment between machines. Native Python and OpenSSL links can point back to the original prefix, which makes resident inference workers fail even when the browser server starts. If the Workspace reports a missing python3.x, missing cv2, or libssl/libcrypto error, recreate or verify the unified env through the public setup command:

bash scripts/setup/bootstrap_worldfoundry.sh --verify-only
source tmp/worldfoundry_unified_env.sh
bash scripts/workspace/run_workspace.sh

Open:

http://127.0.0.1:7870/

The workspace is the preferred UI for release validation. Use it to submit inference jobs, inspect model-specific forms, review artifacts, and compare runs without switching between multiple one-off scripts.

Workspace also supports evaluation jobs. For VBench, select Evaluation, choose benchmark vbench, put generated videos in Dataset Root (or an official VBench results JSON in Results Path), and set Metrics to one or more dimensions such as aesthetic_quality, motion_smoothness, or the preset custom_supported. VBench-specific runtime options such as {"mode":"custom_input","load_ckpt_from_local":true} go in Eval Runtime JSON.

Create Job configuration

There is no separate Settings page in the Workspace UI. Configure each job in the Create Job dialog before submission.

When you open Create Job or switch task profiles, the form is prefilled from Workspace defaults loaded at startup. Model-specific official demo values from the catalog and task profile still take precedence over those shared defaults.

Default values when no settings file is present:

Optional persisted defaults

To reuse the same defaults across Workspace restarts, point the server at a JSON settings file:

export WORLDFOUNDRY_STUDIO_SETTINGS_FILE=/path/to/studio-settings.json

On startup, the Workspace loads that file into /api/settings. The Create Job form reads those values through GET /api/settings. Operators can also update the file programmatically with POST /api/settings:

curl -sS -X POST http://127.0.0.1:7870/api/settings \
  -H 'Content-Type: application/json' \
  -d '{"values":{"fps":12,"num_frames":49,"attention_backend":"torch"}}'

When WORLDFOUNDRY_STUDIO_SETTINGS_FILE is unset, defaults live only in the running Workspace process and reset on restart.

Fast jobs and preflight checks

Workspace inference defaults to a fast job path. It assumes the selected conda environment, checkpoint roots, and viewer dependencies were prepared before the server was launched. Keep this default for normal repeated inference.

Turn checks on only for setup debugging or release preflight:

For the fastest finish path, model runtimes should return explicit preview_image, preview_video, artifact_paths, rrd_path, and output_dir values in their run record. Use always scanning only as a compatibility mode for older runtimes that write files without reporting them.

Resident workers keep the conda child process alive across jobs so the child StudioManager can reuse its pipeline cache. Set WORLDFOUNDRY_STUDIO_RESIDENT_WORKERS=0 when debugging process startup or when you need every job to release GPU memory immediately after completion.

Use Visualizers, Gallery, and Artifacts for quick visual QA. They expose recent generated videos, images, manifests, and direct artifact links. Before making release claims, record the verified run in the model inference validation page.

Workspace Visualizers

The 7870 Workspace is the browser entrypoint for the visualization stack. Open the Visualizers tab, then use each card's Launch, Open, and Stop buttons. The iframe under each card embeds the live viewer, so users do not need to copy URLs during normal QA. The Embodied Simulator Bridge card is marked Planned: unlike Viser, Spark, Rerun, and World Rollout, it does not yet launch an in-tree embodied viewer.

World Rollout Viewer

• Port7868
• Aliasesinteractive-world, world-model, world-rollout

Buffered realtime target for action-conditioned roaming. Controls are sampled continuously while generated chunks are prefetched.

Artifactsimage, video, action_trace

Launch routepython -m worldfoundry.studio.app matrix-game-2 --frontend world --port 7868

Viser Geometry Viewer

• Port18590
• Aliasesviser, geometry, pointcloud

Point clouds, depth-derived point sets, meshes, camera poses, and trajectories.

Artifactsply, pcd, xyz, glb, gltf, obj, npz

Launch routepython -m worldfoundry.studio.app pi3 --frontend points --asset /path/to/scene.ply

Rerun Timeline Viewer

• Port9876
• Aliasesrrd

`.rrd` timelines and recordings with synchronized cameras, frames, tracks, and 3D entities.

Artifactsrrd

Launch routepython -m worldfoundry.studio.app vggt --frontend rerun --asset /path/to/recording.rrd

Spark Gaussian Splat Viewer

• Port8765
• Aliases3dgs, splat

3D Gaussian splats via the in-tree Spark frontend. Live preview uses the Spark.js Hello World butterfly demo.

Artifactssplat, spz, ksplat, sog, splat-ply

Launch routepython -m worldfoundry.studio.app vggt --frontend spark --asset /path/to/scene.splat

Embodied Simulator Bridge

Planned

• Port18610
• Aliasessim, simulator

Planned integration. Studio only registers an external simulator URL and tunnel hints today; there is no in-tree embodied viewer yet. Start the simulator separately, then paste its URL in Visualizers.

Artifactsaction_trace, trajectory, simulator_url

Launch routepython -m worldfoundry.studio.app openvla --frontend embodied --simulator-url http://127.0.0.1:18610

world defaults to port 7868. Set a port manually only when your deployment requires a fixed proxy route.

The realtime frontend requires worldfoundry[studio_realtime] (also included by worldfoundry[ui]). Model loading begins in the background when the server starts. A session then keeps one pipeline context and one execution thread resident; WASD/IJKL key edges are resampled at chunk boundaries while RGB frames are delivered over a long-lived WebRTC track. Per-run manifests, artifact scanning, and MP4 encoding remain Workspace/offline responsibilities and are intentionally absent from the control-to-frame hot path.

Latency-oriented defaults can be tuned without changing model catalog quality profiles:

For remote workstations, expose the Workspace port and the viewer ports you use. For example:

ssh -L 7870:127.0.0.1:7870 \
    -L 18590:127.0.0.1:18590 \
    -L 8765:127.0.0.1:8765 \
    -L 9876:127.0.0.1:9876 \
    user@host

Artifact Visualization Routing

When an inference job finishes, the Workspace scans the run record and output directory for visualizable artifacts. The Artifacts tab shows Open and Open in ... actions for supported files. Job Detail keeps inline preview video/image only; it does not duplicate viewer launch shortcuts.

Base-model perception outputs

WorldFoundry keeps one visualization capability registry for in-tree 3D and perception models. Query it before writing model-specific viewer code:

worldfoundry-eval models visualizations
worldfoundry-eval models visualizations --family perception_core
worldfoundry-eval models visualizations --model raft --json

Structured 2D results share one conversion command. Supported kinds are detection, mask, flow, tracks, feature-pca, depth, normal, keypoints, and text:

worldfoundry-eval models visualize \
  --kind flow \
  --artifact outputs/flow.npz \
  --output outputs/flow.png

worldfoundry-eval models visualize \
  --kind detection \
  --artifact outputs/detections.json \
  --media inputs/frame.png \
  --output outputs/detections.png

This shared path covers the official output schemas used by the Depth Anything, SAM, RAFT, DINO, GroundingDINO, YOLO World, CoTracker, and related in-tree families. Native image/video outputs open in the media viewer; point clouds and meshes use Viser, Gaussian splats use Spark, and synchronized scene recordings use Rerun. The registry command is the authoritative per-model mapping.

Open the run in Workspace and use the Artifacts tab to confirm viewer routing for files that already exist on disk. The table above is the routing reference; the Artifacts actions are authoritative for a specific run.

For model authors, the most reproducible path is to write outputs under the Studio run output directory and include stable paths in the run record:

• preview_image or preview_video for quick media review.
• artifact_paths for raw generated assets.
• rrd_path for Rerun recordings.
• output_dir for additional files that should be discoverable from the run.

3D reconstruction models should export browser-readable geometry such as .ply, .glb, .gltf, .obj, .pcd, .xyz, or compact point-cloud/depth .npz files. 3DGS models should export a native splat file, or a binary little-endian Gaussian-splat PLY whose header contains splat attributes such as scale, rotation, opacity, and spherical-harmonic color fields. The Workspace distinguishes ordinary PLY geometry from Gaussian-splat PLY files before choosing Viser or Spark.

Coordinate-system contract

Viewer launch is part of the geometry contract. A valid artifact can still look upside down or tilted when its world frame is interpreted twice. Keep points, meshes, camera-to-world transforms, frustums, and trajectories in one declared frame:

• An official GLB that already applies OpenCV-to-OpenGL alignment must use coordinate_preset=asset-native and the exporter-defined up axis. VGGT and VGGT-Omega official GLBs use +Y up and alignment=none.
• A raw VGGT/VGGT-Omega prediction NPZ remains in the model's OpenCV frame, so Viser applies coordinate_preset=opencv-to-opengl once and uses +Y up.
• Raw UniDepth V2 and UniK3D model-point NPZ files also use the OpenCV camera frame. Their Workspace PLY exporter instead persists x-right/y-up/z-forward, so those PLY files use asset-native, +Y up, and alignment=none without a second flip.
• DUSt3R, LoGeR, and Pi3 PLY point clouds keep their predicted camera-to-world sidecars. Viser applies first-camera-opengl to points and cameras together, uses +Y up, and frames the central 5-95% geometry range so sparse outliers do not shrink the useful scene.
• Rerun recordings must log the world ViewCoordinates and use the same camera convention in every Pinhole. Cut3R and MonST3R use RDF for both world and camera entities.
• Never fix an orientation mismatch by flipping only the point cloud. Apply the same rigid transform to geometry and camera-to-world poses, then verify reset, orbit, and timeline interaction in a browser.

Workspace resolves the proven VGGT, DUSt3R, LoGeR, and Pi3 defaults automatically. The Visualizer defaults also cover UniDepth V2 and UniK3D raw/Workspace assets. The Visualizer Params JSON remains an explicit override for other exporters. Viser reports the actual bound port when its requested port is occupied. Rerun uses WebGPU by default; set WORLDFOUNDRY_STUDIO_RERUN_RENDERER=webgl only when the target browser requires that fallback.

Choosing a 3D viewer

Keep the default route artifact-driven so model integrations do not duplicate viewer code:

An inference run is not visually verified merely because a viewer process starts. Release evidence for a 3D/4D method should confirm finite geometry, camera data, and the declared coordinate frame, then open the actual artifact in a browser, check a nonblank and readably framed canvas, exercise reset/orbit or timeline controls, and review console errors. Preserve an official viewer as a small visualization plugin; do not copy its inference runtime or build a per-model frontend when shared Viser/Rerun scene layers are sufficient.

Studio Screenshots

These screenshots are from the 7870 workspace used for model release validation.

Inference Jobs

The queue shows submitted runs, final status, and inline preview video or image when the run record exposes them.

Create Job

The form should expose the model's actual input contract. The screenshot below is only a UI-shape example; release demo claims should be checked against the validation reference and the matching run manifest.

Catalog

Catalog filtering is useful for checking which world-model entries are visible to the workspace before submitting jobs.

Gallery

The Gallery gives a compact visual pass over recent inference outputs and preview artifacts.

Inference Demo Results

The gallery below is for visual orientation. It shows representative artifacts from in-tree Studio or Studio-compatible API runs. It is not a release-claim table; exact parameters, environment notes, run ids, and remaining gaps live in the validation reference. Only curated nontrivial examples are embedded here.

Launch matrix

Recommended local start:

cd /path/to/WorldFoundry
PYTHONPATH=. WORLDFOUNDRY_STUDIO_PORT=7868 conda run -p "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}" python -m worldfoundry.studio.app \
  lingbot-world \
  --variant fast \
  --model-ref /path/to/checkpoints/lingbot-world-fast \
  --device cuda:1

For an editable checkout that is not installed into the active environment, keep PYTHONPATH=. in the command.

Studio prints the resolved workspace, model, cache, and Hugging Face cache roots at startup, so path issues are visible before a run begins.

Open:

http://127.0.0.1:7868

For remote GPU nodes, use the SSH tunnel printed by Studio.

The world process serves the page and WebRTC signaling on the configured HTTP port. Video and controls then use the ICE-selected WebRTC path. A plain SSH forward only carries signaling; for a browser that cannot directly reach the GPU host, configure a TURN relay with WORLDFOUNDRY_REALTIME_ICE_SERVERS_JSON.

ssh -N -L 7868:127.0.0.1:7868 user@gpu-host
# local browser:
# http://127.0.0.1:7868

The status strip reports RTC playback FPS, generation time, control latency, queue depth, and dropped stale frames. An ICE timeout is surfaced explicitly instead of leaving the page indefinitely in a connecting state.

Interaction modes

Keyboard/live controls:

WASD = move
Arrow keys / IJKL = look

Keyboard and stick edges are sent immediately over the current realtime channel. The resident runtime samples the latest effective control state at model chunk boundaries, so input does not create a second inference request or reload the pipeline.

Live text events, idle steps, and output resolution

The world frontend exposes three ordered controls beside the keyboard and joysticks:

• Text events: open INITIAL SCENE, choose ADD, and enter an event ID, button label, and prompt. APPLY installs the catalog before a session or replaces it while the session is live. Event buttons activate one text condition at the next model chunk boundary; CLEAR EVENT restores the base prompt. Models without a state-preserving text-update path disable this surface instead of acknowledging an update they cannot apply.
• STEP: queues exactly one generated chunk with no keyboard direction. It is useful for an idle camera or for checking a text event. It does not enable an autonomous generation loop. Queued-segment models keep STEP disabled because their next segment requires new control videos.
• OUTPUT: selects the transmitted frame size. NATIVE preserves the model output size. Smaller choices resize and letterbox only the WebRTC/WebSocket stream; they do not change model inference resolution, sampling steps, causal state, or future world quality.

Event IDs contain 1-64 letters, numbers, or _.:-; labels are at most 64 characters, prompts are at most 1,000 characters, and a catalog contains at most 12 events. Catalog updates carry a revision so a stale editor cannot silently overwrite a newer live catalog.

The reliable control channel returns event_catalog_ack, event_ack, step_ack, and output_config_ack. An event or output ACK means the ordered request was accepted and reports where it takes effect (next_chunk or next_segment); chunk_done remains the completion signal for generated output. Resolution changes are frozen at a complete transport-chunk boundary, so an already announced chunk is never truncated or mixed across dimensions. Each request and ACK carries the same request_id, so rapid clicks cannot let an older response overwrite the current UI state.

The same protocol can be driven without the built-in controls:

{"type":"event_catalog","request_id":"catalog-1","base_revision":0,"events":[{"event_id":"rain","label":"Rain","prompt":"Heavy rain begins.","category":"weather"}]}
{"type":"event","request_id":"event-1","event_id":"rain","state":"trigger"}
{"type":"event","request_id":"event-2","state":"clear"}
{"type":"action","request_id":"step-1","action":{"event":"step"}}
{"type":"output_config","request_id":"output-1","resolution":{"mode":"fixed","width":640,"height":360}}

Fixed transport dimensions must be even, at least 160x90, no larger than the model's native/generated frame on either axis, and within the realtime 1920x1080 pixel budget. Studio advertises only source-safe presets derived from the model catalog; if the native size is unknown, it does not guess a fixed preset. Switching back to {"mode":"native"} removes the transport scaler. WebRTC resize work runs before frames enter the media track, so held frames do not repeatedly pay the resize cost or block the control event loop.

Draggable and collapsible panels

STATUS, CONTROLS, INITIAL SCENE, and RUNTIME share the same panel behavior. Drag a panel header to float it above the viewport, click − to collapse it, and click + to reopen it. The browser clamps floating panels to the visible stage, raises the selected panel, and saves positions and collapsed state locally. Dragging uses one animation-frame transform per browser frame and does not add work to the model/frame transport path. On narrow mobile layouts, panels stay in document flow but remain collapsible.

Immersive fullscreen and live runtime log

The fullscreen view is part of the shared world frontend and is available to every interactive model routed to that frontend. It reuses the current peer or socket, frame buffer, and resident model session. Entering or leaving fullscreen does not reconnect the browser or restart generation.

1. Prepare the model input and start the session normally.
2. Click FULLSCREEN in the upper-left corner of the viewport, or press F.
3. The viewport fills the screen and the live runtime log opens automatically.
4. Click LIVE LOG or press Shift+L to hide or show the log without leaving fullscreen.
5. Press Esc or click EXIT to return to the standard Studio layout.

The log includes client and model readiness, realtime transport state, segment queue/generation/playback transitions, errors, and per-chunk performance data. A completed chunk reports its frame count, generation time (gen), playback queue depth (queue), and discarded stale frames (dropped). These entries are session telemetry for interactive debugging; they are not persisted run artifacts or a copy of the server's complete stdout/stderr stream.

Live log events use the existing realtime connection. The browser batches DOM updates on animation frames and retains at most the latest 120 entries, so a long session does not accumulate an unbounded log tree or add a polling loop to the frame path. When the browser permits the Fullscreen API, the button or keyboard shortcut enters native fullscreen. If native fullscreen is unavailable or blocked by browser policy, Studio keeps the same session and falls back to a viewport-filling layout. The Workspace visualizer iframe already grants fullscreen permission.

Launch flags

Environment

Only WORLDFOUNDRY_* Studio environment variables are supported.

Release Evidence

Only add public demo media after the matching official configuration has run through the in-tree Studio or Studio-compatible API path. Keep the run manifest, decoded media metadata, and visual QA record with the validation artifacts so the embedded video can be traced back to an actual release candidate run.