# Studio Internals (/docs/maintainers/architecture/studio-internals)



Studio is a catalog-driven shell over the same pipeline and artifact contracts used by CLI inference and evaluation. It does not own model weights, scoring logic, or benchmark manifests. Its job is to discover runnable entries, prepare inputs, invoke the right runtime driver, write a `RunRecord`, and pick preview assets for the UI. The same execution core can serve Gradio (`app.py`), Workspace jobs, conda child processes, and torchrun workers; only the presentation layer changes.

```text
launch args
  -> StudioLaunchConfig
  -> discover_catalog()
  -> StudioManager
  -> prepare_inputs()
  -> runtime_driver_for()
  -> run()
  -> materialize_run()
  -> pick_preview_assets()
  -> Gradio / Workspace UI update
```

Keep inference semantics in `studio/execution.py` and catalog discovery in `studio/catalog.py`. `studio/app.py` should stay thin: layout, callbacks, and Gradio component updates. If a change must also work from `runtime_job.py` or `workspace_app.py` without Gradio, it belongs in execution, not in the UI file.

## Files [#files]

| File                  | Role                                                                                                            | Important symbols                                                                            |
| --------------------- | --------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `studio/catalog.py`   | Discovers model entries from pipeline files, canonical runtime entries, and overrides.                          | `CatalogEntry`, `discover_catalog()`, `filter_catalog()`, `find_entry()`, `catalog_stats()`. |
| `studio/execution.py` | Prepares inputs, loads pipeline classes, chooses runtime driver strategy, writes run records, exports previews. | `PreparedInputs`, `RunRecord`, `PipelineContext`, `BaseRuntimeDriver`, `StudioManager`.      |
| `studio/app.py`       | Gradio UI, launch config, preset handling, callbacks, run buttons, recent-run loading.                          | `StudioLaunchConfig`, `parse_launch_config()`, `build_demo()`, `main()`.                     |
| `studio/theme.py`     | Theme constants and UI styling hooks.                                                                           | Used only by the app shell.                                                                  |

## Catalog Discovery [#catalog-discovery]

Catalog build is intentionally static. `discover_catalog()` walks pipeline modules with AST helpers so Studio can list models without importing heavy CUDA stacks or optional dependencies. It merges three sources: `pipeline_*.py` classes, component-pipeline factories, and action-synthesis classes, then overlays curated overrides and canonical runtime entries (aliases, default kwargs, checkpoints, `runtime_kind`). Dedup keys are `(module_path, class_name)`. Hidden or abstract ids stay out of the UI list.

Import happens later, only when a run needs a concrete class: `StudioManager.import_pipeline_class()` loads `module_path` / `class_name` from the selected `CatalogEntry`. That split is why a model can appear in the catalog yet still fail at LOAD if the current environment lacks its runtime imports or local checkpoint.

| Step              | Function                                         | Meaning                                                                                  |
| ----------------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------- |
| Find project root | `_project_root()` / `_project_root_candidates()` | Allows Studio to run from repo root or package context.                                  |
| Inspect pipelines | AST helpers around `_AstPipelineInfo`            | Reads pipeline modules without importing heavy model dependencies.                       |
| Normalize ids     | `_normalize_model_id()`                          | Stable lookup key for UI, CLI launch args, and recent runs.                              |
| Build entries     | `discover_catalog()`                             | Produces `CatalogEntry[]` with family, class path, category, backend, aliases, examples. |
| Filter entries    | `filter_catalog()`                               | Applies search and category filters for the UI list.                                     |

## Execution Core [#execution-core]

`StudioManager` is the orchestration hub. A typical `run` / `stream` / `init` / `reset` path prepares a request, stages inputs into a run directory, selects a driver from `entry.runtime_kind`, loads the pipeline (LRU-cached), invokes the driver, then materializes a `RunRecord`. Drivers exist because not every family is a single `__call__`: two-stage 3DGS reconstructs then orbits, point-cloud nav reconstructs then renders views, WorldFM needs custom camera and panorama handling. Everything else uses `BaseRuntimeDriver`.

`PreparedInputs` is the normalized payload after staging prompts, images, videos, interactions, and runtime kwargs. `PipelineContext` holds the loaded class, device, variant, and output root. `RunRecord` is the durable unit the UI reloads: ids, inputs, outputs, artifact paths, preview picks, status, and manifest JSON. Workspace inference may also pass an `infer_metadata` contract so materialization can enforce required output kinds (video, image, splat, and so on).

| Class / function             | Role                                                                                                       |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `PreparedInputs`             | Captures staged prompt, images, videos, interactions, runtime kwargs, and metadata.                        |
| `RunRecord`                  | Stores run id, model id, inputs, outputs, artifacts, preview paths, status, and manifest export.           |
| `PipelineContext`            | Keeps imported pipeline class, model ref, device, variant, load kwargs, and output root.                   |
| `BaseRuntimeDriver`          | Default driver: load pipeline, run fresh request, optionally initialize/continue streaming, reset state.   |
| `TwoStage3DGSRuntimeDriver`  | Specialized driver for two-stage 3DGS flows.                                                               |
| `PointCloudNavRuntimeDriver` | Specialized driver for point-cloud/navigation previews.                                                    |
| `WorldFMRuntimeDriver`       | Specialized driver for WorldFM camera pose and panorama workflow.                                          |
| `StudioManager`              | Orchestrates import, driver selection, input preparation, run execution, recent-run storage, unload/reset. |

## Preview and Artifact Helpers [#preview-and-artifact-helpers]

After the driver returns, `materialize_run()` turns a heterogeneous result into files under the run directory: metadata JSON, known artifact keys, optional frame-to-video export, mesh conversion, and optional Rerun recordings. When previews are still empty, Studio may scan the output directory depending on `WORLDFOUNDRY_STUDIO_ARTIFACT_SCAN_MODE`. `pick_preview_assets()` then ranks artifacts for the UI—preferring generated videos and named previews over staged inputs—and returns the primary video, image, splat/model, gallery, and optional `.rrd`.

Blank previews usually mean the pipeline wrote files outside expected keys, returned an in-memory object without `save()`, or produced a video the decoder rejects. Fix the pipeline contract first; only then widen scan mode or conversion helpers.

| Function                              | Purpose                                                           |
| ------------------------------------- | ----------------------------------------------------------------- |
| `export_frames_to_video()`            | Converts frame sequences into a preview video.                    |
| `collect_artifact_paths()`            | Finds generated files under the output directory.                 |
| `pick_preview_assets()`               | Chooses primary video/image/model/json assets for UI preview.     |
| `maybe_extract_video_preview_image()` | Saves a representative image from a video preview.                |
| `convert_model_for_preview()`         | Converts 3D assets when browser preview needs a different format. |
| `maybe_build_rerun_rrd()`             | Builds a Rerun recording when available.                          |
| `write_json()`                        | Writes run manifests and input/output records.                    |

## UI File Boundaries [#ui-file-boundaries]

Treat `app.py` as presentation glue. Launch flags that only affect Gradio stay there; values that change non-UI execution belong in `StudioLaunchConfig` consumption inside execution. HTML status blocks and tray cards stay in the UI. Button handlers may call `StudioManager`, but they should not load pipelines, stage files, or invent artifact layouts themselves. Presets that are purely demo defaults can live in the UI; presets that must match evaluation or Workspace contracts should be shared data, not Gradio-only strings.

| Area           | Keep in `app.py`                                            | Move to `execution.py` if                                |
| -------------- | ----------------------------------------------------------- | -------------------------------------------------------- |
| Launch parsing | CLI flags, default UI selections, gradio launch parameters. | The value affects non-UI execution.                      |
| HTML helpers   | Header, tray, cards, status blocks, summaries.              | The same output is needed by tests or CLI.               |
| Callback glue  | Button handlers and component updates.                      | It loads models, touches files, or transforms artifacts. |
| Presets        | UI-specific prompt/interactions defaults.                   | Preset semantics must be shared with evaluation.         |

## Debug Path [#debug-path]

Most Studio bugs are contract mismatches, not Gradio bugs. A missing model is almost always catalog discovery, aliases, or a hidden id. Wrong device or variant usually means launch config and `PipelineContext` overrides. Inputs that never reach the model fail in `prepare_inputs()` or driver kwargs binding. Streaming failures often mean `init`/`stream` was called without memory state or without a prior reconstruction for two-stage drivers. Recent-run reload failures mean the manifest or artifact paths drifted after `materialize_run()`.

| Symptom                  | Check                                                                                                  |
| ------------------------ | ------------------------------------------------------------------------------------------------------ |
| Model missing from UI    | `discover_catalog()` output, `CatalogEntry.aliases`, category filter, pipeline file naming.            |
| Wrong variant/device     | `StudioLaunchConfig`, `_variant_payload()`, `_apply_variant_runtime_overrides()`, `PipelineContext`.   |
| Input not reaching model | `prepare_inputs()`, `PreparedInputs`, driver `run_fresh()` kwargs.                                     |
| Preview blank            | `collect_artifact_paths()`, `pick_preview_assets()`, conversion helpers, generated file extension.     |
| Streaming broken         | `BaseRuntimeDriver.run_init()`, `BaseRuntimeDriver.run_continue()`, pipeline `stream()`, memory state. |
| Recent run cannot reload | `RunRecord.to_manifest()`, `materialize_run()`, `load_run()`, artifact paths.                          |

For end-user launch and frontend routing, see [Studio](/docs/guides/studio). For the pipeline / operator / `GenerationResult` contract Studio sits on, see [Model Runtime](/docs/maintainers/architecture/model-runtime).