Core inference runtime
Model inference specs, task/variant contracts, process-wide policy, autocast, compilation, torchrun commands, devices, and timers.
Core runtime makes a model's runnable surface explicit before loading its implementation. ModelInferenceSpec describes model-family identity, variants, checkpoints, task profiles, fields, artifacts, streaming support, and supported call parameters. The process helpers then apply shared execution policy without embedding one model's behavior in Core.
Build or retrieve an inference spec
from worldfoundry.core import model_inference_spec
spec = model_inference_spec(
model_family_id="my-world-model",
display_name="My World Model",
default_model_ref="org/my-world-model",
workload_type="video",
supported_call_params=("prompt", "seed", "num_frames"),
)
assert spec.model_family_id == "my-world-model"
assert spec.variant().variant_id == "default"
assert spec.task().task_idFor a curated built-in family, model_inference_spec returns the registered spec. For an unknown family it builds a generic spec from the supplied hints. Use get_model_inference_spec when “not registered” must remain distinguishable from a fallback.
Process-wide inference policy
install_worldfoundry_inference_infra configures attention policy, float32 matmul precision, TF32 flags, and an optional SDPA compatibility patch. It is idempotent process state, not a per-request object. worldfoundry_inference_context installs that state and runs under torch.no_grad().
autocast_context returns a CUDA autocast manager only for CUDA devices; CPU and unavailable-Torch cases return a no-op context. compile_module_if_enabled is explicitly opt-in and returns the original module if compilation is disabled, unsupported, or fails in non-strict mode.
from torch import nn
from worldfoundry.core import compile_module_if_enabled, torchrun_module_command
module = nn.Linear(4, 2)
assert compile_module_if_enabled(module, enabled=False) is module
command = torchrun_module_command(
"my_package.worker",
nproc_per_node=4,
args=("--checkpoint", "/models/run-42"),
)
print(command) # builds the command; it does not start a processUse run_torchrun_module only when the current process should own launching and capturing a bounded single-node job. It returns CompletedProcess even on a non-zero child exit, so callers must inspect returncode and logs.
Complete reference
The blocks below are the generated signatures for this category. Use the on-page symbol index to jump; source links open the defining implementation behind each lazy export.
26 public symbols
autocast_context
funcdef autocast_context(device: Any,dtype: Any | None = None,enabled: bool = True) -> Anyworldfoundry.core.autocast_contextfrom worldfoundry.core import autocast_contextOverview
Return a CUDA autocast context and a no-op context for non-CUDA devices. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: Any.
Parameters
deviceAnydtypeAny | None- default:
None enabledbool- default:
True
Returns: Any
def compile_module_if_enabled(module: Any,enabled: bool | None = None,label: str | None = None,backend: str | None = None,mode: str | None = None,fullgraph: bool | None = None,dynamic: bool | None = None,options: dict[str, Any] | None = None) -> Anyworldfoundry.core.compile_module_if_enabledfrom worldfoundry.core import compile_module_if_enabledOverview
Compile one module with `torch.compile only when explicitly enabled. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: Any`.
Parameters
moduleAnyenabledbool | None- default:
None labelstr | None- default:
None backendstr | None- default:
None modestr | None- default:
None fullgraphbool | None- default:
None dynamicbool | None- default:
None optionsdict[str, Any] | None- default:
None
Returns: Any
def cuda_visible_devices_from_device(device: str | torch.device | None,inherited: str | None = None,map_inherited: bool = True,default_cuda: str = '0') -> str | Noneworldfoundry.core.cuda_visible_devices_from_devicefrom worldfoundry.core import cuda_visible_devices_from_deviceOverview
Convert a device string into a `CUDA_VISIBLE_DEVICES value. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: str | None`.
Source docstring
Convert a device string into a `CUDA_VISIBLE_DEVICES` value.
`cuda:N is interpreted as a local index into an inherited CUDA_VISIBLE_DEVICES` list by default, which is the behavior expected by subprocess launchers nested under a scheduler or torchrun process.
Parameters
devicestr | torch.device | Noneinheritedstr | None- default:
None map_inheritedbool- default:
True default_cudastr- default:
'0'
Returns: str | None
class CudaSyncTimer(name: str | None = None,flag_env: str = 'SYNC_TIMER',log_fn: Callable[[str], None] | None = None)worldfoundry.core.CudaSyncTimerfrom worldfoundry.core import CudaSyncTimerOverview
Optional CUDA-synchronized timer usable as a context manager or decorator. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Parameters
namestr | None- default:
None flag_envstr- default:
'SYNC_TIMER' log_fnCallable[[str], None] | None- default:
None
Methods
def generic_model_inference_spec(model_family_id: str,display_name: str | None = None,default_model_ref: str = '',default_load_kwargs: Mapping[str, Any] | None = None,default_call_kwargs: Mapping[str, Any] | None = None,supports_stream: bool = False,workload_type: str = '',supported_call_params: Sequence[str] | None = None) -> ModelInferenceSpecworldfoundry.core.generic_model_inference_specfrom worldfoundry.core import generic_model_inference_specOverview
Build a conservative fallback spec for models not yet curated. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: ModelInferenceSpec.
Parameters
model_family_idstrdisplay_namestr | None- default:
None default_model_refstr- default:
'' default_load_kwargsMapping[str, Any] | None- default:
None default_call_kwargsMapping[str, Any] | None- default:
None supports_streambool- default:
False workload_typestr- default:
'' supported_call_paramsSequence[str] | None- default:
None
Returns: ModelInferenceSpec
def get_model_inference_spec(model_family_id: str) -> ModelInferenceSpec | Noneworldfoundry.core.get_model_inference_specfrom worldfoundry.core import get_model_inference_specOverview
Return a curated inference spec for a model family when one exists. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: ModelInferenceSpec | None.
Parameters
model_family_idstr
Returns: ModelInferenceSpec | None
def inference_infra_state() -> WorldFoundryInferenceInfraStateworldfoundry.core.inference_infra_statefrom worldfoundry.core import inference_infra_stateOverview
Return the process-global inference infra state. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: WorldFoundryInferenceInfraState.
Returns: WorldFoundryInferenceInfraState
class InferenceArtifactSpec(artifact_id: str,kind: str,required: bool = False,preview: bool = False,description: str = '')worldfoundry.core.InferenceArtifactSpecfrom worldfoundry.core import InferenceArtifactSpecOverview
Output artifact contract emitted by an inference task profile. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
artifact_idstrkindstrrequiredbool- default:
False previewbool- default:
False descriptionstr- default:
''
Methods
class InferenceCheckpointRef(role: str,uri: str,required: bool = True,status: str = 'unknown')worldfoundry.core.InferenceCheckpointReffrom worldfoundry.core import InferenceCheckpointRefOverview
Checkpoint reference used by a concrete inference variant. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
rolestruristrrequiredbool- default:
True statusstr- default:
'unknown'
Methods
class InferenceFieldSpec(field_id: str,label: str,kind: str = 'string',target: str = 'call_kwargs',required: bool = False,default: Any = None,choices: tuple[str, ...] = (),description: str = '')worldfoundry.core.InferenceFieldSpecfrom worldfoundry.core import InferenceFieldSpecOverview
User-facing input field contract for one inference task profile. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
field_idstrlabelstrkindstr- default:
'string' targetstr- default:
'call_kwargs' requiredbool- default:
False defaultAny- default:
None choicestuple[str, ...]- default:
() descriptionstr- default:
''
Methods
class InferenceParams(max_batch_size: int, max_sequence_length: int)worldfoundry.core.InferenceParamsfrom worldfoundry.core import InferenceParamsOverview
State container used to cache key/value tensors during inference. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Parameters
max_batch_sizeintmax_sequence_lengthint
Methods
Overview
Public method on this type.
Parameters
batch_idx
Returns: None
class InferenceTaskProfile(task_id: str,label: str,inputs: tuple[InferenceFieldSpec, ...],outputs: tuple[InferenceArtifactSpec, ...],description: str = '',default_call_kwargs: Mapping[str, Any] = <dict factory>,aliases: tuple[str, ...] = ())worldfoundry.core.InferenceTaskProfilefrom worldfoundry.core import InferenceTaskProfileOverview
Runnable inference task profile for a model family or variant. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
task_idstrlabelstrinputstuple[InferenceFieldSpec, ...]outputstuple[InferenceArtifactSpec, ...]descriptionstr- default:
'' default_call_kwargsMapping[str, Any]- default:
<dict factory> aliasestuple[str, ...]- default:
()
Methods
class InferenceVariantSpec(variant_id: str,label: str,checkpoints: tuple[InferenceCheckpointRef, ...] = (),status: str = 'unknown',load_kwargs: Mapping[str, Any] = <dict factory>,call_kwargs: Mapping[str, Any] = <dict factory>,aliases: tuple[str, ...] = (),notes: tuple[str, ...] = ())worldfoundry.core.InferenceVariantSpecfrom worldfoundry.core import InferenceVariantSpecOverview
Concrete checkpoint/runtime variant under a model family. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
variant_idstrlabelstrcheckpointstuple[InferenceCheckpointRef, ...]- default:
() statusstr- default:
'unknown' load_kwargsMapping[str, Any]- default:
<dict factory> call_kwargsMapping[str, Any]- default:
<dict factory> aliasestuple[str, ...]- default:
() notestuple[str, ...]- default:
()
Methods
Overview
Public property on this type.
Parameters
self
Returns: str
Overview
Public method on this type.
Returns: dict[str, str]
def install_worldfoundry_inference_infra(attention_backend: str | None = None,matmul_precision: str | None = None,enable_tf32: bool | None = None,patch_sdpa: bool | None = None) -> WorldFoundryInferenceInfraStateworldfoundry.core.install_worldfoundry_inference_infrafrom worldfoundry.core import install_worldfoundry_inference_infraOverview
Install WorldFoundry core inference optimizations for this process. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: WorldFoundryInferenceInfraState.
Source docstring
Install WorldFoundry core inference optimizations for this process.
Environment controls: - `WORLDFOUNDRY_USE_CORE_INFRA=0 disables installation. - WORLDFOUNDRY_ATTENTION_BACKEND=auto|flash|cudnn|efficient|math selects the SDPA backend policy. - WORLDFOUNDRY_MATMUL_PRECISION=highest|high|medium selects PyTorch float32 matmul precision. - WORLDFOUNDRY_ENABLE_TF32=0 disables TF32 backend flags. - WORLDFOUNDRY_PATCH_SDPA=0` avoids monkey-patching PyTorch SDPA calls.
Parameters
attention_backendstr | None- default:
None matmul_precisionstr | None- default:
None enable_tf32bool | None- default:
None patch_sdpabool | None- default:
None
Returns: WorldFoundryInferenceInfraState
def list_model_inference_specs() -> tuple[ModelInferenceSpec, ]worldfoundry.core.list_model_inference_specsfrom worldfoundry.core import list_model_inference_specsOverview
Return curated model inference specs with explicit variants and task profiles. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: tuple[ModelInferenceSpec, ...].
Returns: tuple[ModelInferenceSpec, ...]
def model_inference_spec(model_family_id: str,display_name: str | None = None,default_model_ref: str = '',default_load_kwargs: Mapping[str, Any] | None = None,default_call_kwargs: Mapping[str, Any] | None = None,supports_stream: bool = False,workload_type: str = '',supported_call_params: Sequence[str] | None = None) -> ModelInferenceSpecworldfoundry.core.model_inference_specfrom worldfoundry.core import model_inference_specOverview
Return curated spec for a model family, or a generic fallback. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: ModelInferenceSpec.
Parameters
model_family_idstrdisplay_namestr | None- default:
None default_model_refstr- default:
'' default_load_kwargsMapping[str, Any] | None- default:
None default_call_kwargsMapping[str, Any] | None- default:
None supports_streambool- default:
False workload_typestr- default:
'' supported_call_paramsSequence[str] | None- default:
None
Returns: ModelInferenceSpec
class ModelInferenceSpec(model_family_id: str,display_name: str,variants: tuple[InferenceVariantSpec, ...],tasks: tuple[InferenceTaskProfile, ...],default_variant_id: str = 'default',default_task_id: str = 'default',aliases: tuple[str, ...] = (),notes: tuple[str, ...] = ())worldfoundry.core.ModelInferenceSpecfrom worldfoundry.core import ModelInferenceSpecOverview
Inference contract shared by Studio, CLI, manifests, and eval. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
model_family_idstrdisplay_namestrvariantstuple[InferenceVariantSpec, ...]taskstuple[InferenceTaskProfile, ...]default_variant_idstr- default:
'default' default_task_idstr- default:
'default' aliasestuple[str, ...]- default:
() notestuple[str, ...]- default:
()
Methods
Overview
Public method on this type.
Parameters
variant_idstr | None- default:
None
Returns: InferenceVariantSpec
Overview
Public method on this type.
Parameters
task_idstr | None- default:
None
Returns: InferenceTaskProfile
RealtimeSpec
clsclass RealtimeSpec(fps: int = 16,first_chunk_frames: int = 9,steady_chunk_frames: int = 9,controls: tuple[str, ...] = DEFAULT_REALTIME_CONTROLS,transport: str = 'in-memory-rgb',stateful: bool = True)worldfoundry.core.realtime.RealtimeSpecfrom worldfoundry.core.realtime import RealtimeSpecOverview
Model-owned playback and generation cadence for one resident session. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
fpsint- default:
16 first_chunk_framesint- default:
9 steady_chunk_framesint- default:
9 controlstuple[str, ...]- default:
DEFAULT_REALTIME_CONTROLS transportstr- default:
'in-memory-rgb' statefulbool- default:
True
Methods
Overview
Parse a model result without letting malformed metadata break play.
Parameters
valueAnyfallback'RealtimeSpec | None'- default:
None
Returns: 'RealtimeSpec'
def resolve_inference_device(device: str | torch.device | None = 'cuda',allow_cpu_fallback: bool = False) -> strworldfoundry.core.resolve_inference_devicefrom worldfoundry.core import resolve_inference_deviceOverview
Resolve a concrete inference device without silently selecting the wrong GPU. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: str.
Source docstring
Resolve a concrete inference device without silently selecting the wrong GPU.
Bare `cuda resolves to the process-local cuda:0. Explicit indices are preserved, which is important when a caller deliberately selects (for example) cuda:4` under an eight-GPU workspace.
Parameters
devicestr | torch.device | None- default:
'cuda' allow_cpu_fallbackbool- default:
False
Returns: str
def resolve_inference_dtype(device: str | torch.device,dtype: str | torch.dtype | None = 'auto',strict: bool = True) -> torch.dtypeworldfoundry.core.resolve_inference_dtypefrom worldfoundry.core import resolve_inference_dtypeOverview
Resolve an inference dtype using the selected accelerator's capability. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: torch.dtype.
Source docstring
Resolve an inference dtype using the selected accelerator's capability.
`auto` selects bf16 on Ampere/Hopper/Blackwell-or-newer CUDA devices, fp16 on older CUDA devices, and fp32 on CPU. Explicit unsupported bf16 is rejected in strict mode instead of producing a later kernel failure.
Parameters
devicestr | torch.devicedtypestr | torch.dtype | None- default:
'auto' strictbool- default:
True
Returns: torch.dtype
def run_torchrun_module(module: str,nproc_per_node: int,args: Sequence[str] = (),env: Mapping[str, str] | None = None,python_executable: str = sys.executable) -> subprocess.CompletedProcess[str]worldfoundry.core.run_torchrun_modulefrom worldfoundry.core import run_torchrun_moduleOverview
Run a single-node Python module under torchrun and capture its logs. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: subprocess.CompletedProcess[str].
Parameters
modulestrnproc_per_nodeintargsSequence[str]- default:
() envMapping[str, str] | None- default:
None python_executablestr- default:
sys.executable
Returns: subprocess.CompletedProcess[str]
def torchrun_module_command(module: str,nproc_per_node: int,args: Sequence[str] = (),python_executable: str = sys.executable) -> list[str]worldfoundry.core.torchrun_module_commandfrom worldfoundry.core import torchrun_module_commandOverview
Build a single-node torchrun command for a Python module. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: list[str].
Parameters
modulestrnproc_per_nodeintargsSequence[str]- default:
() python_executablestr- default:
sys.executable
Returns: list[str]
utc_now_iso
funcdef utc_now_iso() -> strworldfoundry.core.utc_now_isofrom worldfoundry.core import utc_now_isoOverview
Return the current UTC timestamp in ISO-8601 form. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: str.
Returns: str
def worldfoundry_inference_context() -> Iterator[None]worldfoundry.core.worldfoundry_inference_contextfrom worldfoundry.core import worldfoundry_inference_contextOverview
Run model inference under the shared WorldFoundry core runtime policy. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: Iterator[None].
Returns: Iterator[None]
class WorldFoundryInferenceInfraState(installed: bool = False,sdpa_patched: bool = False,attention_backend: str = 'auto',matmul_precision: str = 'high',tf32_enabled: bool = True)worldfoundry.core.WorldFoundryInferenceInfraStatefrom worldfoundry.core import WorldFoundryInferenceInfraStateOverview
Observable process-wide inference acceleration state. Belongs to Core inference runtime (process setup, compile, timers, realtime).
Attributes
installedbool- Whether core inference hooks were installed.default:
False sdpa_patchedbool- Whether the compatibility SDPA patch is active.default:
False attention_backendstr- Normalized attention backend policy.default:
'auto' matmul_precisionstr- Current float32 matmul precision setting.default:
'high' tf32_enabledbool- Whether CUDA TF32 matmul/cudnn execution is enabled.default:
True
def wrap_runner_for_worldfoundry_core(runner: Any) -> Anyworldfoundry.core.wrap_runner_for_worldfoundry_corefrom worldfoundry.core import wrap_runner_for_worldfoundry_coreOverview
Wrap a runner instance so `generate always uses core inference infra. Belongs to Core inference runtime (process setup, compile, timers, realtime). Annotated return type: Any`.
Parameters
runnerAny
Returns: Any