Signal IR And Capability Runtime

Signal IR is the contract layer for turning any imported or generated file into typed signals before timeline, node graph, render, or export adapters decide how to use it.

Signal IR includes:

• src/signals/ defines SignalKind, SignalAsset, SignalRef, SignalArtifact, SignalGraph, SignalOperatorDescriptor, guards, normalization helpers, and mappings from legacy media/node graph types.
• src/runtime/capabilities/ defines fail-closed runtime capabilities such as file.read, artifact.write, network.fetch, gpu.compute, and timeline.mutate.
• src/extensions/ defines provider manifests and a registry for discovering importer/analyzer/operator/renderer/exporter providers by file signature, signal kind, runtime, or capability.
• src/importers/ defines the universal import orchestrator. CSV files become table/metadata/binary SignalAssets; JSON and JSONL files become metadata/binary SignalAssets; unsupported files become binary SignalAssets; and known legacy video/audio/image/model/vector paths remain on the established media pipeline.
• src/artifacts/ defines content-addressed SHA-256 artifact storage with project-local Cache/artifacts/... storage, IndexedDB manifest indexing, and an IndexedDB byte fallback for imports that happen outside an open project folder.
• src/runtime/worker/ defines the capability-ready worker job host/client protocol for long-running runtime providers.
• src/runtime/wasm/ defines the Wasm importer host adapter for direct and jco-style component exports.
• src/runtime/renderers/signalTimelineRendererAdapter.ts is the builtin timeline renderer dispatcher. It routes renderable SignalAssets to existing model or gaussian-splat clip paths when their artifacts can be materialized as files, and falls back to the text summary adapter otherwise.
• src/runtime/renderers/signalTextRendererAdapter.ts defines the fallback text renderer adapter. It materializes SignalAssets as timeline text clips that summarize the selected signal ref and metadata, so previews, nested compositions, save/load, and export reuse the existing text render path.
• wit/masterselects/runtime.wit defines the versioned Wasm Component ABI starter package for importer providers.

The universal importer is connected to the Media Panel import flow. Signal imports appear as signal project items, can be organized, renamed, labeled, deleted, saved, and loaded with the project. Signal assets are draggable to video tracks through builtin renderer adapters. Renderable mesh/geometry artifacts with OBJ/glTF/GLB payloads create real model clips. Renderable point-cloud/geometry artifacts with PLY/SPLAT/KSPLAT/SPZ/SOG/LCC/ZIP payloads create gaussian-splat clips. Everything else falls back to the text-summary adapter. The resulting timeline clip keeps signalAssetId, signalRefId, and signalRenderAdapterId metadata while rendering through established clip render paths.

SignalAsset represents an imported file, generated asset, operator result, timeline output, or node graph output.

SignalRef represents a typed output of an asset or operator. Its kind can describe concrete media (texture, audio, mesh, point-cloud, table, document, vector) or control/data signals (metadata, event, time, number, boolean, string).

SignalArtifact represents persisted content-addressed output. The artifact contract includes a hash, size, MIME type, encoding, storage location, producer, source references, and creation time.

The artifact store returns an ArtifactManifest, which extends SignalArtifact with store bookkeeping (blobId, manifestHash, retention, hashAlgorithm). Project state keeps only the SignalArtifact fields: imports pass stored manifests through toSignalArtifact() (src/artifacts/signalArtifact.ts). The project repository’s field ownership check rejects the extra fields, so before this every signal import into a stored project (STL, CSV, JSON, binary fallback) failed with Unclassified nested DTO field: ProjectSignalState/assets/0/artifacts/0/blobId.

SignalOperatorDescriptor describes importer, analyzer, operator, renderer-adapter, and exporter providers without binding those providers to the main thread.

The compatibility mappings keep current concepts bridgeable:

Existing type	Signal kinds
Video	texture, audio, metadata
Audio	audio, metadata
Image / Solid	texture, metadata
Text	text, texture, metadata
Model	mesh, geometry, metadata
Gaussian Splat / Avatar	point-cloud, geometry, metadata
Lottie / Rive	vector, texture, metadata
Composition	timeline, scene, metadata
CSV	table, metadata, binary
Unknown file	binary, metadata

Unknown files become valid binary SignalAssets. JSON and JSONL use the builtin structured-data importer; malformed JSON falls back to binary preservation.

Signal timeline placement is adapter-driven:

Adapter	Signal refs	Timeline source
masterselects.renderer.signal-model	mesh, geometry, scene, binary with .obj, .fbx, .gltf, or .glb artifacts	model
masterselects.renderer.signal-gaussian-splat	point-cloud, geometry, scene, binary with .ply, .splat, .ksplat, .spz, .sog, .lcc, or .zip artifacts	gaussian-splat
masterselects.renderer.signal-text-summary	fallback for all signal kinds	text

The file-based adapters require the artifact bytes to exist in the project artifact cache or IndexedDB. memory or external artifacts that cannot be resolved still produce a usable text-summary clip instead of failing the drop/import workflow.

Capabilities default to denial. A provider can only run a job if its manifest grants every requested capability. Unknown providers and unknown capabilities fail closed.

The worker runtime provides the off-main-thread job boundary for worker providers. The Wasm importer host adapts direct and jco-style component exports. The builtin host-side importers are CSV, JSON/JSONL, and binary fallback.