Node Catalog

Node Workspace

Codex Direct and Fast do not carry the node inventory in every conversation. The first prompt of a new thread only states how to fetch it: searchNodeCatalog({ list: true }) returns every addable registered operator, visual effect, Flock node, audio effect, parameter source, Color node and common field-backed clip stage as a plain-text list, one line each (exact catalog ID, input/output signal types, short purpose), grouped by owning context. Fixed graph anchors are marked; internal-only stages are omitted. Turns without node work therefore skip its ~8k tokens on every model step. Project-local custom definitions are outside this base inventory. (The kernel Fast V2 request keeps its JSON editorNodeCatalog contract.)

Two read-only tools provide more detail:

getNodeDefinitions returns compact definitions by default: ports as id:type (! required, * repeated) and parameters as id:type [min..max] {options} =default, with a legend field; detail: "full" returns the complete JSON contracts. The Direct instructions ask the agent to request every needed ID in one call, because each extra model step rereads the whole conversation.

• searchNodeCatalog({ query?, kind?, context?, inputType?, outputType?, offset?, limit? }) returns compact summaries, a total count and nextOffset. The default page has 12 entries, with at most 30. Exact IDs and names rank first; all whitespace-separated search terms must match.
• getNodeDefinitions({ ids }) reads up to eight exact IDs together, returning underlying type IDs, ports, parameter defaults, ranges, choices and declared animation support. Unknown IDs appear in missingIds.

Catalog IDs distinguish parameter-control versions from image operators with the same underlying ID. For example, control:values.number and values.number retain their own parameter contracts. Effect entries carry an effect: prefix (their typeId is the plain registry ID); addEffect, directly and inside a node stream, accepts either form. Built-in clip stages have source-dependent contracts; their empty static port/parameter lists do not imply that they have no editable controls. Fixed anchors and internal render primitives are marked explicitly. Signal filters are discovery aids, not proof that a connection is valid for a particular graph owner. Normal domain validation still applies when editing.

focusNodeGraph({ clipId }) selects an existing timeline clip and pins a Nodes panel to it beside Preview, so a separately docked AI Studio stays visible while the graph changes. The assignment survives selection changes. Existing panels pinned to other clips and detached windows remain untouched. Every panel can return to selection-following Active through its source selector. It requires the editor and a docked Preview. Clip creation remains a separate operation when no suitable clip exists.

These discovery/focus tools do not create or mutate graphs. Shaders, implementation functions and internal composition bodies are excluded from discovery results. Existing Codex Direct conversations start a fresh protocol thread after this catalog-tool update so the new tools are available.

Timeline content (tracks, clips, figures, keys, MIDI) streams through the sibling format ms-scene-v1; see Scene Streams.

Direct receives the nodeGraphStream version-1 contract alongside the inventory. An exact ms-nodegraph-v1 fenced block contains newline-delimited JSON. Its first record is {"op":"begin","schemaVersion":1,"clipId":"<existing clip ID>"}. Only after receiving that complete record does the browser select the clip and open Nodes beside Preview automatically; no separate focus call is necessary. If a Nodes panel is already visible elsewhere (the active tab of a docked group, a floating panel or a detached window), that panel is switched to the clip in place and no additional Nodes panel is opened or moved. A new clip must be created before the stream, using its returned ID.

The fenced text protocol is optional: Codex Direct can also call the atomic editor tools. Both paths yield a browser presentation frame after each edit, so consecutive connections, node moves, and other tool mutations become visible one step at a time even when the provider emits several tool calls together. executeBatch reports and presents each contained action before starting the next. A registered compound operator may still expand into many visual nodes in one atomic edit; its nodes and cables enter in a short staggered wave without splitting the saved edit or undo point. Prompt guidelines alone cannot guarantee that a provider chooses the text protocol.

Each subsequent tool record has a consecutive seq starting at 1, a unique ref (optional; defaults to s<seq>), an allowed tool and args. The browser supplies the pinned clipId. Earlier scalar tool results can be referenced with {"$ref":"alias","field":"effectId"} or nodeId. {"op":"end","lastSeq":N} and a closing fence terminate the stream. Partial lines never execute, and ordinary prose/code fences do not trigger graph changes.

Supported operations are visual-effect add/update/remove, effect keyframes (addKeyframe { effectId, param, keys: [{ time, value, easing? }] } animates one effect parameter or exposed <nodeId>_value as one undoable sequence; effectId may reference an effect created earlier in the block), Flock node add/update/remove/connect/disconnect, createImageNodeGraph and editOperatorGraph, using deterministic handlers, policy, audit and undo boundaries. Color and parameter-source graph authoring are not yet supported by this stream. The Fast/kernel route still uses its normal operation plans. Malformed or disallowed operation records and failed steps are skipped: later records still run, and each skipped step is reported to the model with its next tool result. A missing ref defaults to s<seq>, and seq/lastSeq written as numeric strings are accepted. Lost clip ownership or cancellation stops later work; completed steps remain visible and undoable. A reload does not replay streamed mutations. Limits are 128 operations, 64 KiB per line and 1 MiB per block.

createImageNodeGraph({ clipId, name? }) creates a neutral source-to-output graph using the existing image-effect runtime. editOperatorGraph adds/removes/moves registered operators, sets parameters, connects/disconnects ports and configures local numeric sliders. Each call is one atomic edit, also usable in node-code records. The same tool edits existing effect-owned graphs. Incomplete intermediate wiring is explicitly reported and execution pauses until repaired. Native domains retain their existing owner restrictions.

Node IDs start with a letter and use only letters, digits, _ and -. Dots are mapped to - in editOperatorGraph and getOperatorGraph arguments (for example key.split becomes key-split), consistently for adds and later references, and the result lists the mapping in renamedNodeIds. Invalid and duplicate IDs are rejected with separate messages. Compound nodes (registered compositions) accept a caller-chosen nodeId like any other node. They are expanded while editing, so connect resolves the compound ID, or the @compound-<id> handle an automatic add returns, together with a public port ID from getNodeDefinitions to the matching inner endpoints; an unknown public port is rejected with the available port list. Image-graph signal types stay strict (alpha is not number, rgb is not vec3). When a connection mismatches and exactly one parameterless conversion node exists, such as the identity convert.alpha-to-scalar or convert.scalar-to-alpha, connect inserts it between the ports in the same undo step and reports insertedConversion; otherwise the error names the available conversions. remove with a compound ID deletes the whole expanded compound (all members, nested groups and cables).

Inside a stream block, editOperatorGraph records may omit effectId; it defaults to the graph the block last created or named explicitly. When several same-typed outputs fit an input (for example value and weight), an omitted source port uses the first declared output. The agent builds graphs linearly in dataflow order: one add per node carries its wiring and values, and references only nodes that already exist. inputs is either { targetPort: "node" | "node.port" } or an ordered list of sources that fill the first free compatible inputs; params sets parameter values; min/max/step (with label) configure a Value slider. A source port may be omitted when the source has one output or exactly one output fits the target, and connect accepts the same inference (a missing toPortId takes the first free compatible input). Without position, a wired node is placed right of its sources. A composite add is one undo step and leaves the graph unchanged if any part fails; its result lists the connected cables and the new node’s openInputs. Separate connect records remain for rewiring, feedback and the final output cable.

Streamed records execute from text deltas, so the model does not receive their individual results. Failed stream steps are therefore attached, once, to the model’s next editor tool result, with the step number, target and error, and an instruction to inspect and repair the affected graph. If the turn ends with failed stream steps and an edited graph still paused as incomplete (or failures the model never saw), the chat answer gets a visible warning naming the incomplete effect and its reason. Steps rejected before execution (malformed records, missing or unknown effect arguments) also appear as failed calls in the chat history and activity log, and their error names the expected argument, e.g. addEffect needs effectType.

getOperatorGraph({ clipId }) lists effect graph owners and status. Add effectId for actual nodes, values and edges. Add nodeIds, hops: 0..4 and direction: upstream|downstream|both to inspect only a region. Boundary cables and omitted-node counts make the selection explicit. It reads saved graph state, not GPU telemetry, and remains available in plan mode. Graph results are compact for agents: expanded compounds are folded back into one node with their public ports (unless an inner node ID is requested), ports are id:type strings (! required, * repeated), and parameters are id:type [min..max] {options} strings; a legend field explains the format.

A Value node (values.number / values.integer) can be published to the clip’s Effects tab. Select the node and enable Effects tab in its inspector; an optional Exposed name labels the row. The effect then shows a keyframeable row per exposed node. Rows appear in a section named after the node group that directly contains the node, and that section carries the group’s bypass switch when the group has one. Ungrouped rows appear under Graph values. Each row’s keyframes drive the node output through the effect parameter <nodeId>_value (animatable property effect.<effectId>.<nodeId>_value). Turning the toggle off in an image graph writes the current base value back into the node as a literal and removes that parameter’s keyframes; the edit is undoable. Audio graphs and values already owned by a built-in effect parameter cannot be exposed.

The agent uses editOperatorGraph with action: "expose", nodeId, exposed: true|false and an optional label, or passes exposed: true (and optional label) when adding a Value node. slider on an exposed node sets the Effects tab row range. getOperatorGraph returns the node’s exposed field.

In the chat, a stream block is shown as one line ([Node-Stream: N Schritte], or [Node-Stream läuft: …] while it is still arriving); the stored message and copy keep the raw records. Streamed answer text is applied to the chat at most every 100 ms, so rendering a long answer does not delay the edits waiting for a presentation frame.

The chat Work Log records the received text-delta count, operations applied before turn/completed, and relative timestamps for the first operation and provider completion. The CodexNodeStream logger adds the first 16 chunk sizes/timestamps without prompt or answer contents. An operation preceding provider completion proves incremental execution; the visible Nodes panel is the rendering check.

Open Nodes → Catalog to search current definitions by name, ID or signal type. Search also matches signal format descriptions. Filter by supported context and expand an entry for inputs, outputs, default parameters, keyframe support, canonical family/variant, backend, fusion, state and invalidation. Where a registry exposes numeric ranges or signal formats, the same values flow into the catalog. Remaining local implementations are marked from their owning registry rather than maintained as a parallel migration checklist. The inventory reads the registries at runtime; it does not duplicate definitions. It is a reference; add operators from the selected node’s inspector or the appropriate domain’s Add menu.

The parameter-control menu includes Audio envelope (control.audio-envelope). It reads mixed RMS, momentary LUFS or short-term LUFS from existing analysis; it does not use live FFT or the playback AudioContext. The selected clip ID, time basis, interpolation and dB normalization are stored in the control graph. Slit Scan exposes Delay, Map mix and Noise amount as scalar targets. Use Remap to convert the normalized audio value into a desired delay range. Audio source and export integration have unit coverage; live end-to-end verification is pending.

Timing and rig controls in the same menu: Smooth Noise (control.noise), Envelope (control.envelope) and Marker Trigger (control.marker-trigger) under Time & Motion, Two-Bone IK (control.ik-two-bone, seven outputs) under Geometry. They also drive the clip Transform; see Node-Workspace.md.

Rig and motion controls: Ballistic (control.ballistic) and Gait Cycle (rig.gait-cycle) under Time & Motion, Limb IK (rig.limb-ik) and Attach to Joint (rig.attach) under Geometry. They drive the Stick Figure effect (effect:stick-figure, Generate) and clip transforms; see Node-Workspace.md. Contact Trigger (rig.contact-trigger, Time & Motion) times effects to a figure’s action contacts.

Image effects expose six reusable motion operators through Add node:

Optical Flow uses bounded low-resolution image passes. Source Motion uses shared source decoding and a separate bounded analysis atlas. Its DIS cached source pairs option uses real adjacent source PTS instead of the connected interval: an independent WebGPU fast DIS implementation with Gaussian pyramids, overlapping patch search, dense aggregation and forward/backward consistency. Cached fields are sampled at each output pixel’s source time. The original local estimator remains the default for existing consumers. Slit Scan enables DIS for its stretch mask and disables the red overlay during export. These estimates can fail at occlusions, weak texture or large motion; they do not reconstruct missing frames. Earlier local-estimator probes do not validate DIS. Build/test execution is paused; live DIS quality and performance have not yet been verified.

The catalog includes the 15 Fisheye processing groups, the shared coordinate compositions, the colour recipe blocks of the colour effects and the HSV-based Hue Shift composition of older saved graphs. To insert one, select an image effect or an internal node and use right-click → Reusable Nodes, or the inspector’s Add node selector. Each entry exposes its typed boundary and can be expanded after insertion.

The Sampling category adds Texel Offset, Gaussian Weight, and Normalize Weighted RGBA as versioned compositions of existing operators. Their boundary inputs are explicit: pixel offset/resolution, offset/sigma, and accumulated RGBA/total weight. They have no hidden bindings to an effect’s Radius or Samples controls. Gaussian weight requires nonzero sigma; normalization requires nonzero total weight. Kernel indices still need a surrounding reducer scope. These contracts are shared building blocks, not replacements for the existing kernel or sequence reducers.

Saved graphs keep packed composition instances and wire their public ports by name. When a definition’s ports change, OperatorCompositionBody.legacyInputs (extractImageComposition({ legacyInputs })) lists the earlier input ports and the inner endpoints they fed. Expansion routes such a port to exactly those endpoints and leaves them out of the current ports for that instance, so the saved wiring is reproduced instead of failing. Soft Bright Pass declares sample-energy-value (the alpha-weighted sample of the earlier alpha-aware Glow recipe, feeding bright-sample.a); Glow graphs saved in that period failed to load with “Unknown composition input: sample-energy-value” and now open and upgrade. A port that is neither current nor legacy still fails expansion; recipe recognition (matchesEffectRecipe) treats that as “not this recipe” instead of throwing, and a failed effect migration names the effect in the error.

Text Atlas renders its own typed characters (up to 256, in order) with a selectable font and weight into a cached glyph atlas. It needs no glyph effect or effect-owned ramp, so it can be added to every general image graph, including graphs created with createImageNodeGraph. Its outputs are the atlas image, glyph count, columns and rows.

Glyph Sample draws one glyph from any connected atlas: it takes the atlas and its columns/rows, a glyph Index (0 = first character, rounded to the nearest glyph) and a Local UV in 0–1 (top-left origin), and returns the glyph coverage. The atlas row is derived from the glyph centre, so computed integer indices stay exact on the GPU. Typical use: cell ID → index math → Glyph Sample → mix a text color over the background. Compute image effects (Voronoi, Pixel Sort, Quadtree Zoom, Contour) and Analog Signal Lab do not offer Text Atlas.

Further reusable processing recipes are available in the same menu:

These versioned definitions extract existing primitive regions rather than adding effect-specific GPU implementations. Exact structural recognition can share them across compatible effects without crossing saved group boundaries. Expanding a block exposes its ordinary nodes; editing its interior detaches that instance. Supplying an atlas or previous frame still requires a compatible resource-owning effect; inserting a processing block does not enable unsupported resources.

CRT Screen reuses Sine Gain twice, for its scanlines and flicker. Glitch can reuse the same Clamped Image Sample as CRT wherever its exact clamp/sample boundary matches. The recipe extractor can keep selected literal controls as public inputs, so curvature and mask levels are not hidden effect-specific bindings. The screen recognition revision does not reapply earlier detached recipes.

All graph editors use the shared node canvas and connection contract. Domain registries remain responsible for executable operators and parameter schemas; adapters map existing saved definitions to the common ports and endpoints. Shared validation checks signal semantics, supported formats, single versus repeated inputs, recorded dependencies and cycles before a connection is changed.

Math and vector split/combine families can declare adaptive ports. Connecting a different signal selects an executable variant for the current owner and, where necessary, propagates through connected adaptive nodes. Existing cables and fixed boundaries constrain the choice; incompatible edits are rejected atomically. No image/audio/geometry conversion is inferred. The Add selector shows one entry per adaptive family, while the catalog retains its concrete variants.

Audio Math Graph lifts the shared Add, Subtract, Multiply, Divide, Min, Max, Clamp, Abs, Sin, Cos, Floor, Fract and Mix operations over audio samples. Constants broadcast across the samples in each channel. Sample processing has its own audio executor; matching labels do not route image shaders into audio or mesh processing.

listNodeCatalog combines:

The catalog reports actual supported contexts. A common port name does not imply that every domain can execute every operator. Color grading, audio analysis and field-backed camera/light settings remain documented in their own feature pages; they are visible in the clip graph but are not yet registered in this inventory. Model/splat/Flock/voxel renderers have not been replaced by the image-surface executor. This inventory deliberately distinguishes what exists from what still needs a compatible executor adapter.

The matrix reports unspecified or unknown when an older registry definition has not yet declared fusion, state, unit, value format, consumer, or implementation-ownership metadata. It does not infer capabilities from labels. Port formats come directly from shared signal contracts; numeric ranges and steps come directly from parameter definitions. “Local” means an owning compiler or effect registry still has an implementation pending consolidation, while “shared” is only shown when its registry declares or references one. Existing effect entries expose their registered pipeline kind and feedback flag; their “local” marker means the effect registry still owns that implementation, not that a compatible operator migration has already been completed.

The image context includes three reusable definitions with explicit public ports: Cartesian to Polar (position and center to radius and angle), Mirror Repeat (value and period to a reflected value), and Polar to Cartesian (radius, angle and center to position). Angles use radians. Their bodies contain ordinary coordinate and math nodes and expand inline in the image compiler. Matching primitive patterns can adopt the same definitions across image effects. The numeric Value family exposes Float and Integer variants in the inspector; Integer truncates toward zero and keeps the number-port contract.

Image graphs offer two deterministic hashes of a Vector 2 key, both returning a number in [0, 1): Random (Hash) (noise.hash2d.vec2), the original sine hash that older effects keep, and Random (Integer Hash) (noise.hash2d-pcg.vec2), a lowbias32 bit mix plus PCG2D on the exact float bits. Use the integer hash for new graphs: the sine hash shows diagonal stripes on cell grids and loses quality on large keys, the integer hash stays uniform for any key (Glitch and Film Prism use it), and for the same key it gives bit-identical values on the GPU and in the CPU evaluator. Feed it whole numbers (cell indices, counters, integer salts): the GPU computes keys in 32-bit floats and the CPU in 64-bit, so only exact integers below 2^24 reach the hash as the same bits on both. Both are one Random node: its inspector’s Algorithm choice switches between Sine and Integer (PCG).

Random (Cell) (noise.cell-hash.vec2) is a separate node for one value per whole-number cell: it rounds its input down and hashes the cell with PCG2D, identical on every GPU and in the software renderer. Film Grain blends four of them into smooth value noise; Vignette uses two per pixel as its anti-banding dither. Use Random (Integer Hash) when the key itself is already the exact integer to hash, Random (Cell) when a coordinate should give one value per cell.

Fractal Noise (noise.fractal2d.vec2) is smooth gradient noise at a Position in noise cells (one cell is one blob), summed over Octaves (1-8; each doubles the frequency, turned by 37.5° so the octaves never line up) with Roughness (0-1) as each octave’s share. It returns 0-1 around a 0.5 mean. Its lattice hash is the integer PCG2D hash, so it stays clean at large positions, where Random (Hash) (a sine hash) turns into diagonal streaks; use it for paper, clouds and organic masks (Acuarela builds its paper, wet patches and blooms from it). GPU and CPU evaluation match within float rounding.

Cache Image (image.materialize) has a Resolution setting (Full, 1/2 … 1/1024): the cached pass renders at that fraction of the frame (at least one pixel) and later passes read it with normalized UV, so it acts as a smooth, cheaper copy. It is a fixed setting: it decides texture sizes before rendering, so it cannot be animated or bound to a parameter (the compiler says so). Glow builds its bloom pyramid from ten such levels.

src/services/operators/operatorTaxonomy.ts is the one presentation taxonomy for operator nodes; IDs never change for it. Menus are filtered by the owning graph first (what addableEffectOperators offers for an Image, 3D, Splat or Audio graph), then grouped by category: Inputs, Values & Time, Math, Logic & Switch, Vector & Convert, Color & Mask, Coordinates & Lens, Sampling & Filter, Time & Motion, Patterns & Fields, Text & Glyph, Analog Signal, Geometry, Splat Transform, Splat Attributes, Crop & Fade, Shading, Forces & Simulation, Tracking & Depth, Output & Render. Node cards show a short form of the same category (“Math”, “Color Group”).

• Visibility: menus list every addable node, including effect building parts (advanced: most Fisheye areas, kernel and sequence tools, Cache Image); internal ones (anchors, compiler resources) never appear. The Node catalog’s Technical details switch shows compiler facts such as family, execution and state.
• Families appear as one entry and switch their saved variant in the inspector: Value (Float/Integer), Switch (Value, Vector 2, Image), Luminance (Rec.709/Rec.601), sRGB to Linear/Linear to sRGB (exact sRGB transfer), RGB to OkLCh/OkLCh to RGB, Primitive (Box/Sphere/Cylinder), Derivative, Smallest Component, Temporal Smooth and the adaptive math and vector families.
• Names are English Title Case nouns without type suffixes or implementation terms: Divide instead of “Divide (IEEE)”, Split Vector instead of “Split VEC2”, Face Tracker instead of “MediaPipe Face Tracker”. The same concept has the same name in every graph (Value, Oscillator, Time, Minimum, Sine, Gravity, Drag, Turbulence).
• Domains in the catalog (context) are the graphs that actually offer a node. Controls, Color Grade, Clip, Clip effects, Audio effects and Flock complete the list.

Effects are grouped by look in src/effects/effectCatalogGroups.ts (Color & Tone, Blur & Sharpen, Light & Stylize, Lens & Distort, Pixel & Mosaic, Print & Halftone, Lines & Engraving, Textile & Craft, Text & Glyph, Analog & Glitch, Keying, Time, Tracking & Overlays, 3D & Particles); the registry category stays internal. Looks of one engine are styles of each other: the Effects tab shows a Style selector that switches in place and keeps the effect ID and the values and keyframes of the controls both looks have. A control only the new look has starts at its default; one only the old look has keeps its value for a switch back (a customized node graph restarts from the new look’s default). Rom1, the watery-smoke look, is its own effect again since Acuarela became a watercolour. Audio effects use six groups: Level & Channels, EQ & Filter, Dynamics, Repair, Time & Space, Character & Advanced.

Descriptions and search synonyms live in src/services/nodeGraph/catalogText.ts, keyed by catalog ID. Menus, the effect pickers, the Node catalog and searchNodeCatalog search them; the agent’s text inventory is grouped as ## <graph> › <category>. tests/unit/nodeTaxonomy.test.ts keeps every operator categorized, every effect grouped and described, and graph menus free of duplicate names.

Cache Image (image.materialize) stores an image expression in an internal rgba16float pass so later nodes can sample it. Max size (px) (default 0 = full size, at most 8192) renders the cache smaller, with its longer side at most that many pixels. It can be a constant or bound to an effect parameter; a bound value is read once per compile, and a change recompiles the plan. Use it for statistics, not for pictures: Chroma Key keeps its 64 px Screen Grid and its 1 px automatic screen colour this way. Later reads filter it like any image, and a resolution node inside that pass reports the reduced size.

Temporal Smooth (image.temporal-smooth for images, image.temporal-smooth.scalar for per-pixel values such as key masks) blends each frame with the node’s own previous result: mix(current, previous, amount). Amount (0-0.98, default 0.6) is the previous result’s share. Higher values calm flicker more but follow motion more slowly. A connected Amount input overrides the parameter, and the parameter can be bound and keyframed.

Typical use: compressed phone video changes its artifacts at every codec keyframe (often once per second), which makes a hard key mask jump. Place the value variant between the mask and its threshold.

The compiler lowers each node to ordinary mix and conversion operators plus a materialized rgba16float result. TemporalSmoothHistory in the effect pipeline keeps that result per node, effect and history scope. It follows the frame-history policy: advance on the next frame, hold on a re-render of the same frame, and restart from the current frame after seeks, loops, export start, graph edits or reverse steps. Cleared history is transparent and contributes nothing. Contexts without a history owner, such as other preview producers, pass the current frame through. Node previews read the committed history but never advance it.

source.face-landmarks and source.saved-depth are executable references to existing source artifacts. The canvas exposes them as Video Source outputs; connecting one creates a clip/effect-local reference inside the receiving graph. Landmarks retain normalized XYZ; saved scene depth is already calibrated and connects to geometry.depth, bypassing estimation and calibration. No payload is copied into the reference. Saved depth keeps its original calibration and must pass the bake’s source/timing/pose/mapping checks. It currently belongs to its owning cable effect; copying that effect keeps the reference local to the copied bake.

media.source → tracking.face → tracking.smooth → tracking.anchors → simulation.rope
                                     └───────→ geometry.face ─────────────────┐
media.source → depth.estimate → depth.calibrate → geometry.depth ──────────────┤
                               ↑ reference                                  ↓
                               └── geometry.face                 geometry.merge-surface
                                                                            ↓
                                                          collision.mesh / render.cables

The geometry functions have no clip/store/effect ownership. The bake adapter maps landmarks and depth into a common coordinate system. Native rendering and depth collision both consume the same merged exterior, including clipping at image edges. Primary geometry retains its exact tracked coordinates and UVs. This is a surface-specific stitch, displayed as Stitch Surfaces, not a general solid-mesh Boolean union or a simple concatenation of meshes. The stable saved operator ID geometry.merge-surface is unchanged.

portContracts.ts defines semantic types, supported intermediate representations and per-port constraints. The same contracts feed canvas labels/colors, hover and keyboard details, catalog entries, connection dropdowns and operator graph validation. For example, a relative depth grid and a calibrated depth grid both have type Depth but are not interchangeable at the depth-to-mesh input. Stitch Surfaces requires a primary face mesh with a closed UV outline and background depth geometry in the same coordinate space. Collapsed boundary ports preserve these restrictions.

Transform and smoothing belong to typed families that preserve their input signal. Current executable variants are UV transform (UV to UV), Clip Transform (scene to scene, applying the owning clip’s matrix/keyframes) and Smooth Landmarks (landmarks to landmarks, temporal motion-adaptive filtering). The landmark filter uses neighboring samples, preserves fast motion and avoids missing detections or timing gaps. These definitions do not yet provide arbitrary mesh transforms, selectable smoothing methods, mesh smoothing or image smoothing through one polymorphic operator. Such variants require their own supported data contracts and executable adapters; adding a catalog label alone is insufficient.

The cable executor currently supports one tracked face, one relative depth branch and a rope solver. Additional surface instances can reuse incoming values and be connected within those constraints. Inconsistent landmark sources or collision geometry are rejected instead of silently executing another graph.

image.frame provides the decoded image. texture.uv composes UV transforms; texture.image maps the frame to a texture. material.surface applies tint and opacity or a solid color; its Roughness, Metallic and Emission (color, strength, from texture) are used by the path tracer, where a plane with roughness or metallic becomes a lit surface instead of showing its texture unlit. geometry.plane creates a sized plane; geometry.source references the clip’s baked geometry. scene.mesh pairs geometry with material. scene.clip-transform applies the clip matrix and keyframes once; scene.render outputs the object using timeline camera/light references.

compileSceneGraph follows connected ports, independent of node order/layout. The resulting plan reaches both native plane and Face Cables shaders. Definitions are stored under clip.nodeGraph.scene; Face Cables definitions and groups remain in effect.params.operatorGraph. Only definitions and portable bake data are saved, never GPU/runtime handles.

The Weave effect owns an operator graph with domain geometry. Curves flow between general curve nodes; number and vector inputs of a modifier are evaluated once per curve point, exactly as image math is evaluated once per pixel. Both executors lower the same registered Math, Vector and Convert operators through the shared pointwise table (src/services/operators/fields/), so a Multiply or Sine node is one node in every graph rather than a per-domain copy.

The default Weave graph is Weave Pattern → Set Position (Handmade) → Thread Along → Yarn Profile → Flyaways → Surface Bind → Strand Render: a plain weave of fuzzy three-ply yarns with handmade irregularity whose threads are pulled in over the first four seconds of the clip and then billow in the wind (see Weave). Weave Pattern is the only weave-specific node; crimp is analytic (cosine transitions between crossings, with the draft deciding which thread lies in front). Yarn Profile is general: plies circle the curve and fibers circle each ply, with both angles driven by arc length along rotation-minimizing frames, so radius changes never spin the twist (the Houdini sweep issue). Its Radius Scale input is a per-point field: a scale of zero also thins the fibers to nothing, values above one swell them.

Flyaways. Flyaways add stray fibers at render time, with no extra geometry. Along each yarn, every curve cell holds one hashed window per flyaway channel. Inside that window one fiber leaves the outer fiber ring and rises Lift yarn radii above it. A loop returns to the yarn; a Free Ends share stops at the peak. Density counts flyaways per unit of curve length. Flyaways follow the Radius Scale field, so a reveal hides and grows them with the yarn. They need a Yarn Profile to leave from.

Reveal. The default graph grows the sheet from its center: Shape Distance (sphere, cube or plane; an unconnected Position reads the curve point) plus Noise (fractal lattice noise, the same deterministic noise Flock uses) feed a three-key Ramp whose front key swells the yarns before they settle. The group Reveal by Shape exposes Reveal (0–1) in the Effects tab; keyframe it to animate the growth.

Weave In. Thread Along pulls every thread in along its own path. Progress moves a tip from the start to four lift lengths past the end; behind the tip the thread rises Lift along +Z (the rest sheet’s normal) and settles as lift · e^(−Settle·x) · cos(πx/2), with x in lift lengths behind the tip, so the swing has died down when the thread is complete. Ahead of the tip the thread is hidden through its radius scale (Yarn Profile multiplies an incoming radius scale). With Ahead: Trail it stays visible instead and runs straight from the lifted tip in the Trail direction, keeping its length, as if pulled in from a spool; trailing threads are evaluated on the CPU. Stagger spreads the threads’ starts over the progress range in curve order: warps first, then the wefts row by row. Progress is Clip Time × Weave Speed (exposed, 1 = four seconds), capped at 1 by Min, wired into Thread Along’s Progress input. Bypass the group for a finished weave from the first frame.

Handmade. General nodes only: a Noise of (Curve Param × 3, strand key) moves each thread across itself, so spacing varies per thread; a Noise of the strand key scales its crimp (tension); and a Ramp of a third Noise adds slubs to the radius scale. Irregularity (exposed, 1 = default look, 0 = machine-perfect) scales all three.

Knots. Knot and Celtic Knot are general curve generators. The reef knot is an alternating six-crossing diagram of two ropes with both ends of each rope on one side. The Celtic plait traces 45° billiard paths over a lattice of 2·Columns × 2·Rows half cells; crossings sit on interior points with an odd coordinate sum, and like a plain weave the rising thread lies on top at even x. Over and under therefore alternate along every thread, across border loops too. Knit follows the stockinette yarn path x = (t + Lean·sin 2t)·Width/2π, y = Height·cos t, z = Depth·cos 2t per stitch: below twice the loop height of row spacing each head lies in front of the legs of the row above and behind its sinker loops. Extend adds Points per End straight points beyond each end, along the second-order end tangent 3·p0 − 4·p1 + p2, so a curve that ends on a turning point (a knit loop head) continues level instead of along its tilted last segment.

Add and Multiply read 0 and 1 through an unconnected operand in curve graphs. A bypassed field group that feeds one of them therefore leaves the other factor unchanged. Scalar operations on constants are folded while lowering, and unused instructions are dropped. A clock that has settled, such as the capped weave progress, therefore compiles to the same program every frame, and the time-independent curves stay cached.

Wind Cloth. Cloth Sheet simulates a regular grid with XPBD: a fixed number of substeps per step, one pass per substep, stretch/shear/bend links and double precision. Stretch and shear links barely resist compression, so the fabric buckles instead of chattering. Forces are the shared Wind, Gravity, Turbulence and Drag nodes that cables and particles use. Wind acts as air speed across the sheet with strong air drag, so the cloth follows gusts smoothly. The cloth reads the parameter values of these nodes; wiring into their inputs is rejected rather than ignored. Turn (degrees) turns the finished grid about its upright axis after the simulation, so the motion is the same at every angle; Weave’s Woven Cloth uses 32° so wind blowing at the camera shows as a sideways bow. Graphs saved before Turn existed read 0 and stay head-on. The clock is the source time of the host clip, as with Flock: split and trimmed clips continue the motion instead of restarting it. Speed keyframes are not followed yet. Time runs at 60 fixed steps per second, and Pre-roll starts the simulation before the clip. Exact double-precision states are checkpointed every half second. Scrubbing therefore resumes from the nearest checkpoint, and every path to a frame gives bit-identical positions on the same device. After ten minutes of source time the sheet holds still. Surface Bind maps the flat rest sheet onto the simulated grid with bicubic (Catmull-Rom) interpolation and extrapolates past the edges. Curve height becomes offset along the cloth normal, so crimp follows the fabric. Curves before the first Surface Bind are cached per stage, and per-point field expressions whose inputs did not change are reused, so a new frame only advances the cloth. When Surface Bind is the last stage, the renderer binds the cached rest curves on the GPU and builds the rotation-minimizing frames there; a Thread Along with one progress for all points and the Yarn Profiles right before the bind run there too (radius fields compiled to WGSL, constants passed as data). The CPU reference (threadAlong, bindToCloth, packStrandPoints) serves previews and other chains. With the default 40 × 27 grid and 6 substeps, one second of simulation costs about 70 ms of CPU; an animated frame of the default weave costs about 5 ms of CPU. The Cloth Sheet’s preview shows the simulated grid. The default Wind Cloth group holds the sheet at its corners like a sail. Bypassing it leaves the weave flat.

Rod Simulation. Any curves become rods of a given Radius, simulated with XPBD in small steps on the same clock, step rate and checkpoints as Cloth Sheet. Stretch links run along each segment; bending is the vector difference of neighbouring edge directions (rest straight, compliance scaled by length, so the stiffness does not depend on resolution). With a straight rest shape and position-only pins, twist does not move the centre line, so the rods carry no frames. Segments collide as capsules against each other and against non-neighbouring parts of their own rod; candidates come from a hashed grid sorted by counting sort and kept as a Verlet list until a node has moved a quarter radius, and positional friction removes the sliding of each substep up to Friction × penetration. Node speed is capped at half a radius per substep, so rods do not tunnel. Pins hold curve starts, both ends or the points of a Pin field above 0.5; Pull moves them outward along their end tangent over Pull Time from each node’s Pull Start (interpolated from a per-point field, default 0), which tightens knots or, with Extend tails and row-wise start times, unravels a fabric row by row. Start: Straight lays every open rod along its chord through its centroid with its own segment lengths; from its Form Time (a per-point field, seconds of source time, interpolated along the rod) each node is moved toward its place in the input during the prediction, eased in over Form Ease (smoothstep) and capped at a quarter radius per substep, so the contacts and constraints that follow still separate the threads while they fold. The input curves are the rest state (rings stay closed) and the output keeps every input point: Segment Length (0 = one radius) only sets the rod resolution, and finer detail rides along. Stretch and bend constraints are solved colour by colour and contacts as one averaged Jacobi pass; followed only by Yarn Profiles, the renderer runs this scheme on the GPU (RodGpuSimulation, f32, deterministic per device) and the CPU solver serves previews. A Rod Simulation must come before Surface Bind.

Groups. Geometry groups can be bypassed from their Byp header button or their Effects tab section. A curves output passes the group’s incoming curves through: bypassing Yarn draws the bare woven curves. A field output (number or vector) without a unique incoming source of its type is disconnected, and its consumer falls back to its default. Bypassing Reveal by Shape therefore leaves the Radius Scale at 1 and shows the whole sheet. Bypass changes only the compiled view: the saved wiring, values and keyframes stay intact. These field nodes are general per-element operators; Time reads the composition clock, as in image graphs. An example graph of general nodes only (createWaveStrandsGraph) builds an alternating wave from Value, Multiply, Sine, Fraction and Add nodes. Curve ports show a CPU wireframe preview (curve and point counts). Value nodes use literals and can be exposed to the Effects tab like image Value nodes.

Rendering. Effects → 3D & Particles → Weave works on any clip; the strands are drawn as an extra 3D layer above the clip with the clip’s transform, in preview, export and nested compositions, on the main thread or in the Worker render host. The layer carries the geometry program sampled at the frame time (plain data); the renderer evaluates the curves on its own thread, caches them until the program changes and expands every segment into a camera-facing ribbon in the vertex shader. Width is in world units and follows the layer scale. A strand narrower than one pixel keeps one pixel of geometry with a deterministic hashed coverage instead of blending, so dense strands need no sorting and export reproduces the preview; width 0 draws nothing. Visibility is decided per segment, so a thread that ends inside a segment tapers to nothing instead of stretching a sliver across the frame. Curve graphs are authored Y-up: +Y is the top edge and the opposite of gravity, and +Z faces the viewer, the same axes as the shared 3D scene.

Antialiasing. Strand Render offers Hashed (the default, including older graphs) and 4x Coverage. The latter draws every fiber without stochastic fiber thinning or pixel rejection, using four-sample alpha-to-coverage in a strand-only color/depth target. The target starts with the existing scene depth, so opaque meshes and planes in front still hide the strands. Covered samples are resolved once as premultiplied color, preserving layer opacity. Preview, export, nested compositions and both render hosts use the selected mode. This costs additional GPU memory (about 32 bytes per viewport pixel) and draws more fibers. Coverage has only four samples: extremely thin fibers can still lose coverage, and sample masks are device-dependent. Later scene passes use single-sample depth; the resolve writes the nearest strand depth when at least two samples are covered, otherwise retaining the incoming scene depth. Analytic rasterizes the strands with a tile compute pass: each fiber piece (segment × spline piece × fiber) is projected, counted per 16 × 16 pixel tile and written to fixed slots after a prefix sum; a stable radix sort orders the (tile, depth) keys, and one workgroup per tile blends its pieces front to back with exact pixel coverage (box-filtered across and along each piece, so neighbouring pieces of one fiber add up at their joints) until 1 % transmittance remains. Shading uses the same function as the fragment shader. The result is deterministic on a device and composited into the scene color and depth; a layer exceeding the device’s buffer limits falls back to 4x Coverage.

Shading. Every fiber is lit as a round tube.

• Wide fibers (a few pixels or more, as in close-ups): wrapped Lambert diffuse on a cylinder normal reconstructed across the ribbon. Thin fibers fall back to Kajiya-Kay.
• Highlights: two shifted highlights after Marschner/Karis, a white R and a TRT in the fiber color, plus forward scattering when a light is behind the fiber.
• Light clips light the strands like native meshes. Point and panel lights use the same falloff and panel direction, up to four direct lights; environment lights add ambient color. Without a light clip a fixed upper-left key light applies.
• Smooth curves: in close-ups each segment is split into up to eight Catmull-Rom pieces, depending on how many pixels it spans at the nearest point of the layer. Curves stay round without resampling them, and distant views keep one piece per segment.
• Level of detail: where a segment’s fibers are thinner than a pixel, only a hashed share of them is drawn, each with proportionally more coverage (stochastic simplification, Cook et al. 2007). Distant yarns keep their density with fewer fragments. The choice is made per segment and strand, so zooming lets fibers fade in and out one by one.
• Self-shadowing: fibers shadow each other through deep opacity maps. From the shadowing light, a depth pass records the nearest fiber per texel. An opacity pass then adds every fiber’s coverage into four layers behind it (1024², additive, deterministic). The main pass reads the opacity in front of each fragment, with the nearest depth filtered bilinearly so thin fibers do not band.
  • Without light clips the key light casts an orthographic shadow at strength 0.8.
  • With light clips, the point or panel light brightest at the layer casts, if its clip has Casts Shadows on (at its Shadow Strength).
• Shadow exchange with meshes: strand shadow maps are rendered before the opaque passes. With a shadow-casting scene light, lit meshes receive the strands’ deep opacity shadow, and opaque meshes drawn from the same light into an occluder depth shadow the strands (2 × 2 percentage-closer filter). One strand layer passes its shadow to meshes; unlit planes do not receive shadows, and the key light without light clips does not exchange shadows.
• Not yet supported: a dedicated generator clip.

The cloth simulation runs on the CPU of the rendering thread (the Worker render host in the default mode); a long jump into an unsimulated range blocks that thread while it catches up.

1. Search the live catalog and these registries before adding a new operator.
2. Reuse the existing pure implementation or extend its contract with explicit compatible defaults. Add a domain adapter only where execution is supported.
3. Define typed ports, parameter defaults, keyframe support and invalidation in the registry. Add controls through shared inspector primitives.
4. Validate graph structure and executor constraints before committing a mutation. Keep history, persistence, preview and export on the same definition.
5. Test a real output change, project compatibility and relevant UI connections.

Legacy surface.hybrid, collision.face and collision.surface IDs are retained only for migration. They are expanded/replaced on read, hidden from the catalog, and saved in the new form on the next graph mutation or successful bake. Migration does not replace a saved depth artifact or run MediaPipe again.