Web export — resource channel

This document describes the resource channel: the out-of-band upload stream that carries textures, meshes, and future resource kinds from the Python runtime (in Pyodide) to the browser-side WebGPU renderer. It complements the per-frame scene binary produced by DrawSerializer / Scene3DSerializer.

Status (April 2026): frozen for M5. The wire format, kind registry, and Python/JS entry points in this document are the contract between the two sides. Adding a new resource kind means claiming a reserved byte — it does not require a version bump or a format negotiation.

Why a separate channel

Frame binaries carry per-frame state — viewports, lights, materials, draw groups, post-process parameters. They regenerate every tick.

Resources (texture pixels, mesh vertex / index buffers) are upload-once, reuse forever lifecycle data. Embedding them in every frame binary (pre-M2) wasted bandwidth, forced the JS side to re-detect “is this texture new?” on each frame, and made the protocol noisier than it needed to be.

Separating the two means:

  • Python hands the JS side a tiny TLV blob only when something new exists.

  • JS knows every byte in the blob is a fresh upload and needs handler dispatch.

  • The scene-frame header stays small (20 bytes + viewports + materials + lights

    • draw groups + optional post-process — no resources section).

Wire format

Little-endian, no version field. Format changes happen in place and both sides ship in the same HTML bundle, so mismatch is impossible.

count(u32)
  [ kind(u8) + id(u32) + length(u32) + payload(length bytes) ] × count

An empty drain is encoded as b"" (zero bytes) — not as count=0. The JS router treats any zero-length ArrayBuffer as a no-op.

Kind registry

Kind

Hex

Meaning

Status

Consumer

0x01

KIND_OVERLAY_TEXTURE

2D overlay texture (Sprite2D / NinePatchRect / AnimatedSprite2D)

used

Renderer2D

0x02

KIND_MESH

3D mesh (vertex + index buffers)

used

Renderer3D

0x03

KIND_MATERIAL

Material SSBO entry

reserved

—

0x04

KIND_MSDF_ATLAS

MSDF font atlas

reserved

—

0x05

KIND_MESH_TEXTURE

3D PBR texture (albedo / normal / MR / emissive / AO)

used

Renderer3D

Why 0x03 and 0x04 are reserved but unused — see What’s not on the channel.

Per-kind payloads

KIND_OVERLAY_TEXTURE and KIND_MESH_TEXTURE

width(u32) + height(u32) + rgba(width × height × 4 bytes)

Both kinds share the same payload layout; they only differ in which GPU cache the handler populates. The texture is rgba8unorm, one mip level, no sampler state (samplers are owned by the renderer).

KIND_MESH

vertex_count(u32) + index_count(u32)
  + vertex_bytes(vertex_count × 32)
  + index_bytes(index_count × 4)

Vertex layout matches VERTEX_DTYPE (position vec3 + normal vec3 + uv vec2 = 32 bytes). Indices are uint32.

Python API

Helpers live in simvx.graphics.web.resource_kinds:

from simvx.graphics.web.resource_kinds import (
    KIND_OVERLAY_TEXTURE, KIND_MESH, KIND_MESH_TEXTURE,
    pack_overlay_texture, pack_mesh_texture, pack_mesh,
    encode_tlv, decode_tlv,
)

# Build a single texture entry
entry = pack_overlay_texture(tex_id=7, w=16, h=16, pixels=rgba_array)
# entry is (kind, id, payload_bytes)

# Build a TLV blob from a queue of entries
blob = encode_tlv([entry, another_entry, ...])

# Round-trip / debug
decoded = decode_tlv(blob)  # [(kind, id, payload_bytes), ...]

WebApp.drain_resources()

payload = webapp.drain_resources()
# payload is b"" when nothing pending, else a TLV blob.

Drains the unified pending queue — both the 2D overlay textures gathered by prepare_2d_overlays and the mesh / texture entries accumulated by WebRenderer3D during submit_scene. Each (kind, id) pair is emitted exactly once per app lifetime.

For Python-side tests / diagnostics:

payload, summary = webapp.drain_resources(debug_log=True)
# summary: [(kind, id, payload_len), ...] — cheap to inspect, no payload copy.

JavaScript API

Implemented in packages/graphics/src/simvx/graphics/web/resource_router.js and consumed by both Renderer2D and Renderer3D.

import { ResourceRouter } from './resource_router.js';

const router = new ResourceRouter();
router.label = 'Renderer2D';
router.register(ResourceRouter.KIND_OVERLAY_TEXTURE, (id, payload) => {
    // payload is Uint8Array viewing the TLV entry's payload bytes (zero-copy).
});

router.drain(arrayBuffer);  // returns number of entries delivered

Renderer2D registers KIND_OVERLAY_TEXTURE. Renderer3D registers KIND_MESH and KIND_MESH_TEXTURE. CombinedRenderer.drainResources(buffer) hands the same buffer to each sub-renderer; the router silently skips kinds with no registered handler, so a combined 2D+3D export with a 3D texture and a 2D overlay in the same drain blob needs no special routing logic — each sub-router picks up what it cares about.

Debug logging

Set window.__simvxDebug = true (or append ?debug to the URL) to enable debug logging at boot. This turns on:

  • Per-entry router logs: [Renderer3D] kind=MESH_TEXTURE id=5 len=4104 delivered

  • Per-frame renderer stats (throttled to once per second): [Renderer3D] viewports=1 materials=12 lights=2 drawGroups=38 textures=7 meshes=4 bloom=on fog=mode=1 useLDR=false

Flip at runtime from devtools:

renderer.debug = true;           // renderer stats + router logs on both subs
renderer._router.debug = false;  // router logs only (Renderer2D standalone)

What’s not on the channel

Deliberately scoped out, even though it could technically fit:

  • MSDF atlas — delivered via the dedicated WebApp.load_atlas(...) boot call, because JS needs the GPU texture and Python needs the glyph metrics for text measurement. Routing through the drain channel would require either a two-stage handoff or a virtual-FS round trip. The current path runs once at boot and is optimal as-is.

  • Materials — per-frame state, not lifecycle data. Material SSBO content is rebuilt every tick; bundling with draw groups matches Three.js / Babylon / Bevy conventions. Lives in Scene3DSerializer’s per-frame binary.

  • Shaders — compiled at export time and embedded as WGSL strings in the HTML. No runtime upload.

  • Audio — when web audio support lands it will need its own channel (streaming playback ≠ one-shot asset upload).

Adding a new kind

  1. Reserve a byte in resource_kinds.py and in this document’s kind table.

  2. Write a pack_XXX(id, ...) -> (kind, id, payload_bytes) helper alongside the existing helpers so call sites stay declarative.

  3. Push entries onto WebApp._pending_resources (2D / 2D-overlay path) or a renderer-local queue drained by WebApp.tick (3D path).

  4. JS side: register a handler on the appropriate renderer’s ResourceRouter.

  5. Add a round-trip test to test_resource_channel.py.

That’s it — no version bump, no format change.