# Web export — resource channel

This document describes the **resource channel**: the out-of-band upload stream
that carries textures, meshes, and future resource kinds from the Python runtime
(in Pyodide) to the browser-side WebGPU renderer. It complements the per-frame
scene binary produced by `DrawSerializer` / `Scene3DSerializer`.

The wire format, kind registry, and Python/JS entry points in this document are
the contract between the two sides. Adding a new resource kind means claiming a
reserved byte — it does not require a version bump or a format negotiation.

## Why a separate channel

Frame binaries carry *per-frame state* — viewports, lights, materials, draw
groups, post-process parameters. They regenerate every tick.

Resources (texture pixels, mesh vertex / index buffers) are *upload-once, reuse
forever* lifecycle data. Embedding them in every frame binary (pre-M2) wasted
bandwidth, forced the JS side to re-detect "is this texture new?" on each
frame, and made the protocol noisier than it needed to be.

Separating the two means:

- Python hands the JS side a tiny TLV blob only when something new exists.
- JS knows every byte in the blob is a fresh upload and needs handler dispatch.
- The scene-frame header stays small (20 bytes + viewports + materials + lights
  + draw groups + optional post-process — no resources section).

## Wire format

Little-endian, no version field. Format changes happen in place and both sides
ship in the same HTML bundle, so mismatch is impossible.

```
count(u32)
  [ kind(u8) + id(u32) + length(u32) + payload(length bytes) ] × count
```

An empty drain is encoded as `b""` (zero bytes) — **not** as `count=0`. The JS
router treats any zero-length `ArrayBuffer` as a no-op.

### Kind registry

| Kind | Hex    | Meaning                    | Status      | Consumer     |
| ---- | ------ | -------------------------- | ----------- | ------------ |
| 0x01 | `KIND_OVERLAY_TEXTURE` | 2D overlay texture (Sprite2D / NinePatchRect / AnimatedSprite2D) | **used**    | `Renderer2D` |
| 0x02 | `KIND_MESH`            | 3D mesh (vertex + index buffers) | **used**    | `Renderer3D` |
| 0x03 | `KIND_MATERIAL`        | Material SSBO entry       | **reserved** | —            |
| 0x04 | `KIND_MSDF_ATLAS`      | MSDF font atlas           | **reserved** | —            |
| 0x05 | `KIND_MESH_TEXTURE`    | 3D PBR texture (albedo / normal / MR / emissive / AO) | **used**    | `Renderer3D` |
| 0x06 | `KIND_CUBEMAP`         | Environment cubemap (6 RGBA8 faces, +X -X +Y -Y +Z -Z) | **used**    | `Renderer3D` |
| 0x07 | `KIND_AUDIO_PCM`       | Decoded float32 interleaved PCM | **used**    | `AudioBridge` |
| 0x08 | `KIND_AUDIO_FILE`      | Raw audio file bytes for `AudioContext.decodeAudioData` | **used**    | `AudioBridge` |

**Why 0x03 and 0x04 are reserved but unused** — see *[What is not on the channel](#what-is-not-on-the-channel)*.

### Per-kind payloads

**`KIND_OVERLAY_TEXTURE` and `KIND_MESH_TEXTURE`**

```
width(u32) + height(u32) + rgba(width × height × 4 bytes)
```

Both kinds share the same payload layout; they only differ in which GPU cache
the handler populates. The texture is `rgba8unorm`, one mip level, no sampler
state (samplers are owned by the renderer).

**`KIND_MESH`**

```
vertex_count(u32) + index_count(u32)
  + vertex_bytes(vertex_count × 32)
  + index_bytes(index_count × 4)
```

Vertex layout matches `VERTEX_DTYPE` (position vec3 + normal vec3 + uv vec2 =
32 bytes). Indices are `uint32`.

**`KIND_AUDIO_PCM`**

```
sample_rate(u32) + channels(u8) + frame_count(u32)
  + pcm(frame_count × channels × 4 bytes)
```

PCM is float32 interleaved in the range `-1..+1`. Spaceinvaders-style
procedurally synthesised numpy buffers ship through this kind without a
file-decode round trip. JS calls `AudioContext.createBuffer(channels,
frame_count, sample_rate)` and writes deinterleaved channel data.

**`KIND_AUDIO_FILE`**

```
mime_hint(u8) + file_bytes(...)
```

`mime_hint`: `0` auto / `1` wav / `2` ogg / `3` mp3 / `4` flac. The hint is
advisory — browsers sniff the format from the bytes themselves. JS feeds the
payload to `AudioContext.decodeAudioData`, which produces the same
`AudioBuffer` as the PCM path. File-backed `AudioStream`s use this kind so the
Python runtime never has to ship a decoder.

## Python API

Helpers live in `simvx.web.runtime.resource_kinds`:

```python
from simvx.web.runtime.resource_kinds import (
    KIND_OVERLAY_TEXTURE, KIND_MESH, KIND_MESH_TEXTURE,
    pack_overlay_texture, pack_mesh_texture, pack_mesh,
    encode_tlv, decode_tlv,
)

# Build a single texture entry
entry = pack_overlay_texture(tex_id=7, w=16, h=16, pixels=rgba_array)
# entry is (kind, id, payload_bytes)

# Build a TLV blob from a queue of entries
blob = encode_tlv([entry, another_entry, ...])

# Round-trip / debug
decoded = decode_tlv(blob)  # [(kind, id, payload_bytes), ...]
```

### `WebApp.pending_resources()`

```python
payload = webapp.pending_resources()
# payload is b"" when nothing pending, else a TLV blob.
```

Drains the unified pending queue — both the 2D overlay textures gathered by
`prepare_2d_overlays` **and** the mesh / texture entries accumulated by
`WebRenderer` during `submit_scene`. Each `(kind, id)` pair is emitted
exactly once per app lifetime.

For Python-side tests / diagnostics:

```python
payload, summary = webapp.pending_resources(debug_log=True)
# summary: [(kind, id, payload_len), ...] — cheap to inspect, no payload copy.
```

## JavaScript API

Implemented in `packages/graphics/src/simvx/graphics/web/resource_router.js`
and consumed by both `Renderer2D` and `Renderer3D`.

```js
import { ResourceRouter } from './resource_router.js';

const router = new ResourceRouter();
router.label = 'Renderer2D';
router.register(ResourceRouter.KIND_OVERLAY_TEXTURE, (id, payload) => {
    // payload is Uint8Array viewing the TLV entry's payload bytes (zero-copy).
});

router.drain(arrayBuffer);  // returns number of entries delivered
```

`Renderer2D` registers `KIND_OVERLAY_TEXTURE`. `Renderer3D` registers `KIND_MESH`
and `KIND_MESH_TEXTURE`. `CombinedRenderer.drainResources(buffer)` hands the
same buffer to each sub-renderer; the router silently skips kinds with no
registered handler, so a combined 2D+3D export with a 3D texture and a 2D
overlay in the same drain blob needs no special routing logic — each sub-router
picks up what it cares about.

### Debug logging

Set `window.__simvxDebug = true` (or append `?debug` to the URL) to enable
debug logging at boot. This turns on:

- **Per-entry router logs**: `[Renderer3D] kind=MESH_TEXTURE id=5 len=4104 delivered`
- **Per-frame renderer stats** (throttled to once per second):
  `[Renderer3D] viewports=1 materials=12 lights=2 drawGroups=38 textures=7 meshes=4 bloom=on fog=mode=1 useLDR=false`

Flip at runtime from devtools:

```js
renderer.debug = true;           // renderer stats + router logs on both subs
renderer._router.debug = false;  // router logs only (Renderer2D standalone)
```

## What is not on the channel

Deliberately scoped out, even though it could technically fit:

- **MSDF atlas** — delivered via the dedicated `WebApp.load_atlas(...)` boot
  call, because JS needs the GPU texture *and* Python needs the glyph metrics
  for text measurement. Routing through the drain channel would require either
  a two-stage handoff or a virtual-FS round trip. The current path runs once at
  boot and is optimal as-is.
- **Materials** — per-frame state, not lifecycle data. Material SSBO content is
  rebuilt every tick; bundling with draw groups matches Three.js / Babylon /
  Bevy conventions. Lives in `Scene3DSerializer`'s per-frame binary.
- **Shaders** — compiled at export time and embedded as WGSL strings in the
  HTML. No runtime upload.
- **Streaming audio chunks** — `AudioStreamPlayer.stream_mode = "streaming"`
  feeds PCM in small (~50 ms) bursts to an `AudioWorkletNode`. The chunks
  are not lifecycle data and would balloon the TLV blob if shipped through
  this channel; instead `WebApp.drain_audio_feeds()` returns a packed
  binary feed payload alongside the resource drain (see
  [Audio](export.md#audio)).
- **Per-frame audio updates** — `(channel_id, gain, pan, pitch)` quads from
  `update_audio_2d/3d` ride a separate `Float32Array` channel via
  `WebApp.drain_audio_updates()`. They change every frame for every active
  voice, so they don't fit the upload-once shape of this channel.

## Adding a new kind

1. Reserve a byte in `resource_kinds.py` and in this document's kind table.
2. Write a `pack_XXX(id, ...) -> (kind, id, payload_bytes)` helper alongside the
   existing helpers so call sites stay declarative.
3. Push entries onto `WebApp._pending_resources` (2D / 2D-overlay path) or a
   renderer-local queue drained by `WebApp.tick` (3D path).
4. JS side: register a handler on the appropriate renderer's `ResourceRouter`.
5. Add a round-trip test to `test_resource_channel.py`.

That's it — no version bump, no format change.