Visual Testing¶

SimVX provides headless Vulkan rendering and pixel-level assertions for automated visual testing. Tests run with a hidden window — no display required beyond a Vulkan driver.

Testing Layers¶

SimVX has three testing layers, each suited to different needs:

Layer	Module	GPU Required	What It Tests
Logic	`simvx.core.testing`	No	Game logic, node lifecycle, input actions
UI draw calls	`simvx.core.ui.testing`	No	Widget layout, draw commands, focus/input
Visual rendering	`simvx.graphics.testing`	Yes	Actual GPU output — shaders, z-order, colors

This page covers the visual rendering layer.

Quick Start¶

from simvx.core import Node, Camera3D, MeshInstance3D, Material, Mesh, Vec3
from simvx.graphics import App
from simvx.graphics.testing import assert_pixel, assert_not_blank

class RedCube(Node):
    def ready(self):
        cam = Camera3D(position=Vec3(0, 2, 5))
        cam.look_at(Vec3(0, 0, 0))
        self.add_child(cam)

        cube = MeshInstance3D()
        cube.mesh = Mesh.cube(size=2.0)
        cube.material = Material(color=(1, 0, 0, 1), unlit=True)
        self.add_child(cube)

app = App(width=320, height=240, visible=False)
frames = app.run_headless(RedCube(), frames=3, capture_frames=[2])
frame = frames[0]  # (240, 320, 4) uint8 RGBA

assert_not_blank(frame)
assert_pixel(frame, 160, 120, (238, 0, 0, 255), tolerance=20)  # center is red

Headless Rendering¶

`App.run_headless()`¶

Runs the full engine pipeline (physics, process, draw, render) for a fixed number of frames with a hidden window, returning captured framebuffer contents as numpy arrays.

App.run_headless(
    root_node,
    *,
    frames: int = 1,
    on_frame: Callable[[int], None] | None = None,
    capture_frames: list[int] | None = None,
) -> list[np.ndarray]

Parameters:

root_node — Scene root (Node subclass). ready() is called before the first frame.
frames — Total number of frames to simulate and render.
on_frame — Called with the frame index before each frame. Use this to inject input between frames via InputSimulator.
capture_frames — Which frame indices to capture (0-based). None captures every frame.

Returns a list of (height, width, 4) uint8 numpy arrays in RGBA order.

Each call creates a fresh Vulkan context and tears it down on completion — no state leaks between runs.

Hidden Window¶

App accepts visible=False to create an invisible window:

app = App(width=640, height=480, visible=False)

This uses GLFW_VISIBLE=FALSE under the hood. The window still has a real Vulkan surface and swapchain — rendering is identical to a visible window.

Frame Capture¶

Engine.capture_frame() copies the last-rendered swapchain image to CPU memory. It handles the full Vulkan pipeline: barrier transition, image-to-buffer copy, BGRA-to-RGBA swizzle. The returned array is a detached copy — safe to store, compare, or save.

Window Resize¶

For testing resize behavior:

app = App(width=320, height=240, visible=False)
# ... render some frames ...
app.set_window_size(640, 480)
# next frame will trigger swapchain recreation

Pixel Assertions¶

All functions in simvx.graphics.testing operate on (H, W, 4) uint8 RGBA numpy arrays. No external dependencies.

`assert_pixel(pixels, x, y, expected_rgba, tolerance=2)`¶

Check a single pixel. Fails if any channel differs by more than tolerance.

assert_pixel(frame, 160, 120, (255, 0, 0, 255), tolerance=10)

`assert_not_blank(pixels)`¶

Fails if every pixel is the same color. Catches “rendered nothing” bugs.

assert_not_blank(frame)

`assert_color_ratio(pixels, color, expected_ratio, tolerance=0.02, color_tolerance=10)`¶

Assert that approximately expected_ratio of all pixels match color.

# Red should cover ~25% of the screen
assert_color_ratio(frame, (255, 0, 0), 0.25, tolerance=0.05, color_tolerance=30)

`color_ratio(pixels, color, tolerance=10) -> float`¶

Returns the fraction of pixels matching color. Useful for flexible assertions:

ratio = color_ratio(frame, (255, 0, 0), tolerance=30)
assert ratio > 0.1, f"Not enough red: {ratio:.3f}"

`assert_region_color(pixels, rect, expected_color, tolerance=5)`¶

Assert all pixels within a rectangle (x, y, width, height) match a color.

# Top-left 100x100 should be blue
assert_region_color(frame, (0, 0, 100, 100), (0, 0, 255, 255), tolerance=15)

`assert_no_color(pixels, color, tolerance=5)`¶

Assert a color is absent from the image.

# Nothing should be bright green
assert_no_color(frame, (0, 255, 0), tolerance=10)

`save_image(path, pixels)` / `save_diff_image(path, actual, expected)`¶

Debug helpers that write PPM files (no Pillow required). save_diff_image amplifies differences 5x for visibility.

from simvx.graphics.testing import save_image, save_diff_image
save_image("/tmp/actual.ppm", frame)
save_diff_image("/tmp/diff.ppm", frame, expected_frame)

Testing Patterns¶

Z-order verification (no baseline needed)¶

Render overlapping objects with distinct colors and check which color wins at the overlap point:

class ZOrderScene(Node):
    def ready(self):
        cam = Camera3D(position=Vec3(0, 0, 5))
        cam.look_at(Vec3(0, 0, 0))
        self.add_child(cam)

        # Blue cube at z=0 (far)
        back = MeshInstance3D(position=Vec3(0, 0, 0))
        back.mesh = Mesh.cube(3.0)
        back.material = Material(color=(0, 0, 1, 1), unlit=True)
        self.add_child(back)

        # Red cube at z=1.5 (near)
        front = MeshInstance3D(position=Vec3(0, 0, 1.5))
        front.mesh = Mesh.cube(2.0)
        front.material = Material(color=(1, 0, 0, 1), unlit=True)
        self.add_child(front)

frames = app.run_headless(ZOrderScene(), frames=3, capture_frames=[2])
center = frames[0][120, 160]
assert center[0] > 150  # red wins at center
assert center[2] < 50   # not blue

Input injection¶

Use on_frame with InputSimulator to test interactive behavior:

from simvx.core.testing import InputSimulator
from simvx.core import Key

sim = InputSimulator()

def inject(frame_num):
    if frame_num == 2:
        sim.press_key(Key.SPACE)
    elif frame_num == 3:
        sim.release_key(Key.SPACE)

frames = app.run_headless(MyScene(), frames=5, on_frame=inject, capture_frames=[4])

Color coverage for render-order verification¶

When you don’t have a baseline, verify that each rendered element covers the expected proportion of the screen:

frames = app.run_headless(ThreeColorScene(), frames=3, capture_frames=[2])
frame = frames[0]

red = color_ratio(frame, (255, 0, 0), tolerance=80)
green = color_ratio(frame, (0, 255, 0), tolerance=80)
blue = color_ratio(frame, (0, 0, 255), tolerance=80)

assert red > 0.01, "Red cube missing"
assert green > 0.01, "Green cube missing"
assert blue > 0.01, "Blue cube missing"

Pytest Fixtures¶

The graphics test suite provides fixtures in packages/graphics/tests/conftest.py:

def test_my_scene(capture, px):
    frames = capture(MyScene(), frames=3, capture_frames=[2])
    frame = frames[0]
    px.not_blank(frame)
    px.pixel(frame, 160, 120, (255, 0, 0, 255), tolerance=20)

Fixture	Type	Description
`headless_app`	`App`	Pre-configured `App(320x240, visible=False)`. Skips if no Vulkan.
`capture`	`callable`	Shorthand for `headless_app.run_headless(...)`.
`px`	`PixelCheck`	Namespace with all assertion functions as methods.

The px fixture provides: px.pixel(), px.not_blank(), px.region(), px.ratio(), px.no_color(), px.get_ratio().

CI Setup¶

Visual tests require a Vulkan driver. For headless CI, use a software renderer:

Linux (GitHub Actions / Docker)¶

- name: Install software Vulkan
  run: sudo apt-get install -y mesa-vulkan-drivers xvfb

- name: Run visual tests
  run: xvfb-run pytest packages/graphics/tests/test_visual.py
  env:
    VK_ICD_FILENAMES: /usr/share/vulkan/icd.d/lvp_icd.x86_64.json

mesa-vulkan-drivers provides lavapipe, a CPU-based Vulkan 1.4 driver. xvfb-run provides a virtual X display for GLFW.

Arch Linux / Manjaro¶

sudo pacman -S vulkan-swrast
xvfb-run pytest packages/graphics/tests/test_visual.py

Running Tests¶

# All visual tests
cd packages/graphics && pytest tests/test_visual.py -v

# Specific test
pytest tests/test_visual.py::TestRedCube::test_center_is_red