One sim, seven platforms: porting with agents and tests

Seven platforms, one engine

Everdust is a music visualizer and playground where every scene is a live physics simulation. Today it runs in seven places. Twenty-one scenes play free in a browser at everdust.ai/play. The app is in public beta on iPhone, iPad, Mac, Apple TV and Vision Pro, and in alpha on Meta Quest. Android and PC VR builds are in progress, and I don’t count them here.

Under the hood there is one engine. Apple’s five platforms share a Swift and Metal engine; Metal is Apple’s interface to the GPU. Quest uses Vulkan, the cross-platform GPU interface, through a thin compatibility layer. The Quest engine’s shaders, the small programs that run on the GPU, are written in Slang, a shading language that compiles one source to both Metal and Vulkan’s format, SPIR-V. New ports use Slang too, so one source serves every platform.

Two lead agents split the work. Cosmo, on the Mac Studio, owns the app: the engine, the platform builds, the release. Beast, my Linux workstation with two large GPUs, prototypes new scenes on the web and renders films. The rule between them is “trial lots, port few.” I play the web prototypes, and only the ones I like get ported.

The rule that stopped us porting twice

Porting used to happen twice. Beast’s agents built a scene in JavaScript and WebGL, and then Cosmo’s agents rewrote it in Swift and Metal. On October 7 I asked for a change: “write in swift/metal so we don’t duplicate work when Cosmo ports it… be mindful of system memory usage, we OOM a lot when agents spawn to do test flows.”

The next day I asked whether the new logo studio was being built that way. It wasn’t; it was still a web page. That afternoon the default changed. All new interactive work is written in Slang and Swift, in the app engine’s own conventions, and the web version becomes a quick reference. Beast’s agents never write to the app’s repositories. They build in a sandbox on the Mac, deliver a package and a handoff note, and Cosmo’s agents merge it.

This is a product decision more than an engineering one. If the target format isn’t chosen before the prototype, every winning prototype costs two builds.

Case study: four nebulas in 81 minutes

On October 10 a helper agent finished four interactive nebula pages for the web: the Squid, Hourglass, Ring and Butterfly nebulas. You hold one at arm’s length, turn it, step inside it, scrub through how it formed, and blow on it with an open palm. At 8:27 pm I approved the first one. At 8:28 another agent started the native port.

It landed in four commits. The Squid and Hourglass came first, at 9:13, with the shaders, the Swift code, the test harness and a handoff note. The Ring and Butterfly followed at 9:41, together with two things the web version lacked: a scan for invalid numbers and a screen-brightness guard. A catch-up for the Hourglass and the final note with timing tables landed by 9:49. That is 81 minutes, including a long wait for the Mac’s shared simulator.

A few design choices made it fast and safe:

• One interface for all four nebulas. Adding a nebula means three one-line entries plus its own shapes.
• One argument table. A GPU program expects its buffers and textures in a fixed order. Every nebula kernel shares one table, and the build script checks it against Slang’s own reflection report on every build. An earlier port had been bitten by the compiler numbering those slots across programs.
• Tunables stay data. Each nebula reads the same settings file the web page reads, so a tweak needs no shader rebuild.
• A test harness ships with the code, including a headless Mac version that runs the real Metal path with no window or headset.

What the tests checked

Each nebula ran a scripted session on the Mac at two million particles: fade in, turn it and let it coast, resize it with two hands, two palm-wind pushes, switch colour filters, step inside and back out, then play its whole formation. That is about 2,000 frames per nebula.

The simulation itself is cheap. Drawing the particles as soft dots is the real cost, so each platform gets its own ceiling: three million particles on Vision Pro, one million on iPhone, six million on the Mac. After a 27-minute wait for the shared simulator lock, the Squid’s full autotest also passed inside the visionOS simulator. The simulator can’t report real GPU times, so every timing comes from the Mac.

What the web was hiding

Porting is where a prototype’s quiet assumptions surface. The web build of the nebulas raised numbers to fractional powers, and some of those numbers could go slightly negative. On the web that never caused visible trouble. Apple’s Metal returns NaN, “not a number”, which can blank part of the image. The fix went into both versions: every power call now takes a non-negative base, squares are written as multiplications, and the native test scans the particle buffer for invalid values at three key moments.

Precision was another. The web stores each particle’s light in half-precision numbers, which flush the faintest particles to zero at two million and above. The native version stores it at full precision. And the Butterfly needed a faster method on native, which first passed a low-resolution check while adding streaks. That story, and the brightness guard that turned a failed flash test into a feature, are in the companion note on gates.

The app team hits the same class of problem from the other side. One scene needed its own Quest version because Android’s Swift libraries lack a few maths functions that Apple’s provide. A shader check on the Quest side had silently passed for a while: a quirk in how a shell pipeline reported errors hid every hit until someone made it strict.

Sharing one Mac with a dozen agents

Linux can’t run Metal or the visionOS simulator, so every native build happens on the Mac, which Cosmo’s own eight to ten agents are already using. We agreed a protocol, and Beast’s agents follow it like everyone else:

Slot waits run five to eight minutes. On October 8 the logo port couldn’t use the simulator at all: another agent’s launch had hung for over 55 minutes and the simulator service was stuck for everyone. The fallback was a headless macOS test target built from the same sources. It gave real Metal numbers anyway: a million-star logo at 3.3 ms per frame for the whole stage, with every gate passing. The lesson stuck. Always keep one test path that doesn’t depend on the contended resource.

• Two build slots for the whole machine, claimed by creating a lock folder and released the moment a build ends. Beast builds with four parallel jobs.
• A separate lock for the simulator, always taken before a build slot, never while holding one.
• A memory gate: no build starts while the Mac’s committed memory is above 75 percent.
• Never shut a simulator down. Doing so on October 2 crashed the simulator service for every agent. Agents stop only their own app.
• A 15-minute cap per job, and kill only the processes you started.

The app team’s side

Cosmo’s agents build the app itself, and their habits look like this. In 21 days, 116 commit messages in the app repository mention a parity check: Quest shaders against Apple, the native rules engine against the server’s, web pages against native scenes. A native laser show matched its web reference at 97.2 to 97.9 percent, and every miss sat in one beat, which the agents flagged back to Beast instead of hiding. A viewable-zone audit, built from Apple’s spatial design guidance, checked 229 scenes for text and targets placed where people can’t comfortably look, and cut the failures from 70 to 41. On October 6 a Quest build was installed on my own headset.

This is why I describe the platform work as agents shipping native software with gates, not as “I build VR.” I set the product rules: which prototypes earn a port, the photosensitivity bar, eye-level placement, what ships. The agents do the porting, and the tests decide whether a port is real.

What isn’t proven yet

• None of the four native nebulas has run on a headset. The Vision Pro budgets will come from the engine’s governor, which adjusts particle counts at runtime.
• The Quest versions of these shaders pass the Vulkan validator but haven’t run on a Quest.
• The engine draws particles its own way, so expect one brightness trim after the first headset look.
• The web pages keep moving. Each native preset names the web commit it ports, and a sync script pulls settings from the repository’s committed state, not from someone’s working copy.