hraness

web audio graphs that stay deterministic

two clocks, one graph, no surprises

Drafted by an AI agent at Ben Guo's direct request from the Hraness source repositories, and checked against those sources before publication.

Web Audio gives you a hard real-time problem wrapped in a friendly API. The graph runs on a thread you do not control, in quanta of roughly three milliseconds, and any work that misses its quantum is not a dropped frame; it is a click, a gap, or a note that never plays. Meanwhile the browser’s other clocks (the frame clock, the timer queue, performance.now()) run on a different schedule and can be throttled, stalled, or absent entirely depending on which thread you ask from.

Soundfish’s answer is to treat the audio engine as a data problem with two clocks and one immutable program. The document (bars, tracks, notes, automation) is deterministic data. The audio clock decides when sound happens. The scheduling clock decides when the queue gets refilled. The UI renders whenever the frame clock lets it. Determinism comes from keeping those concerns in separate lanes, and this lesson walks through each lane.

two clocks, one program

The first discipline is acknowledging that there are two clocks and they disagree. AudioContext.currentTime advances with the audio hardware and is the only clock that decides when a sample is audible. Everything else (the scheduler pass, the React tree, the playhead you draw) runs on the main thread’s sense of time, which pauses when a background tab is throttled and stalls when a long commit or a garbage collection runs.

So the score and its playback plan live as pure data. transport-position.ts is the pure transport model: anchorForStep is the single formula behind play-from-step, seeks, and live retimes, and rebaseAnchorForTempo rebases the anchor when the BPM changes instead of restarting playback. Scheduling converts document steps into currentTime offsets; rendering reads the transport back for display. Neither side owns the other, which is why a tempo edit mid-playback is a math problem rather than a teardown.

the render thread is not yours

The AudioWorklet quantum is the budget that governs everything. If the processor’s process call does work beyond rendering (decoding a sample, say), that work happens inside a quantum and a quantum lasts about three milliseconds. Soundfish’s own measurement says a lazy in-worklet decode can cost about twenty milliseconds in one quantum with fifteen other channels playing, which is several quanta of dropout for one note.

So the rule is: never decode a sample on the audio render thread for a program a consumer announced. primeProgram decodes the exact (pitch, velocity) pairs a channel will voice, inside a dedicated decode Worker, transfers the PCM to the worklet through a patched port command, and resolves only on the worklet’s acknowledgement. The consumer gates its warm-and-resume on that promise, bounded by its own deadline. Priming is per-pair rather than per-preset because the Grand Piano spans 84 samples, roughly 130 MB decoded, across velocity layers a track may never touch.

The decode Worker mirrors the worklet’s bank stack from the same verified bytes and drops its copy once the PCM is transferred. That mirroring is only safe because decoding is itself pinned: spessasynth_lib 4.3.11 and spessasynth_core 4.3.15 move together, and decode-equivalence.test.ts loads the patched worklet bundle and proves its decoded PCM is bit-identical to the Worker’s for base, kit, and override samples. The test must pass before either pin moves, which is what “deterministic” costs in practice: you do not get to upgrade a decoder casually.

banks are data

A sound bank in Soundfish is not ambient synth state; it is layered data with a declared order. SoundBankManager.priorityOrder is [kits…, overrides…, base]: PCM drum kits at bank 128, melodic SF3 overrides at bank 0, and the base bank underneath both. A kit and an override never shadow each other because they live on different bank numbers, and an override is never a drum preset.

Layers are fetched only for the programs a document uses. Only the base bank is required at creation; loadDrumKits and loadOverrides pull the optional layers by name, and preloadGeneralMidiSoundfont({ drumKits, overrides }) warms them without creating an AudioContext at all. Every blob is integrity-verified before it reaches the graph, and browser Cache Storage entries are keyed by content SHA-256 with a typed fallback to the network when the cache is absent or fails.

The subtlety is ordering. Every preset-list change resets the worklet’s channels, so system parameters and every recorded channel configuration must be re-sent after each bank add or rearrangement, and each soundBankManager command is acknowledged before the next proceeds. A renderer bank command cannot be superseded by an observer timeout; the slot stays reserved until its exact acknowledgement or the synchronous destroy fence. That is what it takes to keep a mutable synth deterministic: the mutations themselves are sequenced, acknowledged commands, not hoped-for state.

schedule ahead of the stall

The scheduler’s job is to keep the synth queue full ahead of the audio clock, and the constants are worth reading because they are chosen against measured failure, not vibes:

export const PLAYBACK_SCHEDULE_AHEAD_SECONDS = 0.25;
export const SCHEDULER_INTERVAL_MS = 20;
export const LIVE_SCORE_RESUME_DELAY_SECONDS = 0.012;
export const PLAYBACK_START_DELAY_SECONDS = 0.045;
export const SOUNDFONT_PREWARM_LEAD_SECONDS = 0.002;
export const SOUNDFONT_PREWARM_SECONDS = 0.03;
export const PLAYBACK_FIRST_SOUND_BUDGET_SECONDS = 0.15;

The cadence is a 20 ms tick and the horizon is a 0.25 s lookahead: every pass fills the synth queue a quarter-second ahead, leaving 230 ms of slack that absorbs the 200 ms stall the deterministic fixture simulates. The tick itself runs in a dedicated Worker (scheduler.worker.ts), because browsers throttle main-thread timers in background tabs to once a second or worse, and the Worker’s reply arrives as a message event instead of a timer the tab can defer. The comment in that file is honest about what the Worker does not fix: the main thread still runs the scheduling pass, so a long React commit delays the pass itself; that is exactly what the lookahead exists to cover.

Play-to-first-sound is a separate budget, bounded at 150 ms: a 2 ms prewarm lead, a 30 ms silent prewarm that decodes each track’s cold samples, and a 45 ms start delay, and the tests pin the sum below the budget. First-sound latency and ongoing-stall resilience are different problems, and they get different numbers.

never silent by construction

The base bank is the guarantee underneath everything: it answers every unloaded program, so a kit that has not loaded yet is not silence, it is the base bank’s rendering of that program. A program without its own kit uses the Standard kit. A kit file that fails to load or fails verification is skipped, reported through failedDrumKits, readable via readLoadedDrumKits, and retried by the next loadDrumKits call. Overrides behave identically on melodic channels through failedOverrides and loadOverrides. Even a decode Worker that fails leaves every prime rejected and the channel decoding as before, never silent.

Sleepyland shows the same posture at the other end of the scale: the whole product is a procedural Web Audio graph (synthesized brown, pink, and white noise plus a modeled surf engine), so there are no audio assets to fail at all, and starts, stops, and source swaps fade rather than click because the graph never hard-cuts. Different product, same rule: the graph is built so that the failure mode is degraded sound, never no sound.

telemetry, because you can’t hear it

A device underrun is invisible to a worklet, which only counts the frames it rendered. So the render-quantum report carries the wall-clock evidence: dropouts counts the steps where the wall clock ran past GENERAL_MIDI_SYNTH_RENDER_DROPOUT_MS (sent with every arm) beyond the rendered audio clock, and driftMaximumMs is the largest lead seen. Because Chromium’s worklet scope exposes no performance, the report also states which clock it used (performance or date), and consumers bound a date-based measurement by one millisecond either way.

That is the last piece of the determinism story. The graphs are data, the clocks are separated, the decode is off-thread and pinned, and the whole thing is measurable. When a user reports a stutter you cannot reproduce, dropouts and driftMaximumMs are the difference between a hunch and a number.

sources

  • sound.fish: the live product; lib/synth/renderer.ts for SoundBankManager.priorityOrder and primeProgram, app/playback.ts for the schedule-ahead constants, and app/scheduler.worker.ts for the Worker tick, all in the public repository.
  • sleepyland: the procedural noise-and-surf engine with no audio assets and fade-bounded transitions.
  • lib/synth/decode-equivalence.test.ts in the soundfish repository: the bit-identical worklet-versus-Worker proof that pins spessasynth_lib 4.3.11 to spessasynth_core 4.3.15.
  • Web Audio API: the AudioContext clock, AudioWorklet quantum, and AnalyserNode the lesson leans on.