A piano roll is a stress test disguised as a widget. Sixteen tracks of sixteen bars of sixteen steps is hundreds of interactive cells, each of which must hit-test a pointer, preview a drag, and repaint on every frame a pointer moves. Rendered as DOM, every pointermove is a candidate React commit, and at sixty frames per second the framework becomes the bottleneck the user can feel.
Soundfish renders its sequencer on canvas and then rebuilds, deliberately, the two things canvas throws away: an accessible control surface and a verifiable interaction contract. The result is a component an agent can drive, because every surface the agent needs is expressed as stable data rather than as pixels. This lesson is about the lanes the code draws, what is allowed on the fast one, and how the whole thing stays measurable.
the ephemeral lane
The sequencer’s own rules state the fast path as a list of prohibitions. Pointer movement is an ephemeral high-frequency lane, and per raw sample nothing durable may happen: no React state, no event-node traversal, no layout reads, no score parsing, and no durable callbacks. A moving pointer produces exactly one thing, a retained latest-sample record (piano-roll-drag.ts keeps the injected-frame drag controller fed with the freshest position), and the frame coordinator decides what to paint.
Two locks keep the lane honest. Gesture intent is fixed at pointer-down: the tool and the modifier semantics for the whole gesture are read once, so a Shift key arriving mid-drag cannot quietly turn a move into a marquee. And a gesture invokes at most one durable callback: the component previews continuously but commits exactly once, at pointer-up, across one semantic boundary. The consumer-facing contract reads the same way; the roll’s props are a union of read-only and editable modes, so a surface that never edits cannot be handed editing callbacks it would misuse:
<PianoRoll
tool="select"
cycleSteps={64}
events={scene}
onMoveCommit={commit}
onResizeCommit={commit}
onSeek={seek}
ariaLabel="Bass pattern"
/>The evidence that this works is measured rather than asserted. The Direct browser harness records layoutReads: { down: 1, move: 0 } for a drag: one layout read when the pointer lands, zero while it moves. That pair of numbers is the entire philosophy of the lane in two fields.
preview and commit are separate
What the user sees during a drag is not the document changing; it is a preview on a retained compositor path. Move, duplicate, group, marquee, edge-resize, and creation previews all draw through paths that stay on the compositor (an overlay element driven by translate3d, a dedicated selection canvas), so repainting a preview is a transform, not a reconciliation. The recorded preview for a moved event reads translate3d(40px, -10px, 0), and the harness asserts projectionMatchesPreview: what the commit computes is what the preview showed.
The commit is the single durable event at the end of the gesture. It crosses the semantic boundary once, expressed as a document operation (a move-event with from and to and an { trackId, eventId } address, a marquee’s selection set, a creation’s interval), and the store applies it as one revision. A cancelled gesture produces zero commits, not a rollback. This is the property that makes the component legible to history and undo: the score only ever changes in whole, named edits, regardless of how many frames of preview preceded them.
a constant dom proxy
Canvas trades away the DOM, and with it everything assistive technology reads, so the component rebuilds that surface as a constant-size semantic proxy. The roll exposes one role="grid" host per track canvas, one Tab stop across the stacked set (a roving tabIndex with adjacent vertical focus handoff), aria-activedescendant pointing at the active cell, and an aria-describedby that combines the help text with a durable selected-count description. The DOM node count does not grow with the score: the harness samples semanticProxyNodesBefore, During, and After a pointer storm and expects the same 32 throughout, one proxy cell per track, with exactly one event proxy across the stack when a controlled selection needs it.
The seams an agent or a test reads are data attributes, not pixel scrapes: data-music-piano-roll-tool names the active tool, -event-count and -selected-count report the model sizes, -seekable marks a live ruler, and -start-step, -loop-from, -loop-to project the transport markers. The ruler itself stays aria-hidden; keyboard seeking is the consumer’s command surface (onSeek, onLoopRangeChange), not a widget pretending. Selection renders through explicit forced-color-safe tokens so a high-contrast user still sees it, and every selected event stays visibly distinct after a gesture rather than collapsing into a uniform block.
geometry is the hit test
Because there is no DOM to hit-test, the scene builds its own deterministic geometry. piano-roll-scene.ts parses events into normalized rectangles and a sparse spatial index; piano-roll-hit-testing.ts resolves overlaps deterministically and clamps pointer targets inward so a resize edge is grabbable even on a short note. Events are identified by caller-owned string IDs paired with their track at every scene, hit, drag, and commit boundary, because an event ID is never assumed unique outside its track.
The same determinism governs what a gesture means. Empty space resolves from the explicit PianoRollTool, never from whether a callback happened to be provided. Select’s defaults are written as rules: blank click clears, blank drag marquees, blank double-click creates exactly once, event double-click deletes through the ordinary commit boundary, and creation is never tunneled through an existing event or a completed lasso. Scheduling semantics are pinned too: windows are half-open, and a note-off sorts before a note-on at the same instant so a repeated pitch retriggers at a cycle seam rather than disappearing.
The viewport follows the same data-first posture. minimumStepWidth can pin the grid at cycleSteps × width CSS pixels, marking the frame data-scrollable, sticking the axis columns, and letting createPlayheadPager page the frame so the playhead stays visible during playback. On touch, the canvas keeps pan-y so vertical scrolling still belongs to the browser while the horizontal lane belongs to the gesture.
the lanes compose
The roll is not the whole surface; it is the reference implementation of a contract the neighbors reuse. The velocity lane under each expanded track is its own canvas, mounted through the roll’s laneSlot in a third frame row, and its gesture sessions follow the same shape: paint sets every crossed event, Alt-drag ramps from an anchor, a drag that starts on a selected event sets the whole selection, and the lane previews from a ref and commits once on pointer-up as replace-event operations, so undo covers it like any other edit.
Keyboard editing is deliberately renderer-neutral. piano-roll-keyboard-edit.ts expresses fine, beat-sized, and octave-sized transforms as intents over event addresses, and at the product level the verifier drives a fixed eight-edit keyboard script (six one-step moves, a paste into the next bar, a loop doubling) against the mounted Studio, so the keyboard path and the pointer path converge on the same commit boundary. The ruler module is likewise renderer-neutral geometry: the step under a pointer, whether a drag left its bar, the bar-aligned range a drag selects. The pattern to take away is that canvas bought the frame budget, and everything the framework would have provided, hit-testing, selection, keyboard semantics, is rebuilt as pure functions over declared geometry that a test can call without a browser at all.
measured, not believed
The reason to call this agent-operable rather than just canvas-rendered is that the contract is exercised, end to end, in a real browser. The Direct harness binds 32 track canvases to 2,048 scene events in the compact matrix and proves constant DOM through pointer storms; it asserts that pointer-down plus pointer-move produce no React reconciliation and no move-time layout reads; and it mounts the production Studio to drive real pointer and keyboard gestures through studio-tasks.ts: a blank document to a four-bar drums, bass, and chords loop in at most 40 gestures, a doubling to eight bars with a varied bar 8 in at most 4 while playing, and a wrong note fixed during playback in at most 3.
Around the gestures, a separate budget holds the frame honest: every edit must stay within two React commits in every mode and zero long tasks under the blocking 6× CPU profile. The artifacts (direct.browser-bridge v2 samples, soundfish.studio-task-benchmark/v1 gesture logs with before-and-after scores) are retained per run, so the claim “sixty frames without layout thrash” is a measured property with evidence, not a performance aspiration.
sources
- sound.fish: the live product;
lib/sequencer/forpiano-roll-scene.ts,piano-roll-hit-testing.ts,piano-roll-drag.ts,piano-roll-selection.ts,react.tsx, andplayhead-paging.ts, anddirect/studio-tasks.tsplusdirect/verify-browser.tsfor the gesture benchmarks, all in the public repository. direct/verify-browser.test.tsin the soundfish repository: the recordedlayoutReads, proxy-node, and commit budgets quoted above.- ARIA Authoring Practices: grid pattern: the
role="grid"andaria-activedescendantcontract the semantic proxy rebuilds over canvas.