A wire protocol is the part of a system you cannot take back. Code you can refactor; bytes another process already wrote are fixed forever. Peer-to-peer systems feel this hardest: there is no deploy order, no flag day, no server that can demand everyone upgrade. The format has to survive versions of the software that do not exist yet, and refuse versions that should not.
Valhalla’s wire formats are small, fixed-width, and ruthless about their own boundaries. The lessons are in the details.
a frame is a contract
The base envelope is vhalla-wire’s Envelope, and its layout is a fixed 78-byte header followed by an opaque body:
version (1) | kind (1) | author (16) | realm (16) | room (16)
| event (16) | sequence (8) | body length (4) | body (n)Every integer is big-endian. Every field is a fixed type at a fixed offset: u128 identifiers, a u64 sequence, a u32 length. HEADER_BYTES is a compile-time usize, and encoded_len() computes the exact byte count without allocating. The body is opaque by construction: Envelope stores it as a private Vec<u8>, exposes only body() -> &[u8], and a compile-fail doctest proves you cannot push to it through a mutable reference. The frame is a contract: the encoder and decoder are the only places the layout exists, and the type cannot be coerced into holding a body that would violate the invariant.
The size bound is derived, not parallel: MAX_BODY_BYTES = MAX_ENVELOPE_BYTES - HEADER_BYTES, where MAX_ENVELOPE_BYTES comes from vhalla-core (64 KiB). One limit defined once at the trust boundary, one subtraction at compile time: there is no second constant that can drift out of sync with the first. Construction enforces it too: Envelope::new returns None for a zero kind or an oversized body (Envelope::chat is the KIND_CHAT convenience over it), so a valid Envelope value is always encodable, and decode’s only job is to refuse bytes that never crossed the constructor.
version bytes and no downgrades
The first byte is WIRE_VERSION = 1. That is the upgrade path: a decoder that sees 2 fails with UnsupportedVersion { found: 2 } before reading another byte. There is no negotiation, no best-effort parse, no “try both.” The version is the first field, the refusal is immediate, and the error names the version it saw.
The signed layer shows the same discipline in reverse. vhalla-crypto’s signed transport frame is SIGNED_VERSION = 2, and its doc comment is explicit: “Legacy unversioned v1 frames are rejected.” The v2 frame wraps the unchanged canonical v1 unsigned envelope inside its own structure: version byte, 4-byte envelope length, the v1 envelope verbatim, then audience, epoch, session, expiry, and a 64-byte signature. The fixed overhead is a named constant: SIGNED_OVERHEAD_BYTES = 1 + 4 + 16 + 8 + 16 + 8 + 64, 117 bytes, every field accounted for. The old format did not get edited; it got enclosed. A v1 unsigned envelope is still valid inside a v2 signed frame, while a v1 signed framing is rejected outright: no downgrade, no ambiguity about which rules a peer is playing by.
The signature itself binds the whole frame through a domain string: vhalla/signed-envelope/v2 is prepended to the signed transcript, so a signature produced for one purpose cannot be replayed as another. The claims layer namespaces further (ClaimDomain::{Session, Provenance, Capability, Receipt}) and the domain constants throughout the workspace are all versioned strings (vhalla/paired-chat/handshake/v1, vhalla/ledger/event/v1, vhalla/owner-invitation/v1). When the protocol changes, the domain changes, and cross-version signature confusion is impossible by construction.
canonical or rejected
A wire format earns its keep by having exactly one byte representation for each logical value. The workspace enforces this everywhere it can:
- Trailing bytes are fatal.
Envelope::decodecomputes the expected end and returnsTrailingBytes { count }if the input is longer.vhalla-native’sBoundedCodecdoes the same on a socket: after reading the declared body it reads one more byte and rejects if anything arrives. A peer cannot smuggle a second message inside the first’s padding. - Declared length is checked before allocation.
BoundedCodec::read_framereads the 4-byte length, compares it toMAX_FRAME, and only then allocatesvec![0; length]. The length is untrusted input, so it is a bound to check, not a promise to believe. - Non-canonical forms are rejected.
vhalla-social’s tag grammar is ASCII lowercase[a-z0-9_][a-z0-9_-]{0,47}; foreign wire keys “must already be canonical”; no normalization happens at the boundary.vhalla-rooms-consensus’seligible_listrequires strictly ascending, duplicate-free owner IDs and rejects any set that isn’t: “each set has exactly one byte form.”
The goal is that equality of bytes means equality of value. If two encodings of the same record can differ, you cannot hash the bytes to get the identity, and everything downstream, from signatures to content addressing to the journal’s deduplication, assumes the bytes are the identity.
unknown fields versus closed fields
The usual advice for protocol evolution is “ignore unknown fields”: protobuf’s whole model rests on it. Valhalla mostly does the opposite, and the contrast is instructive.
The formats here are closed: Invitation::decode requires exactly INVITATION_BYTES; Pin::decode requires exactly 4 + 104 bytes; decode_head in cert.rs requires the VC2 magic and an exact length for the signature count. Nothing is skipped or tolerated. The reason is that these are not extensible messages; they are evidence. A certificate, an invitation, a journal pin are values where an extra byte is not forward compatibility; it is a parse ambiguity or an attack surface. Closed formats make “this exact structure” checkable.
Where the protocol does extend, it extends by adding versions, not fields. vhalla-rooms-consensus batches moved from VRB1 to VRB2 when an eligible transition field was added, and the decoder still reads VRB1, treating it as eligible: None, so “retained journals remain readable.” A VRB2 batch must actually carry a transition; None canonically encodes as VRB1. The magic is the version; the old format is never rewritten.
The social records show the third option: PostFaceted/ReviseFaceted added opcodes 8 and 9 that append a facet array to existing post/revision fields, while “existing v1 opcodes/bytes/domains remain unchanged.” Older clients reject the new opcodes and lose writer/control closure; they don’t misparse it. The rule the README states is the discipline: “never strip annotations, re-sign history, or silently rewrite stored records.”
There is one deliberately open field, and it is instructive: the kind byte. Envelope::decode rejects only kind == 0 (InvalidKind); values 3 and up decode fine. The wire layer carries the kind without interpreting it, because interpretation is the consumer’s job: vhalla-session admits only KIND_CHAT, vhalla-policy authorizes only KIND_READ_MEMORY_REQUEST, vhalla-retrieval ignores anything that isn’t chat. Unknown kinds are the extension space: a future message type doesn’t need a wire-format change, only a consumer that knows what to do with it, and every existing consumer’s closed check is the safe default.
the format outlives the code
The strongest evidence that the format is a contract is that it has a second implementation. vectors/social-v1.json is a transcript generated by an independent Python implementation (hashlib and struct, not Rust), and the social crate’s tests pin canonical bytes and content IDs against it. vhalla-crypto’s v2 framing has the same treatment: a fixed-width fixture built with struct.pack, hashed with hashlib, checked against the Rust encoder’s output byte for byte. The test pins an exact encoded length (200 bytes) and the SHA-256 of the encoded frame: if the layout, endianness, or field order ever drifted, the digest would change.
That is the test a wire protocol actually needs. A Rust round-trip test proves encode and decode agree with each other; an independent implementation proves the format is what the code says it is: that the field is really big-endian, the length is really u32, the domain string is really vhalla/signed-envelope/v2. The code will change. The bytes are already out there.
sources
- vhalla
crates/vhalla-wire/src/lib.rs: the 78-byte envelope,WIRE_VERSION, andTrailingBytescrates/vhalla-crypto/src/lib.rs:SIGNED_VERSION = 2wrapping the unchanged v1 envelopecrates/vhalla-native/src/codec.rs: length-before-allocation and trailing rejection on a live socketcrates/vhalla-rooms-consensus/src/lib.rs:VRB1/VRB2batch versioningcrates/vhalla-social/README.md: the facet opcodes and “never rewrite stored records”vectors/social-v1.json: the independent Python transcript