Laya — 33ms Multilingual System 1 Decision Engine

gist

Convai Innovations ships Laya, an open Apache-2.0 family of non-autoregressive System 1 decision models that answer choice, score, and boolean questions over text or JSON in one forward pass (~33 ms on GPU). Three ModernBERT and mmBERT checkpoints cover English, 100+ languages, and typed enterprise decisions, with Unicode-script routing before inference because English-model confidence stays high even when accuracy collapses outside Latin script. Versus TypeSafe Jev, Laya claims faster latency, better calibration, and free self-hosting, with honest ceilings above roughly 20 choice options and weak zero-shot without fine-tuning.

ideas

• Reflex decisions do not need chatbots. Ticket routing, spam, jailbreak checks, and ordinal urgency are structured System 1 jobs; streaming LLMs add latency, cost, and fake confidence text.
• Three primitives, one pass. Choice, score, and noul return calibrated probabilities and numbers only, so the model never emits free text and cannot emit malformed schema JSON.
• Route before you trust confidence. English ModernBERT shreds non-Latin scripts while reporting ~0.9 confidence; script and stopword routing (sub-millisecond) picks the multilingual checkpoint first.
• Open weights beat a closed decision API on speed and cost. Routed Laya reports ~33 ms P50, ECE 0.081, and $0 self-host versus Jev's ~250 ms and metered API pricing on the team's head-to-heads.
• Ceilings are architectural. Choice quality falls past ~20 options, out-of-box zero-shot is near chance until fine-tuned, and temperature calibration on your domain is required for honest ECE.

quotes