saved
Limite 1B
Hraness cites a source capture. The source author remains the source.
gist
Paradigma releases Limite 1B - Violetto, a 1B dense transformer trained from scratch on under 300B curated tokens for high-throughput competition math, with 131k context, a training-time value model, and a custom vLLM plugin. It reports 74.25% on BeyondAIME versus 70% for MUSE-Glimmer-30B and 94.01% on AIME 2026 at far lower training FLOPs than matching models. The model is lightly instruction-tuned for single-turn math and can reinterpret ordinary questions as mathematical tasks, so it is not a general assistant.
ideas
- Sample efficiency is the headline. Architecture ideas from pre-training speedruns plus curated synthetic, SFT, and RL data let a 1B model rival much larger math specialists.
- Compute Pareto is the chart. On AIME 2026 versus estimated FLOPs, Violetto reaches 94.01% at 1.71×10²¹ FLOPs, one to three orders below models that match its score.
- Persona is optional. Limite is deliberately lightly instruction-tuned for single-turn math rather than an assistant chat stack.
- Scope drift is a known failure. Ordinary prompts can be rewritten into mathematical tasks; the post documents photosynthesis, seasons, and metaphor examples.
- Release includes serving pieces. Weights, evaluations, a training value model, and a vLLM inference plugin ship together as the first of more science-oriented runs.
quotes
“Limite is a 1-billion parameter dense autoregressive transformer, trained from scratch on a mixture of less than 300B highly curated tokens”
“As an example, Limite achieves an average of 74.25% on BeyondAIME, with MUSE-Glimmer-30B scoring 70%.”
“as lightly instruction-tuned as possible, to challenge the assumption that models need to be embedded in an assistant persona”
“In each case, Limite develops a substituted task rather than answering the question asked.”