saved
Positron Vision
Hraness cites a source capture. The source author remains the source.
gist
Positron argues that generative AI is now infrastructure-bound, not intelligence-bound: grids limit power, inference is memory-bound and latency-sensitive unlike batchable training, and GPUs leave most theoretical memory bandwidth on the table. Their bet is memory-first silicon (Asimov) and US-built systems (Atlas) that favor commodity LPDDR over HBM, specialize in decode-heavy phases beside GPUs, and optimize tokens per dollar and joules per request rather than peak FLOPs.
ideas
- Power and memory beat affordability. Data centers are grid-limited, and most accelerators realize under 30% of theoretical memory bandwidth on real inference.
- Training silicon is the wrong tool for serving. Training is compute-bound and batch-tolerant; inference is memory-bound and latency-sensitive, so purpose-built decode hardware is the step-function bet.
- Inference is a portfolio of subworkloads. Prefill versus decode, attention versus feed-forward, and techniques like MoE or speculative decoding keep reshaping the mix that hardware must adapt to.
- Heterogeneous fleets, memory-first chips. Positron targets decode and high-bandwidth capacity with Asimov, claiming over 90% realized memory bandwidth versus under 30% for GPUs, while GPUs keep compute-dense work.
- LPDDR, US supply, and TCO over vanity metrics. Commodity LPDDR trades HBM sticker bandwidth for capacity, cost, and power; Atlas is US-built; customers are scored on latency, tokens per dollar, and joules per request.
quotes
“Data centers are no longer constrained by what they can afford—they're constrained by what the grid can deliver.”
“Yet most AI accelerators achieve less than 30% of their theoretical memory bandwidth on real inference workloads.”
“Training is compute-bound and tolerates batching. Inference is memory-bound and latency-sensitive.”
“The result is over 90% realized memory bandwidth utilization versus under 30% for GPUs”