turbovec

gist

turbovec implements Google Research’s TurboQuant as a local Rust vector index with Python bindings. It compresses vectors to two or four bits per coordinate, searches codes directly with architecture-specific SIMD, accepts online inserts, filters within the scoring kernel, and persists incremental changes. The repository reports stronger speed than matched FAISS baselines across its ARM and x86 test matrix while documenting recall tradeoffs, calibration, benchmark data, and the distinction between its published suite and its synthetic optimization harness.

ideas

• Quantize without a corpus-training phase. A shared random rotation makes coordinate distributions predictable enough for precomputed Lloyd-Max codebooks, with optional per-coordinate calibration for finite-dimensional drift.
• Score compressed vectors directly. ARM and x86 kernels use lookup tables and SIMD instructions instead of reconstructing every database vector before comparison.
• Keep mutation online. New vectors enter the index immediately, stable external IDs support constant-time removal, and the corpus can grow without rebuilding a trained quantizer.
• Push filters into the kernel. Candidate allowlists skip empty 32-vector blocks before lookup and discard individual excluded slots before heap insertion.
• Publish the measurement boundary. The repository exposes result files and reproducible suites, explains low-dimensional recall limits, and labels its fast synthetic harness as an optimization screen rather than evidence for published claims.

quotes