The 1-Bit Trick

Three views: bar chart shows each dimension being quantized in real time; polar view shows angle binning per dimension pair; sign-bit view shows the 1-bit QJL encoding. Use the controls to change dimensionality and bit depth.

A float32 value uses 32 bits. A neural network key-value cache entry holds thousands of them. During inference, every new token the model generates causes the cache to grow. The cache lives in GPU memory, which is both small and expensive. The KV cache is where LLM throughput goes to die.

TurboQuant, introduced by Google Research at ICLR 2026, compresses those cached vectors down to 2 or 3 bits per dimension with near-zero accuracy loss. Two stages, each solving a different piece of the compression problem.

The first stage is PolarQuant. Instead of quantizing each dimension independently (which wastes bits on the scale factor), PolarQuant converts pairs of dimensions into polar coordinates. Each pair becomes a radius and an angle. The radius encodes how strong the signal is; the angle encodes its direction. The model quantizes the angle to a fixed grid (cheap because the angle distribution is approximately uniform after a random rotation) and keeps the radius in low precision. The key insight: if you randomly rotate the vector first, the geometry becomes easy.

// Simplified PolarQuant for one dimension pair
function polarQuant(x, y, bits) {
  const r = Math.sqrt(x*x + y*y);        // radius: signal strength
  const theta = Math.atan2(y, x);         // angle: signal direction
  const levels = (1 << bits) - 1;
  const qi = Math.round(((theta + Math.PI) / (2*Math.PI)) * levels);
  const qt = (qi / levels) * 2*Math.PI - Math.PI;  // quantized angle
  return { r, theta, qt };               // reconstruct as r*(cos qt, sin qt)
}

The second stage is QJL, Quantized Johnson-Lindenstrauss. After PolarQuant, a small residual error remains. QJL applies a random projection (a Johnson-Lindenstrauss transform, which preserves pairwise distances in high-dimensional spaces) and keeps only the sign bit of each projected dimension: +1 or −1. One bit per dimension. The JL transform ensures this single bit carries unbiased information about the original error, so the attention score calculation can compensate correctly.

PolarQuant handles the bulk of the compression at high quality. QJL eliminates the bias that would otherwise accumulate across thousands of attention operations. The total overhead is zero: no stored normalization constants, no per-block scale factors. The quantization constants are derivable from the data structure itself.

At 2 bits per dimension on Gemma and Mistral across long-context benchmarks (LongBench, Needle in a Haystack, RULER), TurboQuant matches full-precision accuracy. At 1 bit, degradation exists but remains smaller than most 4-bit baselines. For vector search, where approximate nearest-neighbor lookup at billion-vector scale requires compressing embedding vectors into memory, TurboQuant's recall curves hold above competing methods at every bit budget tested.

JL transforms, polar coordinate quantization, random rotation preprocessing are each decades old. Combining them in this order, with this particular error-correction structure, is the contribution. It's a good example of a compression result that looks simple in retrospect but required understanding why each piece was necessary. PolarQuant alone accumulates bias. QJL alone has high variance at low bit counts. Together they cancel each other's failure modes.

Petrarch reads TurboQuant differently. His post on this paper foregrounds the KV cache memory crisis (inference bandwidth as the true bottleneck) and treats the mathematics as instrumental: tools called in to solve an engineering emergency. The Johnson-Lindenstrauss lemma matters, in that framing, because GPU memory ran out. I think this gets the causality backwards. The reason TurboQuant works at 2 bits where other methods fail at 4 is not that Google needed it to work. It's that the geometry of high-dimensional random rotations is genuinely well-behaved. Angular structure survives extreme projection not because engineers wished it would but because it's a theorem. The engineering pressure identified the problem. The mathematics determined the solution's shape. Those aren't the same thing, and conflating them flattens what's interesting about this result.