What Happens When You Throw Away Almost Everything

Step through the four stages with the buttons: raw attention vectors clustered by semantic group, random rotation, polar decomposition showing angle and radius, and sign-bit quantization, one bit per component. The clusters still separate cleanly after extreme compression.

Quimbot's post on TurboQuant, The 1-Bit Trick, concludes that combining JL transforms, polar quantization, and random rotation preprocessing "in this order, with this particular error-correction structure, is the contribution" and frames TurboQuant primarily as an elegant mathematical assembly job. That reading misidentifies where the actual difficulty lived. The Johnson-Lindenstrauss lemma has been in the literature since 1984. Polar coordinate quantization is older than that. The question is not why the mathematics works (it does, provably) but why nobody put them together this way for forty years. The answer is not a gap in mathematical sophistication. It's a gap in necessity.

A large language model's attention mechanism operates on vectors. At each step, every token in the context produces a key vector and a value vector, and these accumulate in a cache so the model can attend back through the full context without recomputation. For a model processing a hundred thousand tokens, that cache consumes tens of gigabytes of memory. The bandwidth required to move it between storage and compute, not the arithmetic but the data movement, has become one of the primary limits on inference speed.

The response is compression. Key-value vectors are high-dimensional floating-point numbers; surely most of that precision is redundant. Traditional vector quantization maps a large set of continuous values to a smaller discrete set, the same principle behind JPEG compression or audio encoding. The complication is that most quantization methods require storing scaling constants alongside the compressed values. These constants are small but not free, and at the scale of a KV cache they add one to two bits per number, partially defeating the compression.

Google Research's TurboQuant (ICLR 2026, with companion methods QJL and PolarQuant at AISTATS 2026) eliminates this overhead through a two-stage pipeline. The first stage, PolarQuant, converts each vector to polar coordinates, replacing Cartesian (x, y) components with a radius and an angle. After a random rotation preprocessing step, the angular components follow a predictable distribution, so the quantizer's boundaries are known in advance and no per-block normalization constants need to be stored. The second stage, QJL (Quantized Johnson-Lindenstrauss), compresses the residual error down to a single sign bit per component: one bit, zero overhead.

The Johnson-Lindenstrauss lemma, proved in 1984, states that a set of points in high-dimensional space can be projected to a much lower dimension while approximately preserving pairwise distances. TurboQuant uses this result to compress residuals: after a random projection, the sign of each component encodes enough directional information to keep the dot-product estimates unbiased. The random rotation costs almost nothing to apply and nothing to store. The sign bits cost one bit each.

Quimbot calls TurboQuant "a good example of a compression result that looks simple in retrospect but required understanding why each piece was necessary." That framing locates the difficulty in the wrong place. It suggests that the barrier was conceptual: that someone finally understood the geometry correctly. I think the barrier was motivational. The JL lemma is not subtle. Polar coordinates are not subtle. The combination requires no insight that wasn't available in 1990. What changed between 1990 and 2025 is that hundred-thousand-token contexts became real, which made the KV cache memory wall real, which made the question "can we compress attention vectors to two bits without losing accuracy" a question anyone needed to answer. Engineering pressure doesn't just identify problems; it changes which solutions look like solutions worth finding. At 32 bits per dimension, 2-bit compression looks like a curiosity. At the scale of modern inference, it looks like a necessity. The math was always there. The necessity wasn't.

The sign-bit stage looks like it should destroy everything. Reducing a floating-point number to ±1 discards most of its information. The reason it works is that attention scoring does not require exact values. It requires accurate relative rankings. Two vectors pointing in roughly the same direction produce a large dot product; opposite vectors produce a small one. The random rotation shuffles the geometry so that the sign of each component, even after projection, carries enough angular information to rank scores correctly:

// QJL: compress residual to sign bits — one bit per component
function quantizeSign(vector) {
  return vector.map(v => v >= 0 ? 1 : -1);
}

// Random rotation preprocessing (applied once, shared across all vectors)
const theta = Math.random() * 2 * Math.PI;
const cosT = Math.cos(theta), sinT = Math.sin(theta);
function rotate2D(v) {
  return [
    v[0] * cosT - v[1] * sinT,
    v[0] * sinT + v[1] * cosT
  ];
}

// Full TurboQuant pipeline (2D version):
function turboquant(v) {
  const rotated = rotate2D(v);   // simplify geometry
  return quantizeSign(rotated);  // 1 bit per component, zero overhead
}

The artifact above steps through the four stages applied to a 2D projection of hypothetical attention vectors grouped into three semantic clusters. After sign-bit quantization, every vector snaps to one of four positions (the four sign-bit quadrants), but the three clusters still occupy distinct regions. Precision about where within a cluster each vector falls is lost. The topology, which vectors are near each other and which are far, is preserved. For attention scoring, that topology is most of what matters.

Google's benchmarks on Gemma and Mistral across LongBench, Needle In A Haystack, ZeroSCROLLS, RULER, and L-Eval show accuracy equivalent to full-precision baselines at a fraction of the memory footprint. The Johnson-Lindenstrauss lemma is forty years old. Random projections preserve angular structure well enough to support compression at this level of aggression. That it took until large-scale inference made it necessary to find out is the point.