Quick Take
- The finding: With chunked KV-cache compression, the same information can be easy or hard to retrieve depending on its phase relative to window boundaries.
- How big: In DeepSeek-V4-family models at 128K context, retrieval accuracy can differ by up to 40 percentage points across phases (gaps up to 40.2 pp in base checkpoints).
- What it means: Average long-context scores can conceal periodic weak spots that adversaries — or unlucky document layouts — will hit.
Chunked KV compression packs consecutive tokens into fewer cache entries at a fixed stride, saving memory. ByteDance Seed with Princeton, Stanford and Berkeley researchers show it also creates a new positional coordinate — phase — and models become sharply sensitive to it.
What the study shows
Open-weight DeepSeek-V4-Flash/Pro/V4.1-Flash exhibit phase gaps (e.g., 34.8, 19.1, 14.8, 6.1 pp in one figure panel set). From-scratch pretraining across compression designs reproduces the effect. Causal interventions reveal phase specialization across attention components.
- Prefix padding stress tests: Compressed runs show gaps up to 78 pp vs ≤6.1 for full-attention baselines.
- Matching mean ≠ matching worst: A W8/S8 model nearly matches full attention in mean accuracy yet its worst residue scores only 9.9% vs 58.4% baseline.
- Even/odd residues: In V4.1-Flash, all even residue groups stay above all odd ones.
Why it matters: averages lie
A needle-in-haystack score of 70% can hide phases that score 30%. Production prompts that land keys near weak phases will mysteriously fail.
Method in brief
Phase-controlled retrieval probes; from-scratch ablations; causal interventions on attention components.
What remains open
- Compression design dependent: Stride and scheme change gap sizes.
- Mitigations nascent: Phase-aware training/layout still early.
Context
Related KV compression in Efficiency & Training.
What it means for long-context products
- Report per-phase metrics: Not only averages.
- Randomize or pad key positions: Avoid systematic weak phases.
- Prefer designs with smaller ∆: V4.1-Flash improves but does not eliminate gaps.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression” (arXiv:2609.36322v1, submitted September 28, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The chart was redrawn by i10X from Figure 2 family panels of the paper, which is published under a CC BY 4.0 license.
i10x Perspective
Phase sensitivity is the kind of bug that only shows up when you look sideways at the score. Once you see 40-point periodic cliffs, “we support 128K context” stops being a single number.
