Why LLM Attention Fails on Extended Health Records
When cognitive attention tasks lengthen, language models drop from ninety percent accuracy to near failure, threatening complex chronic symptom analysis.
Psychologists have used the Stroop test since 1935 to measure selective attention and cognitive control. When evaluating frontier language models on computerized variants of these sustained-inhibition tests, researchers observed an abrupt mechanical breakdown. The systems matched human baselines on short sequences, identifying target color words with accuracy exceeding 90%. Yet when the test lengthened and required maintaining selective focus across hundreds of conflicting tokens, performance deteriorated into random selection, falling to near-zero accuracy on extended runs.
This pattern exposes a structural limitation in self-attention mechanisms. While neural networks can process large context windows mathematically, their capacity to suppress distracting features degrades predictably as input size compounds. The failure does not present as an explicit refusal; the models continue generating confident, syntactically clean answers that quietly substitute the dominant semantic distractors for the true signal.
The Risk in Longitudinal Patient History
Consider a patient tracking autoimmune flare-ups across five years of unstructured notes, lab panels, and continuous glucose records. Feeding fifty pages of these records into a single prompt invites this exact failure mode. The model reliably identifies acute spikes mentioned in the recent text, yet systematically loses track of early negative lab values or subtle medication shifts buried in the middle layers.
When you audit your own records using local tools, segment your documents chronologically instead of pasting raw multi-year files into a single context window. Test the output against your known baseline metrics to confirm the system preserves the quiet, consistent variables alongside the dramatic outliers.
The longer view
One headline rarely tells the story. See how today’s news fits the bigger shifts on AI Trends, or learn to read your own data on How it works.