When Long Contexts Break AI Attention in Clinical Reasoning

A standard assumption in digital health is that longer context windows make models better at reading medical histories. When researchers applied the Stroop task—a psychology benchmark used since 1935 to measure selective attention and cognitive control—to frontier neural networks, that assumption cracked. While systems identified incongruent items accurately across short sequences, performance collapsed as the lists lengthened. In controlled evaluations, top-tier architectures saw accuracy rates plummet from above 90% to near zero on prolonged runs.

The mechanism resembles computational attention fatigue. Rather than maintaining selective suppression over thousands of tokens, transformer-based systems lose the ability to filter competing signals as the context expands. The model begins to default to statistical token frequency rather than the active rule set.

In a 2024 analysis of context fidelity, retrieval reliability dropped sharply once prompt lengths exceeded 32,000 tokens, even in models advertised for 128,000-token capacity. For anyone feeding a decade of diagnostic PDFs into an automated interface to understand a chronic condition, the model will sound confident while silently ignoring early baseline tests.

Treat large context windows as marketing until proven otherwise. If you evaluate your own long-term health metrics with these tools, break your queries down by year or single clinical episode, and verify every medication cross-reference yourself.