Stroop test exposes attention failures in large models

Psychologists have long used the Stroop task to quantify cognitive control and selective attention under interference. Researchers recently administered an adapted version of this classic test to prominent large language models, presenting them with sequences of colored words where text and ink colors deliberately conflicted. When given short sequences, leading architectures identified the target attributes with accuracy scores above 90%.

As the sequence length grew and task complexity expanded across sustained test runs, performance collapsed. Several advanced systems deteriorated toward total failure, misidentifying attributes or cycling through repetitive errors. The degradation exposed a systemic inability to maintain stable selective attention across long horizons, mirroring human cognitive fatigue but driven by mathematical attention dilution across token windows.

If you use conversational assistants to synthesize your multi-year lab reports or extended wearable histories, break your inquiries into small, isolated segments. Do not assume a model tracks every detail across a long conversation window; verify key values manually against the primary source documents.