AI Models Fail Standard Psychology Attention Tests on Long Tasks
Automated clinical screeners degrade severely as task duration stretches, raising error rates when models process long patient records or continuous behavioral feeds.
When researchers administered the classic Stroop color-naming test to leading large language models, accuracy plunged from 94% on brief sequences to single digits on extended strings. The failure reveals that artificial attention mechanisms do not maintain uniform cognitive focus over long operational sequences, behaving unevenly under sustained distraction.
This degradation presents clear risks for health workflows. Many clinical intake tools now process multi-page health histories, long psychotherapy transcripts, and uninterrupted voice logs. If an underlying model exhibits severe attention decay after a few hundred continuous tokens of conflicting inputs, it drops subtle behavioral indicators and pharmaceutical interactions situated deeper in the record.
Cognitive scientist Colin MacLeod (1991) documented how human selective attention manages conflicting semantic signals through inhibitory control. Current language models lack an analogous mechanism; they merely distribute attention weights, leaving them prone to catastrophic interference as document length expands.
When you use consumer tools to parse your own multi-year lab reports or psychotherapy notes, feed the system single visits rather than entire medical histories. Demand exact line citations for every summary metric it returns.
The longer view
One headline rarely tells the story. See how today’s news fits the bigger shifts on AI Trends, or learn to read your own data on How it works.