Stroop Tests Expose Severe Attention Decay in LLMs

Language models lose processing accuracy on sustained cognitive tests, casting doubt on their reliability for parsing long-term personal health records.

By Sabin · Wellness & AI3 min read
AI News
Stroop Tests Expose Severe Attention Decay in LLMs

Researchers evaluating large language models using the Stroop test, a psychological assessment designed to measure selective attention and cognitive control, identified a sharp decline in processing reliability over extended sequences. While top systems accurately resolved conflicting color-word prompts in brief evaluations, accuracy collapsed from above 90% to near-zero as prompt length and task complexity increased.

In human psychology, the Stroop effect measures how easily your executive function suppresses automatic semantic reading to name conflicting font colors. In language models, however, the collapse reflects architectural limitations in transformer attention mechanisms: token distraction compounds over long input windows, causing the system to abandon system instructions in favor of raw statistical token frequency.

This degradation pattern exposes a critical vulnerability for anyone delegating complex health management to an automated interface. A model that correctly identifies an allergen or drug interaction in a five-line prompt often drops identical details when processing a dense, thirty-page laboratory history.

Do not paste raw, multi-year medical histories into a single consumer AI prompt expecting an accurate clinical synthesis. Break your records into discrete, chronologically constrained questions, and independently verify every medication dosage against the original laboratory sheet.

One headline rarely tells the story. See how today’s news fits the bigger shifts on AI Trends, or learn to read your own data on How it works.

Keep reading

Based on what you've been reading — always learning.

See all →