Why LLM Health Copilots Stumble on Longitudinal Lab Histories
When clinical attention tests drag on, frontier model accuracy plummets from 90% to near zero, raising urgent reliability flags for chronic health tracking.
If you feed an automated health assistant a single blood panel from Tuesday, it will parse your triglycerides with reasonable precision. The difficulty starts when you hand it five years of metabolic panels, lipid ratios, and endocrine notes. New evaluations using standard psychological attention tests demonstrate that leading language models experience severe cognitive collapse as sequence length and task duration increase.
In structured evaluations based on classic color-naming attention tasks, top commercial language models sustained greater than 90% accuracy across short prompts. Once the sequence lengthened into sustained, multi-step tracking, performance degraded rapidly, with some premier models dropping to near-complete failure. The systems did not simply slow down; they lost track of the underlying rule.
Large models process context through statistical attention heads rather than sustained focus. When a prompt swells to accommodate years of patient data, noise accumulates. The model begins favoring local word associations over global instructions, creating blind spots in multi-year medical chronologies.
Do not paste your entire medical history into an open-ended chat interface and ask for a complete metabolic diagnosis. Break your data into isolated, yearly comparisons, verify the raw numbers yourself, and treat automated summaries as preliminary drafts rather than verified longitudinal charts.
The longer view
One headline rarely tells the story. See how today’s news fits the bigger shifts on AI Trends, or learn to read your own data on How it works.