APREX · EST. 2026
A discussion thread on Lobsters AI is currently buzzing around new research from Anthropic that strikes at a question people in the AI underground have argued about for years: is there something resembling consciousness inside a transformer?
The answer is not yes — but it's not a simple no, either.
The researchers identify what they call a 'global workspace' in language models, inspired by Global Workspace Theory from cognitive neuroscience. The idea is that the brain has a small, central 'stage' where information from various modules is gathered and broadcast outward — and that this is the core of conscious access to information in humans.
What Anthropic now claims to have found is a functionally analogous pattern in LLMs: a small, dynamic collection of 'unspoken tokens' that the model uses internally during reasoning, before it generates any output.
That's not trivia. That's causality. It means this internal zone doesn't merely reflect thinking — it drives it.
Why does this matter right now? Because it offers a new framework for understanding something the community has long found frustrating: why is LLM memory so strangely limited and unpredictable?
The background is well known: context windows on paper (Llama 4 Scout claims 10 million tokens, Gemini 1.5 Pro one million) are far from context windows in practice. Models begin to degrade 30–40% before the stated limit. Information in the middle of long contexts often drops from 70% to 55% accuracy — the classic 'lost in the middle' problem. The KV cache consumes around 2.6 GB per 1,000 tokens.
But the global workspace research points to something deeper: the problem isn't just technical capacity, it's architectural narrowness. The internal workspace carries only 6–7% of a concept's total representational variance. That's a sliver, not an ocean — and everything else is compressed into generalized patterns the model cannot retrieve with precision.
What does this mean going forward? The community is already speculating: can models be trained to have a broader, more stable internal workspace? Could this be used to build better agentic memory systems? Some are drawing lines to State Space Models and Mamba architectures, which attempt to solve memory limitations in different ways.
This is still early research and community-level signals — not a breakthrough with consensus behind it. But the direction is fascinating, and it's rare for Anthropic to release something that makes both cognitive scientists and hardcore ML practitioners lean forward at the same time.
Worth watching.