AI models have more than they show
Work in progressIt was the summer of 2025 when I started going deep into AI. Up until then, they had been presented to me as next-word predictors. However, the more I used them, the more I noticed that there was much more to them. Next-word prediction just happens to be the way we ask them to produce output.
This belief was reinforced by Anthropic's interpretability study of these models, published on its Transformer Circuits blog.
On the Biology of a Large Language Model
Anthropic investigates the internal mechanisms used by Claude 3.5 Haiku through attribution graphs.
July 7, 2026 · 11:34 PM
Today, Anthropic released new research showing that models have an internal thinking space, much like humans do.
New Anthropic research: A global workspace in language models
Anthropic shared new research exploring a global workspace in language models and how information becomes available for verbalization.
Verbalizable Representations Form a Global Workspace in Language Models
If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of cognition happens out of view.
This reinforces my view that:
- We need more people investigating the inner workings of these models.
- We need a better output paradigm than next-token prediction.