AI Models have more than they show
Work in progressIt was the summer of 2025 when I started going deep into AI. Up until then they were just presented as next word predictors. Although the more I used them, I couldn't help but notice that there's way more to them. The next word prediction just happens to be the way we ask them to give their output.
This was solidified by the fact that Anthropic did an interpretation study on these models and wrote a blog called Transformer Circuits.
On the Biology of a Large Language Model
Anthropic investigates the internal mechanisms used by Claude 3.5 Haiku through attribution graphs.
July 7, 2026 · 11:34 PM
Today Anthropic released new research on the fact that models have an inner thinking space just like humans.
New Anthropic research: A global workspace in language models
Anthropic shared new research exploring a global workspace in language models and how information becomes available for verbalization.
Verbalizable Representations Form a Global Workspace in Language Models
If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of cognition happens out of view.
This solidifies my thinking direction that:
- We need more people looking into the interpretation of these models
- We need a better output paradigm than next token prediction