Whenever a model hits a breakthrough score on a benchmark, the same complaint inevitably resurfaces: "People are just moving the goalposts. Yesterday it was chess, then Go, then coding, and now we say that's still not AGI."
I disagree. The goalpost hasn’t moved; people are just conflating accumulated skill with actual intelligence.
This is the exact premise François Chollet built the ARC-AGI benchmark upon: Intelligence is not crystallized performance on a known set of tasks; it is the sample efficiency with which an agent acquires new skills on unfamiliar problems.
When we examine how recent models tackle novel or complex challenges, a glaring structural pattern emerges that explains why we aren’t at AGI yet:
1. The Human Engineer is the "External Meta-Learner"
When a human encounters an unfamiliar problem in the real world:
- They observe and understand the core challenge.
- They identify their own skill deficiencies ("I don't know the theory/tools needed for this").
- They actively seek information, gather data, invent practice drills, and self-train.
- They return to the original problem with acquired mastery and solve it.
Now, look at how frontier models tackle novel benchmarks (like ARC or domain-specific reasoning):
- The model fails at the task.
- Human engineers analyze why it failed.
- Human engineers write specialized simulators, design curriculum pipelines, and generate massive synthetic datasets tailored to that task.
- The model runs gradient descent on human-curated training data and produces a high score.
The model didn't learn how to adapt; the human engineers did the meta-learning on its behalf. The AI remains a passive student executing SGD on scaffolding built by people.
2. The "AlphaGo Fallacy" on Repeat
We saw this with AlphaGo: trained on millions of games, yet it couldn't play Go on a 20x20 board without retraining.
Pre-training or post-training on months of synthetically generated task-specific environments to ace a benchmark is impressive engineering, but it is not autonomous adaptation. If achieving a high score requires an army of researchers spending months curating data specifically targeting the failure modes of that benchmark, you haven’t built a general learner—you’ve built an over-engineered specialist through external brute-force curriculum design.
3. What Does Real Meta-Learning / AGI Look Like?
True AGI cannot rely on an external team of OpenAI or Anthropic engineers to design its curriculum whenever it faces an out-of-distribution domain.
For a system to demonstrate genuine general intelligence, it must close the loop itself:
- Autonomous Diagnosis: Identifying its own blind spots when faced with an open-ended intellectual problem.
- Autonomous Curriculum Generation: Deciding what data or simulation it needs to bridge that gap, and generating/gathering that data itself.
- Self-Directed Skill Acquisition: Training and updating its own weights, context, or internal modules, then applying that new capability to the original problem.
We may see primitive versions of this through Recursive Self-Improvement (RSI)—where a model benchmarks itself, diagnoses weaknesses, and produces training datasets for its next iteration. But even that is initially confined to well-defined coding and math loops.
Conclusion
Current state-of-the-art models are exceptionally good at executing reasoning patterns within the bounds of what they have been optimized for. But they have zero capacity to reason about how they should learn.
Demanding that an AGI be able to autonomously diagnose, explore, and teach itself to solve unseen problems isn't "moving the goalposts." It was the definition of General Intelligence from day one.
Curious to hear your thoughts: Do you think in-context reinforcement learning / test-time compute can eventually bridge this gap, or do we need an entirely different architectural paradigm to achieve true autonomous meta-learning?
