Croesus, king of Lydia, was the richest man in the known world. He was so rich that his name became an expression. Even today we say “rich as Croesus.” Before making the most important decision of his life, he did something that should sound familiar to us. He went to get a second opinion from the thing that, in his time, most resembled an answer-giving machine: the Oracle of Delphi.
And he did something we would still consider sensible today. He didn’t trust blindly. First, he tested.
He sent messengers to the great oracles of the world, all with the same agreed-upon question, to see which one answered correctly. Only Delphi got it right. Convinced he had a reliable source in front of him, Croesus then asked the question that actually mattered to him: should he attack Cyrus’s Persian empire?
The answer went down in history. If he attacked the Persians, the oracle said, Croesus would destroy a great empire.
He heard what he wanted to hear. He gathered his army, crossed the border, and went to war convinced of victory. The great empire he destroyed was his own. Cyrus defeated him, pursued him to Sardis, the capital, and took the city in 546 BC. Lydia was finished. Croesus ended up a prisoner.
Later, as a captive, Croesus sent word to complain to the oracle for having deceived him. The answer he received is at the center of everything I want to say here. The oracle hadn’t deceived him. The prophecy was right, a great empire really was destroyed. Croesus should have asked which empire, and should have asked again before marching. The error was never in the oracle’s answer, it was in Croesus’s question.
Notice how uncomfortable this is. Croesus did everything we’d today call the right thing. He was skeptical. He tested the source. He confirmed it was reliable. And he still fell, because the reliability of the answer was never the problem. The problem was that he didn’t know there was a better question waiting to be asked.
The Blind Spot Is in the Question
There’s an appealing idea that’s becoming more and more common. Give people time and freedom, let each one ask AI for whatever they need, and let them learn along the way, as they build. There’s a lot of truth to it. You learn a huge amount this way, and fast.
But it has a blind spot, and it’s exactly Croesus’s blind spot.
An AI model teaches well whatever you ask it. What it doesn’t do, by default, is reveal to you what you don’t yet know you should be asking. It answers the literal question it receives. If your question doesn’t reach what actually matters, the model doesn’t know it should have, because it isn’t evaluating your competence, it’s responding to the sentence you wrote.
A human mentor, even a bad one, works differently. When you ask a question that reveals a gap, they notice the gap. They might interrupt you and say “wait, before that there’s something you need to understand.” Your naive question gives you away, and an attentive human uses that to redirect you. The model doesn’t. Your naive question gets a competent answer, and you move forward convinced you’re on the right track.
This isn’t a technical flaw in the LLM that the next version will fix. It’s structural to how the interaction works. And it helps explain why exploring freely, alone or in a group, might not solve the problem. It can even amplify it, because a group of people exploring together tends not to discover precisely the thing none of them knew they should be looking for.
The Website That Looked Perfect
Let me give you an example of my own, recent and concrete.
I asked an AI to build me a website. The result was excellent at first glance. Beautiful, functional, done in a fraction of the time it would have cost me. It used to easily take me two weeks to do it, and now it took twenty minutes. The productivity gain is astonishing.
Only later did I realize the problem. All the content had been inserted directly into the WordPress theme, not as blocks or editable text. The LLM coded a theme and asked me to upload it, and so I did. In practice, the site was almost impossible to edit afterward through the normal route. To change a simple paragraph, I had to go dig into the theme’s code.
The AI did exactly what I asked, but I hadn’t asked it the right thing. I didn’t ask “will this stay editable in the CMS afterward?”, and I didn’t ask because I didn’t even know there was a decision to be made there. It’s a question only someone who has already built websites before, and already been burned by this, thinks to ask.
And that’s when I realized something that feels central to me. The greater the AI’s ability to execute, the greater the importance becomes of the knowledge needed to validate what it produces. The machine got better at doing, and that didn’t reduce what I need to know, it increased it.
The Lego Car
There’s an image that explains this better than any theory, and it comes from a box of Lego.
Imagine you decide to build a Lego set without the instruction book, purely by exploration. The question that matters isn’t whether you can. It’s which Lego.
If it’s a Lego Duplo car, the kind for small children, with four or five enormous pieces, you can build it with your eyes closed. There’s nothing you could not know. The piece fits one way and that’s it. Exploring is more than enough.
If it’s a Lego City car, with a few dozen pieces, you can still manage. By trial and error, looking at the picture on the box, you end up with a car that holds together and looks like a car. It might not come out perfect, but it’s good enough. Exploration still covers the problem.
Now imagine a Lego Technic. One of these advanced models, like the replica of the Peugeot 9X8 from Le Mans, has around one thousand seven hundred and seventy-five pieces. And they’re not just pieces. Inside there’s a V6 engine with pistons that move when you push the car, a steering system that links the wheel to the front wheels, independent suspension on all four wheels, a differential on the rear axle.
Try building this without instructions and having never done anything like it before. You won’t manage. And the reason runs deeper than “it’s hard.” It’s that you don’t even know those things exist to be built. You don’t know what a differential is, or that it’s needed, or where it goes. You don’t know that the steering has a geometry that has to line up correctly. The sub-assemblies that make the model work are invisible to anyone who has never come across them. The instruction book does more than speed you up. It’s what tells you that those pieces of the problem even exist.
And here’s the part that really interests me. Whoever has already built a piston engine in Lego before might be able to build the Technic without instructions, through exploration. Not because they’re smarter, but because they already have the map of the invisible pieces in their head. Exploration works for them precisely because they’re no longer exploring blind.
This is the difference that worries me when people talk about learning AI purely through exploration. For simple tasks, the Duplo and City kind, exploring is enough. For complex tasks, the Technic kind, exploring without a foundation doesn’t get you there, because it doesn’t show you the pieces of the problem you don’t know exist. Exploration is enough for the simple, but for the complex, without a foundation, it isn’t.
Two Ways to Learn, and What Each One Solves
It’s worth separating two things that often get confused.
One is learning by building and experimenting. You define what you want, ask the machine, adjust along the way. It’s fast, it’s motivating, and it develops real skills in whatever you touched.
The other is acquiring structured knowledge before building. Studying the capabilities and limits, understanding the shape of the problem, knowing what tends to go wrong. It’s slower and less exciting, and feels like a waste of time, until the day it saves you from a mistake you didn’t even know you were making.
Both are legitimate, and they solve different problems. But only the second protects you against “I didn’t know I had to ask that.” The first, alone, always leaves you at the mercy of your own field of view.
I recently wrote about a related idea. As producing gets cheap, the value shifts to judgment, to knowing what’s worth doing and what gets left out. This piece is the other side of the same coin. In that one, the question was judging what the machine produces; in this one, it’s knowing what to ask it. Both live in the same place: the knowledge that lets you see what isn’t right in front of your eyes.
What Endures Is Reasoning
There’s good news in all of this, and it’s why I’m not a pessimist.
The tools will keep changing fast. Today’s models will be obsolete within a year. If what you learned was only how to operate one specific tool, you age along with it. But there’s one skill that doesn’t age, and that’s reasoning. Understanding the problem, projecting it, planning, structuring, evaluating, deciding. That’s what lets you pick up whatever tool exists at any given moment and, more importantly, know what to ask it.
Using AI well looks less and less like mastering a program and more and more like delegating work well to someone. Understanding what you want, giving the right context, deciding what to hand off and what to hold onto, following up, and critically evaluating what comes back. Whoever knows how to do this with people starts with an advantage. And whoever doesn’t know the problem still doesn’t know they should be asking about it.
The Question Before the Journey
Croesus had the best AI in the ancient world in front of him. A source he himself had tested and confirmed to be reliable. The answer he received was correct. He lost the empire, the crown, and his freedom all the same, because what he lacked was never on the machine’s side. It was on his, in the question he didn’t know how to ask.
AI will always give you an answer to what you asked. Fast, competent, convincing. What it doesn’t do is warn you when your question is the wrong one, or when there’s an entire question that slipped past you.
Exploring teaches you to ask better questions within what you already know. It doesn’t teach you what’s outside your field of view, because that, by definition, doesn’t occur to you to look for. For that, you still have to study the map before you set out.
Croesus didn’t study the map. He destroyed a great empire.
Sources
The story of Croesus and the Oracle of Delphi comes from Herodotus, Histories, Book 1. The decisive detail, the oracle’s answer that Croesus should have asked which empire and consulted again, is in Herodotus 1.91. For a reliable introduction: Croesus (Encyclopaedia Britannica), Croesus (World History Encyclopedia) and Herodotus and the invention of history (The Open University).
On Lego: an example of functioning engines built in Lego, a simple car to build, and the review of the Lego Technic 42156 Peugeot 9X8, with a V6 engine, steering, independent suspension, and a differential, around 1775 pieces.