The bitter lesson of AI
And the sweetness of humanity
The Bitter Lesson is a commonly cited text in AI circles.
It’s core thesis is that trying to plug gaps in AI model performance with human knowledge only ever produces short-term gains, which are gradually eradicated as compute gets cheaper and more powerful.
It gives the example of chess. Originally researchers tried to programme rules and heuristics derived from human play. The methods of Deep Blue, which beat Kasparov in 1997, were based on massive, deep computational search.
In other words, the brute force of raw computer power ended up beating the elegance of human-based chess knowledge and strategy.
When it comes to AI, and in particular AI-based startups, we’re seeing the bitter lesson play out in real time.
Areas of human knowledge, previously thought to be uniquely human or layered on top of AI to create startups, are being swallowed by scale based compute.
AI is hitting 80% on SWE Bench tests.
Gemini can now ingest and transcribe historical documents as well as any proficient human.
AI-based legal startups are finding their lunches are being eaten by foundation models.
In essence, we’re finding that so much of what we value as being uniquely human could well be consumed by the scaling laws of AI.
Which begs the question, what remains?
Is anything safe?
The real question is - what does it mean to be human?
I don’t believe humans are meant to be bound to screens at a desk writing boilerplate code for half of their waking hours.
I don’t believe humans are meant to sift through reams of legal documents searching for specific information.
I don’t believe humans are meant to painstakingly transcribe hundred year old documents into a digital format.
I do believe we’re meant to create.
I do believe we’re meant to inquire into the nature of truth.
I do believe we’re meant to understand ourselves, our ancestors, and the wider story of how and why we came to be here.
AI will force us to do what only humans can do, to be what only humans can be.
For too long we have cosplayed as cogs in machine-like systems.
It is time to remember who we are, to free our spirits and embrace the unknown.
The bitter lesson is a reminder: a call to reclaim the divine sweetness of humanity.
Chess is a really interesting piece. It's true that DeepBlue beat Kasparov through its computational brute force calculation power; but the game tree in chess is still too deep and too wide to really compute it all the way through to the end. It relied on heuristics for which branches to explore and which branches to ignore. And those were absolutely created by human chess experts: They programmed in that a queen is worth more than rook, and countless other heuristics related to development, space advantage, piece mobility, king safety etc
And THEN came AlphaZero Chess, where nothing other than the rules of the game itself were programmed into it. All by itself thorugh reinforcement learning it learned about the relative value of pieces, but in a much more subtle way than a strict heuristic. Chess experts agree that AlphaZero Chess plays much more creative, mind-blowing moves than the traditional brute-force heuristic engines.