The Bitter Lesson By incompleteideas.net
The key lesson from 70 years of AI research is that leveraging computation through general methods is the most effective approach. Many AI researchers initially focused on human knowledge, but the true progress comes from scaling computation through search and learning techniques. Embracing the power of general-purpose methods over human-centric approaches is crucial for long-term success in AI research.
Highlights
The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.
There are psychological commitments to investment in one approach or the other.
We have to learn the bitter lesson that building in how we think we think does not work in the long run.
breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning
The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex
We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.