Why is AI hard and Physics simple?

The Big Picture

Here’s a puzzle that should bother you more than it probably does: the same civilization that can predict the precise moment an electron changes energy levels, to eleven decimal places, cannot reliably explain why a neural network learned to recognize cats. We can split atoms and model the birth of the universe, but we mostly don’t know why deep learning works. Physics: comprehensible. Intelligence: mysterious. Why?

That’s the central question MIT theoretical physicist Daniel A. Roberts takes on in this provocative essay. His answer is not just philosophical. It’s a call to action. Roberts argues that the tools physicists have spent centuries developing, their instinct for finding hidden structure in apparent chaos, are exactly what AI needs. He identifies a single unifying principle linking both fields: sparsity, the idea that a small number of deep rules govern a vast range of phenomena, and that finding those rules is the whole game.

Key Insight: The same reason physics is tractable (nature is sparse, governed by a small number of rules) is why machine learning works at all. Exploiting sparsity is the secret shared by both fields.

How It Works

Start with why AI is hard. Roberts opens with the no-free-lunch theorem, a result from mathematical statistics that sounds like a death sentence for machine learning: if you average over all possible problems, no algorithm does better than random guessing.

Consider images. An n-pixel black-and-white image is one of 2ⁿ possibilities. The number of ways to assign labels to all those images is 2^(2ⁿ), doubly exponential. For just 9 pixels, that’s more possible labeling schemes than atoms in the observable universe. And yet humans learn. Deep learning works. Cats get recognized. The tension demands explanation.

The resolution is structure. Random labelings of random images are mathematically possible but humanly meaningless. Real tasks have deep regularities: cat images share features with other cat images, and labels correlate with those features. The actual functions describing meaningful phenomena occupy a tiny, structured corner of all possible functions. This is sparsity, and it’s the only reason generalization is possible at all.

Physics discovered this long ago. Effective field theory, one of the physicist’s most powerful tools, is sparsity in action: you don’t need to know every detail of a system to make accurate predictions. You identify the few relevant degrees of freedom and build your theory around them.

The Standard Model describes all known fundamental particles with roughly 20 parameters. Thermodynamics reduces the behavior of 10²³ molecules to temperature and pressure. Physics works because nature is sparse.

Machine learning is doing the same thing, whether it knows it or not:

• Training a neural network means fitting a model to data, exactly what physicists do when tuning parameters to match experiment
• Generalization works only when the target function has sparse structure the model can discover
• Deep learning succeeds in part because hierarchical architectures mirror the hierarchical structure of physical reality: edges compose into shapes, shapes into objects, objects into scenes

The gap is clear: physics has a mature theoretical framework for understanding when and why its models work. Machine learning mostly doesn’t, yet.

Why It Matters

Roberts is making a disciplinary argument, not just a mathematical one. He’s urging theoretical physicists to enter AI not as consultants, but as physicists, bringing their characteristic interplay between theory and experiment, their comfort with approximation and scaling, their nose for the right variables.

The physicist doesn’t demand a rigorous proof before engaging with a problem. She builds intuition, makes predictions, tests them against data, and refines. That cycle is what deep learning theory needs.

The stakes are real. AI systems are increasingly embedded in consequential decisions (scientific discovery, medical diagnosis, infrastructure) and we still lack a principled understanding of what makes them work or fail. Roberts points toward developing a genuine theoretical physics of deep learning: predictive, falsifiable, built from first principles the way a physicist would build it, with controlled approximations validated against real networks.

The connection to physics isn’t merely metaphorical. Techniques from statistical mechanics, including mean-field theory (approximating large systems by focusing on averages), the replica method (a mathematical tool for studying disordered systems), and the renormalization group (a framework for understanding how physical laws change across scales), are already finding direct application in understanding neural network training and generalization. The influence runs both ways: AI problems are reshaping how physicists think about complexity itself.

Bottom Line: AI lacks the hard-won insight that made physics tractable: the world is sparse. Roberts’ prescription, to bring physicists into AI to work as physicists rather than just as mathematicians, could be the missing piece in building a genuine science of intelligence.