Generating Interpretable Networks using Hypernetworks

The Big Picture

Imagine trying to understand how a calculator works by only looking at its output. No buttons, no display, just the answer. That’s roughly what researchers face when trying to decode how a neural network actually solves a problem. The field of mechanistic interpretability exists to crack open these networks and read the algorithm hidden inside.

There’s a catch: most existing work starts with a known algorithm and builds a network specifically designed to reproduce it. That guarantees interpretability, but it also guarantees you’ll never discover anything genuinely new.

A team from MIT (Isaac Liao, Ziming Liu, and Max Tegmark) decided to flip the script. Instead of asking “how do we build a network that implements this algorithm?”, they asked: “Can we build a system that generates interpretable networks, and then figure out what those algorithms are?” The answer was yes, and the algorithms it found included ones nobody had thought to look for.

Their tool is a hypernetwork: a neural network that designs the internal parameters of another, smaller neural network. By training this hypernetwork carefully, the team produced a family of tiny networks that all solve the same simple problem (computing the L1 norm, or sum of absolute values of a list of numbers) but through surprisingly different and previously unknown methods.

Hypernetworks can systematically explore the space of interpretable algorithms, discovering computational strategies that humans never thought to try but can still understand in hindsight.

How It Works

The researchers chose a deliberately simple test case: the L1 norm. It sounds almost trivial. But simplicity was the point. If you can’t fully understand how a network computes something this basic, what hope is there for complex tasks?

The target networks are compact two-layer MLPs (multi-layer perceptrons, a standard architecture where neurons are organized in stacked layers that pass information in sequence) with 16 input neurons, 48 hidden neurons, and a single output. They use the swish activation function, a smooth curve governing how strongly each neuron fires in response to its input. Conventional training produces networks that technically solve the problem but look like noise: no discernible structure, no interpretable pattern.

The hypernetwork changes the game in two ways. First, it generates weights with structured, repeating motifs that are the fingerprints of an underlying algorithm. Second, it includes a parameter β that controls model complexity by balancing how well the network fits data against a penalty for unnecessary complexity. High β means simpler networks; low β allows more complex ones. By sweeping β and collecting models across 33 random seeds, the team assembled a rich library for analysis.

To read the algorithms out of these networks, the team used force-directed graph drawings, a visualization technique that arranges neurons on a plane according to their connection strengths, making symmetries visible at a glance. Three distinct algorithms emerged:

• The double-sided algorithm: The expected one. Build an absolute value from two ReLU neurons, one firing for positive inputs and one for negative, then sum across all dimensions. Clean, familiar, unsurprising.
• The pudding algorithm: Not expected. This method exploits a mathematical identity involving a limit as a constant approaches infinity, squeezing the L1 computation through a different algebraic path. It comes in two signed variants (“positive pudding” and “negative pudding”) which look visually distinct in the force-directed drawings.
• The convexity algorithm: Also unexpected. This one uses the convexity properties of the absolute value function through a completely different computational route.

Only the double-sided algorithm was anticipated before running the experiments. The other two emerged purely from what the hypernetwork chose to do.

Why It Matters

The real significance here isn’t that a hypernetwork found two new algorithms for computing absolute values. It’s what this reveals about the space of interpretable algorithms. Neural networks, even tiny ones, don’t necessarily rediscover human-familiar methods. They find their own paths, paths that are sometimes describable once you see them but that humans simply hadn’t thought to explore.

This has direct implications for mechanistic interpretability. Researchers currently hunt for structure in networks that weren’t designed to be interpretable. The MIT team’s work suggests an alternative: use hypernetworks to systematically generate and catalogue interpretable algorithms, building a library that could guide future decoding efforts.

The team also showed that their trained hypernetwork generalizes to input dimensions it never saw during training, constructing correct models for vector sizes outside the training distribution. That kind of systematic generalization suggests the hypernetwork has learned something genuinely algorithmic, not just memorized specific configurations.

There’s a bigger question lurking here, one the team explicitly raises: do large language models, when they solve math problems, rediscover human algorithms or invent alien ones? The pudding and convexity algorithms hint that the latter is entirely possible, even for elementary computations. Understanding those alien algorithms, rather than forcing networks into human-shaped molds, may be the key to truly trustworthy AI.

By training a hypernetwork to generate interpretable networks, the MIT team discovered two entirely new algorithms for computing absolute values that nobody thought to look for. Neural networks may harbor a richer algorithmic imagination than we’ve given them credit for.