Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models

The Big Picture

Imagine trying to understand why a car engine makes a strange noise by looking only at the whole engine block, never at individual pistons, valves, or fuel injectors. That’s what AI interpretability researchers have been doing for years: explaining why a language model behaves a certain way by pointing to broad internal components with names like “attention head 7” or “layer 4.” These components are like engine blocks. Too big, too tangled, and frustratingly vague.

The deeper problem is polysemanticity: individual processing units inside AI language models respond to dozens of unrelated concepts at the same time. A single unit might activate for “banana,” “yellow objects,” and “tropical countries” all at once. Trying to explain a model’s behavior using these blurry units is like describing a symphony by pointing at sections of the orchestra. Technically accurate, but missing everything that matters.

A team from Northeastern University, MIT, and the Technion built a sharper tool. Their sparse feature circuits, published at ICLR 2025, reveal the fine-grained, human-interpretable causal pathways actually responsible for specific model behaviors. They also built a pipeline that discovers thousands of these circuits automatically.

Key Insight: By replacing blurry processing units with sharp, meaningful “features” learned by a specially trained neural network, researchers can now trace exactly which concepts cause a language model to make a specific prediction, and surgically edit those pathways.

How It Works

The method rests on two technical pillars.

The first is sparse autoencoders (SAEs), neural networks trained to decompose a model’s internal representations into interpretable building blocks. When a language model processes text, each layer produces a stream of numbers encoding what the model “knows” at that point. SAEs re-express those numbers as a combination of features, each corresponding to a single recognizable concept: “the word ‘doctor’ in a medical context” or “a plural subject in a relative clause.” The researchers trained SAEs for every sublayer of Pythia-70M and used the publicly available Gemma Scope SAEs for Gemma-2-2B.

The second pillar is attribution patching, a fast mathematical technique for estimating how much each feature causally contributes to a specific model output. Given a “clean” input (“The teacher…”) and a “patch” input (“The teachers…”), attribution patching estimates how much each feature’s difference shifts the model’s probability of outputting “is” versus “are.” Because the technique scores thousands of features in parallel, it scales.

Combining both pillars, the pipeline works like this:

1. Define a behavior, for example, subject-verb agreement on sentences with an intervening noun phrase.
2. Run SAEs on every sublayer to decompose activations into sparse features.
3. Score features using attribution patching to find which ones causally influence that behavior.
4. Threshold and connect: keep only the most important features and trace how they communicate across layers, forming a directed graph.

The result is a sparse feature circuit: a graph where nodes are interpretable features and edges represent causal influence.

One honest touch: the SAE error term (the gap between what the SAE captured and what the model actually computed) gets its own node in the circuit. Researchers can see exactly how much of the model’s behavior is explained by interpretable features versus unexplained residuals.

Why It Matters

The most direct application is SHIFT (Sparse Human-Interpretable Feature Trimming), a technique for removing bias from AI classifiers by eliminating irrelevant features. The team tested it on a worst-case scenario: a model trained to predict profession where gender perfectly predicts the correct answer in training data.

Standard bias-removal methods require carefully curated examples where gender and profession are independent. SHIFT doesn’t. A human examines the discovered circuit, identifies features clearly related to gender rather than profession, and removes them. The model’s spurious gender sensitivity drops without any special data.

Beyond debiasing, the team built a fully unsupervised pipeline that discovers thousands of narrow language model behaviors (“predicting ‘to’ as an infinitive object,” “predicting commas in dates”) using clustering, then automatically computes sparse feature circuits for all of them. Results are publicly browsable at feature-circuits.xyz. That scalability matters most here: interpretability no longer has to be a hand-crafted, one-behavior-at-a-time endeavor.

Open questions remain. Better SAEs will shrink the unexplained residual. Attribution patching is an approximation, so complex interactions between multiple features could be missed. And the method has so far been validated on smaller models; scaling to frontier systems is still ahead.

Bottom Line: Sparse feature circuits give researchers a genuinely interpretable, causally grounded window into language model behavior, and a practical tool for editing that behavior without needing carefully curated datasets.