In a sweeping new blog post, Dan Hendrycks, a leading AI safety researcher and director of the Center for AI Safety, makes a provocative case: it may be time to scale back our expectations for mechanistic interpretability. While recent months have seen calls to double down on the effort—notably from Anthropic CEO Dario Amodei—Hendrycks argues that the field's central promise has not held up to scrutiny.

Mechanistic interpretability aims to "reverse-engineer" AI models, mapping out the role of individual neurons or circuits to explain a model's behavior. It is, in essence, an attempt to dissect large models like machines: locate the wires, trace the connections, and understand how they think. But Hendrycks says this framing may be misguided, and worse, ineffective.

The Problem of Complexity

The article's critique is grounded in complexity science. Just as it's impossible to predict the weather by tracking every air molecule, Hendrycks argues, it may be futile to understand an advanced AI by trying to track a complete chain of thought. AI systems, like brains or ecosystems, are complex systems: they exhibit emergent behavior, context sensitivity, and distributed causality. The whole is more than the sum of its parts.

The piece draws on analogies from psychology and neuroscience. Humans often don't understand their own decisions—they rely on intuition, tacit knowledge, and subconscious processing. Why should we expect a deep learning model, orders of magnitude larger and trained on data far beyond any human, to be interpretable in a simple, linear fashion?

Hendrycks also highlights the problem of compression. The behaviors of today’s frontier models are encoded in weight files that can be hundreds of gigabytes or even terabytes in size. Expecting these behaviors to be distilled into short, human-comprehensible explanations may be like trying to summarize a novel using a single sentence. Worse, if that kind of compression were feasible, it might suggest we didn't need large models at all.

A Track Record of Disappointment

Hendrycks doesn't just critique the theory; he reviews the field's empirical track record. He cites numerous techniques that once promised interpretability but failed to deliver:

Feature visualizations showed compelling but inconsistent neuron activations.

Saliency maps do not seem to accurately capture what a trained model has learned or is paying attention to.

BERT neuron studies found that supposedly interpretable patterns disappeared on new datasets.

Sparse autoencoders (SAEs) struggled to compress activations in a meaningful or robust way.

In each case, initial enthusiasm was met with sobering results. Even DeepMind’s own efforts to apply interpretability to its 70B-parameter Chinchilla model yielded only partial insights, after months of labor, and those insights proved brittle when the task format shifted.

A Better Path: Top-Down Interpretability

Rather than continue to chase a neuron-by-neuron understanding, Hendrycks advocates for a shift to a "top-down" strategy. Drawing from fields like meteorology and biology, he argues we should focus on high-level patterns and emergent properties. This doesn’t mean giving up on understanding AI systems—it means studying them more like psychologists study behavior or physicists study fluid dynamics.

This perspective is the foundation of a new field Hendrycks and others are advancing: representation engineering (RepE). Rather than trying to explain how every neuron works, RepE examines distributed representations across many neurons and uses them to modify model behavior. In recent work, researchers have used RepE techniques to suppress harmful concepts, improve honesty, reduce susceptibility to adversarial attacks, and align model values—all without needing a mechanistic map of the model's internals.

In Hendrycks' view, this is reason for optimism. Even if we never achieve full transparency through mechanistic means, we can still make models safer, more controllable, and more aligned. Safety doesn't require total understanding. It requires leverage points—and RepE may offer them.

Neats vs. Scruffies

Part of what makes mechanistic interpretability so appealing, Hendrycks suggests, is aesthetic. Many researchers in the field are "neats"—a term from early AI history describing those who favor clean, formal theories over messy, heuristic-driven approaches. Hendrycks draws a contrast to the "scruffies," who embrace complexity and empirical messiness.

This debate isn't new. In fact, Leo Breiman's now-famous 2001 paper made a similar case: that the most powerful learning systems would be opaque and resistant to simple statistical explanation. Hendrycks suggests that interpretability researchers, consciously or not, may be drawn to the mechanistic vision because it fits their preferred intellectual style—not because it's the most promising path forward.

Conclusion: Change Expectations, Not Just Tools

Hendrycks is not calling for an end to all mechanistic interpretability research. But he argues that its prominence should be a function of its track record, not its conceptual elegance. And after more than a decade of high-investment, low-return research, he believes it’s time to shift priorities.

If we're going to pursue the dream of something like an "MRI for AI," we need to accept what real MRIs do: they show patterns, clusters, and correlations—not line-by-line explanations of thought. For AI, that may be good enough. It may, in fact, be the only thing that's tractable.

As the field continues to mature, Hendrycks’ essay is a reminder that interpretability is not just a technical problem. It's also a strategic and philosophical one. And if we want to understand how machines think, we might first have to rethink how we try to understand them.