šŗ šļø Watch: Opening AIās black box
The Neuron Ā· 2026-07-16
The Neuron interviews Goodfire CEO Eric Ho about mechanistic interpretability research, explaining how Goodfire extracts internal model featuresāincluding geometric representations and uncertainty signalsāto make AI model training more like deliberate engineering and less like opaque trial and error.
Appears in
Extraction
Topics: mechanistic-interpretabilityai-safetyneural-network-geometryhallucination-reductiongoodfire
Claims
- Goodfire can extract internal model representations of language, style, arithmetic, biology, and model uncertainty by analyzing neural network features and circuits.
- Researchers at Goodfire identified internal hallucination mechanisms and successfully trained models to avoid them by using those features directly as reinforcement-learning reward signals.
- Models appear to represent concepts as curved, high-dimensional geometric structuresānot flat linear encodingsāwith practical implications for how training and steering should be approached.
- Only a few hundred full-time industry researchers are currently working on interpretability, despite the field containing significant unexplored and potentially high-value research directions.
- Goodfire's platform Silico aims to enable 'intentional design'āshifting model training from guess-and-check toward a practice of inspection, targeted debugging, and deliberate improvement.
Key quotes
The answer is only the tip of the iceberg: A single token can emerge from an enormous hidden world of knowledge and computation the user never sees.
Goodfire is building tools that use AI to interpret AI. Its long-term goal is what Eric calls intentional design: training models less like mystery creatures fed enormous piles of data, and more like software you can inspect, debug, edit, and improve on purpose.
Eric says only a few hundred full-time industry researchers may be working on interpretability... the field is sitting on an enormous pile of low-hanging fruit.