New Formal Methods for Reading Model Internals From Weights

Two research programs are independently pursuing methods to read behavioral properties of neural networks directly from their weights, without running the model on inputs.

Lucius Bushnaq's adversarial Parameter Decomposition (VPD) claims to decompose model weight matrices — including attention layers — into causally distinct subcomponents, and asserts production readiness on frontier models [1]. ARC's Jacob Hilton published mechanistic estimation, a cumulant-propagation approach achieving O(width⁻²) MSE scaling on random MLPs versus O(width⁻¹) for Monte Carlo sampling, framed explicitly as a base case toward detecting deceptive alignment [2].

Secondary coverage of the VPD work has appeared at Goodfire.ai [3][4], MATS Research [5], and via a LessWrong linkpost [6], extending its reach beyond the Alignment Forum, though no new substantive claims have emerged from that coverage.

If either approach matures to cover frontier trained models, it would enable safety audits that a deceptively aligned model cannot game by behaving well only on observed inputs — the core threat model behind deceptive alignment. VPD's adversarial ablation finding, if it holds, also implicitly challenges the faithfulness of a large body of prior interpretability work built on non-adversarial ablation methods.

A small cluster of AI safety researchers is converging on a shared intuition: the most robust way to audit a trained model may be to read its behavior directly from its weights, bypassing the need to run it on inputs at all. Two distinct technical programs are pursuing this goal, arriving from different directions and sitting at very different points on the maturity curve.

The parameter decomposition effort, announced in a linkpost by Lucius Bushnaq on May 5, 2026, introduces adversarial Parameter Decomposition (VPD) as the third iteration of a research line that previously produced Stochastic Parameter Decomposition (SPD) and Attribution-based Parameter Decomposition (APD) [1]. VPD breaks a model's weight matrices into subcomponents each implementing a small piece of the model's learned computation, such that any given input activates only a fraction of those subcomponents. The central technical claim is that VPD can decompose attention layers — a target that prior interpretability methods including sparse autoencoders (SAEs), transcoders, and cross-layer transcoders (CLTs) have historically failed to handle cleanly [1]. A secondary finding is that adversarial ablation — using worst-case rather than stochastic or zero ablations when assessing whether a subcomponent matters — is critical to identifying causally important nodes faithfully, and that its absence calls into question the faithfulness of subnetworks identified by the majority of prior methods in the literature. Bushnaq positions VPD as production-ready: 'the parameter decomposition approach is now more-or-less ready to be applied at scale to models people care about' [1].

Two days later, on May 7, ARC researcher Jacob Hilton published a complementary but independently motivated paper on mechanistic estimation [2]. Where VPD is an interpretability tool aimed at understanding what a model computes, mechanistic estimation is framed explicitly as a step toward catching deceptive alignment. The key idea is that a model which behaves well on all training inputs might still harbor dangerous capabilities that stochastic gradient descent never encounters. Mechanistic estimation sidesteps this by computing behavioral properties analytically from the weights using a cumulant propagation algorithm. For wide random ReLU MLPs with 4 hidden layers and width 256, ARC's algorithms outperform Monte Carlo sampling across FLOP budgets spanning seven orders of magnitude, with MSE scaling as O(width⁻²) versus O(width⁻¹) for Monte Carlo [2]. The longer-term vision — 'mechanistic training' — would run gradient descent on mechanistic estimates rather than sampled outputs, penalizing dangerous behaviors too rare to appear in finite training runs. Hilton is careful to frame the current result as a base case: the method applies to randomly initialized networks only, and extension to trained networks requires tracking higher-order statistical deviations that remain an open problem [2].

The two programs share the underlying premise that weights contain recoverable behavioral information, but diverge sharply in method, maturity, and immediate application. VPD is claimed ready for deployment on real models; mechanistic estimation is explicitly foundational. Neither paper cites the other, suggesting parallel rather than coordinated efforts. Secondary coverage of the VPD work has since appeared at Goodfire.ai [3][4], MATS Research [5], and via a LessWrong linkpost [6], indicating the paper is attracting attention beyond its original Alignment Forum venue, though no new substantive claims or external validations have yet emerged from that coverage.