Why Static Interpretability Fails on Multi-Step Agentic Decision Chains

1. What "static" actually means in mechanistic interpretability

Mechanistic interpretability (MI), as practiced over the last several years, is built almost entirely around a single unit of analysis: one forward pass, on one prompt, producing one output. The standard toolkit - activation patching, causal tracing, path patching, attribution patching, the logit lens - all operate inside that frame.

Activation patching (also called causal mediation analysis or interchange intervention) runs a model twice: once on a "clean" input that produces the correct output, once on a "corrupted" input that produces the wrong one, then splices a clean activation into the corrupted run at a chosen layer and token position to see whether the output recovers. Causal tracing, introduced for locating factual associations, extends this by measuring how the effect of a single patched activation propagates to intermediate layers before it reaches the output. Path patching generalizes the intervention from a single activation to a specific computational path through the network. Attribution patching approximates all of this with a linear, gradient-based estimate so it can scale past the cost of running one forward pass per component per prompt pair.

Every one of these methods shares a structural assumption worth stating precisely: causality is assumed to be local - input tokens directly shape the output within one pass, and the object being explained is a fixed computational graph evaluated once. This is not an incidental simplification; it is stated as a limitation directly in the literature. One recent paper applying sparse autoencoders to chain-of-thought reasoning notes that its activation patching "targets only the residual activation of the final token and does not trace causal effects through the reasoning process; this is fundamentally due to the static, snapshot-based nature of the [method], which is incompatible with token-level or path-level causal tracing" across a reasoning trajectory. That is the static paradigm being honest about its own boundary, in exactly the setting - chain-of-thought - that is the smallest possible step toward an agent.

2. Four ways the static frame breaks when the subject is an agent

2.1 The unit of analysis is wrong

Static MI explains a mapping from input tokens to output tokens within one context window. An agent is not one mapping - it is a *policy* applied repeatedly against a changing environment: plan, act, observe a tool result or a webpage or a test failure, update, act again. The object that needs explaining is not "what did this forward pass compute" but "why did this multi-step process end up here," and no single forward pass contains the answer, because the answer is distributed across passes separated by real-world observations the model did not generate and could not have predicted in advance.

2.2 Causality stops being local - temporal credit assignment becomes unavoidable

In a static setting, attribution methods can assume the causally relevant input is nearby: the token whose activation you patch is a few dozen or few hundred positions from the logit you're measuring. In a long-horizon agent trajectory, the decision that actually caused a failure often occurred many steps before the failure became visible. This is structurally the same problem reinforcement learning calls temporal credit assignment: distributing "blame" or "credit" across a long sequence of actions when only a sparse, delayed signal (task success or failure) is available at the end. Recent work on long-horizon interpretability names this explicitly as the reason static attribution cannot transfer: "in static contexts, attribution methods assume that most causality is local... but in long-horizon agentic loops, outcomes often depend on actions taken many steps earlier." Early decisions shape later success or failure in ways that only become legible once the trajectory is treated as the unit of analysis, not the token.

The reinforcement-learning literature on process reward models makes the same point from the training side rather than the interpretability side, which is useful corroboration from an independent research direction: outcome-only, trajectory-level reward "fails to quantify the behavioral contributions of individual steps," creates "ambiguous credit assignment," and leaves models "vulnerable to reward hacking" precisely because a trajectory can fail despite many individually correct intermediate decisions, or succeed despite flawed ones. If credit assignment across steps is hard enough that it has spawned its own subfield for *training* agents, there is no reason to expect a method built to explain a single pass can retroactively solve it for *interpreting* them.

2.3 Planning is a persistent internal state, not a property of one pass

Long-horizon tasks force a model to do more than produce locally plausible text - it has to commit to a strategy, revisit it as new evidence arrives, and sometimes abandon it. Static, single-pass interpretability can characterize the logits and attention pattern *for one edit or one step*, but it has no native way to answer the questions that actually determine whether an agent succeeds: which internal state supports planning several steps ahead, and when - at which specific step, in response to which specific piece of evidence - does the model revise or fail to revise its strategy. A coding agent can keep applying locally reasonable patches while remaining committed to an incorrect hypothesis about the underlying bug, even as new test failures contradict that hypothesis, and eventually loop without progress. Analyzing the activations for any single patch explains *what* the agent did at that step. It cannot explain *why* external, contradicting feedback stopped updating the agent's internal beliefs, because that question is only well-posed at the level of the trajectory, comparing internal state before and after the evidence arrived.

2.4 Failure cascades and structured failure modes are invisible below the trajectory level

Long tasks exhibit failure patterns - misinterpreting a tool's response, misreading a goal constraint, looping, dead ends - that compound over time and simply do not exist as objects in a single forward pass. You cannot observe a cascade by inspecting one link in the chain. Interpreting these failures requires detecting when internal representations lose coherence and how a model's latent trajectory diverges from what a successful run looks like - a comparison that is undefined without a notion of "the trajectory so far" as the unit under analysis.

3. Empirical confirmation: static trace-reading measurably underperforms dynamic, causal probing

The argument above is architectural. It is also directly measurable, and a controlled benchmark exists that isolates exactly this comparison rather than leaving it as intuition.

TraceElephant, a 2026 benchmark for failure attribution in LLM-based multi-agent systems, evaluates the same attribution task - identify which agent and which step caused a task failure - under two conditions. Static attribution gives a method the complete execution trace: every step's inputs, outputs, tool calls, and metadata, all at once, exactly analogous to reading a full static log or a full forward-pass activation cache after the fact. Dynamic attribution gives the method the same static trace *plus* a replayable execution environment, so it can re-run the system from a candidate failure point and issue counterfactual checks - actually intervening on the trajectory and observing whether the failure reproduces, rather than only reading what already happened.

The results are unambiguous in direction, if not overwhelming in magnitude. Averaged across three agent architectures (a dynamically-assembled team, a fixed-role orchestrated system, and a single-agent tool-centric coding scaffold) and evaluated with ground truth available, the best static method reached 65.9% agent-level accuracy and 30.3% step-level accuracy. The dynamic method - identical in every respect except for the addition of counterfactual replay - reached 66.7% and 33.3% respectively, a roughly 10% relative improvement specifically in step-level attribution, the finer-grained task of identifying *which exact decision* caused the failure rather than merely *which agent* was broadly responsible. The paper's own explanation for why the dynamic method's gain concentrates in step-level rather than agent-level accuracy is instructive: dynamic replay "actively verif[ies] candidate failure steps through controlled re-execution and counterfactual probing," which "filter[s] out spurious candidates identified from static traces." Static analysis alone generates the right *suspect list* most of the time; it takes an actual intervention - rerunning the trajectory from a candidate point and checking whether the failure still occurs - to confirm which suspect on that list is the actual cause.

A second, sharper result from the same benchmark reinforces the general argument in Section 2: step-level accuracy degrades far more than agent-level accuracy when information is removed from the static trace (76% relative accuracy drop for step-level attribution versus a 22% drop for agent-level attribution, when metadata and input fields are stripped from the trace). Identifying *roughly who was responsible* is comparatively robust to incomplete static information; identifying the *exact causal step* is not - which is precisely the distinction Section 2.2 draws between coarse trajectory-level accounting and genuine temporal credit assignment.

*(Figures are averaged across the benchmark's three agent architectures, ground-truth-available condition, best-performing static technique per row.)*

4. Why temporal order alone is also not enough - the symptom/root-cause confound

A natural first fix, once you accept that a single forward pass isn't the right unit, is to reach for temporal order: walk the trajectory step by step, or binary-search over steps, and find the *earliest point* at which something looks wrong. This is a real improvement over pure static, all-at-once analysis, but it turns out to have its own structural blind spot, independently identified by researchers building automated failure-tracing systems for multi-agent deployments.

GraphTracer, a 2026 framework built specifically to trace root causes in multi-turn, multi-agent search systems, states the problem with unusual precision: "traditional failure attribution approaches seek the earliest action in the temporal sequence whose correction leads to success... [but] this temporal view conflates the manifestation of failure with its underlying cause." The distinction matters because temporal order and causal dependency are not the same relation. A node can be a root cause even though it occurs *early* in the trajectory, with its damage only becoming visible much later through a chain of dependencies - and conversely, the step where a failure first becomes *observable* is frequently not the step that actually caused it, because an earlier, silent error can sit dormant until a downstream step happens to depend on the corrupted information.

The fix GraphTracer proposes is to stop treating "temporal order" and "causal order" as interchangeable and build the second one explicitly. It constructs an Information Dependency Graph (IDG): a directed acyclic graph where each node is a specific piece of information produced during execution, and a directed edge from node *i* to node *j* means node *j*'s content explicitly depends on node *i*'s output. This is a genuinely different object from a transcript or a log - a transcript only tells you *when* something was produced; the IDG tells you *what it actually depended on*, which can skip over long stretches of temporally-intervening steps that had no causal bearing on the failure at all. Root-cause localization then becomes a graph-traversal problem, not a search over positions in a sequence: a node is confirmed as a root cause by checking, counterfactually, whether correcting that node's information and re-propagating the correction through everything causally downstream of it actually resolves the failure - the same "intervene and observe" logic that made dynamic attribution outperform static attribution in Section 3, but now applied along dependency edges instead of naive chronological order. On the Who&When benchmark, this graph-structured, RL-trained tracer outperformed the prior temporal state of the art by up to 18 percentage points in attribution accuracy, and produced 4.8–14.2% measured performance improvements when integrated back into production multi-agent frameworks as a debugging tool - evidence that the causal reframing is not just more elegant, it is more useful.

A closely related independent line of work, agent-level contagion analysis in multi-agent reinforcement learning, arrives at a compatible picture through a different mechanism: it uses gradient-based sensitivity of each agent's policy to upstream agents' actions, aggregated over a moving window, to build directed contagion graphs that separate genuine originating causes ("Patient Zero") from agents that merely amplify or relay an error that started elsewhere. The paper's stated motivation for the two-stage design - detect a local policy anomaly, then trace *upstream* influence via causal derivatives rather than trusting temporal order - is explicitly to avoid flagging a downstream agent as the cause when it was in fact only the first place the damage became statistically visible.

5. What a temporal/causal interpretability method actually needs to do differently

Putting the failures in Sections 2–4 together yields a reasonably concrete specification for what replaces single-pass activation patching when the subject is an agent, rather than a leaving it as a vague appeal to "look at the whole trajectory."

5.1 The unit of analysis moves from the forward pass to the trajectory. Instead of a clean/corrupted pair of prompts, the object under study is an ordered sequence of steps, each with its own inputs, outputs, and - where accessible - internal activations, spanning real interactions with tools, environments, or other agents. Any method that only ever looks at one step at a time inherits every limitation in Section 2 by construction, regardless of how sophisticated its within-step analysis is.

5.2 Attribution has to be counterfactual and interventional across steps, not just observational within one. The TraceElephant results in Section 3 show concretely that reading the full trace (observational, static) reliably narrows the search but does not reliably confirm the cause; only replaying the trajectory from a candidate step and checking whether the failure reproduces (interventional, dynamic) closes that gap. A temporal interpretability method needs the equivalent of activation patching's clean/corrupted swap, but applied at the level of *replaying from a step with a corrected input* and observing whether the downstream trajectory recovers - the direct trajectory-level analogue of the token-level intervention that made static MI causal in the first place, rather than merely correlational.

5.3 Dependency, not chronology, is the graph that has to be built. Section 4's central lesson generalizes past any one benchmark: whatever structure a temporal method uses to reason about "what caused what" needs to be a genuine dependency graph - which step's output was actually read and used by which later step - rather than an implicit assumption that the causally relevant predecessor is whichever step happened to come immediately before. This is a real construction cost (GraphTracer builds it by extracting, at each step, which prior outputs a model reports having used, which requires either models that can report this reliably or an external tracer trained to infer it), but it is the piece of infrastructure that converts "the earliest place things looked wrong" into "the actual cause," which Section 4 shows are frequently different steps entirely.

5.4 Latent state needs a notion of "coherent trajectory" to diverge from. Detecting the failure cascades in Section 2.4 requires a reference - what does the internal representation trajectory look like when this class of task *succeeds* - so that a live trajectory's latent state can be compared against it and flagged as diverging before the divergence becomes visible in the output. This is a meaningfully different object than anything static MI produces, because static MI has no notion of "trajectory" to build a reference distribution over in the first place; it would need to be constructed the way process reward models construct step-level supervision - by sampling many trajectories, scoring their steps, and building the reference from what characterizes the steps that lead to success versus failure.

5.5 Full observability of inputs, not just outputs, is a precondition, not a nice-to-have. The ablation in Section 3's table is the sharpest evidence here: stripping input fields and metadata from an otherwise-complete trace collapses step-level accuracy by 76% relative to the full-trace condition, far more than it hurts agent-level accuracy. A temporal/causal method that only has access to what each step *produced*, without the context each step actually *received*, is working with something closer to the "output-only" setting that the same paper shows underperforms badly - which is a direct warning against building agent interpretability tooling around whatever a deployment platform happens to already log (typically outputs and tool-call results) rather than around what the attribution task actually needs (the full input context at every step).

6. Closing synthesis

The mismatch is not that static mechanistic interpretability is wrong, or that its results on single forward passes are unreliable - activation patching, causal tracing, and their attribution-based approximations remain the right tool for the question they were built to answer, which is how one computation, run once, produces one output. The mismatch is that an agent is not one computation run once. It is a policy applied repeatedly against a world that talks back, where the thing that needs explaining - a hypothesis the model refuses to abandon, a silent error that only becomes visible five steps later, a plan that was quietly wrong from step one - is a property of the *trajectory*, not of any single step examined in isolation. The empirical record so far, thin as it still is, points in a consistent direction: reading the whole static trace helps, correctly narrowing where to look; but only intervening on the trajectory - replaying it, correcting a candidate step, and checking whether the failure actually resolves - and only building genuine causal-dependency structure rather than trusting chronological order, closes the remaining gap between "here is roughly where it went wrong" and "here, provably, is what caused it." That combination - trajectory as the unit of analysis, counterfactual replay as the causal test, and dependency graphs instead of timelines as the object being reasoned over - is what a temporal/causal alternative to static interpretability concretely looks like, and it is being built, in pieces, by researchers who arrived at the same conclusion from RL credit assignment, from failure-attribution benchmarking, and from production multi-agent debugging, independently of one another.