# The Stream, Its Writers, and the Assumption in Appendix E URL: https://www.msuiche.com/posts/the-stream-its-writers-and-the-assumption-in-appendix-e/ Date: 2026-09-11 Author: Matt Suiche Tags: Activation Steering, Abliteration, Control Vectors, GLP, Refusal Direction, LoRA, Weightless, Mechanistic Interpretability > Arditi et al. proved inference-time directional ablation and weight orthogonalization equivalent, under one assumption most readers skip: the stream must arrive clean, which puts the embedding matrix in the edit set. A baked adapter that leaves embeddings alone inherits exactly that gap — it projects each layer's new contributions, while a runtime hook projects the accumulated stream. The linearity argument, the measured 71.9-vs-6.2 writer asymmetry, and where the single-direction claim survives contact with MoE, looped models, and a cos −0.32 transfer. --- [Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717) (Arditi et al., 2024) gives the field two ways to erase the refusal direction. The first is directional ablation at inference (§2.4, §3.1): during the forward pass, zero out the component along $\hat{r}$ from every residual-stream activation, $$x \leftarrow x - (x \cdot \hat{r}) \hat{r}$$ The second is weight orthogonalization (§4): rewrite every matrix that writes into the stream so the model can never produce that component in the first place. Appendix E proves the two equivalent. The proof opens with an assumption most summaries skip: > Supposing that directional ablation was similarly applied after all > previous contributions to the residual stream, we have that > $\hat{r}^{\intercal} x_{\text{pre}} = 0$. Everything the stream already carries must arrive clean. That one line decides what you have to edit, and it is where the two forms of "abliteration" in circulation today quietly diverge. ## What the assumption costs A transformer's residual stream is a running sum: embeddings at the start, then every layer adds its attention and MLP contributions, $$x_{\text{post}} = x_{\text{pre}} + \sum_i W_i a_i$$ The matrices $W_i$ — `o_proj`, `down_proj`, their hybrid-arch cousins — are the *writers*. The accumulated $x$ is the *stream*. No single weight matrix sits behind the stream; the writers each have a concrete, editable $W$. Two vocabularies collide in that sentence, and both are worth pinning down. First, the names: full self-attention calls its output matrix `o_proj`, linear attention calls the same role `out_proj`, and the MLP calls its own `down_proj` — one concept, three dialects, and hybrid models speak two of them at once (Qwen3.8-27B has 16 full-attention layers with `self_attn.o_proj` and 47 linear-attention layers with `linear_attn.out_proj`). Second, "projection" itself: the output projections are learned matrices that map between spaces, while the steering operation $h \leftarrow h - \alpha(h\cdot\hat{d})\hat{d}$ is the geometric act of measuring and removing a component along $\hat{d}$, computed per token from the live activation. The writers are the matrices. The steering is the operation. Baking folds the second into the first. For Appendix E's assumption to hold, $\hat{r}^{\intercal} x_{\text{pre}}$ must vanish before each writer fires. Arditi et al. buy that by orthogonalizing **the embedding matrix, the positional embedding, the attention outs, the MLP outs, and the output biases**. The embedding term does the heavy lifting: strip the $\hat{r}$-component at embed time, from every token, and the stream enters layer 1 already clean. Skip it, and the stream arrives at each layer carrying a component no writer edit can reach — the equivalence breaks exactly there. ![The residual stream with its writers, and the three interventions: the runtime GLP hook projects the accumulated stream; the baked LoRA edits each writer; Arditi et al.'s orthogonalization additionally rewrites the embedding, which is what their Appendix E equivalence requires.](./images/stream-vs-writers.svg) The price of their completeness: the embedding edit touches every token, including tokens that would never trip a refusal. The component is gone at embed time, permanently, for benign and harmful text alike. ## What a bake edits instead Our `captain-vector bake` takes the same math to a different stopping point. At a writer $h = Wx$, the projection is a weight edit $\Delta W = -\alpha \hat{d}(\hat{d}^{\intercal} W)$, which is exactly a rank-1 LoRA with `lora_B = d̂`, `lora_A = −α·d̂ᵀW`. Bake emits that adapter for the per-layer writers and leaves the embeddings alone. Linearity does the rest: $$\sum_i \big(W_i x_i - \alpha(\hat{d}\cdot W_i x_i)\hat{d}\big) = \Big(\sum_i W_i x_i\Big) - \alpha\Big(\hat{d}\cdot\sum_i W_i x_i\Big)\hat{d}$$ Projecting every writer of a layer at α equals projecting that layer's *new contributions* at α. The component the stream carried in — from the embeddings, or from any writer outside the edit set — passes through, where the runtime hook removes it regardless of origin. The baked form and the runtime form coincide when the writers re-inject the direction every layer, which is the regime these models live in: on the Qwen3.8 hybrid, steering the MLP writer moves delivery 71.9 points while the attention writer moves 6.2. The direction is written, layer after layer, mostly by the FFN. Close, measurable, and *not* bit-identical — which is why bake ships as a troubleshooting and interop form (validate that a direction lands, probe a merge for survival) while the hotfixes steer the stream at runtime. The paper's own [blog post](https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction) describes the inference-time version per-writer ("every time a component writes its output to the residual stream, we can erase its contribution"). Appendix E is the reconciliation: writer-level erasure, applied after *every* contribution including the embedding, coincides with stream-level ablation and with weight orthogonalization that includes the embedding. One operation, three surfaces, provided the embeddings are in the edit set. ## Counting the writers A dense layer has two writers: attention's `o_proj` and the MLP's `down_proj`. An MoE layer swaps the single MLP for a router plus N experts, and each expert carries its own output projection — DeepSeek V4 runs 256 routed experts per MoE layer, so one layer there has roughly 258 writers and the model has thousands. The math survives the headcount. $\Delta W_e = -\alpha\hat{d}(\hat{d}^{\intercal} W_e)$ is exact for every expert, and linearity covers the router itself: only the top-k experts fire per token, but a router-weighted sum of cleaned outputs is already clean. What breaks is everything around the math — thousands of rank-1 adapters computed against per-expert quantized weights, no serving machinery that attaches LoRA to routed expert matrices inside a fused MoE kernel, and a fail-closed verification burden multiplied by the same thousands. The runtime hook's cost scales with none of it: it sits after the router-weighted sum, where the hundreds of writers have already collapsed into one tensor, and projects once. That asymmetry — bake's cost grows with the writer count, the hook's does not — is the concrete content of "architecture-blind." ## The three forms, side by side | | runtime projection (GLP native) | weight orthogonalization (Arditi et al.) | baked LoRA (`bake`) | |---|---|---|---| | edits | the accumulated stream, live | every writer **and** the embeddings | per-layer writers only | | stream's incoming component | removed, any origin | never forms (embeddings cleaned) | passes through | | α | tunable at serve time | frozen into the weights | frozen into `lora_A` | | MoE | architecture-blind | per-expert explosion | per-expert explosion | | checkpoint binding | none (width check only) | bakes `W` implicitly | `lora_A` carries `W`; pinned revision enforced | | failure mode on drift | hook refuses (spec gates) | silent | silent — bake fails closed upstream instead | One more row's worth of asymmetry: the runtime hook and the spec's reader-conformance gates fail loud, while any weight-space form fails silent, so the weight-space forms carry their verification with them — the pinned-revision check in bake, the per-writer roundtrip in `bake-report.json`. The table's three rows are all projection. The paper itself ran the addition column, in Appendix I.1: adding $-\hat{r}$ at a single layer bypasses refusal about as well as directional ablation (their Figure 23 against Figure 1), and pays for it in increased CE loss over harmless data (Table 9). Their Figure 22 shows the mechanism — a constant negative shift moves harmful activations toward the harmless cluster and pushes the harmless cluster off-distribution, where ablation moves the harmful points and leaves the harmless ones where they are. Addition is input-independent; the same shove lands on tokens that carry none of the component. That asymmetry, measured by the original authors a year before the llama.cpp control-vector ecosystem standardized on addition, is the one our own add-vs-project measurements keep reproducing. ## Where "a single direction" meets our measurements The claim the paper is named for has now survived two years of everyone else's experiments, most of ours included, with caveats that keep growing. Our current scoreboard: - **Transfer at cos −0.32 (DSV4).** A third-party direction recovered from an abliterated DeepSeek V4 checkpoint delivers 31/32 on refusal32 despite sitting at cosine −0.32 to our derived direction — closer to orthogonal than aligned. Two directions sharing that little orientation should not both erase the same behavior under a strict rank-1 theory. They do. Refusal reads as a low-rank subspace whose dominant component does most of the work, which is why rank-1 per-layer vectors carry the engineering while the GLP format reserves rank-k. - **The site matters more than the slogan.** If one direction lived in "the stream" as a flat object, the hook point would be bookkeeping. The 71.9-vs-6.2 writer asymmetry above says otherwise, and on DSV4 the calibrated site is the pre-fold FFN write (`ffn_out_pre_residual`), a label we carried wrong for months before a shape probe corrected it. - **Dose is model-specific, sometimes non-monotonic.** Inkling garbles at α=1.0 and ships at 0.25; Hy4's collateral damage is *worst* at α=1.0–1.5 and zero at 2.0; GLM-5.3-Flash falls off a cliff at 2.5; the 743B flagship refuses *more* above α=1. Four dose curves, four entanglement structures, one axis label. - **Removing the gate reveals the doorman.** After the dominant direction goes, residual behaviors remain — hedging, disclaimers, premise-rejection near the decision boundary — and that residue has its own derivable direction (the [GLP-63 work](https://huggingface.co/msuiche/Qwen3.8-27B-hedging-GLP-63-L1-63-a0.5), covered in [the membership-vs-mass post](/posts/2026-09-09-glp-gcd-membership-vs-mass/)). One behavior, at least two orthogonal axes. - **Looped models re-decide per pass.** Same-layer directions across Nanbeige4.2's two loop passes agree at only cos 0.25–0.6, so the vector ships 44 directions, one per execution step ([the looped-model post](/posts/abliterating-a-loop-control-vectors-meet-the-looped-transformer/)). "The direction" is per *execution context*, not per weight tensor. The follow-up literature landed in the same place from different directions. Zhao et al. show harmfulness and refusal are encoded separately, at different token positions ([arXiv 2507.11878](https://arxiv.org/abs/2507.11878)) — two concepts the single "refusal direction" label conflates. Piras et al. model refusal as a manifold of closely related directions and suppress it better with several directions than with any single one ([arXiv 2511.08379](https://arxiv.org/abs/2511.08379)). The numeral in the original title keeps shrinking under replication; the linear accessibility underneath it keeps holding. None of this dents the paper's real contribution, which runs deeper than the numeral: refusal is *linearly accessible* — causal in both directions, derivable from 64 prompts, erasable by a rank-1 edit. That is the load-bearing finding, and every artifact we ship stands on it. Our operating estimate, after all the lanes above: a rank-1 treatment of the dominant direction carries roughly ninety percent of the behavior, and the residual ten percent is where the interesting things live — the hedging axis, the transfer anomalies, the dose curves. The "single" in the title names the dominant eigenvector of something bigger, and the paper says as much in its own limitations section. ## Pointers for the reader The inference-time form: paper §2.4 (definition, Equation 4) and §3.1 (results, Figure 1); blog post sections "Ablating the 'refusal direction' to bypass refusal" and "Feature ablation via weight orthogonalization". The equivalence: paper §4 and Appendix E. Activation addition compared honestly (approximately as effective at bypassing refusal, measurably worse on harmless-data loss): Appendix I.1. The code: [andyrdt/refusal_direction](https://github.com/andyrdt/refusal_direction). Our forms: the [GLP spec](https://github.com/msuiche/weightless/blob/main/spec/GLP.md), [captain-vector](https://github.com/msuiche/weightless/tree/main/tools/captain-vector) for derive/inspect/bake, and the [llama.cpp compatibility notes](https://github.com/msuiche/weightless/blob/main/docs/llama-cpp-compat.md) for what happens when a projective vector meets an additive runtime.