Testing J-Lens on Qwen3-4B: A Near-Zero Result Was Not Enough

TL;DR. I tested whether writing out reasoning makes Qwen less dependent on information stored inside the model. The study did not establish this. The method affected a simpler factual task, but the math results could not tell whether the underlying mechanism was absent or the interpretability tool had missed it. A direct causal validation is needed before near-zero results can support a strong conclusion.

I began with a simple calculation: (2 + 4) × 3. If Qwen3-4B writes 2 + 4 = 6 before giving the final answer, the written 6 can act as external memory. This is the idea behind explicit chain-of-thought (CoT): an intermediate result in the context may reduce what the model must retain in its activations.

An Anthropic study, Verbalizable Representations Form a Global Workspace in Language Models, introduced the Jacobian lens, or J-Lens. J-Lens reads token-indexed directions from intermediate activations. The authors removed highly ranked directions, an intervention they call J-space ablation, and found that it hurt direct math answers much more than explicit CoT. I tested whether that result transferred to Qwen3-4B.

The first comparison was inconclusive, and the follow-ups became a test of construct validity: did the J-Lens readout and the ablation target the arithmetic representation named by the hypothesis? The public lens was fitted on activations from the same model name, Qwen/Qwen3-4B, rather than transferred from another model. Yet known arithmetic intermediates appeared in its top-ranked readout less often than known intermediate facts did. Larger tests then measured near-zero accuracy effects without resolving whether the intervention had reached the contextual arithmetic representation. Methods, prompts, scoring, controls, and study timing are in Section 5.

The Hypothesis and Its Direct Test

Why Written Chain-of-Thought Might Help

Written CoT externalizes an intermediate result into the context, where later tokens can attend to it. It may therefore serve as memory rather than merely displaying a solution.

At each token position, successive transformer layers read and update a shared vector called the residual-stream activation. J-Lens maps that activation to a ranked vocabulary-token readout. Each J-Lens vector is a token-indexed direction in residual-stream space, such as a direction indexed by Mexico or 6.

The source paper defines J-space using sparse combinations of J-Lens vectors. Its ablation procedure selects the ten highest-ranked eligible vectors at each position and projects the activation away from their span across a band of layers. J-Lens produces the readout; J-space ablation is the intervention. A token-indexed vector for 6 is not automatically the contextual representation of the value six in a particular calculation. That distinction became central to the follow-up experiments.

Following the source paper, I excluded from ablation any J-Lens vector whose token was already among the unablated model’s ten most likely next-token predictions at that position. The rule was intended to avoid directly suppressing a token the model was about to write.

On GSM8K, the source result differed sharply between direct answers and explicit CoT:

These are Claude Sonnet 4.5 scores after ablation divided by the unablated score for the same answer form, not raw accuracies. The public figure does not provide the underlying correct-answer counts. The authors interpreted the 13-point gap between the two normalized losses as evidence that CoT reduced dependence on an internal workspace.

I used the post-trained Qwen/Qwen3-4B and a public lens fitted on activations from the same model name using WikiText. The early experiments used layers 23–31, selected based on Qwen readout measurements; later validation used layers 19–28, a proportional mapping of the source paper’s medium range. Section 5.1 records the exact checkpoint, intervention, layer choices, and timing.

The transfer prediction was straightforward: J-space ablation should cause a larger accuracy drop for direct answers than for explicit CoT.

The First Comparison Was Inconclusive

I ran the same 150 GSM8K questions in four conditions: direct answers and explicit CoT, each with an unablated baseline and J-space ablation. The direct condition forced an immediate boxed answer with a short assistant prefill; the CoT condition requested a written solution. Section 5.2 gives the exact prompts and scoring rules.

Because the two prompts produced very different baseline accuracies, I compared the ablation effect within each answer form. I define accuracy drop as baseline accuracy minus post-intervention accuracy. The interaction contrast is the direct-answer drop minus the CoT drop; a positive value means the intervention was less harmful under CoT.

The scale matters. The frozen plan used absolute percentage-point drops, which avoid dividing by the low 27.3% direct-answer baseline. On that scale, the interaction contrast was 0.7 points, with a 95% paired-bootstrap interval from -7.3 to +8.7. The source paper instead normalized each score to its own baseline. A post-hoc sensitivity check on that scale estimated relative accuracy drops of 14.6% for direct answers and 3.8% for CoT, a 10.9-point interaction with a wide interval from -15.3 to +32.2.

Normalization therefore changes the point estimate but not the inference: the 150 questions do not determine which answer form was better protected. The frozen analysis and the post-hoc sensitivity both permit effects in either direction.

This motivated two lines of follow-up. Section 2 tests whether the ablation can selectively change behavior and whether the CoT interaction survives new questions. Section 3 investigates why the math effect remained small. These were sequential follow-ups, not one preregistered study; Section 5.4 records when each choice was made.

Testing Whether J-space Ablation Can Selectively Change Behavior

I used three random controls, and they are not interchangeable. A random-direction control matched the number of directions and intervention sites but not the perturbation norm. A position-wise matched-norm control matched the L2 norm at every tested token and layer on its own autoregressive rollout. A layer-calibrated matched-norm control matched only the mean relative norm in each layer. The fact comparisons used the position-wise control; several broader math diagnostics used the weaker layer-calibrated control. Section 5.3 defines all three.

A Behavioral Positive Control on Reused Fact Questions

A two-step fact question is built around a known intermediate fact. For example, the intended chain for “What is the capital of the country where Frida Kahlo was born?” is Frida Kahlo → Mexico → Mexico City. The word Mexico need not appear in the final answer.

I used these 40 two-step questions as a behavioral positive control for the intervention. Sentiment and text copying, 20 questions each, served as negative controls for nonspecific language degradation. Each fact question ran under a baseline, J-space ablation over layers 19–28, and three position-wise matched-norm controls.

The selected directions would count as important for the two-step answers only if J-space ablation caused a clearly larger accuracy drop than the position-wise controls, the interval for that difference contained only positive values, and the negative-control tasks remained intact.

The reused 40-question check showed a 32.5-point accuracy drop, with a 95% interval from 17.5 to 47.5 points. Sentiment and text copying both remained 20/20 correct under the baseline and J-space ablation.

The simpler random-direction controls scored 24/40, 25/40, and 26/40. The position-wise controls had a larger mean perturbation norm, about 15% of the activation norm versus about 9% for J-space ablation, because the norm was matched on each control’s own rollout. Even so, their mean accuracy drop was only 9.2 points. J-space ablation caused 23.3 points more drop, with a 95% interval from 11.7 to 35.8.

Because these 40 questions and prompts had already been used, this was not an unseen test. It establishes a selective behavioral effect on one known fact set.

The CoT Result Did Not Survive New Questions

An early experiment used 150 two-step fact questions. It compared direct answers and explicit CoT under a baseline, J-space ablation, and a position-wise matched-norm control. The interaction contrast asked how much CoT reduced the accuracy drop. The claim required this contrast to be larger under J-space ablation than under the position-wise control.

The first result looked positive, but 40 questions had already shown a large direct-answer drop during development. The 110-question row is the remainder of that same set, not a third experiment. Once the reused questions were removed, the interval included zero.

The strongest test used 300 separately constructed questions checked before the run. The interaction contrast was 4.3 points under J-space ablation and 8.0 points under the position-wise control. Their difference was therefore -3.7 points: the new-question test did not show a J-space-specific CoT interaction. A later blinded LLM re-scoring agreed with the automatic scorer on 98.4% of usable reviews and left the conclusion unchanged. Section 5.4 gives the audit procedure.

I therefore retracted the J-space-specific CoT claim. One procedural limit remains: these comparisons used the earlier layers 23–31 setup, while the behavioral positive control in Section 2.1 used the later layers 19–28 setup. I did not rerun all six conditions with the later procedure, so the positive control does not directly validate these CoT results.

Diagnosing the Small Math Effect

J-Lens Top-10 Coverage Was Lower for Math Intermediates

J-space ablation followed the highest-ranked J-Lens vectors, so I measured top-10 coverage: the fraction of questions whose known intermediate token form entered the top ten in any tested layer. The check used 55 short order-of-operations questions at the final prompt token, with eligible numeral and number-word forms of the known intermediate. For comparison, I used the source evaluation’s separate set of 93 two-step fact questions.

Arithmetic coverage was lower in both ranges. This is the strongest descriptive clue about the small math effect: ablating the top-ranked vectors is unlikely to matter when the known intermediate rarely enters that set.

The check used one token position, restricted token forms, and different math and fact sets. It also invalidated my planned 50% threshold, because fact coverage stayed below 50% even though ablation affected facts elsewhere. This measures the readout, not causal use.

Broader Checks Still Found No Math-Specific Effect

I tested five common explanations for the small math effect. Where a check used a random comparison, it used the layer-calibrated control defined in Section 5.3.

• The intervention was inactive or much weaker on math. J-space ablation changed the exact completion on about 58% of paired math runs. Its mean perturbation norm was about 9% to 10% of the activation norm on math and about 9% on the two-step fact task.
• Too few layers were changed. No tested range made J-space ablation more harmful than the layer-calibrated control. At the widest range, accuracy was 26.0% at baseline, 20.3% after J-space ablation, and 13.8% after the control. The same wide ablation was active on the fact task, lowering accuracy from 64% to 8%, while the control left it near 53%. See the layer-range analysis.
• Familiar wording hid the effect. The changed-number test was inconclusive: only 14.0% of baseline answers were correct, below the planned 15% minimum.
• The prompt gave too little support. Eight answer-only examples did not help. Baseline accuracy was 25.0%, versus 26.5% without them, far short of the planned 10-point improvement.
• The samples were too small. On 4,600 new questions, J-space ablation caused a 0.52-point drop [-0.59, +1.65]. Relative to the layer-calibrated control, its extra drop was -0.26 points [-1.46, +0.96]. After 583 capped outputs were rerun in all three conditions, the estimate moved from an apparent 1.02-point drop to 0.52 points.

Together, these checks ruled out an inactive intervention and tightly bounded the accuracy effect of this procedure.

Program-Specified J-Lens Vectors Still Had Almost No Effect

The coverage check suggested that top-10 selection might miss arithmetic intermediates. I therefore generated 384 controlled expressions spanning four operations and one to three later calculation steps. A program computed one correct intermediate for every expression. Instead of using the top-10 readout, the intervention ablated J-Lens vectors indexed by eligible token forms of that value, such as 6 and six. I call this the program-specified ablation.

These expressions used a short written-calculation prompt, so their 94% baseline is not comparable with the immediate-answer GSM8K baseline. Section 5.2 gives the prompt and scorer.

Every question received the planned ablation, and its size met the design requirement. The main run kept the exclusion rule from Section 1.1. Because that rule could exclude the very number forms supplied to the experiment, a follow-up planned in advance repeated the runs at both layer ranges with the rule turned off.

Baseline accuracy was 361/384, or 94.0%, with a 95% interval from 91.7% to 96.1%:

Each setting also had three position-wise matched-norm controls. The program-specified run scored 0.2 to 0.4 points higher than the mean position-wise control in the four comparisons, and every 95% interval included zero.

The 384-question result shows that bypassing top-10 selection did not solve the problem. A context-independent J-Lens vector indexed by 6 may still differ from the contextual representation Qwen uses in this calculation, and written CoT may allow recomputation. The missing causal test was a counterfactual change from 6 to 7, not another ablation.

Conclusions and the Next Step

What the Results Show and What Remains Open

The experiments support four main conclusions:

1. The first math comparison was underpowered. The absolute interaction was 0.7 points [-7.3, +8.7]. The post-hoc normalized interaction was 10.9 points [-15.3, +32.2]. The scale changed the point estimate, but neither interval resolved its sign.
2. J-space ablation produced a selective behavioral effect on a reused fact set. It caused a 32.5-point drop on two-step questions while sentiment and text copying remained correct.
3. The J-space-specific CoT claim failed on new fact questions. On 300 fresh questions, the interaction was 4.3 points under J-space ablation and 8.0 under the position-wise control, a difference of -3.7 points [-12.0, +4.0].
4. Arithmetic construct validity remains open. Top-10 coverage was lower for math than for facts. The 4,600-question comparison and the program-specified ablation both produced near-zero effects, but neither identified the contextual arithmetic representation.

This scope statement applies once to the whole article: the results concern the pinned Qwen3-4B checkpoint, public WikiText J-Lens, tested prompts and layers, and these ablation procedures. They do not show that Qwen lacks internal math information or refute the source paper.

The 4,600-question test answered one statistical question precisely: how much did this intervention change accuracy? It did not answer the construct-validity question. Repeating the same intervention on still more questions would mainly narrow the same behavioral estimate.

This matters for AI-safety monitoring. Fitting a monitor on the correct model is necessary but not sufficient; its readout and intervention must be validated for the target information and task. A negative reading can mean that the information is absent, represented in a form the tool cannot name, or outside the tested locations. Only a successful control on known information can distinguish those possibilities.

The Missing Experiment: Change an Intermediate Result from 6 to 7

The next experiment should test the missing link directly. The source paper uses mean-difference patching, a form of activation patching, for this kind of test. It estimates the average residual-stream difference between examples whose intermediate results are 7 and 6, then adds that difference at a candidate location. This differs from adding one context-independent J-Lens vector.

Before running this experiment, I would identify the exact Qwen revision used to fit the public J-Lens and confirm that it matches the pinned checkpoint. If that cannot be confirmed, I would fit a new lens on the pinned checkpoint and compare the two.

The causal test would then proceed as follows:

1. Build two balanced sets of calculations with known intermediate results and known counterfactual answers. Balance the expression form and later operations, and vary the final answers, so that the average 7-minus-6 difference does not encode some other fixed difference. Use one set to estimate the patch and the other to test it.
2. Use the J-Lens readout to nominate layers and token positions, then estimate the mean 7-minus-6 activation difference at each candidate.
3. Patch that difference into runs whose true intermediate is 6 and test whether the output moves toward the exact counterfactual answer.
4. Compare nominated locations with prespecified nearby layers and repeated matched-norm random patches. The targeted patch should produce the predicted answer more reliably than either control.

For (2 + 4) × 3, the intermediate result is 6 and the final answer is 18. If the experiment genuinely changes the intermediate result from 6 to 7, the predicted final answer is 21. An answer of 21 is much stronger evidence than a general accuracy drop because it tests both what information was found and how the model used it.

A successful patch would show that the nominated activation can affect the answer. I would then apply the validated intervention under direct answering and explicit CoT to test whether CoT reduces reliance on that activation.

Methods and Reproducibility Notes

This section collects implementation details that are necessary for checking or repeating the experiments but are not needed to follow the main argument.

Model, J-Lens, Ablation, and Layer Ranges

In the source paper, “ablation strength” meant the range of layers changed, not a larger subtraction within each layer. Its light, medium, and heavy settings covered 62% to 71%, 54% to 79%, and 42% to 88% of model depth. All three used ten J-Lens directions at each token position.

I used the post-trained Qwen/Qwen3-4B checkpoint, not the separate Base checkpoint. The experiments used non-thinking chat mode, greedy decoding, and float16 inference. This was a transfer test from Claude Sonnet 4.5 to Qwen, not an exact replication.

The public Neuronpedia J-Lens configuration shows that the lens was fitted on activations from Qwen/Qwen3-4B itself, using 479 prompts from WikiText-103. I used this lens and Anthropic’s reference code rather than fitting a new lens.

The configuration names the model but does not record the exact Qwen revision it loaded. Qwen’s public weight files are unchanged between the original upload and my pinned revision, making a weight mismatch unlikely. A documented difference remains: the lens was fitted on general WikiText passages with bfloat16, whereas my experiments used math and fact prompts in non-thinking mode with float16.

At each tested position and layer, the top-10 experiments used ten eligible J-Lens vectors. I orthonormalized them and subtracted the residual-stream activation’s entire projection onto their combined span. If the columns of Q_J form the orthonormal basis and h is the activation, the removed component is:

δ_J = Q_J Q_J^T h

The projection coefficient was fixed at one; intervention strength changed only through the layer range. Percentages such as 9% report the measured L2 perturbation norm after a run. The top-10 experiments also applied the exclusion rule from Section 1.1.

Five numerical layer ranges appear in the article, but only two were main ranges. Their origins were:

• Layers 22–31: a small pilot range.
• Layers 23–31: the early main range. I measured how often J-Lens returned the same top-ranked token at neighboring text positions in each Qwen layer. The selection rule used the median score across layers as a baseline, set a cutoff halfway between that baseline and the highest score, and kept the contiguous block containing the highest-scoring layer. The rule was written before the main comparison, but after the pilot.
• Layers 19–28: the later main range. The source paper’s medium setting covered 54% to 79% of model depth; mapping that proportional range onto Qwen’s 36 layers produced layers 19–28. I chose this range after the early CoT experiments and a small exploratory fact check, but before the later validation. It was not part of the original preregistration.
• Layers 22–25 and 15–32: the narrow and wide diagnostic ranges in Section 3.2, mapped from the source paper’s light and heavy settings.

Layers 23–31 are nine of Qwen’s 36 layers, or 25% of the model, so they approximately matched the width of the source paper’s medium intervention but sat later in the model. The later validation therefore kept that early range as a comparison while using layers 19–28 as the direct proportional mapping. The layer-range diagnostic reused layers 23–31 as its middle condition; it was a test of narrower and wider interventions, not an exact transfer of all three source settings.

Prompts and Scoring

GSM8K immediate answers and explicit CoT.

The two math conditions used these instructions:

• Direct answer: Answer the following problem with the final answer only, inside \boxed{}. Do not show any reasoning or intermediate steps. The assistant turn was prefilled with The final answer is \boxed{.
• Explicit CoT: Solve the following problem step by step, showing your reasoning. Then give the final answer inside \boxed{}. There was no assistant prefill.

An instruction-only pilot failed because Qwen began writing reasoning and did not reach a final answer within the short direct-answer output limit. I added the prefill after that pilot. Direct completions were limited to 32 new tokens; CoT completions were limited to 1,024.

For GSM8K, the expected answer was the value after the dataset’s final #### marker. The primary scorer joined the direct-answer prefill to the generated text and extracted the last complete, brace-balanced answer inside \boxed{}. If an untruncated output had no box, it used the last number. It first checked symbolic mathematical equivalence, then used normalized string or numeric equality if the symbolic check did not return a match. A truncated output with no boxed answer counted as incorrect.

I re-scored the stored outputs in two sensitivity checks. One used the same symbolic scorer but treated truncated text as complete. The earlier scorer used only normalized string or numeric equality and had no truncation rule. These were alternative checks on the same outputs, not three scores combined into one verdict. All three rules supported the same conclusion about the small math effect.

The post-hoc normalized sensitivity check in Section 1.2 jointly resampled the 150 questions 10,000 times and recomputed (1 - ablated/baseline)_direct - (1 - ablated/baseline)_CoT on each sample. It used the stored primary correctness judgments and seed 0.

The 27.3% GSM8K baseline in Section 1.2 and the 26.0% baseline in the 400-question layer-range check used the same immediate-answer prompt and procedure on different question sets. That small difference therefore does not indicate a prompt change.

Two-step fact questions.

The fact questions used a different prompt pair:

• Direct answer: Answer in as few words as possible.
• Explicit CoT: Think step by step briefly, then give your final answer in as few words as possible on a new line beginning with 'Answer:'.

Neither prompt used an assistant prefill. Direct responses were limited to 32 new tokens and CoT responses to 256. The automatic scorer compared the response with accepted answer strings or aliases using complete, word-boundary matches. For a direct answer, it searched the whole short response. For CoT, it checked the text after the last Answer: marker. If an untruncated response had no marker, it searched the whole response; a truncated response without a marker counted as incorrect.

These conditions explain why the 64.0% and 65.0% fact baselines are not comparable with the 27.3% GSM8K baseline. The question set, direct prompt and prefill, and scoring rule all differed. The experiments do not separate the contributions of those differences.

Controlled expressions with program-specified J-Lens vectors.

The 384 controlled expressions used this instruction:

Calculate step by step very briefly. Show only essential arithmetic equalities, without explanatory prose. End with the exact final integer inside \boxed{}. Keep the response concise.

The question appeared as Evaluate exactly: {expression}. The model received no assistant prefill or examples and could generate up to 128 new tokens. The scorer required the last complete \boxed{} to contain exactly one base-10 integer equal to the known final answer. It did not fall back to the last number in the text or to symbolic equivalence.

Random Control Interventions and Perturbation Norm

All three random controls update a residual-stream activation h by subtracting a perturbation δ: h_new = h - δ. I measure its local size with the ratio ||δ||₂ / ||h||₂. A reported percentage such as 9% is the mean over all recorded token-position and layer pairs. Each pair counts once, so longer completions contribute more pairs, and zero-perturbation pairs remain in the mean.

The position-wise control recomputes the matched norm on its own autoregressive rollout rather than replaying the value from the J-space run. Its average norm can therefore differ even when each local match is exact.

Study Timing and Post-Run Checks

The studies formed a sequence of follow-ups, not one preregistered study. Here, “set before” means that a choice was written down before that follow-up’s target outputs were produced. It may still have been motivated by an earlier result.

For the later fact-output audit, Claude Sonnet 5 received each question, its accepted answer strings, and the model response. It did not receive the answer-form label, experimental condition, automatic verdict, or the other responses, although response style could reveal whether reasoning had been requested. Of 2,700 requested reviews, 2,696 were usable. The language-model judge and the automatic scorer agreed on 2,652, or 98.4%. Thirty-five disagreements changed an automatic “wrong” to “correct,” and nine went the other way. This was an alternative scoring check on fact outputs only; it did not audit the GSM8K math scorer.

Project Artifacts and Disclosure

This work began as a focused transfer test and expanded through several rounds of follow-up experiments. Some links below may require repository access; I plan to release a compact public bundle of code and data.

Project artifacts

• Repository and reproduction instructions
• Concise technical report
• Original preregistration and amendments
• Frozen main experiment and analysis plan
• Intervention implementation audit
• Two-step fact test on 300 new questions
• Fresh 4,600-question GSM-Symbolic analysis
• Program-specified ablation: protocol, results, and released data

Role and acknowledgments

I designed the hypotheses and controls, implemented and audited the code, and ran or supervised the experiments. I used AI systems for coding, review, and drafting assistance, and checked their outputs against the prespecified analysis scripts and saved results. The source J-Lens implementation, third-party fitted weights, and public datasets are credited in the repository.