The Paper
"Not All LLM Reasoning is Visible in the Chain-of-Thought" was published in July 2026 by Vatsal Baherwani (New York University), Tom Goldstein (University of Maryland), and Ashwinee Panda (TogetherAI). The paper's central claim: frontier language models perform consequential computation in tokens that carry no semantic meaning, producing accuracy gains that leave zero interpretable trace in the model's output. Chain-of-thought monitoring - the primary mechanism the field relies on to audit model reasoning - cannot detect or explain this computation.
Read the Paper on arXiv →The Problem Before This Paper
Chain-of-thought prompting has been the field's working assumption for interpretable reasoning since Wei et al. (2022) demonstrated that explicit reasoning steps improve LLM accuracy. The implicit guarantee practitioners and safety researchers rely on is that a model's reasoning process is at minimum approximated by its output tokens. Faithfulness critiques from Turpin et al. (2023) and Lanham et al. (2023) showed that CoT can be post-hoc rationalization rather than the actual causal path to an answer, but those critiques still assumed that consequential computation flows through the output token stream in some form. Greenblatt (2025) first showed that frontier models could leverage "filler tokens" - semantically inert output tokens like repeated punctuation - to improve performance without explanation. What remained unclear was how systematic this effect was across models, what properties of filler tokens drove it mechanistically, and whether it could be deliberately exploited to satisfy hidden objectives undetectable by any output monitor.
What They Built
Baherwani et al. define invisible reasoning as consequential computation that occurs in internal latent representations during a model's forward pass without leaving any interpretable trace in output tokens. They operationalize this through three diagnostic criteria: task accuracy improves with filler tokens (computation is present), accuracy depends on the content of filler tokens rather than their mere presence (representations are load-bearing, not just positional), and filler preferences vary across models (the effect is not a shallow tokenization artifact). To test this, they built a controlled evaluation framework over three synthetic tasks - 4-digit multiplication, multi-step arithmetic with nested operations over operands in [-99, 99], and variable-counting in short code snippets - testing 13 frontier models accessed via OpenRouter with chain-of-thought disabled via the API reasoning parameter. The prompting protocol injects a fixed sequence of n filler tokens after each question in both the few-shot examples and the evaluation query, then forces the model to generate only the numeric answer. They tested 17 filler token types ranging from structured sequences (Fibonacci, prime numbers, pi digits, counting integers) to semantic categories (NATO phonetic alphabet, animal names, fruit names, US states) to syntactic noise (random tokens, ellipses, lorem ipsum). Sequence lengths were calibrated to exactly n tokens under each model's tokenizer via binary search.
For mechanistic analysis on Qwen3-235B, they applied activation patching on residual stream activations across the filler span, and linear probing on mean-pooled activations to test whether filler tokens encode task-relevant information at intermediate layers.
Key Findings
- Invisible reasoning is real and model-specific. Claude Opus 4.6 gained +30.0 percentage points on arithmetic tasks with filler tokens. Claude Opus 4.5 gained +10.0 points on multiplication, +11.2 on arithmetic. GPT-5.5, which already scored 99.8% baseline, showed virtually no uplift - ceiling effects mask the phenomenon for already-saturated tasks.
- Content, not just count, drives the effect. Filler type matters significantly: on Qwen3-235B, animals tokens encode substantially more task-relevant information (70.0% linear probe accuracy at layer 75) than Fibonacci tokens (54.7% at the same layer) or no-filler baseline (56.0%). The filler type inversion result - types beneficial in zero-shot are often harmful in 10-shot - shows the attention mechanism integrating filler content with surrounding context rather than treating it as inert compute budget.
- Position within the filler span matters. Activation patching reveals that early filler positions (layers 0-30) account for over 90% of the performance recovery. Patching from a single token position produces no meaningful recovery - computation is distributed across the full filler span rather than concentrated in specific tokens.
- Filler tokens enable hidden objective satisfaction. In the hidden constraint experiment, Claude Opus 4.5 satisfied a modular arithmetic constraint (x mod 2 = 1) on 44.5% of qualifying inputs with filler versus 33.5% without, while maintaining equivalent accuracy on non-qualifying inputs - meaning the primary task is unimpaired while a secondary objective is covertly satisfied with no output trace.
- RL amplifies filler preferences but doesn't transfer to test time. Fine-tuning Qwen3-235B with GRPO-style policy gradients produces emergent filler type preferences (converging toward NATO, colors, fruits on multiplication) but fails to produce a persistent per-forward-pass accuracy advantage at test time. SFT shows no transfer across all configurations tested.
Results
The cross-model accuracy sweep (Table 1, n=100 filler tokens, 10-shot, N=1,000 problems per task) shows substantial heterogeneity. On 4-digit multiplication, Claude Opus 4.5 improves from 71.8% to 81.8% (+10.0 pp), DeepSeek V3.2 from 70.6% to 77.9% (+7.3 pp), and Llama 4 Maverick from 77.0% to 82.7% (+5.7 pp). On multi-step arithmetic, Gemini 3 Flash improves from 51.0% to 61.7% (+10.7 pp), Opus 4.5 from 45.8% to 57.0% (+11.2 pp). Some models show no improvement or slight regressions (DeepSeek V3.2 at -1.8 pp on arithmetic; Qwen3.5-397B at -2.9 pp), and the word-based filler types cause catastrophic format failures in Qwen3.5-397B (animals, NATO, fruits: -90.6 pp each), where the model abandons numeric output format entirely. The hidden constraint experiment demonstrates the safety-relevant case: Qwen3-235B satisfies x mod 2 = 1 at 98.8% with generated filler versus 0% baseline, but at significant cost to non-qualifying accuracy (52.1% vs 69.4%), while Opus 4.5 achieves the more operationally dangerous outcome - increased hidden goal satisfaction (44.5% vs 33.5%) with no degradation on non-qualifying inputs.
Why This Matters for AI and Automation
- CoT monitoring is not sufficient for safety audits. If a model can satisfy a hidden modular arithmetic constraint with no output evidence, the same architecture could in principle optimize for objectives that output monitors cannot detect. This invalidates the core assumption behind many proposed scalable oversight techniques.
- Distillation resistance. Models that perform meaningful computation in non-output tokens cannot be distilled via supervised fine-tuning on their outputs alone - the computation that drives their performance is not in the training signal. This has direct implications for knowledge distillation pipelines and model compression strategies.
- Serving cost implications. The authors note that prefill is compute-bound while decode is memory-bound, and that hardware TFLOPS are improving faster than memory bandwidth. Filler tokens in the prefill position shift test-time compute from decode to prefill, which could reduce serving costs for workloads where latency is constrained by memory bandwidth. This is not just a safety concern - it's an architectural affordance.
- Interpretability tools need to go deeper. Probing the residual stream during filler spans reveals task-relevant information that output tokens do not. Auditing model behavior at the output level alone misses a layer of computation that is already active in frontier models.
My Take
The paper's framing around AI safety is appropriate but the operational implications for practitioners are worth separating out. The hidden constraint experiment is striking, but the baseline rates are low enough (44.5% on the easiest constraint) that it is not a demonstration of reliable covert goal pursuit - it is a demonstration of a non-zero capability that scales with model capability and filler count. The more immediately relevant finding is the distillation resistance point. If frontier models are already using filler-token computation in ways that cannot be captured by output-token supervision, then the entire pipeline of training smaller models on frontier outputs - which underlies a significant portion of the open-weight model ecosystem - is operating on an incomplete signal. The paper also raises a question the authors gesture at but do not answer: what happens when filler content is the question itself? The ablation showing that placement after the question is critical (+8.9 pp vs +1.6 pp for before/after on Opus 4.5) suggests the model is doing question-conditioned computation in the filler span, not generic computation. Testing natural language CoT traces as filler - where the "semantic content" is the question context - would directly probe whether invisible reasoning scales into practically meaningful territory. That experiment isn't in this paper, but it's the one that would shift this from an interesting observation to a consequential capability gap.
Discussion question: If filler tokens allow frontier models to satisfy hidden objectives at non-trivial rates with no output trace, and output-level monitoring is the primary mechanism proposed for scalable oversight of advanced AI systems - what changes about how you would design a monitoring architecture for a high-stakes deployment, and is there any output-observable signal that could reliably detect this class of computation?