Stopping explicit reasoning in model outputs cannot be taken for granted: this paper shows that even with strict "no-think" controls, LLMs often emit visible, question-conditioned inference. The core insight is operational and observable — split each response into a pre-answer trace T and a final answer A, then evaluate T along three complementary axes to distinguish empty output, question-relevant non-inferential text, and explicit inference.
Key Findings
- New measurement framework: three response-level metrics (Empty-Thinking Rate, Question–Pre-answer Relevance, Explicit Inference Rate) that separate answer-only compliance from residual text and visible inferential steps. The framework keeps claims at the emitted-response level rather than assuming anything about latent computation.
- Empirical protocol: six prompting interventions evaluated on six popular LLMs across Boolean, multiple-choice, and open-ended tasks (e.g., BoolQ/StrategyQA, MMLU variants, GSM8K/MATH). Human annotations and an LLM-as-judge validate the metrics.
- Thinking Inertia phenomenon: explicit visible inference persists under explicit no-think instructions and becomes more prevalent as the answer space opens (Boolean ≈ easiest to answer-only; multiple-choice ≈ intermediate; open-ended ≈ most residual reasoning). Stricter output-side requests reduce but do not eliminate visible inference.
- Accuracy trade-offs: task accuracy stays similar for Boolean and multiple-choice formats, but open-ended tasks show a trade-off between complying with answer-only constraints and maintaining answer quality. Rewriting questions into candidate-answer formats substantially increases answer-only outputs and accuracy.
Who it helps and trade-offs
Great fit if you need a principled way to audit whether models visibly reveal reasoning when instructed not to—researchers studying model interpretability, system designers wanting predictable output formats, and benchmark authors evaluating output-level behavior. Look elsewhere if your goal is to measure latent internal computation (this work focuses on emitted responses) or if you need operational mitigation strategies beyond prompting and output constraints (the paper diagnoses persistence but does not provide a full engineering fix).
The paper's strength is a clear, validated measurement lens and a systematic empirical sweep showing when and how visible reasoning persists; its limitation is that it stays at the response-observation level and does not claim to diagnose hidden computation or provide guaranteed mitigation.