Self-Consistency Decoding
One sample isn't a reliable signal for reasoning tasks
For tasks requiring multi-step reasoning — arithmetic word problems, logical deduction, multi-hop question answering — a single greedy or sampled generation from a chain-of-thought prompt can follow a flawed reasoning path and land on a wrong answer, even though the model is capable of reaching the right answer through a different path. Self-consistency (Wang et al., 2022) addresses this by sampling multiple independent reasoning chains for the same question, then taking a majority vote over their final answers, rather than trusting any single chain.
How it works mechanically
Given a chain-of-thought prompt, self-consistency samples $N$ completions (typically 5-40) using temperature-based sampling rather than greedy decoding, so each sample explores a genuinely different reasoning path. Each completion's final answer is extracted (usually via a consistent output format, like 'The answer is X'), and the most frequent answer across all $N$ samples is returned as the final result. The intuition: wrong reasoning paths tend to diverge toward different wrong answers, while correct reasoning paths tend to converge on the same correct answer — so the correct answer is disproportionately likely to be the plurality choice even if no single sample is reliable alone.
from collections import Counter
import re
def extract_final_answer(reasoning_text):
match = re.search(r"[Tt]he answer is\s*([^\.\n]+)", reasoning_text)
return match.group(1).strip() if match else None
def self_consistency_answer(question, llm_call, num_samples=10, temperature=0.7):
prompt = f"{question}\nThink step by step, then state: 'The answer is X.'"
answers = []
for _ in range(num_samples):
completion = llm_call(prompt, temperature=temperature)
answer = extract_final_answer(completion)
if answer:
answers.append(answer)
if not answers:
return None, {}
vote_counts = Counter(answers)
best_answer, votes = vote_counts.most_common(1)[0]
confidence = votes / len(answers)
return best_answer, {"confidence": confidence, "distribution": dict(vote_counts)}The vote distribution is a free confidence signal
Cost and when it's worth it
Self-consistency multiplies inference cost by roughly $N$ — 10 samples means roughly 10x the token cost of a single generation (plus the cost of each full chain-of-thought, which is already longer than a direct answer). It's worth the overhead specifically on tasks where (a) correctness matters a lot more than latency or cost, and (b) the task has a verifiable or extractable final answer that can be cleanly aggregated by majority vote — math problems, multiple-choice questions, classification tasks. It's much less useful for open-ended generation (essays, creative writing, long explanations) where there's no single 'correct' answer to vote on.
Self-consistency vs single-chain reasoning
| Property | Single chain-of-thought | Self-consistency (N samples) |
|---|---|---|
| Inference cost | 1x | ~Nx |
| Robustness to a single bad reasoning path | None | High, via majority vote |
| Built-in confidence signal | No | Yes, via vote agreement ratio |
| Best fit | Low-stakes, latency-sensitive tasks | High-stakes tasks with extractable final answers |
Majority vote can amplify a shared systematic bias
What's Next?
Self-consistency explores multiple independent linear reasoning chains. The next step in this direction is exploring multiple *branching* reasoning paths within a single problem and deliberately searching or backtracking through them, rather than just sampling independent full attempts.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.