Softmax Temperature and Calibration

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

What temperature does inside softmax

The standard softmax function $\text{softmax}(z)_i = \frac{e^{z_i}}{\sum_j e^{z_j}}$ can be generalized with a temperature parameter $T$: $\text{softmax}(z, T)_i = \frac{e^{z_i/T}}{\sum_j e^{z_j/T}}$. Dividing logits by $T$ before exponentiating rescales how peaked or flat the resulting distribution is. As $T \to 0$, the distribution approaches a one-hot vector on the largest logit (effectively argmax). As $T \to \infty$, the distribution approaches uniform over all classes, regardless of the logits' relative values. $T = 1$ recovers ordinary softmax.

temperature_effect.py

python

import numpy as np

def softmax_with_temperature(logits, T=1.0):
    z = np.array(logits) / T
    z = z - z.max()
    e = np.exp(z)
    return e / e.sum()

logits = [4.0, 2.0, 0.5]
for T in [0.5, 1.0, 2.0, 5.0]:
    print(f"T={T}: {softmax_with_temperature(logits, T).round(3)}")

# T=0.5: [0.924 0.072 0.004]   <- sharper, more confident
# T=1.0: [0.780 0.172 0.048]   <- raw model distribution
# T=2.0: [0.605 0.274 0.121]   <- flatter
# T=5.0: [0.429 0.335 0.235]   <- close to uniform

Two very different jobs temperature does in practice

Temperature shows up in two largely unrelated contexts, and it's worth keeping them separate. In sampling (text generation from language models, reinforcement learning action selection), temperature controls the randomness/diversity of draws from the distribution — higher temperature means more varied, less greedy sampling, applied at inference time on an already-trained model. In calibration, temperature is a learned or fit scalar applied *after* training to correct a mismatch between a model's confidence and its actual accuracy, specifically to make reported probabilities trustworthy, not to change what gets predicted (argmax is temperature-invariant).

Why trained networks tend to be overconfident

Guo et al. (2017, 'On Calibration of Modern Neural Networks') showed that modern deep networks — despite often being *more* accurate than older, shallower models — tend to be substantially *less* calibrated, producing overconfident probability estimates. Contributing factors include training with cross-entropy against hard one-hot targets (which rewards infinite confidence, as in the label smoothing discussion), increased model capacity, and the use of batch normalization. A model might assign 95% confidence to predictions that are only actually correct 70% of the time — a 25-point calibration gap that matters enormously for any downstream decision that relies on the stated probability (risk scoring, active learning, selective prediction).

Temperature scaling as a calibration fix

Temperature scaling is deliberately the simplest possible post-hoc calibration method: fit a single scalar $T$ (via gradient descent minimizing negative log-likelihood on a held-out validation set, keeping all other model weights frozen) and divide all logits by it before the final softmax at inference time. Because it's a single parameter applied uniformly, it cannot change which class gets the highest score — accuracy is completely unaffected — it only reshapes how confident the reported probabilities are. Guo et al. found this remarkably simple fix outperformed more complex calibration methods (Platt scaling, isotonic regression, histogram binning) on most of their vision benchmarks.

fit_temperature.py

python

import torch
import torch.nn as nn
import torch.optim as optim

def fit_temperature(val_logits, val_labels, max_iter=50):
    """
    val_logits: (N, K) tensor of logits from the trained, frozen model
    val_labels: (N,) tensor of true class indices
    """
    T = nn.Parameter(torch.ones(1))
    optimizer = optim.LBFGS([T], lr=0.01, max_iter=max_iter)
    nll = nn.CrossEntropyLoss()

    def closure():
        optimizer.zero_grad()
        loss = nll(val_logits / T, val_labels)
        loss.backward()
        return loss

    optimizer.step(closure)
    return T.item()  # typically > 1 for overconfident models

T > 1 is the common finding, not T < 1

In practice, fitted temperatures for overconfident modern networks almost always come out greater than 1 — dividing logits by a number bigger than 1 flattens the distribution, pulling confidence down toward reality. A fitted $T$ close to 1 means the model was already well-calibrated; a very large fitted $T$ (well above 2-3) signals severe overconfidence worth investigating further, possibly alongside training changes like label smoothing.

Calibration is dataset- and shift-sensitive

A temperature fit on in-distribution validation data does not guarantee good calibration under distribution shift — a model can be well-calibrated on its validation set and still badly overconfident on out-of-distribution inputs it wasn't fit against. Temperature scaling corrects calibration within the regime it was tuned on; it is not a substitute for out-of-distribution detection.

What's Next?

Calibration quality is typically visualized and audited with reliability diagrams and quantified with metrics like Expected Calibration Error — tools that make the gap between confidence and accuracy concrete rather than theoretical.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →