Contrastive Loss Functions (Triplet, InfoNCE Compared)

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

The goal: learning a useful embedding space, not a classifier

Contrastive losses train a model to produce embeddings where similar inputs land close together and dissimilar inputs land far apart, without necessarily needing explicit class labels at all. This is the backbone of modern representation learning: face recognition, image/text retrieval, self-supervised pretraining (SimCLR, CLIP), and recommendation systems all rely on some flavor of contrastive loss. The common thread across variants is that the loss is computed over *pairs* or *groups* of examples, not single examples in isolation.

Triplet loss

Triplet loss (popularized by FaceNet, Schroff et al., 2015) operates on triplets: an anchor $a$, a positive $p$ (same identity/class as the anchor), and a negative $n$ (different identity/class). It penalizes the model when the anchor-positive distance isn't smaller than the anchor-negative distance by at least a margin $m$: $L = \max(0, \, d(a,p) - d(a,n) + m)$. The loss is zero once the positive pair is closer than the negative pair by the margin — it doesn't keep pushing positives infinitely close together, only enforces relative ordering.

triplet_vs_infonce.py

python

import torch
import torch.nn.functional as F

def triplet_loss(anchor, positive, negative, margin=0.2):
    d_pos = F.pairwise_distance(anchor, positive)
    d_neg = F.pairwise_distance(anchor, negative)
    return F.relu(d_pos - d_neg + margin).mean()

def info_nce_loss(query, keys, temperature=0.07):
    """
    query: (N, D) embeddings, one per anchor
    keys:  (N, D) embeddings where keys[i] is the positive for query[i],
           and every other keys[j] (j != i) acts as a negative (in-batch negatives).
    """
    query = F.normalize(query, dim=1)
    keys = F.normalize(keys, dim=1)

    logits = query @ keys.T / temperature       # (N, N) similarity matrix
    labels = torch.arange(query.size(0), device=query.device)
    return F.cross_entropy(logits, labels)       # positive = diagonal entry

InfoNCE: contrastive learning as classification

InfoNCE (van den Oord et al., 2018; used in SimCLR, MoCo, CLIP) reframes the problem as classification: given a query and a set of candidate keys (one positive, many negatives), predict which key is the true positive via softmax over similarity scores, scaled by a temperature $\tau$: $L = -\log \frac{\exp(\text{sim}(q,k^+)/\tau)}{\sum_{i} \exp(\text{sim}(q,k_i)/\tau)}$. Unlike triplet loss, which considers one negative at a time, InfoNCE considers all negatives in the batch (or a memory bank) simultaneously in a single softmax, which gives a denser, lower-variance gradient signal per step and scales naturally to large negative pools.

Loss	Negatives per update	Sensitive to temperature/margin	Common use
Triplet loss	One (per triplet)	Margin value matters a lot	Face/metric verification
Contrastive loss (pairwise)	One per pair	Margin for negatives	Early siamese-network metric learning
InfoNCE / NT-Xent	Many (batch-wide)	Temperature strongly affects gradient sharpness	Self-supervised pretraining (SimCLR, CLIP, MoCo)

Why temperature in InfoNCE matters so much

Low temperature ($\tau$ small) sharpens the softmax, making the loss focus heavily on the hardest negative(s) — those closest to the positive in similarity — which gives strong gradients but can be unstable early in training. High temperature flattens the distribution, spreading gradient across all negatives more evenly but with weaker individual signal. SimCLR's ablations found performance quite sensitive to this single scalar, typically tuned in the 0.05–0.5 range — it's one of the highest-leverage hyperparameters in contrastive pretraining.

The hard-negative-mining problem

All contrastive losses share a practical challenge: random negatives are often trivially easy (an image of a cat vs. a car), contributing almost no gradient once the model is minimally competent, similar to the easy-example problem in focal loss. Hard negative mining — deliberately selecting negatives that are close to the positive but wrong — accelerates learning but risks selecting false negatives (items that are actually similar but labeled as negative) or destabilizing training if negatives are too hard too early. Triplet loss implementations commonly use 'semi-hard' negative mining (negatives farther than the positive but still within the margin) to balance this tradeoff.

In-batch negatives couple loss quality to batch size

Because InfoNCE draws negatives from the current batch (or a memory bank, as in MoCo), its effectiveness is sensitive to batch size — too small a batch and there simply aren't enough diverse negatives to learn a well-separated embedding space. This is a major reason SimCLR-style pretraining uses very large batch sizes or memory-bank/momentum-encoder tricks to approximate larger negative pools without paying for them at full batch size.

What's Next?

These losses are what shape an embedding space during training; understanding the geometry of the resulting space — why embeddings cluster the way they do, and what distance metric actually means there — is the natural next layer of the picture.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →