Active Learning Query Strategies
Labels are the bottleneck, not data
In most real deployments, raw unlabeled data is abundant and cheap, while labels — human annotations, expert judgments, expensive experiments — are scarce and expensive. Active learning flips the usual supervised-learning assumption that the training set is fixed; instead, the model itself participates in choosing which unlabeled examples get sent for labeling next, aiming to reach a target accuracy with far fewer labeled examples than random sampling would require. The core bet is that not all unlabeled examples are equally informative — some would barely move the model if labeled, others would resolve genuine uncertainty.
Uncertainty sampling: the simplest query strategy
The most widely used active learning strategy is uncertainty sampling: label the examples the current model is least confident about. For classifiers, common uncertainty measures include least-confidence ($1 - \max_y P(y|x)$), margin sampling (difference between the top two predicted class probabilities — small margin means the model is torn between two classes), and entropy of the full predicted distribution ($-\sum_y P(y|x)\log P(y|x)$). The intuition is straightforward: examples the model already classifies confidently provide little new information; examples near the decision boundary are where labels matter most.
import numpy as np
def entropy_sampling(probs: np.ndarray, n_query: int) -> np.ndarray:
"""probs: (n_samples, n_classes) predicted probabilities from the current model."""
eps = 1e-12
entropy = -np.sum(probs * np.log(probs + eps), axis=1)
return np.argsort(entropy)[-n_query:] # indices of the most uncertain examples
def margin_sampling(probs: np.ndarray, n_query: int) -> np.ndarray:
sorted_probs = np.sort(probs, axis=1)
margin = sorted_probs[:, -1] - sorted_probs[:, -2] # top1 - top2 confidence
return np.argsort(margin)[:n_query] # smallest margin = most ambiguousBeyond single-model uncertainty: committees and diversity
Uncertainty sampling using a single model can over-query a narrow, redundant region of the input space — many uncertain examples might all look alike. Query-by-committee trains an ensemble of models and selects examples where the committee members disagree most (measured, e.g., by vote entropy), which tends to target genuine model uncertainty rather than single-model quirks. Diversity-based / representative sampling explicitly selects a batch of examples that are both informative and spread out across the input distribution, often via clustering in representation space, to avoid wasting a labeling budget on a cluster of near-duplicate hard examples. Expected model change and expected error reduction go further, estimating how much labeling a given example would actually reduce future error — more principled, but far more computationally expensive per query.
Active learning query strategies
| Strategy | Signal used | Cost per query | Main risk |
|---|---|---|---|
| Uncertainty sampling | Single model's confidence | Low | Can over-sample redundant hard examples |
| Query-by-committee | Disagreement across an ensemble | Medium (needs ensemble) | Ensemble training overhead |
| Diversity sampling | Representation-space spread | Medium | May include uninformative but diverse points |
| Expected error reduction | Estimated impact on future loss | High | Expensive to compute per candidate |
Active learning can amplify dataset bias
Where active learning pays off most
Active learning delivers the largest wins when labeling is genuinely expensive relative to inference — medical image annotation by specialists, rare-event detection where positives are scarce, or scientific experiments with physical costs. It delivers less benefit in domains with cheap crowdsourced labels or where unlabeled data itself is scarce, since the whole premise relies on a large unlabeled pool to select from. In production ML pipelines, active learning is often combined with weak supervision — using programmatic or heuristic labels for the easy majority of data and reserving active learning's human-labeling budget for the genuinely hard, informative examples.
What's Next?
Active learning assumes a human (or oracle) is available to supply ground-truth labels on demand; the next natural question is what happens when that assumption is relaxed — when labels instead come from noisy heuristics, rules, or weak signals rather than a reliable oracle.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.