YOLO Object Detection Explained
The object detection problem
Object detection asks for more than classification: given an image, find every object of interest, draw a bounding box around each, and label what it is. The classic approach (R-CNN family) was a two-stage pipeline — first propose candidate regions that might contain an object, then classify each region separately. This works well but is slow, since the classifier runs once per proposed region.
YOLO (You Only Look Once, Redmon et al., 2016) reframes detection as a single regression problem: one neural network pass over the whole image directly predicts all bounding boxes and class probabilities at once. This is the origin of the name — the network only has to 'look' at the image one time to produce every detection.
The grid-cell prediction scheme
YOLO divides the input image into an $S \times S$ grid of cells. Each cell is made responsible for detecting any object whose center falls inside it. For each cell, the network predicts: a fixed number of bounding boxes (each as offset coordinates $(x, y, w, h)$ relative to the cell), a confidence score for each box (how likely it contains an object, and how accurate the box is), and a set of class probabilities for the object if one is present.
Because this is all produced in one forward pass through a single convolutional network, the output is just one big tensor of shape $S \times S \times (\text{boxes} \times 5 + \text{classes})$ — no second-stage classifier, no per-region cropping.
import numpy as np
def decode_grid_cell(cell_pred, cell_x, cell_y, grid_size, num_classes):
"""cell_pred: [tx, ty, tw, th, confidence, class_0, class_1, ...]"""
tx, ty, tw, th, conf = cell_pred[:5]
class_scores = cell_pred[5:5 + num_classes]
# Box center = cell offset (sigmoid-bounded 0..1) + cell's own grid position
box_cx = (cell_x + sigmoid(tx)) / grid_size
box_cy = (cell_y + sigmoid(ty)) / grid_size
box_w, box_h = np.exp(tw), np.exp(th) # scaled by anchor size in practice
class_id = int(np.argmax(class_scores))
final_score = conf * class_scores[class_id] # objectness * class probability
return (box_cx, box_cy, box_w, box_h), class_id, final_score
def sigmoid(z):
return 1 / (1 + np.exp(-z))Confidence is objectness times class probability
Non-max suppression: cleaning up duplicate boxes
A single object near a grid-cell boundary often gets multiple overlapping boxes predicted for it, from neighboring cells or different anchor boxes. Non-max suppression (NMS) removes the duplicates: sort all predicted boxes by confidence, take the highest-confidence box, discard every remaining box that overlaps it above an IoU (intersection-over-union) threshold (commonly 0.5), and repeat with the next highest remaining box. The result is one box per actual object instead of a cluster of near-identical ones.
Two-stage detectors vs YOLO
| Property | Faster R-CNN (two-stage) | YOLO (single-stage) |
|---|---|---|
| Pipeline | Region proposal, then classify each region | One forward pass predicts all boxes |
| Speed | Slower, more accurate on small/overlapping objects | Much faster, suited to real-time video |
| Localization of small objects | Generally stronger | Historically weaker (improved in later versions) |
| Typical use case | Offline, accuracy-critical pipelines | Real-time detection, robotics, video |
The grid constraint is also a limitation
What's Next?
YOLO's single-pass, dense-prediction philosophy has since spread well beyond detection — the same idea of directly regressing structured outputs in one network pass shows up in pose estimation, instance segmentation heads, and real-time tracking systems. The core tradeoff it exposed, between single-stage speed and two-stage precision, remains a recurring design decision across modern vision pipelines.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.