The phrase 'face recognition' covers at least three distinct tasks, and conflating them causes real confusion about what a system can actually do. Face detection answers 'is there a face in this image, and where?' — it draws a box, nothing more. Face verification answers a yes/no question: 'do these two face images show the same person?' — used for unlocking a phone with your own face. Face identification answers an open-ended question: 'who, among a large database of known people, does this face belong to?' — used in surveillance and search scenarios. Identification is dramatically harder and riskier than verification, because it involves comparing against many candidates instead of just one.

Detection first: finding the face before analyzing it

Before any recognition can happen, a face detector (historically Haar cascades or HOG-based detectors, now typically a CNN-based detector similar in spirit to general object detectors) locates face bounding boxes and often facial landmarks — eye corners, nose tip, mouth corners — used to align the face into a standard, front-facing crop. This alignment step matters: recognition models perform much better on consistently cropped, roughly frontal faces than on raw, arbitrarily rotated crops.

Embeddings: turning a face into a point in space

Modern face recognition doesn't classify a face directly into a fixed list of names. Instead, a CNN is trained to map any face image to a point in a high-dimensional embedding space (commonly 128 to 512 dimensions) such that images of the same person land close together and images of different people land far apart. Recognition then becomes a geometry problem: verification compares the distance between two embeddings against a threshold, and identification finds the closest embedding in a database (nearest-neighbor search).

python

Triplet loss: training embeddings to separate correctly

A key innovation (popularized by FaceNet) is triplet loss. Training examples are grouped into triplets: an anchor image, a positive image (same person as the anchor), and a negative image (a different person). The loss pushes the anchor-positive distance down and the anchor-negative distance up, with a margin enforcing a minimum gap between them. Training on millions of such triplets produces an embedding space where distance genuinely tracks identity similarity, rather than incidental factors like lighting or pose — though in practice those factors still leak in to some degree.

text

Note

Face recognition systems have repeatedly been shown, in peer-reviewed studies including NIST's own Face Recognition Vendor Test program, to have measurably higher error rates for women, for people with darker skin tones, for older adults, and for children, compared to the demographic groups most represented in common training datasets. A system reporting 99%+ accuracy on a benchmark like LFW does not mean 99% accuracy for every demographic group in deployment — benchmark datasets are frequently skewed toward certain populations, and error rates can differ by an order of magnitude across groups for the same system. This has had serious real-world consequences, including documented wrongful arrests following false identification matches. Any claim of near-perfect face recognition accuracy should be read with this caveat attached, and identification (open-ended search) carries substantially more risk than verification (one-to-one confirmation) precisely because a false match among thousands of candidates is far more likely than a false match between two specific people.

Detection vs. verification vs. identification

Task	Question answered	Typical risk level	Example use case
Detection	Is there a face here, and where?	Low — no identity claim made	Auto-focus, photo cropping, blurring faces for privacy
Verification (1:1)	Are these two faces the same person?	Moderate — single comparison, user-initiated	Phone unlock, ID document matching
Identification (1:N)	Who is this, among many known people?	High — false matches scale with database size	Surveillance search, law enforcement lookups

Threshold choice is a policy decision, not just a technical one

The distance threshold used to decide 'same person' vs. 'different person' directly trades off false accepts (wrongly matching two different people) against false rejects (wrongly failing to match the same person). Lowering the threshold to catch more true matches also lets in more false ones. For a phone unlock, a false accept is a minor security issue; for an identification system feeding a law enforcement decision, a false accept can lead to wrongful suspicion of an innocent person. The appropriate threshold is fundamentally a judgment about acceptable error cost, not a number a model can determine on its own — which is part of why face recognition deployment in sensitive contexts remains legally and ethically contested.

What's next

Face recognition is a specialized application of embedding-based similarity, a pattern that recurs across computer vision whenever 'is this the same thing' matters more than 'what category is this.' Continue to video action recognition to see how identity and motion information combine across frames, or revisit object detection to see how the underlying detection step generalizes beyond faces.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →

Need production AI or MarTech expertise? I build custom LLM workflows, server-side GTM tagging, and analytics infrastructures professionally.

Need custom AI or MarTech setup? Let's build together.

Let's talk →