Depth Estimation from Images
Recovering 3D from a flat image
Depth estimation predicts, for every pixel in an image, its distance from the camera. A single 2D photograph fundamentally throws away depth information — many different 3D scenes can project to the identical 2D image — so depth estimation is an inherently ill-posed problem that must be solved either with extra hardware (stereo cameras, LiDAR) or with strong learned priors about how the world typically looks (monocular estimation).
Stereo depth: triangulation from two viewpoints
With two calibrated cameras a known baseline distance apart, depth can be computed geometrically. For a point visible in both images, its horizontal pixel offset between the left and right image — the disparity $d$ — relates to depth $Z$ by:
$$Z = \frac{f \cdot B}{d}$$
where $f$ is the camera's focal length and $B$ is the baseline between the two cameras. Larger disparity means the point is closer; disparity shrinks toward zero as depth grows toward infinity. The core algorithmic challenge is stereo matching: finding, for each pixel in the left image, its corresponding pixel in the right image, usually by searching along the same horizontal scanline (epipolar line) and comparing local patches.
Why stereo matching struggles on flat, repetitive surfaces
Monocular depth: depth from a single image using learned priors
Humans infer depth from a single photo constantly, using monocular cues: relative object size, texture gradients, occlusion ordering, and known object scale (a person is roughly 1.7m regardless of how far away they appear). Monocular depth estimation networks learn these same statistical regularities from large datasets of images paired with ground-truth depth (from LiDAR, stereo rigs, or structure-from-motion).
Models like MiDaS and DPT (Dense Prediction Transformer) use an encoder (often a ViT) to extract features at multiple scales, then a decoder that upsamples back to full resolution — the same encoder-decoder-with-skip-connections shape used in U-Net for segmentation, just predicting a continuous depth value per pixel instead of a class label.
import numpy as np
def disparity_to_depth(disparity_map, focal_length_px, baseline_m, eps=1e-6):
"""disparity_map: HxW array of pixel disparities (left vs right image).
Returns a depth map in meters. Zero/near-zero disparity -> clipped to avoid div-by-zero."""
safe_disparity = np.clip(disparity_map, eps, None)
depth_map = (focal_length_px * baseline_m) / safe_disparity
return depth_map
# Monocular networks instead output "relative" or "affine-invariant" depth directly:
# they predict depth UP TO an unknown global scale and shift, since a single image
# alone carries no absolute metric scale information.Stereo vs monocular depth estimation
| Property | Stereo depth | Monocular depth |
|---|---|---|
| Hardware needed | Two calibrated cameras | Any single camera |
| Depth scale | Absolute (metric), from geometry | Often relative/scale-ambiguous unless calibrated |
| Failure mode | Textureless or repetitive surfaces | Unfamiliar scenes outside training distribution |
| Underlying principle | Geometric triangulation | Learned statistical priors |
Monocular depth is usually relative, not absolute
What's Next?
Depth maps are the stepping stone to full 3D scene representations — once you have per-pixel depth across multiple viewpoints, you can project and fuse them into point clouds, meshes, or the implicit volumetric representations that power novel-view-synthesis methods like NeRF and Gaussian splatting.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.