Convolutional networks are excellent at detecting whether certain features are present, but pooling layers — which discard precise spatial information to achieve translation invariance — throw away exactly the information needed to tell whether those features are arranged correctly. A CNN can be fooled by an image containing the right parts (eyes, nose, mouth) scrambled into the wrong arrangement, because max-pooling only asks "is this feature present somewhere nearby," not "is it present in the right place relative to the other features."

Geoffrey Hinton, who had co-invented backpropagation and popularized deep belief networks, proposed Capsule Networks (CapsNets) in 2017 specifically to address this: represent not just whether a feature exists, but its full pose — position, orientation, scale — and explicitly model how parts combine into wholes.

Capsules: vectors instead of scalars

A traditional CNN neuron outputs a single scalar — a confidence that some feature is present. A capsule outputs a vector: the vector's length represents the probability that the entity it detects is present, and the vector's orientation (direction) encodes that entity's instantiation parameters — its pose, such as rotation, thickness, or skew.

A capsule for "eye" doesn't just say "an eye is present with 90% confidence" — its output vector also encodes which way that eye is tilted and how big it is, information a scalar activation simply cannot carry.

Dynamic routing: parts voting for wholes

The mechanism that makes capsules useful for part-whole reasoning is dynamic routing by agreement. Each lower-level capsule (say, representing an eye or a nose) makes a prediction, via a learned transformation matrix, for what the pose of a higher-level capsule (say, a face) should be, given its own detected pose. If several lower-level capsules' predictions agree strongly with each other, that higher-level capsule becomes active, and the agreeing capsules are routed to it more strongly in an iterative refinement process.

This replaces pooling's crude "take the max" with something more like a vote: a face capsule activates when its parts' predicted poses are mutually consistent, and this consistency check is exactly what catches scrambled-feature images that fool ordinary CNNs.

python

Note

The agreement-based routing means a capsule network's internal representation of "face" is tied to a specific, mutually consistent configuration of parts — not just their co-occurrence. In principle this should make capsule networks more robust to viewpoint changes (a rotated face is still recognized because the pose vectors rotate consistently together) and harder to fool with adversarial examples that just scatter the right features in the wrong places.

CNNs vs. Capsule Networks

Property	CNN (with pooling)	Capsule Network
Unit output	Scalar activation	Vector (length = presence, direction = pose)
Spatial relationships	Discarded by pooling	Explicitly modeled via routing
Aggregation mechanism	Max/average pooling	Dynamic routing by agreement
Training/inference cost	Low — highly optimized, hardware-friendly	High — iterative routing is slow
Production maturity	Extremely mature, ubiquitous	Limited — mostly research-stage

Note

Despite the elegant motivation, capsule networks have not displaced CNNs in practice. Dynamic routing is computationally expensive and doesn't parallelize as cleanly on GPUs as convolution does, capsule networks have struggled to scale to large, complex datasets (ImageNet-scale results have lagged well behind CNNs and transformers), and subsequent research found other ways to get CNNs some translation-equivariance benefits without the routing overhead. CapsNets remain primarily a research direction and a widely taught idea for what CNNs miss, rather than a deployed production architecture.

The idea's legacy

Even without broad adoption, capsule networks influenced how the field thinks about equivariance versus invariance — the question of whether a network should throw away positional information (invariance, what pooling does) or preserve it in a structured, transformable way (equivariance, what capsules attempt). That framing shows up in later equivariant neural network research and in vision transformers' design choices around how spatial information is retained through a network.

What's next

Capsule networks model structure using explicit, engineered pose vectors and routing rules. Spiking Neural Networks take structural inspiration from biology in a completely different direction — modeling how biological neurons actually communicate, with discrete spikes in time rather than continuous activations.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →

Need production AI or MarTech expertise? I build custom LLM workflows, server-side GTM tagging, and analytics infrastructures professionally.

Need custom AI or MarTech setup? Let's build together.

Let's talk →