Graph Neural Networks (GNNs)

Data that isn't a grid or a sequence

CNNs assume a grid (pixels). Transformers and RNNs assume a sequence (tokens in order). But a huge amount of real data is neither — it's a graph: nodes connected by edges, with no fixed ordering and no fixed neighborhood size. A social network, a molecule, a road network, a knowledge graph, a citation network — all graphs.

You could flatten a graph into a vector or force it into a grid, but you'd throw away the one thing that matters most: the connectivity structure. Graph Neural Networks (GNNs) are built to consume that structure directly, learning representations that respect which nodes are connected to which.

The core idea: message passing

Every mainstream GNN variant — GCN, GraphSAGE, GAT, GIN — is built around the same loop, called message passing (or neighborhood aggregation):

1. Each node starts with a feature vector (e.g., atom type, user profile, word embedding).
2. At each layer, every node collects messages from its neighbors — typically a transformation of each neighbor's current vector.
3. The node aggregates those messages (sum, mean, max, or a learned attention-weighted combination).
4. The node combines the aggregated message with its own previous vector and passes it through a small neural network to produce its next-layer representation.

Stack k of these layers and each node's final vector encodes information from everything within k hops — its k-hop neighborhood. This is the graph analogue of a CNN's receptive field growing with depth.

Variants: GCN, GraphSAGE, GAT

GCN (Graph Convolutional Network) normalizes aggregation by node degree, approximating a spectral graph convolution — the first widely adopted GNN.

GraphSAGE samples a fixed-size subset of neighbors rather than using all of them, which makes it scale to graphs with millions of nodes (crucial for industrial recommendation systems).

GAT (Graph Attention Network) replaces fixed aggregation weights with learned attention — each neighbor's message is weighted by how relevant it is, the same idea as transformer attention, but restricted to graph neighbors instead of the whole sequence.

GNN variants at a glance

What GNNs are used for in practice

Three task types recur across applications:

Node-level tasks — classify or embed individual nodes (fraud detection on a transaction graph, predicting a user's interests on a social network).

Edge-level tasks — predict whether an edge should exist (link prediction: "will these two users become friends," "does this drug interact with this protein," knowledge-graph completion).

Graph-level tasks — produce one prediction for the whole graph, usually by pooling all final node vectors (molecular property prediction: will this molecule be toxic, what's its binding affinity — this is a mainstay of modern drug discovery pipelines like those used in virtual screening).

Limitations: over-smoothing and scalability

Stack too many message-passing layers and every node's representation converges toward the same average vector — a problem called over-smoothing, since each layer blends a node further with its neighborhood until local distinctions wash out. Most practical GNNs stay shallow (2-4 layers) as a result, which caps how far information can travel compared to, say, a transformer's full-sequence attention.

Scalability is the other persistent issue: naive message passing over a graph with billions of edges does not fit in memory, which is why sampling-based approaches like GraphSAGE and mini-batch training over subgraphs are standard in industry rather than optional optimizations.

What's next

GNNs generalize convolution to irregular structure; the next step for relational reasoning is attention-based architectures without any convolutional prior — look at the Transformers lesson for how self-attention handles a similar flexibility problem for sequences, and consider how a fully-connected attention graph is itself a special case of a GNN where every node is a neighbor.