Hierarchical Classification
When labels form a tree, not a flat set
Flat multiclass classification treats all classes as equally distinct siblings. But many real label taxonomies are hierarchical: an e-commerce catalog has Electronics → Audio → Headphones → Wireless Earbuds; biological taxonomy has Kingdom → Phylum → ... → Species; support tickets might be Billing → Refunds → Duplicate Charge. Hierarchical classification exploits this structure instead of flattening it into one giant label set, which both improves accuracy (coarse categories are easier to get right) and produces errors that are less costly (confusing two sibling leaf nodes is better than confusing two unrelated branches).
Local classifiers per node (top-down approach)
The most common pattern trains a separate classifier at each internal node of the hierarchy, deciding only among that node's immediate children. Classification proceeds top-down: at the root, pick a top-level category; descend into that branch; repeat until reaching a leaf. This decomposes one hard, many-way problem into several easier, few-way problems. The catch is error propagation: a wrong decision at a shallow level can never be corrected lower down, since the model never even considers the other branches. This is sometimes called the 'local classifier per parent node' strategy, and it's the direct hierarchical analogue of a decision tree's recursive splitting.
Global (big-bang) classifiers
An alternative is to train a single flat model over all leaf labels but incorporate the hierarchy into the loss function or the feature representation — for instance, penalizing a misclassification less if the predicted and true leaf share a common ancestor (a hierarchical/tree-distance loss), or concatenating ancestor-path embeddings as extra features. This avoids error propagation but is harder to train and requires a hierarchy-aware loss implementation, which isn't built into most off-the-shelf libraries.
# A simplified hierarchical (local classifier per node) pipeline sketch
from sklearn.linear_model import LogisticRegression
hierarchy = {
"root": ["electronics", "clothing"],
"electronics": ["audio", "computers"],
"audio": ["headphones", "speakers"],
}
class HierarchicalClassifier:
def __init__(self, hierarchy):
self.hierarchy = hierarchy
self.node_models = {} # node -> fitted classifier over its children
def fit(self, X, y_paths):
# y_paths: list of full label paths, e.g. ["root","electronics","audio","headphones"]
for node, children in self.hierarchy.items():
mask_rows, mask_labels = [], []
for i, path in enumerate(y_paths):
if node in path:
idx = path.index(node)
if idx + 1 < len(path) and path[idx + 1] in children:
mask_rows.append(i)
mask_labels.append(path[idx + 1])
if mask_rows:
clf = LogisticRegression().fit(X[mask_rows], mask_labels)
self.node_models[node] = clf
def predict_path(self, x):
path, node = ["root"], "root"
while node in self.node_models:
node = self.node_models[node].predict([x])[0]
path.append(node)
return path
Use hierarchy-aware metrics, not plain accuracy
| Approach | Error propagation risk | Training complexity | Respects hierarchy in loss |
|---|---|---|---|
| Flat multiclass | N/A (no hierarchy used) | Low | No |
| Local classifier per node | High — early mistakes compound | Moderate — many small models | Implicitly, via structure |
| Global hierarchy-aware loss | Low | High — custom loss needed | Explicitly |
What's next?
The local-classifier-per-node pattern is really a special case of a broader systems idea: chaining classifiers so that each one narrows the problem for the next, trading a bit of early-stage accuracy risk for large downstream efficiency gains — a pattern that shows up well beyond label taxonomies.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.