Text Clustering Techniques
Grouping text without predefined categories
Not every text organization problem has known categories ahead of time. Text clustering groups documents by similarity without any labels — useful for discovering emergent themes in customer feedback, deduplicating near-identical support tickets, or organizing a large unlabeled document collection before deciding what taxonomy even makes sense. The pipeline has two independent choices that each matter enormously: how documents are represented as vectors, and which clustering algorithm groups those vectors.
| Algorithm | Needs K upfront? | Handles non-spherical clusters? | Scales to large N |
|---|---|---|---|
| K-Means | Yes | No — assumes spherical clusters | Yes, with minibatch variants |
| Hierarchical (agglomerative) | No — cut dendrogram at any level | Better than K-Means | Poor — O(n²) or worse |
| DBSCAN | No — density-based | Yes | Moderate |
| HDBSCAN | No — density-based, hierarchical | Yes, handles varying density well | Good |
Representation matters more than the clustering algorithm
Clustering TF-IDF vectors directly groups documents by shared vocabulary, which conflates topic similarity with vocabulary overlap — two documents about the same topic using different words won't cluster together. Clustering dense sentence embeddings (from sentence-transformers or pooled transformer outputs) groups documents by semantic similarity instead, which is almost always what's actually wanted for topic discovery. This single choice — sparse lexical vs. dense semantic representation — typically has far more impact on cluster quality than the choice of clustering algorithm itself.
from sentence_transformers import SentenceTransformer
from sklearn.cluster import KMeans
import hdbscan
import numpy as np
encoder = SentenceTransformer("all-mpnet-base-v2")
embeddings = encoder.encode(documents, show_progress_bar=True)
# Option A: K-Means, when you have a reasonable guess for K and expect
# roughly balanced, spherical clusters.
kmeans = KMeans(n_clusters=8, random_state=42, n_init=10)
kmeans_labels = kmeans.fit_predict(embeddings)
# Option B: HDBSCAN, when K is unknown and clusters may vary in density
# and size — it also explicitly labels outliers as noise (-1), rather
# than forcing every point into some cluster.
clusterer = hdbscan.HDBSCAN(min_cluster_size=15, metric="euclidean")
hdbscan_labels = clusterer.fit_predict(embeddings)
n_noise = np.sum(hdbscan_labels == -1)
print(f"{len(set(hdbscan_labels)) - 1} clusters found, {n_noise} noise points")Choosing K and evaluating cluster quality without labels
Without ground-truth labels, cluster quality must be assessed intrinsically. The silhouette score measures, for each point, how much closer it is to its own cluster's points versus the nearest other cluster, averaged across all points (range -1 to 1, higher is better). The elbow method plots within-cluster sum of squared distances against K and looks for the point of diminishing returns. Neither is a substitute for human review of actual cluster contents — in practice, teams almost always sample and manually inspect the top terms or representative documents per cluster (e.g., via BERTopic, which combines embedding-based clustering with an automatic per-cluster keyword extraction step) to judge whether clusters are semantically coherent and actionable, not just numerically compact.
Density-based methods often fit real text data better than K-Means
High-dimensional embeddings make distance metrics behave unintuitively
What's Next?
Clustering groups documents at a coarse level; a closely related but finer-grained question is how to directly measure the similarity between any two specific pieces of text.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.