Cosine Similarity
Why Euclidean distance fails for text
When you embed text into vectors, the magnitude of the vector often just reflects how much text there was — a long document naturally has larger values. Two documents saying the same thing but at different lengths shouldn't be far apart in your search results.
Cosine similarity solves this by measuring the angle between two vectors instead of the distance between their endpoints. Documents that are semantically identical will point in the same direction, regardless of their magnitude. Two vectors are maximally similar (cosine = 1) when they point exactly the same way. Completely unrelated vectors are orthogonal (cosine = 0). Opposite vectors score -1.
The formula
Cosine similarity between two vectors A and B is:cos(θ) = (A · B) / (||A|| × ||B||)
where A · B is the dot product (multiply corresponding elements, sum them up) and ||A|| is the vector's magnitude (square root of sum of squared elements).
In practice: if you normalise both vectors to unit length first (divide each by its magnitude), the dot product becomes identical to the cosine similarity. This is why vector databases normalise embeddings on insertion — the expensive division is done once at index time, and every similarity computation at query time is just a fast dot product.
Note
Where it's used in practice
Semantic search: The user's query is embedded. Cosine similarity against all document embeddings in the index finds the semantically closest matches — the foundation of every RAG pipeline.
Recommendation systems: Netflix, Spotify, Amazon. User preferences and item features are both embedded. Cosine similarity finds items closest to a user's taste vector.
Duplicate detection: Two sentences with cosine similarity above ~0.95 are probably duplicates or near-duplicates. Used to deduplicate training data for LLMs and to detect plagiarism.
Clustering evaluation: Measure cohesion within clusters (average pairwise cosine similarity) and separation between clusters. High cohesion and low inter-cluster similarity = good clusters.
Cosine similarity vs. dot product vs. Euclidean distance
Euclidean distance (L2): measures physical distance between endpoints. Good for spatial data where magnitude matters (e.g., location coordinates). Penalises magnitude differences.
Dot product: fast, but magnitude-dependent. Higher for longer vectors even if they point the same way. Best when you've normalised everything — then it equals cosine similarity.
Cosine similarity: magnitude-invariant. Best for embeddings, text, or any situation where the direction of a vector encodes meaning but the magnitude is an artifact of representation.
Most vector databases let you choose the metric. For text and embedding use cases, always use cosine (or equivalently, dot product after normalising).
Note
1 - cosine_similarity) to turn it into a proper metric when needed. Most vector databases handle this internally — just tell them you want cosine similarity and they take care of the rest.What's next
You've now completed the core ML track — from perceptrons to gradient descent, embeddings, transformers, agents, and the key classical algorithms. The next track covers the practical side: how to build production AI systems with RAG, tool-calling, and orchestration frameworks. Or explore the MarTech track to see how data engineering and AI intersect in marketing systems.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.