Word Sense Disambiguation

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

The same word, different meanings

'Bank' can mean a financial institution or a river's edge. 'Bass' can mean a fish or a low musical frequency. Word Sense Disambiguation (WSD) is the task of determining which sense of a polysemous word is intended given its context — a prerequisite for many downstream tasks (machine translation, question answering, information retrieval) where picking the wrong sense produces nonsensical or incorrect output.

Sense inventories: you need a dictionary of meanings first

WSD requires a predefined inventory of possible senses per word. WordNet is the standard English resource: a lexical database organizing words into synsets (sets of synonymous senses), each with a gloss (definition) and relations to other synsets (hypernyms, hyponyms, meronyms). Given 'bank' has roughly 10 WordNet senses, WSD for a given occurrence is a classification problem over those 10 candidate labels, conditioned on context. The fineness of sense granularity is itself a design choice — WordNet's senses are often finer-grained than practically useful, motivating coarser sense groupings in applied systems.

From knowledge-based to neural approaches

The classic Lesk algorithm disambiguates by computing word-overlap between the context around the ambiguous word and each candidate sense's dictionary gloss, picking the sense whose gloss overlaps most. This is simple and unsupervised but brittle, since dictionary glosses are short and the overlap signal is sparse. Supervised approaches (feature-based classifiers trained on sense-tagged corpora like SemCor) improved accuracy by incorporating surrounding words, POS tags, and syntactic dependencies as features. Modern neural WSD casts the problem as a nearest-neighbor lookup in contextual embedding space: encode the ambiguous word's context with a transformer (BERT), encode each candidate sense's definition similarly, and pick the sense whose embedding is closest to the word's contextual representation — this is the dominant modern approach (e.g., GlossBERT).

wsd_lesk_vs_contextual.py

python

from nltk.wsd import lesk
from nltk.corpus import wordnet as wn

# --- Classic Lesk algorithm ---
sentence = "He went to the bank to deposit his paycheck".split()
sense = lesk(sentence, "bank", "n")
print(sense, "->", sense.definition())
# Expect something like: "a financial institution..."

# --- Contextual embedding approach (sketch) ---
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")
context_embedding = model.encode("He went to the bank to deposit his paycheck")

candidate_senses = [s.definition() for s in wn.synsets("bank", pos=wn.NOUN)]
gloss_embeddings = model.encode(candidate_senses)

scores = util.cos_sim(context_embedding, gloss_embeddings)
best_sense_idx = scores.argmax().item()
print(candidate_senses[best_sense_idx])

Method	Supervision needed	Typical accuracy regime
Lesk (gloss overlap)	None (unsupervised)	Low — brittle on short glosses
Supervised feature classifier	Sense-tagged corpus (SemCor)	Moderate, limited to seen word senses
Contextual embedding + gloss matching	Pretrained LM, optionally fine-tuned	High, generalizes better to unseen contexts
Modern LLM zero-shot prompting	None (in-context)	High on common words, weaker on rare/technical senses

Large language models blur the line between WSD and 'just understanding'

Modern contextual language models don't disambiguate senses as an explicit, separate step — sense distinctions are implicitly resolved as part of producing a contextual embedding or next-token prediction. This is why classic WSD as a standalone benchmark task has become less central in NLP research even as the underlying problem (context-dependent meaning) remains fully relevant inside every modern system.

Sense inventories don't transfer across domains

A general-purpose sense inventory like WordNet may miss domain-specific senses entirely — 'cell' in a biology paper vs. a telecom paper vs. a prison context carries three distinct technical meanings that a generic inventory handles poorly. Domain-specific WSD often requires a custom sense inventory or terminology resource rather than general lexical databases.

What's Next?

Resolving word-level ambiguity is one layer of NLP structure; a related but distinct challenge is discovering the latent topical structure spanning an entire document collection.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →