Intent Classification for Chatbots
What does the user actually want?
Before a chatbot can act, it needs to know what the user is trying to do. Intent classification maps a user utterance to one of a predefined set of intents — book_flight, check_order_status, cancel_subscription, fallback — which determines which downstream logic (dialogue flow, API call, slot-filling form) gets triggered. It's structurally identical to general text classification, but the specific constraints of conversational systems — short utterances, ambiguous phrasing, an ever-growing intent list, and a hard requirement to detect out-of-scope requests — make it its own well-studied sub-problem.
Why short utterances make this harder than typical text classification
Chatbot utterances are often just a few words ('cancel my order', 'where's my package'), giving classifiers far less context to work with than a full paragraph of product review text. This makes intent classifiers unusually sensitive to surface-level phrasing variation, and historically motivated heavy reliance on data augmentation — paraphrasing training utterances via back-translation, synonym substitution, or template generation — to ensure the classifier generalizes beyond the exact phrasings seen in training data. Modern fine-tuned transformer encoders (or few-shot LLM prompting) have substantially reduced, but not eliminated, this sensitivity.
from sentence_transformers import SentenceTransformer
import numpy as np
# Embedding-based few-shot intent classification: works well with very
# little training data per intent, and new intents can be added without
# retraining — just add new example embeddings.
encoder = SentenceTransformer("all-MiniLM-L6-v2")
intent_examples = {
"book_flight": ["book me a flight", "I want to fly to Paris", "reserve a plane ticket"],
"cancel_order": ["cancel my order", "I want to cancel my purchase", "undo my order"],
"check_status": ["where is my package", "track my order", "order status please"],
}
intent_centroids = {
intent: np.mean(encoder.encode(examples), axis=0)
for intent, examples in intent_examples.items()
}
def classify_intent(utterance, threshold=0.55):
emb = encoder.encode(utterance)
sims = {
intent: np.dot(emb, centroid) / (np.linalg.norm(emb) * np.linalg.norm(centroid))
for intent, centroid in intent_centroids.items()
}
best_intent, best_score = max(sims.items(), key=lambda kv: kv[1])
return best_intent if best_score >= threshold else "out_of_scope"
print(classify_intent("can you track my shipment")) # -> check_statusOut-of-scope detection: the problem a naive classifier can't solve
A standard softmax classifier is forced to assign every input to one of its known classes, even gibberish or genuinely unrelated requests ('what's the meaning of life') — it has no built-in notion of 'none of the above.' This is addressed by explicitly modeling an out-of-scope (OOS) class, trained on a diverse set of unrelated utterances, combined with confidence thresholding (reject predictions below a calibrated confidence) or distance-based methods (reject if the input's embedding is far from every known intent's training examples, as in the centroid-based approach above). Getting OOS detection right matters enormously for user trust — a chatbot confidently misrouting an out-of-scope request to the wrong intent handler produces a worse user experience than cleanly saying 'I didn't understand that.'
| Approach | New intent cost | OOS handling | Data needed per intent |
|---|---|---|---|
| Fine-tuned transformer classifier | Requires retraining | Needs explicit OOS training class | Dozens to hundreds |
| Embedding + centroid/kNN | Just add new examples, no retrain | Natural via similarity threshold | As few as 3-5 |
| LLM zero/few-shot classification | Just update the prompt's intent list | Can be instructed explicitly in-prompt | Zero to a handful |
Intent taxonomies grow organically and need ongoing maintenance
cancel_order vs. return_order) accumulate confusion over time. Regularly auditing misclassification confusion matrices and merging or splitting intents based on real usage patterns is as important as the model architecture itself for maintaining classification quality over the system's lifetime.Intent classification alone doesn't resolve the full request
book_flight doesn't tell you the destination, date, or number of passengers — that's a separate job handled by slot filling, operating jointly with (and often in the same model as) intent classification in modern joint intent-slot architectures.What's Next?
Once an intent is identified, extracting the specific parameters needed to actually fulfill that request is the natural next piece of the conversational pipeline.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.