Question Answering Systems
Answering, not just retrieving
A search engine returns documents; a Question Answering (QA) system returns the actual answer — a span of text, a generated sentence, or a structured value. QA spans a spectrum of formulations, each with distinct architectures: extractive QA pulls an answer span directly from a given passage, open-domain QA must first find the relevant passage from a large corpus before extracting or generating an answer, and generative QA synthesizes a free-form answer, potentially combining information from multiple sources.
| QA type | Input | Output | Example architecture |
|---|---|---|---|
| Extractive (reading comprehension) | Question + given passage | Span of text copied from passage | BERT fine-tuned on SQuAD |
| Open-domain QA | Question only, no passage given | Answer, found from a large corpus | Retriever + reader (DPR + BERT) |
| Generative QA | Question (+ optional retrieved context) | Free-form generated text | RAG, fine-tuned seq2seq, LLM |
| Knowledge-base QA | Question, structured KB available | Structured query result | Semantic parsing to SPARQL/SQL |
Extractive QA: predicting answer spans
Extractive QA, popularized by the SQuAD benchmark, frames answering as finding the start and end token positions of the answer within a given passage. A transformer encoder processes the question and passage jointly (typically concatenated as [CLS] question [SEP] passage [SEP]), and two separate classification heads predict, for every token, the probability it's the answer's start and end position respectively: $P_{\text{start}}(i) = \text{softmax}(W_s h_i)$, $P_{\text{end}}(j) = \text{softmax}(W_e h_j)$. The final answer is the span $(i, j)$ with $i \le j$ maximizing $P_{\text{start}}(i) \cdot P_{\text{end}}(j)$. This architecture is fast, grounded (the answer is guaranteed to be verbatim text from the passage, eliminating hallucination), but fundamentally limited to questions answerable by a literal substring of the given passage.
Open-domain QA: retrieve, then read
When no passage is given, the system must first locate relevant text from a large corpus (Wikipedia, an enterprise document store) before answering — the classic retriever-reader architecture. The retriever (traditionally BM25/TF-IDF, now typically a dense bi-encoder like DPR, Dense Passage Retrieval, trained so that question and answer-containing-passage embeddings land close together under cosine similarity) narrows millions of documents down to a handful of candidates. The reader (an extractive or generative QA model) then processes those candidates to produce a final answer. This two-stage design is exactly the pattern behind Retrieval-Augmented Generation (RAG), where the reader is replaced with a generative LLM conditioned on the retrieved passages.
from sentence_transformers import SentenceTransformer, util
from transformers import pipeline
retriever = SentenceTransformer("facebook-dpr-question_encoder-single-nq-base")
passage_embeddings = retriever.encode(corpus_passages, convert_to_tensor=True)
def retrieve_top_k(question, k=5):
q_emb = retriever.encode(question, convert_to_tensor=True)
scores = util.cos_sim(q_emb, passage_embeddings)[0]
top_idx = scores.topk(k).indices
return [corpus_passages[i] for i in top_idx]
reader = pipeline("question-answering", model="deepset/roberta-base-squad2")
def answer_open_domain(question):
candidates = retrieve_top_k(question)
best = max(
(reader(question=question, context=p) for p in candidates),
key=lambda r: r["score"],
)
return best["answer"], best["score"]
print(answer_open_domain("When was SpaceX founded?"))Retriever quality is the real bottleneck in most open-domain QA failures
Generative QA trades grounding for flexibility — and risks hallucination
What's Next?
QA systems embedded in a conversational product need more than a single question-answer turn — they need to track what's been discussed across an entire multi-turn dialogue.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.