Grammar Error Correction Models
Fixing text without changing its meaning
Grammatical Error Correction (GEC) takes a sentence containing grammatical mistakes — subject-verb agreement errors, wrong prepositions, missing articles, tense inconsistencies — and produces a corrected version, while preserving the original meaning and style as closely as possible. It's used in writing assistants (Grammarly), language-learning tools, and as a preprocessing step before other NLP tasks that are sensitive to grammatical noise.
Two competing framings: sequence tagging vs. sequence-to-sequence
GEC can be approached two ways. Sequence tagging (edit-based) approaches, like GECToR, frame correction as predicting a small, discrete edit operation per token — keep, delete, replace with X, insert X before — rather than regenerating the whole sentence. This is fast (single forward pass, no autoregressive decoding) and the edits are directly interpretable, but limited to a predefined vocabulary of edit operations and struggles with errors requiring substantial reordering. Sequence-to-sequence (seq2seq) approaches treat GEC as translation: encode the erroneous sentence, decode a corrected one token by token, using the same encoder-decoder transformer architectures used in machine translation. This handles arbitrarily complex rewrites naturally but is slower (autoregressive decoding) and can occasionally over-correct, rephrasing things that weren't actually wrong.
| Approach | Speed | Handles reordering/rewrites | Risk of over-correction |
|---|---|---|---|
| Edit-tagging (GECToR-style) | Fast — single pass, no decoding | Limited | Low — edits are minimal by design |
| Seq2seq (transformer encoder-decoder) | Slower — autoregressive decoding | Yes, handles complex rewrites | Moderate — can rewrite more than needed |
| LLM prompting ('fix grammar in this text') | Depends on model size/latency | Yes | Higher — may rewrite style, not just grammar, unless constrained |
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("grammarly/coedit-large")
model = AutoModelForSeq2SeqLM.from_pretrained("grammarly/coedit-large")
def correct_grammar(sentence: str) -> str:
prompt = f"Fix the grammar: {sentence}"
inputs = tokenizer(prompt, return_tensors="pt")
output_ids = model.generate(**inputs, max_length=128, num_beams=4)
return tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(correct_grammar("She dont like going to the market yesterday."))
# -> "She didn't like going to the market yesterday."
# Minimal-edit philosophy check: a good GEC system changes only what's
# wrong, not stylistic choices that are merely unusual but grammatical.Evaluation: why exact match isn't enough
GEC evaluation can't rely on simple string match against a single reference correction, because there are often multiple equally valid ways to fix a sentence. Standard metrics instead compute edit-level precision and recall against one or more annotated reference corrections — the ERRANT toolkit, widely used on the CoNLL-2014 and BEA-2019 shared-task benchmarks, extracts the specific edit operations (insertions, deletions, substitutions, categorized by error type — determiner, preposition, verb tense) from both the system output and references, then scores overlap at the edit level rather than requiring the full sentence to match verbatim. This also allows fine-grained analysis of which error types a system handles well versus poorly.
Minimal edits are a deliberate design goal, not a side effect
LLM-based correction needs explicit constraints to avoid scope creep
What's Next?
Correcting grammar is one piece of a broader text normalization problem — getting raw, messy text into a clean, consistent form before any downstream model or system processes it.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.