One-Shot Learning & Meta-Learning (MAML)
Learning from almost nothing
Deep learning's dirty secret is data hunger: a typical classifier needs thousands of labeled examples per class. Humans don't work that way — show a child one picture of an unfamiliar animal and they can usually recognize it again. One-shot (or few-shot) learning is the attempt to give models that same ability: learn a new task or category from one example, or a handful.
Siamese networks (previous lesson) solve one slice of this by learning similarity. Meta-learning tackles it more generally: instead of learning a task, the model learns how to learn a task — a learning algorithm that itself was produced by training.
Learning to learn: the meta-learning framing
Standard training: one dataset, one task, one set of final weights. Meta-learning restructures training around many small tasks instead of one big one. Each "meta-training" episode samples a new mini-task — say, a 5-way classification problem with 5 examples per class (a "5-way 5-shot" episode) — and the model is evaluated on how quickly and well it adapts to that episode using only those few examples.
Across thousands of such episodes, the model's parameters are optimized not to solve any one task, but to be a good starting point for solving any task drawn from the same distribution, after a small amount of adaptation. At test time, a genuinely new task (new classes, never seen in training) is handed a handful of labeled examples, the model adapts briefly, and it's evaluated.
Note
MAML: Model-Agnostic Meta-Learning
MAML (Finn et al., 2017) is the clearest instance of this idea. It directly optimizes the initial weights θ such that one or a few steps of ordinary gradient descent, starting from θ, produce a good model for any new task.
Concretely, for each sampled task: take a few gradient steps on that task's small training set starting from θ, producing task-specific weights θ'. Then measure how well θ' performs on held-out examples from that same task. That performance becomes the loss that updates the original θ — meaning MAML differentiates through the inner-loop gradient steps themselves (a gradient of a gradient). This is what "model-agnostic" means: nothing about the inner adaptation step is specific to a particular architecture, so MAML can wrap around any model trained with gradient descent.
Approaches to learning from few examples
| Approach | How it adapts to a new task | Strength |
|---|---|---|
| Fine-tuning a pretrained model | Many gradient steps on new data | Simple, works with lots of data |
| Siamese / metric learning | No adaptation — compares embeddings | Fast, no retraining at all |
| MAML (optimization-based) | A few gradient steps from a learned init | General-purpose, model-agnostic |
| In-context learning (LLMs) | No weight updates — examples in the prompt | Zero training cost per task |
Note
Where this matters today
Pure MAML is now more of a conceptual touchstone than a production workhorse — few teams run MAML training loops in industry. But its core idea — optimize for fast adaptability rather than final performance — shows up everywhere: in robotics (learning policies that adapt quickly to a new robot body or environment), personalization (adapting a shared model to one user with minimal data), drug discovery (predicting properties for rare molecule classes with few examples), and it directly motivated research into why large language models can do in-context learning — adapting to a task from a few examples in the prompt without any weight updates at all.
What's next
Meta-learning optimizes a model to adapt fast with ordinary gradient descent. Normalizing Flows and Neural ODEs, covered next, take a different angle on "rethink the architecture" — asking what happens if you replace discrete layers or fixed transformations with continuous or exactly-invertible mathematical structure.
I build these systems professionally.
Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.