A genetic algorithm (GA) solves optimization problems by simulating natural selection over a population of candidate solutions. Each candidate is encoded as a "chromosome" — typically a string of numbers or bits representing a possible answer. The algorithm evaluates how good each candidate is, preferentially breeds the better ones, introduces random variation, and repeats across many generations until the population converges on strong solutions.

This is a metaheuristic: it doesn't use gradients, derivatives, or any structural knowledge of the problem. It only needs a way to measure quality and a way to produce variation, which makes it applicable to problems where no clean mathematical gradient exists at all.

The core loop: selection, crossover, mutation

Every generation of a GA runs the same cycle. Selection picks parent candidates, biased toward higher-fitness individuals — common methods include roulette-wheel selection (fitness-proportional probability) and tournament selection (randomly sample a few candidates, pick the best). Crossover combines two parents' chromosomes to produce offspring, usually by splicing segments from each parent, so good partial solutions from both parents can combine into something better than either alone. Mutation randomly flips or perturbs small parts of a chromosome, introducing variation that selection and crossover alone couldn't generate, which prevents the population from getting stuck.

The new generation replaces (or partially replaces) the old one, and the cycle repeats. Over many generations, average fitness tends to rise, though not monotonically — this is stochastic search, not gradient descent.

python

The fitness function is the whole problem

A genetic algorithm is only as good as its fitness function — the scoring function that tells selection which individuals deserve to reproduce. Designing it is usually the real engineering work. It needs to discriminate meaningfully between mediocre and good solutions (a flat fitness landscape gives selection nothing to act on), and it needs to be fast to evaluate, since it runs once per individual per generation, potentially millions of times across a run.

A poorly designed fitness function produces a classic failure mode: the GA optimizes exactly what you measured rather than what you meant, exploiting any loophole in the scoring to maximize fitness without solving the actual problem — a phenomenon sometimes called "specification gaming" or "reward hacking" in the reinforcement learning literature, which evolutionary computation shares.

Note

Unlike gradient descent, which follows a single point downhill, a GA maintains an entire population of candidates exploring different regions of the search space simultaneously. This makes it naturally resistant to getting permanently stuck in one local optimum — a weak region of the population can die out while a stronger region thrives, without needing any restart logic.

When evolutionary search beats gradient-based optimization

Gradient descent needs a differentiable objective function — you must be able to compute how a small change in parameters changes the loss. Many real problems don't offer that: a fitness function might involve discrete decisions, simulation outcomes, non-differentiable control logic, or hardware-in-the-loop evaluation where there is no mathematical formula to differentiate at all.

Genetic algorithms and other evolutionary methods only need to evaluate fitness, not differentiate it, which makes them useful for combinatorial optimization (scheduling, circuit layout, the traveling salesman problem), neural architecture search (evolving network topologies, not just weights), evolving control policies for robots in simulation, and generative design in engineering, where physical simulations stand between a parameter choice and its score.

Genetic algorithms vs. gradient-based optimization

Property	Genetic algorithms	Gradient descent
Requires differentiable objective	No	Yes
Search style	Population-based, parallel exploration	Single-point, follows local gradient
Convergence speed	Slow, many evaluations needed	Fast when gradients are informative
Handles discrete/combinatorial problems	Naturally	Poorly, needs relaxation tricks
Risk of local optima	Lower, due to population diversity	Higher, especially with poor initialization
Typical scale	Thousands of parameters	Billions of parameters (deep learning)

Note

If one individual becomes dominant too early, crossover keeps recombining its genes with themselves, and the population loses diversity before finding a good solution — converging prematurely on a mediocre local optimum. Practitioners fight this with higher mutation rates, diversity-preserving selection schemes, or "niching" methods that explicitly reward novelty, not just raw fitness.

Evolutionary computation beyond simple GAs

The genetic algorithm is one member of a broader family. Genetic programming evolves entire programs or expression trees rather than fixed-length chromosomes, letting evolution discover both structure and parameters. Evolution strategies (ES) and CMA-ES operate directly on continuous real-valued parameters using mutation distributions rather than bit-flipping, and have been used to train reinforcement learning policies at scale, as an alternative to backpropagation-based policy gradients. Neuroevolution specifically evolves neural network weights or architectures, historically important before backpropagation-trained deep learning became dominant, and still relevant for architecture search and small-network control tasks where gradients are unreliable.

What's next

Genetic algorithms are one way to search a space without gradients; another is to search a space of discrete logical possibilities under hard constraints. The next module, Constraint Satisfaction Problems, covers that style of search directly — and shows where backtracking and constraint propagation outperform evolutionary methods for problems like Sudoku and scheduling.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →

Need production AI or MarTech expertise? I build custom LLM workflows, server-side GTM tagging, and analytics infrastructures professionally.

Need custom AI or MarTech setup? Let's build together.

Let's talk →