The Decision Framework I Actually Use to Pick a Model for a Project

Clients regularly ask "which model should we use" expecting a leaderboard answer. The leaderboard answer is almost never the right one, because the leaderboard doesn't know your latency budget, your data residency constraints, or whether your use case is actually a structured-extraction task wearing a conversational costume.

The four questions, in order

• Does this need to run where the data lives, or can it call out to a hosted API? This alone eliminates most of the field for regulated clients.
• Is the task closer to generation (write something new) or extraction (pull structured data out of unstructured input)? Extraction tasks tolerate much smaller, cheaper models than people assume.
• What's the actual latency budget, measured in a real user flow, not a demo? A support widget that needs a response in under two seconds rules out some otherwise-great models immediately.
• What happens when it's wrong, and how often can you tolerate that? This decides whether you need the frontier model or whether a smaller model plus a validation layer gets you the same effective reliability for a fraction of the cost.

Most model-selection debates are actually unresolved disagreements about one of these four questions, dressed up as a benchmark argument.

Only after those four are answered do I pull up pricing and benchmark numbers, and usually the field has already narrowed to two or three realistic options by then. I've seen teams spend weeks debating frontier model choices for a task that turned out to be pure structured extraction, where a much smaller model plus decent prompt engineering performed identically at a tenth of the cost — the debate was happening at the wrong altitude.

The framework isn't exotic, but writing it down in the proposal forces the client to answer questions they usually haven't thought about yet, which does more to set realistic expectations than any benchmark comparison would.