Edge AI Deployment Constraints Surprised Us More Than the Model Accuracy Did

A client needed an object detection model running on fixed cameras in a warehouse environment, inferring entirely on local edge hardware with no cloud round trip. We spent the bulk of our planning time on model accuracy and architecture selection, treating the deployment target as a solved problem we'd sort out once the model was ready. That ordering cost us weeks once we actually tried deploying.

What the deployment environment actually threw at us

The edge devices had far less memory than the spec sheet's headline number suggested once the operating system and other running processes were accounted for. Temperature swings in an unconditioned warehouse caused thermal throttling that degraded inference speed inconsistently throughout the day. Network connectivity for model updates was far less reliable than assumed, which meant our planned remote update strategy needed a complete redesign for offline-first operation.

A model that's ready to deploy and a device that's ready to run it are two separate problems, and the second one doesn't get solved by improving the model.

What we'd do differently

Model accuracy work was, in hindsight, the easier and more predictable half of the project — we knew how to improve it and had clear metrics to chase. The deployment environment's constraints were specific to physical hardware and a physical facility, discoverable only by actually testing on-site, and no amount of careful planning on a spec sheet alone would have surfaced the thermal throttling issue before it happened in the actual warehouse.

• Get real target hardware in hand and test on it as early as possible, not after the model is finalized.
• Test in the actual deployment environment's physical conditions, not a comfortable office — thermal and power constraints are invisible on paper.
• Design the update and monitoring strategy for intermittent connectivity from day one if the environment isn't a reliable network.
• Budget real engineering time for deployment-environment debugging as its own line item, separate from model development time.

The model itself ended up being fine, close to our first good checkpoint. The project timeline slipped because we'd implicitly treated the deployment target as a footnote to the modeling work, when for an edge AI project it's just as much the actual engineering challenge as the model is.