A Domain Mismatch Lesson From a Transfer Learning Project

We picked a pretrained industrial defect detection model as the starting point for a client's quality control project, fine-tuning it on their labeled images of manufacturing defects. Validation accuracy during development looked solid, over 90%. Production accuracy on the factory floor landed closer to 70%, and it took a few weeks of confused debugging to figure out why the gap existed at all.

The mismatch was lighting, not the defects themselves

The pretrained model's source dataset, and most of the client's own labeled training images, had been captured under controlled, even studio-style lighting. The actual factory floor used overhead industrial lighting with harsh shadows and glare that varied by time of day and which machine was running nearby. The model had never seen that lighting variation during pretraining or fine-tuning, so it was effectively being asked to generalize to a visual domain it had no real exposure to — and defect detection models are more sensitive to lighting artifacts than you'd intuitively expect, because shadows and glare can look a lot like surface defects.

The model wasn't bad at detecting defects. It had just never seen the lighting conditions where it was actually going to be asked to work.

The fix was going back to the factory floor specifically to capture training images under the real lighting conditions across different times of day and machine states, rather than relying on the cleaner images that had been convenient to collect first.

• Validation accuracy on data collected under the same convenient conditions as training data doesn't tell you anything about generalization to real deployment conditions.
• Transfer learning inherits the source domain's blind spots, including ones as mundane as lighting, unless the fine-tuning data specifically corrects for them.
• Collect training data under the actual deployment conditions, not the easiest conditions to capture it in.
• A domain mismatch this specific rarely shows up as an obvious error — it shows up as a confusing accuracy gap that takes real investigation to trace.

Once retrained on lighting-matched factory floor images, production accuracy came up to 89%, close to the original validation number. The whole episode was a reminder that 'transfer learning' doesn't mean the model inherits general competence — it inherits the specific conditions of wherever its original training images came from, blind spots included.