A defect model that hits 98% in a pilot cell and then bleeds accuracy on the line is not a broken model. It is an unhardened one. The lab proved the defect is detectable under one lighting rig, one camera angle, one batch of product. The line did not agree to those terms.
That gap has a name — domain shift, or training-serving skew — and it is the ordinary reason industrial vision fails, not a rare edge case. Benchmark photos are taken under controlled light, on a clean belt, with the part facing the lens. A factory is none of those things, and it does not stay still after you ship.
The line changes the pixels. The model does not get a vote.
Ambient light leaks through doors and skylights and moves across the day and the season. Overhead lamps age and shift colour temperature. A matte part from the pilot batch comes back glossy after a supplier change, and specular highlights start looking like defects. Maintenance moves the camera two centimetres to clear a jam. The conveyor runs 15% faster and you get motion blur the pilot never saw.
None of that is exotic. It is the physics of a working floor. A model trained on a narrow lighting and pose distribution treats every one of those changes as a new object. It still returns a label and a confidence. It does not raise a hand.
The model often learned the background, not the defect
Pilot cells are visually boring: one backdrop, one fixture, one pair of gloves. The network is perfectly happy to use those shortcuts. Then someone adds safety tape, a new guard rail, a different tote colour, or a worker in a different sleeve, and the "defect detector" is actually a scene detector. Industry writing on production vision keeps finding the same pattern: the object was never the only thing the model used.
That is why swapping YOLO for a larger backbone rarely saves a failed line trial. The architecture was not the lie. The training distribution was.
Rare defects make a high score cheap
In inspection, the good parts are the job. Defects might be 1–3% of frames, sometimes far less. A model that always says "OK" is already in the high nineties. Lab reports that quote overall accuracy without the miss rate on the expensive class are not a production metric. The number that matters is the cost of a miss versus the cost of a false stop, agreed before you tune the threshold.
If you do not have enough labelled examples of the hard miss — the crack that looks like oil, the scratch under glare — you do not have a model of the defect. You have a model of the happy path.
Silence is the dangerous failure
Software that throws an error is annoying. A vision system that is quietly wrong is a quality escape. Off-the-shelf and pilot models rarely come with a working out-of-distribution signal. They will classify a new SKU, a dirty lens, or a frame of the floor with the same confidence they used on the demo. Operators learn not to trust the banner. Then the system is furniture.
The production version needs a review loop on uncertain cases, a fallback that does not halt the whole line for every maybe, and someone named who retrains when the world moves. Accuracy on last month's set is not ownership.
Treat capture as infrastructure
The cheapest accuracy gain in factory vision is often not more training. It is a lighting fixture that does not change between shifts, a camera mount that does not wander, a trigger that matches line speed, and a process for when the lens gets coated. If the photons are unstable, the weights cannot save you.
Then collect from the live line, on purpose: night shift, the new supplier, the SKU that launched last week, the frames operators already argue about. Prove the signal on a held-out set from that world, not the pilot cell. Deploy with a human path. Watch drift. Expand coverage only where the model has earned it.
The lab is where you learn whether a defect is visible at all. The factory floor is where you learn whether the system can survive a Tuesday.