This is a retail & fmcg problem we approach through our Platform to Platform service line.
The problem
Somewhere in your operation is the person who types products into the back office. New lines from a supplier, seasonal stock, loose produce, the local goods a barcode never reaches — each becomes a form: name, size, category, price, tax code. It is slow, it queues behind whoever owns the permission, and it is why the catalogue describes the shelf as it stood a few weeks ago.
The lag has a cost at the till. A product that is not in the catalogue gets sold under a miscellaneous code, priced from memory, invisible to reporting and reordering. Multiply by an estate and a meaningful slice of sales is flowing through the gap between the shelf and the system.
The obvious fix is to make capture as fast as the shelf changes: photograph the product, get a draft entry, correct it, approve it. Cameras that recognise products are already in production retail — Tiliter’s produce recognition has been deployed at Woolworths self-service scales, and Mashgin’s camera-based checkout was announced for rollout across thousands of Circle K stores. The recognition approach is proven in the market. Getting it to build your catalogue, on your stock, is the engineering.
Why it’s hard
Packaged goods are adversarially similar. The 500ml and the 375ml bottle differ by a line of small print. Flavour variants share nine tenths of their artwork. A model that is nearly right on lookalikes is dangerous precisely because its drafts look plausible — the error survives review by a busy human and becomes a wrong price on a live catalogue.
The open-set problem is the whole game. In a lab, the model chooses among products it was trained on. On a shelf, it constantly meets ones it was not — new lines are the very reason the feature exists. A system that cannot return “no match” does not degrade gracefully; it manufactures confident wrong answers. Knowing when it does not know is harder to build than the recognition itself.
Shop-floor images are nothing like training images. Glare off foil, shrink-wrap, a hand holding the product, half the label facing away, the lighting of a stockroom at 6am. A model benchmarked on clean product shots falls over on the photos staff actually take, so the pipeline has to be built and evaluated on capture conditions, not catalogue photography.
A draft entry is more than a label. Recognising “this is olive oil” is the easy fraction. A usable catalogue entry needs the brand, the variant, the size, a category in your hierarchy, a tax treatment. Some of that is reading the packaging, some is inference against your existing catalogue structure, and all of it needs a human checkpoint before it is allowed to become a record that prices real sales.
How we approach it
The same detect–verify–retrain pattern we have run in production. In 2018 we built and delivered a system that identifies individual animals from live video by coat pattern — 96% identification accuracy, trained on more than 20,000 labelled images — in which an operator confirms each match before it becomes an official record, and every confirmation or correction feeds the next model. That loop cut surveillance workload by around 70% for the operators. Product capture is the same shape: the model proposes, a person disposes, and the corrections are the training pipeline. We reason from that delivery; we do not claim it was retail.
Draft, never commit. The photograph produces a draft entry — proposed name, size, category, matched against the existing catalogue for duplicates — that a staff member corrects and approves before it exists. The correction step is the mechanism that measures accuracy live and generates the labelled data that improves it.
No-match is a first-class answer. Below a confidence threshold the system says so and routes the item to manual entry, pre-filled with whatever it did read. The threshold is tuned against your correction data, per category, because the cost of a wrong draft differs between loose produce and priced spirits.
Designed as a capture surface for the catalogue the POS already owns. Our cloud point of sale, AmshPOS, is designed to take image-based product capture as an onboarding path, so the photo-to-draft flow would land in the same catalogue and estate console the tills run from, rather than in a side system someone has to reconcile.
What we do not do. We do not ship this without the correction loop — recognition without verification is how catalogues rot invisibly. And we would not promise day-one coverage of your full range: the model starts from your product images and grows through use, and the pilot scope should be the categories where manual entry hurts most.
What it takes
Your images, labelled. A starting set built from your actual stock under actual store conditions — supplier artwork helps but does not substitute. The wildlife system’s accuracy stood on 20,000+ labelled images; a retail pilot starts far smaller by scoping to a category, but the principle holds: the labelled set is the asset, and the correction loop is what grows it.
A catalogue structure to land in. Category hierarchy, tax codes, duplicate rules. If the existing catalogue is inconsistent, image capture inherits the inconsistency — cleaning that is part of the work, not a surprise.
Staff time in the loop, priced honestly. Corrections are minutes per day, but they are the system doing its job, and the rollout plan should say so out loud.
A pilot with a measured gate. One category, one site, correction rate tracked from day one, with the expansion decision made on that number rather than on enthusiasm.
Where this has been done
Plainly: nowhere yet, for retail, by us — this is a capability, not a shipped reference, and we are stating that rather than blurring it. The reasoning behind the capability is real delivery. We have built and operated production computer vision in which live imagery is matched against known individuals, an operator verifies every match, and corrections retrain the model — 96% accuracy from 20,000+ labelled images, with operator workload down around 70%, in the wildlife-monitoring case study linked from this page. The detect–verify–retrain loop is the transferable engineering; the staff member correcting a draft entry is the same role as the operator confirming a sighting. If you need a vendor with a named retail vision deployment, we are not yet it, and you should hold whoever claims to be one to the same evidence standard this page is trying to meet.