Ingest one vendor feed and the mess looks bespoke. Ingest several and it starts to rhyme: the same defects recur in catalog after catalog, because they come from how commerce platforms model and export data, not from any one vendor being careless. There are roughly seven of them, and once you can name them you can detect them systematically instead of discovering each one live in front of a shopper. This is the field guide — the seven defects, how to catch each, and the non-negotiable rule about what you do before an AI ever sees the data.
The seven defects
None of these are exotic. Each has a clean detection signal and a definite fix.
| Defect | What it looks like | How to detect | Fix |
|---|---|---|---|
| Numeric attribute values | Brand or colour is an integer — "by 2822" | Value is a bare number where a word belongs | Resolve the platform option ID to its label; reject numerics |
| Attributes hidden in prose | Serving size, weight, capsule count only in the description | Structured field empty but pattern present in text | Parse from text into structured fields deterministically |
| Language soup | Estonian title, English description, Russian flavour | Language detection disagrees across fields | Normalise to canonical terms; generate per-language natively |
| Variant chaos | One protein in 18 flavour/size combos as 18 products | Shared root name, differing size/flavour tokens | Group under one parent; collapse in search |
| Category mismatch | Five vendors, five incompatible category trees | Source category has no clean map target | Map into one shopper-first taxonomy |
| Missing images | Grey placeholder in a grid of real photos | No image URL, or a known placeholder URL | Gate the product out at ingest |
| Stale stock | Buyable product the vendor cannot ship | Stock signal absent or older than a threshold | Safe default plus frequent partial re-sync |
Two of these have production war stories worth reading in full: the numeric-brand bug is dissected in the Magento integration piece, and variant chaos gets its own treatment in handling product variants. The rest are handled below by principle rather than anecdote.
Detect before you fix
The instinct is to reach for a cleanup script the moment you see a defect. The better move is to build the detector first, run it across the whole catalog, and see how common the defect actually is. Detection turns "I saw a weird brand once" into "312 products have numeric brands across two vendors", which tells you whether the fix belongs at ingest, in a one-off backfill, or in the source mapping. It also gives you a metric that trends: the same detectors that find defects today are the data-quality score you watch over time.
Detection also protects you from fixing the wrong thing. A serving size "missing" from the structured field might be sitting in the description — a parse problem, not a data-gap problem. Knowing which defect you have determines which fix is correct. Detectors are cheap insurance against silent regressions, too: a feed that was clean for months can degrade after a vendor upgrades a plugin or re-exports, and the detector that found the original defects is exactly the alarm that catches the new one.
Deterministic validation before any AI
Here is the rule that prevents most catalog disasters: everything computable is validated and repaired deterministically before a language model is allowed anywhere near the product. Numeric brands are rejected, units are parsed, prices-per-serving are computed, variants are grouped, and conflicts between what different passes found are flagged — all by rules, all for free, all repeatable — and only then does an LLM fill the genuine language gaps.
Our enrichment pipeline runs about a dozen deterministic steps before any AI call, and one of them exists purely to catch disagreement: it compares what the deterministic passes extracted against what the model later claims, and flags the mismatch instead of silently trusting either side. The reason this ordering is non-negotiable is a lesson we paid for — a model handed garbage input does not sanitise it, it writes fluent, confident copy around it. Validate the input the model cannot be trusted to question.
This is why the sequence matters more than any single check. An LLM is a language tool; it is superb at the prose and useless as a guarantor of arithmetic or provenance. Put the deterministic gauntlet first and the model's job shrinks to what it is actually good at — which also makes it cheaper and safer, as AI product enrichment lays out in full.
The order operations run in
Sequence the pipeline so each stage can rely on the one before it:
- Validate and reject — kill the disqualifying defects first (numeric brands, missing images), so nothing downstream wastes effort on a product that should not list.
- Parse and normalise — pull attributes out of prose, map local terms to canonical values, so later stages see structured data.
- Group and classify — collapse variant families and map into the taxonomy, so search and browse see coherent products.
- Enrich — let the model fill the remaining language gaps, on clean input.
- Re-validate — check the output against the specs before it publishes; fail closed on anything unsubstantiated.
Get the order wrong and you enrich products you should have rejected, or classify data you have not yet normalised. Get it right and each defect is caught at the earliest stage that can catch it.
Scoring quality so it trends
The last move is to turn the detectors into a running score. Track, per vendor and overall, the share of products with each defect, and watch the trend across syncs. This does three things: it tells you which vendor to talk to, it catches a feed that suddenly got worse (a regression on the vendor's side), and it converts "the catalog feels rough" into a number you can hold a vendor or a pipeline to. And publish the score where the people who can act on it will see it — a per-vendor quality number in the operator console turns an abstract pipeline concern into a conversation, because "your feed's missing-image rate doubled this week" is a message a vendor can act on. Data quality that is measured is data quality that improves; the rest is vibes.
Key takeaways
- The defects rhyme: numeric attributes, prose-hidden fields, language soup, variant chaos, category mismatch, missing images and stale stock recur in every feed.
- Build the detector before the fix; knowing how common a defect is decides whether it belongs at ingest, in a backfill, or in the source mapping.
- Validate everything computable deterministically before any AI touches the product — a model amplifies garbage input, it does not sanitise it.
- Sequence the pipeline so each stage — reject, normalise, group, enrich, re-validate — can trust the one before it.
- Turn detectors into a trended quality score per vendor, so a feed that regresses is caught as a number, not a shopper complaint.
Frequently asked questions
What are the most common product data quality problems?
Should you clean product data with AI or rules?
How do you detect product data defects at scale?
In what order should a product data pipeline run?
Clean catalogs, from messy feeds.
Our managed marketplace runs deterministic validation and AI enrichment over every vendor feed, so shoppers see a coherent catalog and never the defects.
Request early access See the live marketplace →