Entity matching: how do you map supplier data to a catalog?

01Why entity matching for products is harder than it looks

Supplier and manufacturer data arrives in every shape. One site lists a model with its year in the title, another puts it in a separate field, a third uses a marketing name that appears nowhere in your catalog. Specifications use different units and labels. Some fields are missing, some are wrong, and the same product can appear several times with small differences. This is part of our larger guide, AI integration for SaaS: add AI without a rebuild.

Your catalog, on the other hand, has a fixed structure: makes, models, trims or variants, categories and attributes, each with rules your product depends on. Search, reports, pricing and every future AI feature assume that a record in the catalog means one real thing.

Matching connects the two. It is called entity matching, record linkage or entity resolution, and it has a long research history. Open-source tools such as Splink, built on the Fellegi-Sunter model of record linkage, score pairs of records with a match probability rather than a yes or no. That framing is the right one for product data too: every match has a confidence, and confidence decides what happens next.

02Agree on the data model before you match

The first mistake is to treat matching as a data engineering task that starts once data is collected. It starts earlier, with agreement on the target.

Before any matching runs, settle these with the catalog owner:

1. 01

   The target schema.

   Which fields exist, which are required, and what each one means. "Trim" and "variant" mean different things to different teams.

2. 02

   The level of a match.

   Is a match to the model enough, or must it reach the trim and year? A dealer comparing prices needs the second.

3. 03

   Normalization rules.

   Units, capitalization, abbreviations, how marketing names map to catalog names.

4. 04

   What happens to no-match.

   Create a new catalog entry, hold for review, or drop?

5. 05

   Who decides disputes.

   Someone on the client side must own the mapping, or every review round reopens the rules.

This step usually becomes a meeting about how the mapping itself will be managed: where mapping files live, who edits them and how changes are reviewed. Plan for it. It is not a delay; it is the work that makes the rest repeatable.

03Clean and normalize before you compare

Matching raw strings is slow and inaccurate. Clean first, then link.

• Split compound fields. An ingredient line or a listing title often packs several facts into one string: quantity, unit, item and preparation, or year, make, model and trim. Parse them into separate fields.
• Normalize names. Remove branded prefixes, standardize abbreviations and produce a generic name next to the original.
• Merge duplicates inside each source before matching across sources.
• Keep the original. Store the raw value beside the normalized one, so a reviewer can always see what the supplier actually said.

04Generate candidates cheaply, then score carefully

Comparing every supplier record with every catalog record does not scale. Splink's documentation notes that a dataset of one million records produces around 500 billion pairwise comparisons. The standard answer is to split the work into two stages.

For scoring, research has moved quickly. A widely cited paper on Ditto showed that fine-tuning pretrained language models for entity matching beat earlier methods by up to 29% in F1 on benchmarks and needed far less labeled data. That does not mean every product team needs a fine-tuned model. Field-level rules plus a simple similarity model often handle most of a catalog, and a stronger model can be saved for the hard remainder.

Whatever the method, use more than one blocking rule. A single rule misses true matches with a typo in the key field; a small set of rules catches them while keeping comparisons manageable.

05Plan for the long tail

Matching is never finished, because sources keep changing. Manufacturer sites are redesigned, dealer websites change templates without notice, and new brands arrive. Plan for ongoing work:

• Monitor coverage. Track how many records failed to collect or failed to match, by source, every cycle.
• Fix wrong matches, not just missing ones. A wrong product in the catalog does more harm than a missing one, because people trust it.
• Reuse by template. Many sources share a platform. Group them so one fix covers many.
• Keep the mapping versioned so you can tell which rule produced which match.

Clean product data is also what any later AI feature in your product will stand on. If you are planning AI on top of this data, the cost side is covered in how much production AI costs in SaaS.

06What we see in production

For a dealership marketing and software company, we built structured product data capture for powersports and marine manufacturers, feeding the inventory module of its multi-tenant dealership portal. Manufacturer sites differed widely: one used several product layouts, and a dynamic site needed several fetching approaches. We wrote one collector per brand, then per-brand mapping files into the client's own data format. Extraction was planned and built in the first sprints. The later sprints went to mapping files, and the mapping was revised with the client over two rounds of feedback. A later phase added an inventory data API and automatic population of inventory data inside the portal.

For a competitive inventory intelligence product serving RV and automotive dealerships, where we run scoping, engineering and crawler operations end to end, data comes from hundreds of public dealer websites that use different templates, change without notice, block bots or go offline. We group sites into reusable crawler types per site platform rather than one scraper per dealer, and map listings to a consistent make, model and trim catalog with management commands. Listings arrive with messy titles, conditions and missing fields, so offer-to-product mapping and cleanup of wrong products are continuous work, and every sprint includes a crawler health pass so coverage does not decay. The product has been live and in continuous use since at least 2022.

For a food discovery and nutrition startup, we cleaned and linked menu items, recipes and ingredients that nothing connected before. Ingredient lines were split into quantity, unit, item and preparation, and branded menu names were given a generic second name. We trained word2vec on the cleaned names, used unigram and bigram search for lookup and locality sensitive hashing to make similarity search over the large set several times faster. Nutrition values came from the linked recipe and ingredient data, never from a model.

Agreeing on the target included the category list itself: in the menu and recipe linking work, categories such as Starters and Beginnings were folded into Appetizers. We worked from the client's existing exports and its 200,000 labeled menu items rather than collecting new data.

The lesson across all three: the matching method mattered less than agreement on the target catalog and a mapping that both sides could review.

Frequently asked questions