How does natural language search map to product data?

01Why "just ask the model" breaks on real catalogs

A demo of natural language search usually runs on a clean sample: a few hundred items, consistent fields, sensible prices. A production catalog looks different. Fields are missing, duplicated or used differently by each supplier. Prices can be placeholders. One file can mix product types that share no attributes. URLs follow different schemes per seller.

Buyers, meanwhile, do not speak in fields. They describe concepts ("something for weekends with the kids"), constraints ("under a certain monthly payment"), compatibility ("will my vehicle pull it"), and location ("near me"). They also follow up: "show me the cheaper one", "what about used?".

If you pass the whole catalog, or a large slice of it, to a language model and ask it to pick, you get answers that sound right and are sometimes wrong. The model may invent a unit, misread a price or ignore a constraint. In a sales context, that is worse than no answer.

02Build a vocabulary layer for natural language search

The core of the work is a mapping from the language buyers use to the attributes the data contains. We build it in three parts.

Concept mappings are where most of the value sits, and they are a business decision, not a model decision. Someone who knows the catalog has to agree what "family friendly" means in fields. Write those definitions down, version them and test them like code.

03Let the model call tools, not invent results

Once the mapping exists, the model's job becomes much narrower: understand the request, decide which tools to call and with which parameters, and explain the results. Modern model APIs support this directly through function or tool calling, where the model returns a structured call that your code executes. OpenAI's documentation, for example, describes a strict mode that makes the call follow a JSON schema exactly, which is useful when the parameters are filters on real fields.

A typical tool set for catalog search looks like this:

1. 01

   A structured search tool that accepts filters on known fields (type, condition, price range, length, capacity, location).

2. 02

   A keyword and similarity search tool for concepts and free text that do not map cleanly to fields.

3. 03

   A calculation tool for payments or compatibility, so arithmetic happens in code, not in the model.

4. 04

   A lead or hand off tool, so the conversation can end in a real next step.

Hybrid retrieval helps with the second tool. Elastic's documentation describes hybrid search as running lexical and vector search in one request and merging the rankings, which suits catalog text where exact model names and loose descriptions both matter.

04Keep the data honest underneath

The mapping only works if the fields it relies on are reliable. Before tuning prompts, clean the data the filters use:

• Normalize the fields each supplier fills differently, and pick one source of truth when two fields overlap.
• Treat placeholder prices as missing, not as real values, so the bot never quotes them.
• Separate product types that share a file but not attributes.
• Sync the search index on a schedule that matches how often inventory changes.
• Store explicit rules for what may never be shown, such as excluded units or condition rules.

Then decide what the system says when nothing matches. An honest "we do not have that in stock, here is the closest option and why" builds more trust than a stretched match.

05Trade-offs to decide early

A few choices shape the whole system, and they are easier to make on day one than to reverse later.

• Filters versus similarity. Strict filters are precise but brittle when a buyer uses an unexpected word. Similarity search is forgiving but can drift. We use filters for anything the buyer states as a requirement and similarity for preference and description, and we log which path produced each result.
• Index freshness versus load. Syncing inventory more often keeps answers current but costs more and can strain the source systems. Match the sync schedule to how often stock really changes, and show the last sync time to the team.
• One model call versus several. A single call that does everything is simpler to build and harder to debug. Separate steps for intent, filter building and explanation make each failure visible.
• Clarifying questions versus guessing. Asking "new or used?" adds a turn but avoids a wrong list. Ask when the missing value would change the results a lot; otherwise, pick a sensible default and say which one you picked.

Write these decisions down with the reason. When a dealer or a product manager asks why the assistant behaves a certain way, the answer should be a documented rule, not a prompt someone tuned months ago.

06What we see in production

We built a multi-tenant AI sales assistant for a dealership software company that serves RV, marine and vehicle dealers. It runs on dealer websites, recommends real units from live inventory, explains why, qualifies the buyer and hands a complete lead with the transcript to the dealer's CRM.

The inventory data differs by dealer: missing prices, duplicate classification fields, placeholder prices, RVs and boats in one file, and different page URL schemes. Buyers ask in natural language with follow ups, tow vehicle questions, payment budgets that are monthly in the U.S. and biweekly in Canada, and location constraints.

Our early approach read inventory through a generic file based agent. We moved to search backed by an index that syncs from each dealer's inventory feed every hour, and later rebuilt the flow as a tool using agent with a lead tool, vector similarity search and structured filters. We mapped concept and spec queries to inventory attributes so the agent filters on what the feed actually contains, instead of hoping the model guesses.

Payment based filtering and towing capacity matching became explicit features. Answers are grounded in each dealer's own inventory, website and FAQ content, with inventory exclusion and new versus used rules, and an honest fallback when no unit matches. Dealer data is isolated per tenant, and the assistant works in English and Spanish.

On a sales assistant for a retail sales services company, the mapping is set by people: the assistant reads a seller's database only through the tables or SQL view an admin selects, and comparison answers merge database rows and CSV data into tables and cards, so shoppers see products side by side.

07How to test the mapping

Test the vocabulary layer separately from the conversation. Keep a list of real buyer phrases and the filters each one should produce, and check that list every time the mapping or the prompt changes. When a phrase produces the wrong filter, fix the mapping first; tuning the prompt to cover one phrase tends to break others. We cover broader evaluation in why AI agents fail in production.

Frequently asked questions