How do you build an AI recommendation engine?

01Define what a good recommendation is

Teams often start by choosing an algorithm. We start by asking what a user should do after seeing a recommendation, because that decides what good looks like. This is part of our larger guide, AI integration for SaaS: add AI without a rebuild.

On a collaboration product, the answer is often surprising. On a social music collaboration app with over a million users, musicians record multi-track songs and others add their own parts. Finding the right collaborators is the core loop, and the client tied search directly to retention. The most useful recommendation for a track that already has vocals and guitar is not another vocal track. It is a drummer. That is a complementary recommendation: it fills the missing role rather than repeating what is there.

Other products need different definitions:

• A marketplace may want items similar to what the user just viewed.
• A nutrition product may want foods that fit the user's plan and history.
• A B2B workflow product may want the next action that similar accounts took.

Write the definition down with an example of a good result and a bad one. It becomes the basis of your test set later.

02Fix the data before the model

Most recommendation problems in established products are data problems in disguise. On the music app, about a third of the example searches asked for an instrument or role, such as piano, drummer, guitar or vocalist, that no field in the data held. Declared genres were often wrong. No ranking model can recommend a drummer if nothing in the data says which tracks have drums.

So we treated it as a data problem first. Instead of tuning ranking on unreliable metadata, the prototype listened to the audio. A contrastive language-audio model, CLAP, places audio and text in the same space, so a phrase like solo piano can be scored against a track without training a separate classifier for each instrument. A signal processing library added cheap, explainable rhythm and energy features. On a sample of real tracks, the audio model recovered what tracks actually were where the declared genres were wrong.

The same principle held years earlier for a food discovery and nutrition startup we worked with. Menu items had branded names, recipe names were personalized, and ingredient lines mixed quantity, unit, item and preparation in one string. Nothing linked ingredients to recipes or recipes to menu items. We cleaned and normalized first, then linked the records, and only then built recommendations on top. Nutrition values came from the linked recipe and ingredient data, never from a model.

03Use one index for search and recommendations

A common mistake is to build recommendations as a separate system from search. The two then disagree: search says a track is piano, recommendations treat it as rock, and users stop trusting both.

The design we proposed for the music app writes every signal, from metadata and from audio, into one index that serves both search and recommendations. The complementary-role recommendation matches the missing role by sound, from the same index that answers a search for drummers, so the two never disagree. Personalization nudges the order of results but does not override intent: a drummer search still returns drummers.

Google's machine learning course on recommendation systems describes the usual structure in three stages, and it maps well onto a shared index.

For text and catalog data, hybrid search, which combines keyword matching with vector similarity, is a sensible base for candidate generation. Elastic's documentation describes running both in one request and merging the lists with reciprocal rank fusion.

04Build the AI recommendation engine in steps

This is the order we follow:

1. 01

   Write the definition of a good recommendation, with examples, and agree it with the product team.

2. 02

   Audit the data for the signals that definition needs. List which exist, which are unreliable and which are missing.

3. 03

   Create the missing signals, from content, behavior or both, and write them into the shared index.

4. 04

   Start with simple, testable rules for parsing what users ask for. A language model can take over messier input later and emit the same signals.

5. 05

   Build candidate generation, scoring and re-ranking on that index.

6. 06

   Label a baseline set and measure ranking quality, for example precision and nDCG at the top five results, before and after each change.

7. 07

   Release to a subset first, such as the most discoverable part of the catalog, and widen as the metrics hold.

On the food platform, recommendations combined two methods: collaborative filtering through matrix factorization, which learns from what similar users chose, and content-based filtering, which matches on the food itself. Running them on a graph database made it quick to explore relationships between users, foods and nutrition. Combining the two helps with new items and new users, where behavior data alone is thin.

05What we see in production

On the music collaboration app, the work was a paid test project on the client's real tracks, not a production launch. It produced a reference architecture, notebooks for metadata and audio analysis, and a live demo with prompt search, facets with counts, similarity search and a production design. That design kept everything inside the client's own cloud project, extended the existing GraphQL interface rather than forking it, and ran enrichment as another workload on the client's existing cluster. In an established product, the recommendation engine has to fit the stack that is already running, and the design work to make it fit is a large share of the effort.

That project also showed where the limits are. Detecting drums from full mixes is hard because the parts are not separated, so a visual pass over sampled video frames was proposed as a next step. Musical key was the one signal the open models did not replace cleanly. Being clear about those limits up front kept the plan honest.

The production plan also set a definition of done for the first search release: behind the existing API, with explicit filters, and a measured gain on a labeled query set, judged by precision and nDCG at five instead of a claimed improvement.

For an audience data company, our engineers worked inside the client's own team for about two years, supporting the weblog analytics stack that fed its recommendations: data streaming into Spark and written to Elasticsearch and other stores. The work fitted the environment the client already ran rather than replacing it. Recommendations are only as good as the pipelines that feed them, and those pipelines need steady care long after launch.

Across these projects, the pattern is the same. The model choice mattered less than the signals, the shared index and the test set. For the related question of keeping AI features affordable once usage grows, see how to reduce AI infrastructure cost in production.

Frequently asked questions