Designing a Recommendation Engine

The whiteboard for this problem is in excalidraw/9. Designing Recommendation Engine.excalidraw. This file is the full write up of that board.

1. Problem Statement

Recommendation engines are the heart and soul of consumer platforms such as YouTube, Amazon and Spotify. Think about how often you click a recommended video or buy a book Amazon suggested.

The feature changes depending on the platform, but the overall architecture stays similar. We will design one for Amazon, so the items are products and the signals are browsing history and orders.

2. Requirements

2.1 Functional Requirements

2.2 Non Functional Requirements

1. Fast serving. A recommendation read must be a few milliseconds, because it sits on the product page and the home page.
2. Scale. Millions of users scored against millions of items.
3. Freshness. New products and recent behaviour must show up, not wait a week for the next full training run.
4. Availability. If personalization is unavailable, the page must still show something reasonable.
5. Cheap at read time. Computing recommendations on the fly for every page view is not affordable.

2.3 Out of Scope

Ads and sponsored placement auctions, pricing, search ranking.

3. The Naive Approach

Recommend the most popular items. Simple, quick and dirty.

What is wrong with it:

• The most popular items are the same for everybody, so there is no personalization at all.
• It recommends items the user already bought. You bought an iPhone and Amazon keeps showing you iPhones.
• It recommends items that are irrelevant or out of budget. You have never spent more than 10,000 rupees and it suggests a 150,000 rupee laptop.

It is still a useful fallback for a brand new user with no history, which is worth saying out loud, but it cannot be the product.

4. Two Core Machine Learning Approaches

A well designed feed blends two approaches, to balance what the user already likes against fresh content.

4.1 Content Filtering, Exploitation

Recommend items similar to what the user has already consumed, searched for or bought. This exploits known preferences and keeps the user engaged in a niche. Buy a physics book, get more physics books.

Implementation:

1. Cluster the content, meaning products, videos or articles, into groups based on their features such as title, description and type.
2. When an item is interacted with, pick other items from the same cluster.

4.2 Measuring Similarity

The common way to decide whether two items are similar is cosine similarity.

• Items are turned into mathematical vectors in an n dimensional space, where each dimension is a feature such as a token from the product description.
• The angle between two item vectors decides similarity. A smaller angle means they are closer and highly similar, so the cosine approaches 1. A larger angle approaching 90 degrees means dissimilar, so the cosine approaches 0.

Cosine is used rather than plain distance because it ignores magnitude. A long product description and a short one about the same thing point in the same direction, even though one vector is much larger.

4.3 Collaborative Filtering, Exploration

Recommend items that other similar users have purchased. If user A and user B are similar, and A bought an iPhone, recommend an iPhone to B. This encourages exploration, since it suggests things the user has shown no direct interest in.

Implementation:

1. Instead of clustering items, cluster users based on their browsing or order history. A user can belong to several overlapping clusters based on different interests, for example one cluster for cheap electronics and another for physics books.
2. Load the user clusters and order histories into a graph database such as Neo4j. Graph databases are essential here, because the queries are naturally about relationships: find items that candidate users in my cluster bought and I did not.

Cohorts are created from many features, such as price band, categories browsed, purchase frequency and season.

4.4 Querying Candidate Users

Given a user, pick who to compare against:

• Pick a few users from the cluster at random.
• Or pick the top n most similar users to this user.
• Or, if the cluster is small enough, take all of them.

4.5 Querying Candidate Items

For each candidate user, find items they purchased that our user A did not. That is a simple query for a graph database, and a painful multi join for a relational one.

MATCH (a:User {id: $userId})-[:IN_CLUSTER]->(c:Cluster)<-[:IN_CLUSTER]-(peer:User),
      (peer)-[:PURCHASED]->(item:Product)
WHERE NOT (a)-[:PURCHASED]->(item)
RETURN item, count(peer) AS peers
ORDER BY peers DESC
LIMIT 500

4.6 Ranking

Once candidate items are found, the system predicts the rating the user would give each item and sorts them in descending order, to maximize the chance that the user likes the recommendation.

So the pipeline is three stages:

1. Candidate generation, narrowing millions of items down to a few hundred or a few thousand, using collaborative filtering or content based filtering.
2. Scoring and ranking, using a heavier model over richer features such as user history, time of day and context.
3. Re ranking and filtering, removing already purchased items, ensuring diversity so the feed is not twenty variations of one product, and mixing in promoted content.

The split exists because the models have opposite cost profiles. Candidate generation must be cheap enough to touch millions of items. Ranking can be expensive because it only sees a few hundred.

5. System Architecture and Data Pipeline

5.1 Ingestion

Distributed computing, meaning Spark jobs, reads from the product and order databases. It filters out low quality items, for example anything rated below 3, to protect the user experience, and dumps the clean data into a central repository such as S3.

Filtering at ingestion rather than at serving time is deliberate. A badly rated product should never enter a cluster in the first place, otherwise every stage downstream has to remember to exclude it.

5.2 Computation

Spark and ML pipelines run over the S3 data to train models and build the content and user clusters, storing the precomputed results back in S3 or pushing them into the graph database.

5.3 Serving

Because computing recommendations on the fly is expensive, a recommendation generator continuously processes data and stores precomputed recommendations for each user in a fast key value recommendation DB.

The read path then becomes a single key lookup, which is what makes the milliseconds requirement achievable.

5.4 Handling Exhaustion

When a user scrolls through all the generated recommendations, the frontend triggers an event to the backend through Kafka. The recommendation generator acts asynchronously to quickly compute and add more items to that user's list.

This is the same pattern as the Tinder feed: the client knows how many items are left, so the client asks for a refill. The backend never polls.

6. API Design

6.1 Read APIs

GET /api/recommendations?limit=20&cursor=...:

{
  "items": [
    {
      "productId": "p_7781",
      "title": "Feynman Lectures on Physics, Volume 1",
      "price": 129900,
      "rating": 4.6,
      "imageUrl": "https://cdn.shop.dev/p_7781.webp",
      "reason": "BECAUSE_YOU_BOUGHT",
      "reasonDetail": "You bought Six Easy Pieces",
      "score": 0.87,
      "source": "CONTENT"
    },
    {
      "productId": "p_5510",
      "title": "Mechanical Keyboard",
      "price": 549900,
      "rating": 4.4,
      "reason": "SIMILAR_USERS",
      "score": 0.71,
      "source": "COLLABORATIVE"
    }
  ],
  "remaining": 34,
  "nextCursor": "eyJyYW5rIjo1MH0",
  "generatedAt": "2026-08-19T09:00:00Z",
  "modelVersion": 42
}

Returning reason and source matters for two reasons. The user gets an explanation, which measurably improves click through, and the team can compare how content and collaborative sources actually perform.

POST /api/recommendations/refill returns 202 Accepted after publishing an exhaustion event to Kafka. It does not wait for generation.

6.2 Event APIs

{ "userId": "u_1729", "productId": "p_7781", "type": "VIEW", "at": "2026-08-19T10:31:00Z" }

These all publish to Kafka and return immediately. Behavioural events are the raw material for the whole system, and the volume is enormous, so nothing about recording them can be synchronous.

6.3 Internal APIs

7. Data Model

Five stores, each matched to one access pattern.

7.1 Source of Truth, Relational

products

users

orders and order_items

order_items is the most valuable table in the system. It is the strongest signal of real preference, because someone actually paid.

7.2 Behaviour Log, Append Only

user_events (Kafka into S3, queried by Spark)

Never a relational table. This is billions of rows, written constantly and read only in bulk by batch jobs.

7.3 Model Artifacts, Object Storage

item_vectors and item_clusters, in S3

model_versions, relational, small

7.4 User Clusters, Graph Database

Nodes and edges rather than tables:

The IN_CLUSTER edge carries a weight, which is what makes overlapping membership work. A user can be 0.8 in cheap electronics and 0.3 in physics books, and candidate generation weights peers accordingly.

Graph replicas serve reads, because the generator's traversals are heavy and must not compete with the batch writes that refresh cohorts.

7.5 Serving Store, Key Value

recommendations (Redis or DynamoDB)

One row per recommended item, not one document with an array. Same reasoning as the Tinder feed: an array inside one document bloats, costs serialization on every read, and makes marking a single item as served awkward.

7.6 Relationships

In words:

• One user places many orders, one order belongs to one user.
• One order contains many order items, and one product appears in many order items. That makes users and products many to many through orders, which is the matrix collaborative filtering factorizes.
• One category groups many products, one product sits in one category.
• One user generates many events, one event concerns at most one product.
• One user receives many recommendation rows, one row names one product.
• One model version generated many recommendation rows, which is what allows a fair comparison between versions.
• Users and clusters are many to many with a weight, which is the overlapping cohort membership.
• Products relate to other products through similarity, which is a many to many self relationship produced by content clustering.

The two many to many relationships, user to product and product to product, are the ones that justify a graph database. Expressed relationally they are multi hop joins over the largest tables in the system.

8. Key Flows

8.1 Batch Training

8.2 Precomputing One User's Feed

8.3 Serving and Exhaustion

8.4 Reacting to Recent Activity

Batch training runs daily, which is too slow for "I just searched for a tent".

The precomputed list stays the base and the session signal only reorders it. That gives same session responsiveness without any expensive work on the read path.

9. Deep Dives

9.1 Why Precompute

Computing on the fly for one user means graph traversals over peer purchases, similarity lookups over millions of items, a model scoring pass, and filtering. That is seconds of work, and the home page has a budget of milliseconds.

Precomputing moves all of it to a background process. The read becomes a key lookup. The costs are staleness, which the session boost and Kafka refills mitigate, and storage, since we hold 200 rows per active user. For 100 million active users at roughly 100 bytes per row, that is around 2 TB, which is a normal size for a distributed key value store.

9.2 Exploration Against Exploitation

Content filtering alone traps the user in a niche. After one physics book they see nothing but physics books, and they stop discovering anything. Collaborative filtering alone recommends items with no obvious connection to what the user was doing, which feels random.

So blend them, typically 60 to 70 percent exploitation and the rest exploration, and always keep some slots for genuine exploration even when the model is confident. Without those slots the system can never learn about a new interest, because it never shows anything that could reveal one.

9.3 Cold Start

Content filtering handles a new item immediately because features come from the title and description. Collaborative filtering cannot, because nobody has bought it yet. That asymmetry is a good reason to run both.

9.4 Diversity and Re Ranking

A purely score ordered list tends to be twenty near duplicates, because whatever scores highest usually has close siblings. Re ranking fixes it:

• Cap the number of items per category and per brand in one page.
• Penalize an item that is very similar to something already placed above it.
• Reserve slots for exploration and for new arrivals.
• Drop anything already purchased, dismissed, or out of the price band.

The user experience metric here is not score, it is whether the page looks worth scrolling.

9.5 Measuring Whether It Works

Offline metrics decide which models are worth testing. Online A/B tests decide which one ships. Offline numbers routinely disagree with online results, and the online result wins.

9.6 Why a Graph Database

The core collaborative query is "items purchased by users in my clusters that I have not purchased". Relationally, that is a join from users to clusters to users to order items to products, with a not exists over the largest table in the system, and it gets slower as data grows.

In a graph store it is a two hop traversal from one node, and the cost depends on the neighbourhood size and not on the total number of users. Traversal is the access pattern, so the store should be built for traversal.

10. Failure Handling

The pattern here: every failure degrades to a worse but working feed. This is a system where wrong is acceptable and empty is not.

11. Tradeoffs

12. Interview Talking Points

• Start with the naive popular items approach and kill it with the two concrete examples from the board: recommending an iPhone to someone who just bought one, and a 150,000 rupee laptop to someone who never spends over 10,000. Then note that it survives as the cold start fallback.
• Frame the two approaches as exploitation and exploration, and say why you need both.
• Explain cosine similarity in one sentence about the angle between vectors, and say why the angle matters rather than the magnitude.
• Name the three stage pipeline, candidate generation then ranking then re ranking, and give the cost reason for the split.
• Justify the graph database with the actual query. "Items my cluster peers bought that I have not" is the sentence that makes the choice obvious.
• Say that recommendations are precomputed and that the read is a single key lookup, then handle the freshness objection with the session boost.
• Mention that quality filtering happens at ingestion, not at serving.
• Bring up diversity re ranking. Most candidates stop at the score, and a page of twenty near identical products is a visibly bad feed.