Designing a Fraud Detection System

The whiteboard for this problem is in excalidraw/8. Designing a Fraud Detection system.excalidraw. This file is the full write up of that board.

1. Problem Statement

Financial transactions are risky and prone to fraud, so banks deploy a fraud detection system. It has to:

• Detect fraud in real time.
• Block the transaction.
• Inform a customer support executive so they can take the customer's consent.

When a user initiates a transaction, the bank makes an entry, but before proceeding any further it checks for potential fraud.

• If it is not fraud, allow the transaction.
• If it looks like fraud:
  1. Hold the transaction.
  2. Involve a support executive.
  3. The executive calls the customer and confirms.
  4. If the customer says yes, allow the transaction. Otherwise abort it.

2. Requirements

2.1 Functional Requirements

2.2 Non Functional Requirements

1. Low latency. The fraud detection service must respond within 200 ms. It sits inside the transaction path, so every millisecond is on the customer's clock.
2. High throughput. Peak load, such as a salary day or a sale, must be handled without degrading.
3. Adaptable. New fraud patterns appear constantly, so new rules and retrained models must ship without touching the transaction service.
4. Available. The detection service being down must not stop the bank from taking transactions, but it also must not silently let everything through.
5. Auditable. Every decision must be explainable later, since money and regulators are involved.

2.3 Out of Scope

The core ledger and settlement, KYC onboarding, chargeback processing with card networks.

3. Storage for Transactions

We need the ability to register a transaction.

source_acc | target_acc | status | amount | currency | ip | region | location | target_bank | ...

Status values:

INITIATED | BLOCKED | FRAUD | ALLOWED | DONE | FAILED

The schema is simple and clearly structured, so the choice of database is not complicated. We start relational. The data is also shardable, and we shard on source_acc, because every query is about one account's activity.

We can drop foreign key constraints here. At this write volume, and with sharding in play, enforcing referential integrity across shards costs more than it gives. Correctness moves into the application layer.

The bank API is a set of servers behind a load balancer, drawn as one box from here on.

4. The Transaction Handler

For an incoming HTTP request the handler:

1. Gets the source and target account entries.
2. Extracts details from the HTTP request, such as IP, location and user agent.
3. Sends those details synchronously to fraud detection.

Fraud detection is a simple HTTP service that:

1. Receives the transaction and request details over HTTP.
2. Returns whether it is fraudulent or not.

The bank API checks fraud detection synchronously and updates the database based on the answer.

4.1 Why Synchronous

This is the one place in these designs where a synchronous call is correct. The decision has to be made before money moves. Making it asynchronous would mean either completing the transaction and reversing it later, which is exactly the loss we are trying to prevent, or holding every transaction until a worker gets to it, which makes normal payments feel broken.

The 200 ms budget is what makes synchronous acceptable, and it is why the model must be loaded in memory and never fetched per request.

5. Fraud Detection Algorithm

The service uses many parameters to judge whether the current transaction is fraudulent. We generate a lot of decision trees, which is a random forest, from training data and train the model. Every transaction is classified as either fraudulent or not fraudulent.

Different trees look at different feature combinations:

• One tree for location plus time.
• A second tree for location plus bank.
• A third tree for time plus user plus bank.
• And so on.

A new transaction is classified by all the trees and the majority answer wins.

A forest is a good fit here because each tree is a series of cheap comparisons, so scoring is fast enough for the 200 ms budget, and the path through a tree is readable, which matters when a regulator asks why a transaction was blocked.

5.1 Rules Alongside the Model

Some decisions must not be probabilistic. Transactions from a sanctioned region, or above a hard limit, are blocked outright. So the service evaluates rules first and the model second, and a rule hit short circuits the model.

Rules are configuration, not code, so a new one ships without a deploy. The model handles patterns nobody wrote a rule for.

6. Preparing the Training Data

Who knows which transactions were actually fraud? Customer support and the complaint portal. We need the transactions that had a fraud dispute.

We write a job that periodically processes the customer support database, extracts the fraudulent transactions, the communication summary and the details shared on the ticket, and puts them into blob storage such as S3.

The amount of data moved is huge, so we need distributed computing. One machine reading, transforming and writing would be far too slow. We use a big data tool such as Spark.

To train the model we also need the transaction details themselves: who sent to whom, from which IP, location, region, currency and so on. All of that is in the transaction database, so the same kind of Spark job moves it to S3.

6.1 Training

Now that all the data is in S3, we use distributed ML libraries to build the model. For example Spark with MLlib trains the decision trees and puts the model back in S3.

The model is just a file holding the serialized version of the decision trees. That single sentence is what makes the serving story simple.

7. Serving the Model

The fraud detection service is a simple HTTP service that:

1. Loads the model from S3.
2. Creates an in memory structure to classify transactions.
3. Exposes an API to do this synchronously.

The user facing transaction API calls fraud detection to find out whether the transaction is fraudulent. If not, the transaction proceeds. If it is, customer service is involved and a notification is sent to the user.

Loading the model at startup is what keeps the request path fast. No network call, no deserialization per request, just tree traversals over in memory structures.

New model versions are published to S3 with a version marker. Services poll it, load the new model into a separate structure, and swap the reference when it is ready, so no request ever waits on a load. A bad model rolls back by pointing the marker at the previous file.

8. API Design

8.1 Transaction API, Public

Initiate:

{
  "sourceAcc": "ACC7781920",
  "targetAcc": "ACC5510338",
  "amount": 4500000,
  "currency": "INR",
  "targetBank": "HDFC",
  "idempotencyKey": "app-req-99120"
}

Response when it is held:

{
  "transactionId": "t_88120",
  "status": "BLOCKED",
  "reason": "pending_verification",
  "message": "We have put this transaction on hold. Our team will call you shortly.",
  "caseId": "c_4410"
}

Response when it goes through:

{ "transactionId": "t_88121", "status": "ALLOWED", "riskScore": 0.07 }

The customer never sees which rule or feature fired. Telling an attacker exactly which signal tripped is a free tuning loop for them.

8.2 Fraud Detection Service, Internal

Score request:

{
  "transactionId": "t_88120",
  "sourceAcc": "ACC7781920",
  "targetAcc": "ACC5510338",
  "amount": 4500000,
  "currency": "INR",
  "targetBank": "HDFC",
  "ip": "203.0.113.9",
  "region": "IN-KA",
  "location": { "lat": 12.97, "lng": 77.59 },
  "userAgent": "BankApp/4.2 Android/14",
  "deviceId": "d_5512",
  "requestedAt": "2026-08-19T10:31:00Z"
}

Score response:

{
  "transactionId": "t_88120",
  "isFraud": true,
  "riskScore": 0.91,
  "decision": "BLOCK",
  "triggeredRules": ["AMOUNT_ABOVE_ACCOUNT_P99", "NEW_DEVICE_HIGH_VALUE"],
  "topFeatures": ["amount_vs_account_avg", "device_age_hours", "geo_distance_km"],
  "modelVersion": 42,
  "latencyMs": 38
}

The response carries triggeredRules, topFeatures and modelVersion because the internal caller has to be able to explain the decision six months later.

8.3 Case Management, Internal

Resolve:

{ "outcome": "CUSTOMER_CONFIRMED", "executiveId": "e_221", "notes": "Verified on call" }

CUSTOMER_CONFIRMED releases the transaction and labels it as a false positive. CUSTOMER_DENIED aborts it and labels it as confirmed fraud. Both outcomes are training labels, which is the whole reason support is in the loop.

9. Data Model

9.1 Transaction Store, Sharded Relational

transactions

Foreign key constraints are dropped deliberately, as noted on the board. The target_acc may not even exist in this bank, and cross shard constraints are not enforceable anyway.

accounts

fraud_cases

outcome_label is the most valuable column in the whole system. It is the ground truth the next model trains on.

case_notes

fraud_rules

model_versions

9.2 Feature Store, Redis

Not a table. Precomputed per account aggregates that the model needs but cannot compute in 200 ms:

feat:acc:{acc}:avg_amount_30d
feat:acc:{acc}:txn_count_1h
feat:acc:{acc}:distinct_devices_7d
feat:acc:{acc}:last_txn_region
feat:ip:{ip}:txn_count_5m

A streaming job keeps these fresh. The question "how many transactions from this IP in the last five minutes" is exactly the kind of signal that catches an attack in progress, and it is impossible to compute from the transaction table inside the latency budget.

9.3 Relationships

In words:

• One customer owns many accounts, one account belongs to one customer.
• One account sends many transactions, one transaction has one source account.
• One transaction may open at most one fraud case, and a case belongs to exactly one transaction.
• One case has many notes, one note belongs to one case.
• One executive handles many cases and writes many notes.
• One model version scored many transactions, and each transaction records the version that scored it.
• Rules trigger on many transactions, recorded as a list on the case rather than as a join table, because the list is written once and only ever read whole.

10. Key Flows

10.1 A Clean Transaction

10.2 A Suspicious Transaction

10.3 The Training Loop

This loop is what makes the system adaptive. Support outcomes become labels, labels become a model, and the model changes tomorrow's decisions.

11. Deep Dives

11.1 Meeting the 200 ms Budget

Rules run before the model on purpose. A blocked region is a single comparison, so there is no reason to score a forest first.

11.2 Precision Versus Recall

This is the real product decision, not a technical one.

• A false positive blocks a legitimate customer, generates a support call and annoys someone who did nothing wrong.
• A false negative lets actual fraud through, and the bank absorbs the loss.

The threshold is a business choice, and it is usually not one threshold. A practical setup uses bands: below 0.3 allow, 0.3 to 0.8 allow but flag for review, above 0.8 block and open a case. High value transactions get a lower block threshold than small ones, because the loss from a false negative scales with the amount and the cost of a false positive does not.

11.3 Class Imbalance

Fraud is rare, often well under one percent of transactions. A model that always says "not fraud" would be 99 percent accurate and completely useless. So:

• Never evaluate on accuracy. Use precision, recall and area under the precision recall curve.
• Rebalance the training set by undersampling clean transactions or oversampling fraud.
• Weight the classes so a missed fraud costs the model more than a false alarm.

11.4 Why a Forest and Not a Neural Network

11.5 Graph Signals

Individually clean transactions can form an obvious ring when viewed as a graph: many accounts sharing one device id, one phone number or one IP, moving money in a cycle. A graph database over accounts and devices finds these patterns.

Those queries are too slow for the 200 ms path, so they run as a batch job. The output is a precomputed risk flag per account, written into the feature store, where the synchronous path reads it as a single lookup.

12. Failure Handling

The fail open versus fail closed question is genuinely harder here than in a rate limiter, because failing open costs money directly. The compromise above, splitting by transaction value, is what real systems do.

13. Tradeoffs

14. Future Work

• Stream processing with Flink or Spark Streaming for real time windowed features, replacing the periodic feature job.
• Graph database such as Neo4j for synthetic identity rings, shared devices and shared phone numbers.
• Shadow mode, where a new model scores live traffic without affecting decisions, so precision and recall can be compared before the switch.
• Per segment thresholds, since a first time user and a ten year customer should not face the same bar.
• Device fingerprinting and behavioural biometrics, such as typing rhythm and how the app is held, as extra features.
• Feedback on false positives, tracking how often a blocked transaction is confirmed legitimate, and treating that rate as a first class metric.

15. Interview Talking Points

• Say why this check is synchronous, and note that it is the exception. Most other designs in this folder push work off the request path, and here you cannot.
• Name the 200 ms budget early, then show how every design choice serves it: the model in memory, the features precomputed, the rules short circuiting.
• The sentence "the model is just a file holding the serialized decision trees" is what makes deployment and rollback obvious.
• Explain where the labels come from. Customer support is the ground truth source, and most candidates never answer that question.
• Bring up class imbalance without being asked. Accuracy is a useless metric when fraud is under one percent.
• Discuss precision versus recall as a business decision with value based thresholds, not as one magic number.
• Handle the availability question carefully. Neither blanket allow nor blanket block is acceptable, and splitting by transaction value is the answer.