Fraud Detection System Design

1-Hour Interview Roadmap

Time	Topic
0 – 5 min	Requirements clarification
5 – 10 min	Capacity estimation
10 – 18 min	High-level architecture + core services
18 – 28 min	Real-time scoring pipeline — rules + ML model serving
28 – 36 min	Feature engineering — velocity checks, device fingerprinting, graph signals
36 – 43 min	Case management + analyst workflow + feedback loops
43 – 50 min	Database design + API design
50 – 56 min	Model lifecycle — training, deployment, champion/challenger
56 – 60 min	Trade-offs + common interview mistakes

---

What Are We Building?

An enterprise Fraud Detection System (FDS) that evaluates every transaction, login, account change, and payment in real time and produces a risk score with an action recommendation — approve, flag for review, or block — within the latency budget of the originating transaction (typically under 150 ms).

Fraud detection is one of the most technically demanding system design problems because it sits at the intersection of three disciplines: high-throughput low-latency systems engineering, machine learning model serving, and operational case management. Getting any one of those wrong costs the business money — either from fraud losses (false negatives) or from declined legitimate transactions (false positives). The cost of a false positive in payments is not zero: a declined card for a good customer generates churn, support calls, and reputational damage.

Real-world scale context:

Company	Transactions/day	Fraud rate	Latency SLA
Visa / MC	700 M+	~0.05%	< 100 ms
PayPal	60 M+	~0.32%	< 150 ms
Stripe	250 M+	~0.1%	< 100 ms
Retail bank	5–20 M/day	~0.2%	< 200 ms
Fintech app	500 K–5 M/day	~0.5–1%	< 300 ms

The five challenges that make this hard:

1. Sub-100 ms latency at high throughput — a scoring call cannot block the transaction for more than a fraction of the total payment round-trip. Feature computation, model inference, and rule evaluation must all happen in that window.
2. Extreme class imbalance — fraud rates of 0.05–1% mean 99%+ of events are legitimate. Naive classifiers that always predict "not fraud" achieve 99%+ accuracy while being completely useless. Precision/recall at the tail matters far more than overall accuracy.
3. Adversarial adaptation — fraudsters observe block decisions and adapt. A static rule set or a model that does not retrain will degrade over time as fraud patterns shift. The system must detect model drift and retrain continuously.
4. Explainability and regulatory compliance — financial regulators require that adverse decisions (declined transactions, frozen accounts) be explainable to customers. "The model said no" is not a compliant answer. Every decision needs audit trail and human-readable rationale.
5. Feedback latency — confirmed fraud labels arrive days or weeks after the transaction (after chargeback or investigation close). The training pipeline must handle delayed labels without poisoning the feature store with stale signals.

---

Functional Requirements

Core scoring:

• Accept a transaction or event (payment, login, account change) and return a risk score (0–1000) + action (APPROVE / REVIEW / BLOCK) within latency SLA
• Evaluate configurable rule sets before and after ML scoring
• Support multiple event types: card payment, ACH, wire, login, device registration, address change, beneficiary addition

Feature computation:

• Compute real-time velocity features (spend in last 1 min / 5 min / 1 hr / 24 hr / 7 days)
• Evaluate device fingerprint signals (new device, known device, emulator detected)
• Compute graph/network features (shared devices, shared accounts, money mule ring detection)
• Provide feature values to analysts for explainability

Case management:

• Route flagged transactions to analyst queues with risk score, feature breakdown, and decision rationale
• Support analyst decision recording (confirm fraud / clear / escalate)
• Close feedback loop — confirmed labels flow back to model training pipeline

Model lifecycle:

• Support champion/challenger model deployment with traffic splitting
• Track model performance metrics (precision, recall, AUC, false positive rate) continuously
• Trigger retraining when drift detected

Alerts and rules management:

• Allow operations team to create/modify/disable rules without code deployment
• Support A/B testing of rule changes
• Provide rule hit statistics and performance dashboard

---

Non-Functional Requirements

Attribute	Target
Scoring latency	p50 < 50 ms, p99 < 150 ms
Throughput	50,000 TPS sustained, 200,000 TPS burst
Availability	99.99% (< 1 hr downtime/year) — cannot block all payments
False positive rate	< 0.3% for card-present, < 1% for card-not-present
Fraud catch rate	> 85% of confirmed fraud flagged before settlement
Label feedback	Confirmed labels ingested within 24 hours of resolution
Audit retention	7 years per PCI-DSS / financial regulation
Model retraining	Minimum weekly; triggered automatically on drift

---

Capacity Estimation

Baseline assumptions (mid-size fintech):

• 5 million transactions/day
• Peak: 5× baseline = 25 M transactions in a 4-hour peak window
• Average transaction payload: 2 KB
• Feature vector size: ~200 features × 4 bytes = 800 bytes
• Fraud rate: 0.3% → 15,000 fraudulent transactions/day
• Analyst team: 50 investigators

Throughput:

5,000,000 / 86,400 = ~58 TPS average
Peak: 5,000,000 * 5 / (4 * 3600) = ~1,736 TPS sustained peak
Design target: 10,000 TPS (3× headroom)

Feature store reads:

• Each scoring call reads ~50 feature keys from Redis
• 10,000 TPS × 50 reads = 500,000 Redis ops/second
• Redis cluster: 6 nodes × ~100K ops/s = 600K ops/s capacity — fits with headroom

Storage:

Event log: 5M events/day × 2 KB = 10 GB/day = 3.65 TB/year
Feature snapshots: 5M × 1 KB = 5 GB/day
Model scores: 5M × 200 bytes = 1 GB/day
Case records: 15,000/day × 5 KB = 75 MB/day
Total: ~15 GB/day hot storage, archive after 90 days

Model inference:

• Gradient boosting (XGBoost): ~1 ms per inference on CPU
• Neural net (MLP): ~3 ms on CPU, < 1 ms on GPU
• 10,000 TPS requires: 10,000 × 1 ms = 10 CPU-seconds/s → 10 CPU cores for inference alone
• Deploy 20 inference pods (2× headroom), each with 4 vCPUs

---

High-Level Architecture

flowchart TD
    subgraph Ingestion
        API[Transaction API Gateway]
        KAFKA[Kafka: raw-events]
    end

    subgraph Scoring Pipeline
        RULES_PRE[Pre-Score Rule Engine]
        FEAT[Feature Computation Service]
        FS[(Redis Feature Store)]
        MODEL[ML Scoring Service]
        RULES_POST[Post-Score Rule Engine]
        DECISION[Decision Aggregator]
    end

    subgraph Persistence
        PGWRITE[(PostgreSQL - write)]
        S3[(S3 - event archive)]
        CH[(ClickHouse - analytics)]
    end

    subgraph Case Management
        QUEUE[Case Queue Service]
        ANALYST[Analyst Workbench UI]
        LABEL[Label Ingestion Service]
    end

    subgraph ML Platform
        TRAIN[Training Pipeline]
        REGISTRY[Model Registry]
        DRIFT[Drift Monitor]
    end

    API --> KAFKA
    API -->|sync scoring| RULES_PRE
    RULES_PRE -->|not blocked| FEAT
    FEAT --> FS
    FEAT --> MODEL
    MODEL --> RULES_POST
    RULES_POST --> DECISION
    DECISION -->|response| API

    KAFKA --> PGWRITE
    KAFKA --> S3
    KAFKA --> CH

    DECISION -->|REVIEW| QUEUE
    QUEUE --> ANALYST
    ANALYST --> LABEL
    LABEL --> TRAIN
    LABEL --> FS

    TRAIN --> REGISTRY
    REGISTRY --> MODEL
    DRIFT --> TRAIN
    CH --> DRIFT

---

Core Service Responsibilities

Service	Responsibility
Transaction API Gateway	Receive events, route to sync scoring pipeline, enforce auth, return decision
Pre-Score Rule Engine	Fast deterministic checks (blocklist, velocity hard limits) — block immediately
Feature Computation	Pull real-time aggregates from feature store, compute derived signals
ML Scoring Service	Load model from registry, run inference, return score + SHAP explanation
Post-Score Rule Engine	Apply score-threshold rules, business policy overrides, allow-list checks
Decision Aggregator	Combine rule outputs + model score → final action + audit record
Redis Feature Store	Sub-millisecond read/write of user/device/merchant velocity counters
Case Queue Service	Route REVIEW decisions to analyst queues by risk tier and domain
Analyst Workbench	UI for investigators — display features, transaction graph, decision tools
Label Ingestion	Accept confirmed fraud/not-fraud labels, publish to Kafka for training
Training Pipeline	Retrain models on labeled dataset, validate, register new version
Model Registry	Store versioned models + metadata, support champion/challenger traffic splits
Drift Monitor	Track feature distribution and model score distribution, trigger retraining
ClickHouse	OLAP store for rule performance, model metrics, investigator productivity

---

Fraud Detection State Machine

stateDiagram-v2
    [*] --> RECEIVED: transaction arrives

    RECEIVED --> HARD_BLOCKED: pre-score rule match (blocklist / velocity limit exceeded)
    RECEIVED --> SCORING: no hard block

    SCORING --> APPROVED: score < low threshold AND no post-rule flag
    SCORING --> REVIEW_QUEUED: score >= review threshold OR post-rule flag
    SCORING --> AUTO_BLOCKED: score >= block threshold

    HARD_BLOCKED --> [*]: event logged, response returned
    AUTO_BLOCKED --> [*]: event logged, response returned
    APPROVED --> [*]: event logged, response returned

    REVIEW_QUEUED --> UNDER_INVESTIGATION: analyst picks up case
    UNDER_INVESTIGATION --> CONFIRMED_FRAUD: analyst confirms fraud
    UNDER_INVESTIGATION --> CLEARED: analyst clears as legitimate
    UNDER_INVESTIGATION --> ESCALATED: analyst escalates to senior / compliance

    ESCALATED --> CONFIRMED_FRAUD: escalation resolved as fraud
    ESCALATED --> CLEARED: escalation resolved as legitimate

    CONFIRMED_FRAUD --> LABEL_INGESTED: label published to training pipeline
    CLEARED --> LABEL_INGESTED: label published to training pipeline
    LABEL_INGESTED --> [*]

---

Step 1: Event Ingestion and Routing

Every evaluatable event enters through the Transaction API Gateway. The gateway does two things simultaneously:

Synchronous path (< 150 ms SLA):

• Validates the event payload (required fields, schema check)
• Calls the scoring pipeline inline
• Returns { score, action, case_id?, explanation } to the caller before completing the transaction

Asynchronous path (no latency impact):

• Publishes the full raw event to Kafka topic raw-events
• Downstream consumers write to PostgreSQL (durable record), S3 (long-term archive), and ClickHouse (analytics)

The synchronous path must not fail open if the scoring service is unavailable. Define a fallback policy per event type:

• Card payments: fail-open with a configurable "degraded mode" score (treat as medium risk, allow with flag)
• Wire transfers > $10,000: fail-closed (decline until scoring recovers)
• Login events: fail-open (allow login, flag for async review)

This policy is business-defined and stored in configuration, not hardcoded.

---

Step 2: Pre-Score Rule Engine

Before invoking the ML model (expensive), the Pre-Score Rule Engine performs fast O(1) deterministic checks:

Blocklist checks (Redis set lookups, ~0.1 ms):

• Is the card number in the global blocklist?
• Is the device fingerprint in the device blocklist?
• Is the IP in the known fraud IP range (CIDR match)?
• Is the beneficiary account in the mule account registry?

Hard velocity limits (Redis counter lookups, ~0.5 ms):

• More than 10 transactions from this card in the last 60 seconds?
• More than $50,000 total spend from this account in the last hour?
• More than 5 failed PIN attempts in 10 minutes?

Geographic impossibility check:

• Last transaction was in Singapore 3 minutes ago; this transaction is from Germany — impossible travel time given physics.
• Requires reading the last event timestamp and location from the feature store.

If any pre-score rule matches, the engine short-circuits — it returns HARD_BLOCK immediately without invoking the feature computation or ML model. This protects model serving capacity and keeps latency at < 5 ms for the obvious fraud cases.

Rules are stored in a database and evaluated by the engine in-memory after a cache warm-up on startup. Rule changes propagate within 30 seconds via a config reload mechanism.

---

Step 3: Feature Computation

For events not caught by pre-score rules, the Feature Computation Service builds the feature vector used by the ML model. Features fall into five categories:

Velocity Features (Redis, computed real-time)

Feature	Window
user_txn_count_1m	1 minute
user_txn_count_1h	1 hour
user_spend_1h	1 hour
user_spend_24h	24 hours
user_spend_7d	7 days
merchant_txn_count_1h	1 hour
user_failed_auth_count_10m	10 minutes
card_decline_count_24h	24 hours

Velocity counters use sliding window approximation in Redis: a sorted set per (entity, feature) where each member is a transaction timestamp. Count members in the window → velocity. Expire old members via a background job.

Device Signals (from device fingerprint service)

• is_new_device — never seen this device for this user
• device_age_days — how long since device first registered
• is_emulator — emulator/jailbreak detection flag from mobile SDK
• device_txn_count — total transactions ever from this device
• device_user_count — how many distinct users have used this device (mule detection)

Behavioral Features (user profile store)

• avg_transaction_amount_30d — normal spend baseline
• transaction_hour_usual — is this transaction at an unusual hour for this user?
• merchant_category_match — is this merchant category normal for this user?
• recipient_is_known — has this user sent to this recipient before?

Network / Graph Features (computed async, cached)

• recipient_fraud_score — fraud score of the receiving account
• shared_device_risk — max fraud score among users sharing this device
• ip_cluster_risk — historical fraud rate at this IP subnet

Transaction Context

• transaction_amount, merchant_category_code, card_present flag, country_mismatch, currency_mismatch, channel (mobile/web/POS)

The feature service fetches all these in parallel using Redis pipelining and a batch read from the user profile store. Total feature fetch time: 5–15 ms for warm cache.

---

Step 4: ML Scoring Service

The ML Scoring Service receives the feature vector and returns a normalized risk score (0–1000) plus a SHAP-based explanation.

Model Architecture

Primary model: Gradient Boosted Trees (XGBoost)

• Reasons: fast inference (~1 ms), handles tabular data well, produces feature importances natively
• Input: ~200 numeric features
• Output: probability of fraud (0–1), converted to score 0–1000

Secondary model: MLP (for behavioral sequence signals)

• Processes user's last N transaction sequences
• Captures temporal patterns XGBoost misses (e.g., account takeover velocity buildup)

Ensemble: weighted average of XGBoost score and MLP score. Weights are tunable per event type.

Champion/Challenger Deployment

The model registry stores multiple model versions. Traffic is split:

• Champion (current production model): 90% of traffic
• Challenger (candidate new model): 10% of traffic

Both models produce scores; only the champion's decision is used for the real-time response. The challenger's scores are logged for offline comparison. When the challenger's AUC on labeled outcomes exceeds the champion's by a statistically significant margin, it is promoted automatically (or with a human approval step for high-stakes event types).

SHAP Explanations

Every model inference produces a SHAP value vector — one value per feature — representing each feature's contribution to the final score. These are:

• Stored in the case record for analyst use
• Used to generate human-readable explanations ("High risk: 3 failed authentication attempts in the last 10 minutes, transaction is 4× user's average spend, new device never seen before")
• Provided to customers in adverse action notices (regulatory requirement)

SHAP computation adds ~3 ms to inference time. For the highest-throughput event types, SHAP is computed asynchronously and attached to the case record within 1 second.

---

Step 5: Post-Score Rule Engine

After ML scoring, the Post-Score Rule Engine applies policy overlays:

Score threshold rules (configurable):

score >= 800  → BLOCK
score >= 500  → REVIEW
score < 500   → APPROVE

Allow-list overrides:

• Trusted merchant + trusted device + score < 600 → downgrade REVIEW to APPROVE
• Recurring bill payment to known biller → downgrade REVIEW to APPROVE if amount within 10% of typical

Escalation rules:

• Wire transfer > $25,000 → force REVIEW regardless of score
• New beneficiary + international transfer → force REVIEW
• Account age < 30 days + any transaction > $1,000 → REVIEW

Rule performance tracking: Every rule records hit count, false positive rate (when analyst clears), and false negative rate (when confirmed fraud slips through). Rules with FPR > 5% are flagged for review by the ops team.

---

Step 6: Case Management

REVIEW decisions are routed to the Case Queue Service, which assigns cases to analyst queues based on:

• Risk tier: score 500–650 (low review), 650–799 (high review), manual escalation
• Domain specialization: card fraud queue, account takeover queue, wire fraud queue
• Analyst availability and workload balancing

Case Record Structure

Each case contains:

• The original transaction payload
• Full feature vector with SHAP values
• Timeline: all transactions in last 24h for this user
• Device history: all devices used by this account
• Account network graph: accounts linked via shared device/IP
• Previous cases on this user
• Analyst decision tools: confirm fraud, clear, request more info, escalate

Analyst Decision Workflow

sequenceDiagram
    participant FDS as Fraud Detection
    participant CQ as Case Queue
    participant AW as Analyst Workbench
    participant LI as Label Ingestion
    participant TP as Training Pipeline

    FDS->>CQ: route REVIEW case
    CQ->>AW: assign to analyst
    AW->>AW: investigate (transaction history, device, graph)
    AW->>CQ: record decision (FRAUD / CLEAR / ESCALATE)
    CQ->>LI: publish confirmed label event
    LI->>TP: label available for training
    LI->>FDS: update feature store (confirmed fraud flags)

SLA for Case Resolution

Queue	Target Resolution Time
Low risk	24 hours
High risk	4 hours
Wire fraud	1 hour
Account takeover	30 minutes

Cases approaching SLA are auto-escalated to senior analysts.

---

Step 7: Feedback Loop and Model Training

The feedback loop is the most critical — and most neglected — part of a fraud detection system. Without quality labels flowing back to the training pipeline, the model degrades over time as fraud patterns shift.

Label Sources

Source	Latency	Quality
Analyst decision	Hours–days	High
Chargeback (card network)	30–90 days	High
Customer dispute	7–30 days	Medium
Automated rule match	Real-time	Low

Because chargeback labels arrive with significant delay, the training pipeline uses a delayed labeling strategy:

• Training data includes only transactions with labels older than 30 days (to ensure near-complete labeling)
• Near-real-time retraining uses analyst decisions + automated rule matches as proxy labels
• Monthly full retrain uses complete chargeback-confirmed labels

Training Pipeline

sequenceDiagram
    participant LI as Label Store
    participant FE as Feature Engineering
    participant TR as Trainer
    participant VAL as Validator
    participant REG as Model Registry
    participant SCORE as Scoring Service

    LI->>FE: pull labeled events (last 90 days)
    FE->>FE: reconstruct feature vectors at event time
    FE->>TR: training dataset (features + labels)
    TR->>TR: train XGBoost + MLP
    TR->>VAL: candidate models
    VAL->>VAL: compute AUC, precision@K, recall@K
    VAL->>VAL: compare challenger vs champion
    VAL->>REG: register if challenger wins
    REG->>SCORE: deploy as new challenger (10% traffic)
    SCORE->>REG: report live metrics after 48h
    REG->>SCORE: promote to champion if metrics hold

Point-in-time correctness: Feature vectors for training must be reconstructed as they were at the time of the transaction — not using today's feature values. This prevents label leakage where future information contaminates the training signal. The feature store must store historical snapshots of feature values for every transaction.

---

Step 8: Device Fingerprinting

Device fingerprinting generates a stable, unique identifier for a client device without requiring user login. It is one of the strongest fraud signals — a confirmed fraud device contaminates all future transactions from it.

Browser Fingerprinting Signals

• User agent, screen resolution, color depth, timezone
• Canvas fingerprint (hardware rendering signature)
• WebGL renderer and vendor string
• Installed fonts list
• AudioContext fingerprint
• JavaScript engine timing signals

Mobile SDK Signals

• Hardware identifiers (IMEI, advertising ID — where permitted)
• Jailbreak / root detection
• Emulator detection (virtualized environment markers)
• App install timestamp
• Sensor fingerprint (accelerometer/gyroscope calibration signatures)

Fingerprint Stability

Browser fingerprints change when users update their browser or clear data. The fingerprinting service uses a fuzzy matching algorithm — if 80%+ of signals match a known fingerprint, it assigns the same device ID and updates the stored profile. This handles the natural drift of legitimate devices while still detecting fresh installations by fraudsters.

Device Graph

The device graph links:

• Device → User (a device used by multiple users is a risk signal)
• Device → IP addresses historically used
• Device → Transaction outcomes

A device used by 10 different users with high fraud rates is a "mule device" — any new user transacting from it inherits a risk penalty.

---

Database Schema

PostgreSQL: Transaction Events and Cases

-- Core transaction event record
CREATE TABLE fraud_events (
    event_id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    external_id       VARCHAR(128) NOT NULL UNIQUE,  -- caller's transaction ID
    event_type        VARCHAR(50)  NOT NULL,          -- PAYMENT, LOGIN, ACH, WIRE
    user_id           UUID         NOT NULL,
    account_id        UUID         NOT NULL,
    device_id         UUID,
    amount_cents      BIGINT,
    currency          CHAR(3),
    merchant_id       VARCHAR(128),
    channel           VARCHAR(20),
    ip_address        INET,
    event_payload     JSONB        NOT NULL,
    score             SMALLINT,                       -- 0-1000
    action            VARCHAR(10)  NOT NULL,          -- APPROVE/REVIEW/BLOCK
    model_version     VARCHAR(50),
    features_snapshot JSONB,                          -- feature vector at decision time
    shap_values       JSONB,                          -- SHAP per feature
    decision_reason   TEXT,
    created_at        TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    decided_at        TIMESTAMPTZ
);

CREATE INDEX idx_fraud_events_user_id     ON fraud_events (user_id, created_at DESC);
CREATE INDEX idx_fraud_events_account_id  ON fraud_events (account_id, created_at DESC);
CREATE INDEX idx_fraud_events_device_id   ON fraud_events (device_id, created_at DESC);
CREATE INDEX idx_fraud_events_action      ON fraud_events (action, created_at DESC);
CREATE INDEX idx_fraud_events_external_id ON fraud_events (external_id);

-- Fraud cases for analyst workflow
CREATE TABLE fraud_cases (
    case_id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    event_id         UUID NOT NULL REFERENCES fraud_events(event_id),
    user_id          UUID NOT NULL,
    queue_name       VARCHAR(50)  NOT NULL,
    risk_tier        VARCHAR(20)  NOT NULL,  -- LOW, HIGH, ESCALATED
    assigned_to      UUID,                   -- analyst user_id
    status           VARCHAR(20)  NOT NULL DEFAULT 'OPEN',
    analyst_decision VARCHAR(20),            -- FRAUD, CLEAR, ESCALATE
    analyst_notes    TEXT,
    opened_at        TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    assigned_at      TIMESTAMPTZ,
    resolved_at      TIMESTAMPTZ,
    sla_deadline     TIMESTAMPTZ  NOT NULL
);

CREATE INDEX idx_cases_queue_status ON fraud_cases (queue_name, status, sla_deadline);
CREATE INDEX idx_cases_assigned_to  ON fraud_cases (assigned_to, status);
CREATE INDEX idx_cases_user_id      ON fraud_cases (user_id, opened_at DESC);

-- Confirmed fraud labels
CREATE TABLE fraud_labels (
    label_id         UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    event_id         UUID NOT NULL REFERENCES fraud_events(event_id),
    user_id          UUID NOT NULL,
    label            VARCHAR(20)  NOT NULL,  -- FRAUD, LEGITIMATE
    label_source     VARCHAR(30)  NOT NULL,  -- ANALYST, CHARGEBACK, DISPUTE, AUTO_RULE
    label_confidence NUMERIC(4,3),           -- 0.0 – 1.0
    labeled_at       TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    labeler_id       UUID                    -- analyst user_id or system
);

CREATE INDEX idx_labels_event_id  ON fraud_labels (event_id);
CREATE INDEX idx_labels_labeled_at ON fraud_labels (labeled_at DESC);
CREATE INDEX idx_labels_label     ON fraud_labels (label, labeled_at DESC);

-- Rule definitions
CREATE TABLE fraud_rules (
    rule_id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    rule_name        VARCHAR(100) NOT NULL UNIQUE,
    rule_type        VARCHAR(20)  NOT NULL,   -- PRE_SCORE, POST_SCORE
    event_types      TEXT[]       NOT NULL,
    expression       TEXT         NOT NULL,   -- DSL expression evaluated by rule engine
    action           VARCHAR(10)  NOT NULL,   -- BLOCK, REVIEW, APPROVE
    priority         INT          NOT NULL DEFAULT 100,
    is_enabled       BOOLEAN      NOT NULL DEFAULT TRUE,
    created_at       TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    updated_at       TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    created_by       UUID         NOT NULL
);

-- Device registry
CREATE TABLE devices (
    device_id        UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    fingerprint_hash VARCHAR(64)  NOT NULL UNIQUE,
    device_type      VARCHAR(20)  NOT NULL,   -- BROWSER, IOS, ANDROID
    first_seen_at    TIMESTAMPTZ  NOT NULL DEFAULT NOW(),
    last_seen_at     TIMESTAMPTZ,
    risk_score       SMALLINT     NOT NULL DEFAULT 0,
    is_blocklisted   BOOLEAN      NOT NULL DEFAULT FALSE,
    signals          JSONB        NOT NULL
);

CREATE INDEX idx_devices_hash         ON devices (fingerprint_hash);
CREATE INDEX idx_devices_blocklisted  ON devices (is_blocklisted) WHERE is_blocklisted = TRUE;

---

API Design

POST /v1/fraud/evaluate

Synchronous scoring endpoint called by the transaction processor.

Request:

{
  "external_id": "txn_8f3a9b2c",
  "event_type": "PAYMENT",
  "user_id": "usr_7d4e8a1f",
  "account_id": "acc_3c9b2e5a",
  "device_fingerprint": "fp_a1b2c3d4e5f6",
  "ip_address": "203.0.113.45",
  "amount_cents": 149900,
  "currency": "USD",
  "merchant_id": "mch_amazon",
  "merchant_category_code": "5999",
  "channel": "WEB",
  "card_present": false,
  "metadata": {
    "browser_ua": "Mozilla/5.0 ...",
    "session_id": "sess_xyz",
    "referrer": "direct"
  }
}

Response (APPROVE):

{
  "event_id": "evt_c2d3e4f5a6b7",
  "external_id": "txn_8f3a9b2c",
  "score": 210,
  "action": "APPROVE",
  "model_version": "xgb_v42",
  "latency_ms": 38,
  "top_risk_factors": [
    { "feature": "is_new_device", "contribution": 45, "value": true },
    { "feature": "user_spend_1h", "contribution": 12, "value": 420.00 }
  ],
  "decided_at": "2026-07-14T09:23:41.210Z"
}

Response (REVIEW):

{
  "event_id": "evt_d3e4f5a6b7c8",
  "external_id": "txn_9g4b0c3d",
  "score": 620,
  "action": "REVIEW",
  "case_id": "case_e4f5a6b7c8d9",
  "model_version": "xgb_v42",
  "latency_ms": 52,
  "top_risk_factors": [
    { "feature": "user_spend_1h", "contribution": 180, "value": 4500.00 },
    { "feature": "is_new_device", "contribution": 120, "value": true },
    { "feature": "transaction_amount_vs_avg", "contribution": 95, "value": 8.3 }
  ],
  "human_readable_reason": "Transaction is 8× user's average spend; sent from a device not previously seen on this account.",
  "decided_at": "2026-07-14T09:23:42.340Z"
}

Response (BLOCK):

{
  "event_id": "evt_e5f6a7b8c9d0",
  "external_id": "txn_0h5c1d4e",
  "score": 920,
  "action": "BLOCK",
  "block_reason": "RULE_MATCH",
  "rule_matched": "device_blocklist",
  "model_version": "xgb_v42",
  "latency_ms": 4,
  "decided_at": "2026-07-14T09:23:43.110Z"
}

POST /v1/fraud/label

Submit a confirmed label for a past event (called by chargeback ingestion, analyst decisions).

{
  "event_id": "evt_c2d3e4f5a6b7",
  "label": "FRAUD",
  "label_source": "CHARGEBACK",
  "label_confidence": 1.0,
  "labeled_at": "2026-07-14T09:23:41.210Z"
}

GET /v1/fraud/cases?queue=wire_fraud&status=OPEN&limit=20

Returns analyst workbench cases with full context.

PATCH /v1/fraud/cases/{case_id}/decision

{
  "decision": "CONFIRMED_FRAUD",
  "notes": "User confirmed card stolen 3 days prior. Transaction pattern matches known fraud ring targeting electronics merchants."
}

---

Kafka Event Architecture

Topic	Producer	Consumers	Retention
fraud.events.raw	API Gateway	PostgreSQL writer, S3 archiver, CH	7 days
fraud.decisions	Decision Aggregator	Case Queue, Notification Service	30 days
fraud.labels	Label Ingestion	Training Pipeline, Feature Store	90 days
fraud.model.deployed	Model Registry	Scoring Service, Monitoring	30 days
fraud.alerts.rules	Rule Engine	Ops Dashboard, ClickHouse	30 days
fraud.device.blocklisted	Case Management	Pre-Score Rule Engine, Feature Store	90 days

---

Observability

Key Metrics

Metric	Alert Threshold
fraud.scoring.latency_p99_ms	> 150 ms
fraud.scoring.error_rate	> 0.1%
fraud.decision.block_rate	> 5% (sudden spike)
fraud.decision.false_positive_rate	> 1% (rolling 1h)
fraud.model.auc	< 0.90 (weekly eval)
fraud.feature_store.cache_miss_rate	> 5%
fraud.cases.sla_breach_rate	> 2%
fraud.labels.ingestion_lag_hrs	> 48 hours
fraud.model.score_distribution_drift	PSI > 0.2

Distributed Tracing

Every scoring request carries a trace ID through: API Gateway → Pre-Score Rule Engine → Feature Computation → ML Scoring → Post-Score Rule Engine → Decision Aggregator

The trace records latency at each hop, which features were fetched, which rules were evaluated, and the model score. This is essential for debugging unexpected decisions and for SLA compliance audits.

---

Design Trade-offs

1. Rules-First vs Model-First

Option A (Rules-first): Run the rule engine before the model. Fast blocks on obvious cases save model compute.

Option B (Model-first): Run the model on everything, then apply rules as a policy layer.

Decision: Rules-first pre-score, model-first for everything else. Hard blocklist and velocity hard limits are O(1) Redis lookups. Running the model on a card that is on the global blocklist wastes compute and adds latency. But all nuanced decisions go through the model — rules-only systems are too brittle for novel fraud patterns.

---

2. Synchronous vs Asynchronous Scoring

Option A (Sync): Block the transaction response until the score is ready. Full accuracy.

Option B (Async): Return provisional approval immediately, reverse the transaction asynchronously if fraud is detected.

Decision: Sync for all transaction types. Asynchronous scoring has a reversal window problem — the fraudster has already received value (gift card codes, wire funds) before the reversal fires. Sync scoring at < 150 ms is achievable with the right architecture. Async scoring is only acceptable for very low-value, low-risk event types (e.g., login events where the only action is flagging).

---

3. Single Model vs Ensemble

Option A (Single model): One XGBoost model trained on all event types.

Option B (Ensemble): Separate specialized models per event type (card fraud model, ATO model, wire fraud model) plus a meta-ensemble.

Decision: Specialized models per event type with a common infrastructure. Card fraud patterns (velocity, merchant) are very different from account takeover patterns (login behavior, device changes). Training on mixed data causes one signal to dominate. The infrastructure (feature store, model registry, serving layer) is shared, but each event type has its own trained model.

---

4. Feature Freshness vs Storage Cost

Option A (Real-time features only): Compute all features from Redis at scoring time. Ultra-fresh, but Redis is expensive for large history windows.

Option B (Pre-computed batch features + real-time deltas): Nightly batch job computes 30-day and 90-day aggregates into a fast key-value store. Real-time features cover the last 24 hours. The model uses both.

Decision: Hybrid — real-time features for windows < 24 hours, pre-computed features for longer windows. Maintaining 90 days of per-user sorted sets in Redis is prohibitively expensive at scale. Pre-compute longer windows with Spark overnight. This introduces up to 24 hours of staleness in long-window features — acceptable because those features are trend signals, not real-time indicators.

---

Common Interview Mistakes

1. Only designing the happy path. A fraud detection system that doesn't discuss what happens when the scoring service is unavailable is incomplete. Failover policies (fail-open vs fail-closed) are a core design decision.

2. Treating fraud detection as purely a model problem. The model is roughly 40% of the system. Case management, feedback loops, rule engines, and the feature store are equally important and get neglected in interviews.

3. Ignoring label latency. Saying "we retrain the model on fraud labels" without addressing that chargeback labels arrive 30–90 days late is a significant omission. Delayed labeling strategies must be discussed.

4. Designing for precision only. Blocking all transactions above a certain score maximizes precision but destroys recall and creates massive false positive rates. The precision/recall tradeoff and its business implications must be addressed explicitly.

5. Forgetting point-in-time correctness in training. Using today's feature values to train a model on historical transactions leaks future information into the training set. Always reconstruct features as they were at transaction time.

6. Proposing a static rule engine. Rules that cannot be updated without a code deployment are a liability. The operations team needs to add, modify, and disable rules within minutes of discovering a new fraud pattern. A DSL-based rule engine with a management UI is required.

7. Overlooking explainability. "The model said no" is not a compliant response in a regulated financial context. SHAP values and human-readable decision reasons are not a nice-to-have — they are a regulatory requirement for adverse action notices.

8. Underestimating the feature store. Students often propose Postgres for feature storage. At 500K ops/second, you need Redis or a purpose-built feature store. The feature store design (data model, TTL, sliding window approximation) deserves its own detailed discussion.

---

Summary

flowchart LR
    A[Transaction Arrives] --> B{Pre-Score Rules}
    B -->|Hard block| Z1[BLOCK — instant]
    B -->|Pass| C[Feature Computation]
    C --> D[ML Model Inference]
    D --> E{Post-Score Rules}
    E -->|score < 500| Z2[APPROVE]
    E -->|500-799| F[Case Queue]
    E -->|800+| Z3[BLOCK — score]
    F --> G[Analyst Investigation]
    G --> H[Confirmed Label]
    H --> I[Training Pipeline]
    I --> J[New Model Version]
    J --> D

Design Principles:

• Latency is a feature — a scoring system that misses its 150 ms SLA is not production-ready, regardless of its accuracy
• Rules protect the model — fast deterministic rules guard model compute capacity and handle obvious cases cheaply
• Labels are the fuel — a fraud system without a quality feedback loop will drift into irrelevance within months
• Explainability is mandatory — every block decision must produce a human-readable rationale for regulatory compliance
• False positives cost money — declining a good customer has a measurable cost; optimize for the business-defined precision/recall operating point, not raw accuracy
• Adversarial systems degrade — fraudsters adapt; continuous monitoring, drift detection, and retraining are not optional

---