A risk scoring service that stores the factors, not the score

A company applies at 09:14 on a Tuesday. It was incorporated eleven months ago, its registered office is in one country and its sole director lives in another, name screening returned a politically exposed person match at 0.82 similarity, and the declared expected volume is 250,000 EUR a month. The scoring service answers 64, the review band starts at 60, and a case opens. The reviewer's first question is not whether 64 is right but where the 64 came from, and a service that can only answer "the model said so" has already failed her.

This post describes a risk scoring service built for that question. The one constraint that shapes it: the score is never stored as the source of truth. It is the sum of factor rows, each naming the rule version that produced it, the input values it read and the points it added, so any score a reviewer, an auditor or the customer sees two years later can be rebuilt from rows they can open. I have built centralised KYB, KYC and AML dashboards, and this is the shape I would give the scoring behind one.

flowchart TD
  E[Scoring request] --> S[Input snapshot]
  P[Data providers] --> S
  S --> R[Rule engine]:::accent
  V[Rule set versions] --> R
  R --> F[Factor rows]
  F --> A[Aggregator]
  T[Threshold versions] --> A
  A --> D[Decision row]
  D --> C[Review case]

The aggregator is the only place arithmetic happens, and it is small enough to read in one screen. It takes the factor rows and a threshold set and returns everything the decision row needs; it never touches the profile, the providers or the rules.

// simplified: the aggregation step, given sealed factor rows
type Category = 'geography' | 'ownership' | 'screening' | 'volume';

type Factor = {
  ruleId: string;
  ruleVersion: number;
  category: Category;
  points: number;
  status: 'scored' | 'missing';
  snapshotIds: number[];
};

type ThresholdSet = {
  version: number;
  bandFor(score: number): 'approve' | 'review' | 'decline';
};

const CAPS: Record<Category, number> = {
  geography: 30, ownership: 25, screening: 40, volume: 20,
};

function aggregate(factors: Factor[], thresholds: ThresholdSet) {
  const byCategory = new Map<Category, number>();
  for (const f of factors) {
    byCategory.set(f.category, (byCategory.get(f.category) ?? 0) + f.points);
  }
  let score = 0;
  for (const [category, sum] of byCategory) {
    score += Math.min(sum, CAPS[category]);
  }
  const complete = factors.every((f) => f.status === 'scored');
  const band = thresholds.bandFor(score);
  // a run with a missing input is never approved without a person
  const outcome = !complete && band === 'approve' ? 'review' : band;
  return { score, complete, band, outcome, thresholdVersion: thresholds.version };
}

Trade-offs

Failure modes

• The screening provider times out. At 09:14:00 the registry answers in 400 ms; the screening provider has not answered at 09:14:03. The snapshot row for screening is written with status missing, the screening rule emits a missing factor with 0 points, and the other categories sum to 41. Under a plain sum the company is approved below 60 with the 0.82 match never scored; here completeness is false, the outcome is review, and the case reads "Screening unavailable at 09:14:03, provider timed out". The customer sees "your application is under review". When the provider's health check passes, a provider_recovered event opens a new run on a fresh snapshot and its decision supersedes the first.
• Someone edits a weight in place. A rule's points change from 12 to 20 without a new rule set version. Re-summing yesterday's run from its stored factor rows still gives 64, because the rows carry their own points, but a verifier that re-evaluates the rules against the snapshot gets 72 and the hash check fails. The containment is a trigger that refuses UPDATE on published rule set rows, and the verifier job that re-sums a sample of runs each night and pages on a mismatch.
• Two rules score the same evidence. The registered-office rule and the director-residence rule both fire on the same country, 15 points each, and geography would reach 45. The cap holds it at 30, which is visible in the case as "geography 45, capped to 30", so the reviewer can see the overlap rather than wonder why the score is high. A longer fix is an evidence key on the factor row so the second rule on the same snapshot id contributes zero.
• The director's address changes after scoring. The profile now shows a different country, but the case still says the old one. Because the sentence cites snapshot 4181 and the snapshot is immutable, the mismatch is between the profile and the snapshot, not inside the case, and the profile change event is a trigger that opens a new run. The old decision is never edited; it is superseded with a pointer to the new run id.
• A threshold change would reclassify closed cases. The review band moves from 60 to 55. A dashboard that recomputes bands from live thresholds shows yesterday's approvals at 57 as review. The decision row pins threshold version 7, the dashboard groups by that column, and only a periodic review trigger re-scores those subjects under version 8.

Build the snapshot table and the factor table before any rule exists, and write the aggregator as a function over factor rows with a hash check. Add rules one at a time against that, and make the first rule the one that emits a missing factor when a provider does not answer, because every other guarantee in the service depends on it.