Design notes

1. The case has no status column

A servicing case's state is a fold over its append-only event log, computed on read. There is no status field to update.

The alternative — a status column kept in sync with an audit table — has one failure mode, and it is permanent. Any path that writes the status and skips the audit row, or writes the audit row and fails before the status update, leaves the two disagreeing. Nothing detects it. It is found months later by a regulator, or by a borrower who has a screenshot, and by then there is no way to reconstruct which one was lying.

Deriving the state costs a query and removes the possibility. The tests assert that a backfilled older event does not overwrite a newer fact, and that an event kind with no state mapping is recorded without changing anything.

2. Events are append-only, and the database enforces the rest

Event raises on update and on destroy. A fact that happened does not get edited.

Idempotency is a unique index on (lender_id, idempotency_key), not a read-then-write check. Two workers handed the same retried job both pass a "does this exist yet" check and both insert; only one survives a unique index, and the loser is handed the winner's row. The suite replays the same outreach fifty times and asserts one event, and asserts that the same key under a different lender is a different event, because tenancy is part of the key.

3. Imports are idempotent at the file and at the row

Servicing shops re-send tapes constantly: a nightly job fires twice, an analyst re-uploads "the good one". The same bytes for the same lender produce one Import row and write nothing, by unique index on the content digest. Within a run, each row upserts on (lender_id, external_id), and a row whose digest is unchanged is counted and not rewritten — so updated_at stays honest and no spurious "loan changed" event fires downstream.

The three counts the importer returns — created, updated, unchanged — are the reconciliation an operator reads before trusting the run. A run that reports twelve updates on a tape nobody changed is a bug you can see. The imports page lets you paste a tape and watch a replay do nothing.

4. A model reading is evidence; only a person makes it a fact

The classifier writes a Reply row with a typed intent, a confidence, and the verbatim span it relied on. It does not write an event, schedule a draft, or close a case. A servicer promotes it, and the promotion is the event — carrying the confidence and the reply id, so the audit trail records that a model was involved and what it said.

This is the decision the evaluation exists to justify. On the labelled set the model reads "I can do 300 a month, that is genuinely all I have. Take it or take the house" as a promise to pay, at 80% confidence. It is a hardship. Acting on it schedules a draft that returns and charges an NSF fee to the borrower least able to absorb one, which is the harm the product is supposed to reduce. Because the confidence is high, a threshold rule would not catch it. Only the boundary does.

Two guards on the structured output

The model is constrained by a response schema, so the caller gets a typed object or an error — never a paragraph to regex. An intent outside the enum is a failure and is recorded as one, not coerced to the nearest match.

Then both actionable fields are re-derived from the borrower's own text. An amount survives only if its digits appear in the email; a date survives only if the email contains a date-shaped token. A model that confidently supplies a figure the borrower never wrote loses it here. Both guards are tested directly.

When the upstream fails — and it did during this build, on a model deprecation — the classifier degrades to the deterministic baseline, records the error string on the reply, and shows it in the UI. Nothing is silently dropped and nothing is silently wrong.

What is deliberately not here

• Authentication. Tenancy is modelled and enforced on every key; sessions are not, because they would not teach anything about the problem.
• Outbound sending. Outreach is an event with an idempotency key. The SMTP side is a solved problem and would add surface without adding an argument.
• Background jobs. Classification runs inline so it is watchable. In production it is a job — the idempotency keys are already in place for the retries that implies.
• Caching the eval. The recorded run is a JSON file in the repo, on purpose: it is reviewable in a diff, so a regression in the numbers shows up in a pull request.

Stack

Rails 8.1, PostgreSQL, Hotwire, no build step. 40 labelled replies; 37 tests, 251 assertions covering the import idempotency, the append-only guarantees, the derived state, the grounding guards, and the eval harness itself. Source at github.com/tachyurgy/recourse.