DevEx payouts board, ADHD edition

SAY THIS SENTENCE FIRST

"Robux is a closed double-entry economy, and DevEx is the one door out of it. A request escrows Robux atomically at submission, and what happens to that escrow — destroyed (burned) or refunded — is decided by what real dollars actually did."

STEP 1 OF 5

Requirements

4 min

ASK, THEN COMMIT

Four questions. State your assumption for each so the interview moves even without answers.

"Real time" earnings: exact-and-instant, or seconds-stale display over an exact ledger? Assume the latter. Only the balance at submission must be exact.
Can the payment processor (the external service that moves real dollars) pay X out of Y (partials)? Assume yes, and partials must settle cleanly.
Is exchanged Robux destroyed, or returned to a pool? Assume destroyed, via a burn account. Either way it is recorded as a ledger transfer.
Who owns eligibility rules (account age, identity checks)? Assume a separate service; we snapshot its answer, we do not re-implement it.

Numbers to say

Earnings: tens of millions of player purchases/day ≈ hundreds of writes/sec, spiky. Payouts: thousands/day ≈ under 0.1/sec. Completion within 5 business days. All amounts in integer minor units.

Hard rules

No over-withdrawal, ever. The Robux-to-USD rate is locked at submission and can never drift. Every state change lands in an append-only audit trail.

DONE WHEN: interviewer nods at the scale split: earnings are high-volume, payouts are tiny but high-stakes. That split shapes everything.

STEP 2 OF 5

Core entities

1 min

Developer account: holds a withdrawable Robux balance.
Earnings ledger entry: one leg of a Robux movement. A player purchase splits into the developer's share and the platform's cut. This is the truth.
DevEx request: one exchange, carrying a locked rate, an eligibility snapshot, and a lifecycle.
Exchange rate version: an immutable (rate, valid-from) row. Requests point at one.
Payout attempt: one immutable row per processor call, committed BEFORE the call (state SENDING). Closed with the outcome when one arrives; a timeout leaves it SENDING — the missing outcome IS the record.
System accounts: DEVEX_ESCROW holds in-flight Robux; ROBUX_BURN is where completed exchanges destroy it.

DONE WHEN: you have said the opening sentence again, pointing at ESCROW and BURN.

STEP 3 OF 5

API

3 min

POST   /v1/devex-requests        Idempotency-Key: key  // scoped (developer, key)
  { robux: 100000 }
  → 201 { requestId, PENDING, rateId, usdMinor }   // rate + USD locked NOW
  → 422, body: { error: INSUFFICIENT_BALANCE | BELOW_MINIMUM | NOT_ELIGIBLE, reasons }
  → 409, body: { error: KEY_REUSED }   // same key, different body
GET    /v1/devex-requests?cursor=          // history
GET    /v1/devex-requests/{id}             // status + every attempt
DELETE /v1/devex-requests/{id}             // cancel; only while PENDING; 409 once processing
GET    /v1/earnings?game=&granularity=day  // display; eventual; carries asOf (freshness timestamp)

The 201 already contains the locked rate and computed USD, because both are fixed inside the submission transaction. Nothing about the amount can drift afterward.
Earnings displays are allowed to lag and say so. Only submission touches the exact balance.

STEP 4 OF 5

High-level design

10 min, end to end, no dives yet

Whiteboard version: draw order

player purchases → ROBUX LEDGER (truth) → CDC (change data capture: the commit-log stream) → earnings rollups (display)   -- row 1: the economy
developer → API → submission txn: eligibility snapshot + rate lock + escrow debit  -- row 2: the door
worker claims PENDING → processor (idempotency key = requestId) → outcome settles escrow: burn / refund / split
the processor's settlement report → daily reconciliation; separately, CDC → write-once archive   -- row 3: the audit floor

THE WALKTHROUGH, ONE BREATH PER CLAUSE

The earnings side is a closed Robux economy: every purchase writes double-entry legs splitting the spend, and dashboards are derived rollups off the ledger's change stream → dive A (in the reserve list, next section). The payout side is deliberately tiny and serial: submission is one transaction that snapshots eligibility, locks the rate, computes the USD, and moves Robux into escrow under a no-overdraft guard → dive B. Workers claim requests with leases and call the processor using the request id as the idempotency key → dive D; the outcome settles the escrow: burn on paid, refund on failed, split on partial → dive E. Daily reconciliation diffs the processor's report, and every change streams to write-once storage for compliance.

DONE WHEN: the interviewer picks a box to open. Let them steer from here.

If they just nod and wait: "the riskiest part is the submission transaction, shall I open it?"
Running behind: protect dives B and E. Cut A first (it reuses the ledger design from the wallet-transfers board, separate prep material).

STEP 5 OF 5

Deep dives: the core four

~20 min, interviewer steers

TRIGGER: "two submissions race for the same balance" / "walk me through submit"

B · The submission transaction

ONE TRANSACTION · everything the money depends on is fixed inside it

Before anything: the replay check

Look up (developer, key). Seen, body hash matches: return the stored answer verbatim. Retries must replay even if eligibility has since changed.
Seen, body differs: 409 KEY_REUSED. A key names one exact request, forever.

Then, before the transaction

Call the eligibility service (slow, external) and keep its full answer.
Not eligible: return 422 with the reasons. Nothing was started.

Inside one COMMIT

1. Insert the request row, PENDING. ON CONFLICT (developer, key) catches two racing FIRSTs: the loser re-reads and replays.
2. Read the current rate row (immutable) and compute the USD. Store both on the request.
3. Debit the developer and credit DEVEX_ESCROW, with the guard fused into the debit: balance minus robux must stay at or above zero.
4. Write the two ledger legs and the eligibility snapshot.
Zero rows from the guard: mark REJECTED, commit, 422. No Robux ever moved, so there is nothing to refund. The anchor (the request row, our idempotency record) survives for retries.

The double-spend, traced

Two submissions race for a 100K balance, 80K each.
The second waits on the row lock, then re-checks against the already-debited balance.
It matches zero rows and is rejected. No check-then-act window exists.
Say the punchline: "strongly consistent balance" is one line of SQL here, not a distributed protocol.

TRIGGER: "which rate applies" / "rules changed after they submitted"

C · Freeze the inputs: rates and eligibility

ONE IDEA, APPLIED TWICE · decisions are written down at submission, so later change cannot touch them

Rates

A rate change is a NEW immutable row, never an update.
The request stores which rate row it used and the USD it computed.
Processing days later reads the request and cannot see any other rate.
"Why did this pay $350?" is answered by the request row itself: it stores rateId and usdMinor.

Eligibility

The full check result (rules passed, identity status, checked-at) is stored on the request.
Compliance asks "why was this approved": the answer is the snapshot, immune to later rule changes.
Re-check policy, explicit: once, at submission. The snapshot IS the decision; re-checking later would let a rule change rewrite an already-priced request.

The minimum

Checked at the door: static validation before the eligibility call, 422 BELOW_MINIMUM. Nothing racy about it, so it needs no guard; the balance check is the only racy one.

TRIGGER: "the processor is slow/down" / "your worker crashes mid-payout"

D · The payout pipeline

ONE PIPELINE · built for crash-safety, not throughput, and say so

Claiming work

Workers sweep PENDING rows. A claim locks its row; others SKIP locked rows instead of waiting (FOR UPDATE SKIP LOCKED).
The claim also writes a lease and an attempt number, and sets PROCESSING.
A reaper flips expired-lease rows back to PENDING (lease expiry is the one PROCESSING → PENDING loop) and stamps the next retry time.
At thousands/day, ONE worker is enough. The machinery exists for crash-safety. Saying that is the judgment point.

Calling the processor

The idempotency key IS the request id.
So every retry, across crashes and expired leases, collapses to at most one real disbursement.
Per-attempt keys are the double-payout bug. Name it before they do.

The ambiguous moment

We called the processor, then crashed before recording the answer.
The row is still PROCESSING; when the lease expires, the next attempt retries with the SAME key, or asks the processor's read-only status endpoint what happened.
Only a definitive processor answer moves the row.
The 5-day budget is why this is calm: retries at 15m, 1h, 6h, 1d, 2d all fit inside it.

The audit trail

Every attempt is an immutable row, committed BEFORE its call: attempt number, key (= the requestId, always), state, outcome once one arrived, processor reference, timestamps.
The history endpoint returns this table verbatim.

TRIGGER: "the processor paid half" / "what ends a request"

E · Terminal states settle the escrow

ONE RULE · every ending is a ledger transfer, written WITH the status change

The endings — five with escrow to settle, one without

COMPLETED: escrow → ROBUX_BURN, all of it. Robux leaves the economy exactly when dollars left the platform.
FAILED: escrow → developer, full refund. Notification out (email/webhook, after COMMIT, outside the money path).
CANCELLED (only while PENDING): same full refund, guarded so it cannot race a claim.
PARTIAL (paid X of Y): burn floor(robux × X/Y), refund the rest.
Ambiguous X: CAS to ON_HOLD, park in the DLQ (dead-letter queue, the holding table). Alerts fire, NO legs move, a human picks one of the endings above.
REJECTED is the odd one out: born at submit when the guard says no. Nothing was ever debited, so there is nothing to refund — an ending with no legs.

The invariant that audits everything

ESCROW's balance = the sum of all in-flight requests' Robux, checkable continuously.
And every burn has a matching line in the processor settlement report.
One sentence: money left the platform if and only if Robux left the economy.

The state machine

PENDING → PROCESSING → COMPLETED | PARTIAL | FAILED | ON_HOLD. PENDING → CANCELLED.
Lease expiry loops PROCESSING back to PENDING. REJECTED is born terminal at submit; it never enters the loop.
Every transition is an atomic compare-and-set on the status, committed together with its settlement legs. No transition can skip the money.

STEP 5, CONT.

Three more dives

rehearse after the core four

TRIGGER: "how do earnings get tracked at scale"

A · The earnings ledger

ONE ANSWER · the wallet design, reused, and say so

One purchase, three legs

A player spends 400 Robux in game G: minus 400 player, plus 280 developer share, plus 120 platform cut.
The split percentage is config, versioned. Legs sum to zero; conservation holds.

"Real time," split honestly

Earnings panels are derived rollups off the ledger change stream: seconds stale, labeled asOf.
The withdrawable balance is the account row: exact, and read strongly only where it gates money.

Where the scale lives

Hundreds of purchase writes/sec, sharded by account. DevEx adds under 0.1/sec.
Sizing the two sides separately is the first judgment point of the whole question.

TRIGGER: "processor is down for 6 hours" / "their report disagrees with you"

F · Failures and reconciliation

4 SEPARATE SCENARIOS · each self-contained

Processor downtime

Trip a circuit breaker; leave rows PENDING; no attempt churn.
On recovery, drain oldest first, paced to the processor's limits.
The 5-day budget absorbs multi-hour outages. Dashboard: age of the oldest PENDING.

Rejected payout

A definitive no from the processor maps to FAILED. (Different word, different thing: REJECTED is the submit-time guard saying no.)

Our crash, any point

Before submission commits: nothing exists. After: escrow holds the Robux, PENDING survives.
Mid-payout: lease expiry plus the same idempotency key. No state is ever torn, because every transition is one transaction.

Daily reconciliation, both directions

Their report has a line we do not: repair from processor truth, page the on-call at SEV-2.
We say COMPLETED and their report is silent: page at SEV-1. Until explained, that is missing money.
Plus, continuously: the escrow invariant and no-negative-balances, checked off the ledger stream.

TRIGGER: "should this be event sourced" / "what do you watch"

G · Audit and operations

2 SEPARATE TOPICS

The event-sourcing question, answered directly

The ledger legs and the attempts table already ARE an append-only event log; every balance and status derives from them.
Full event sourcing (state only by replay) adds cost without adding auditability here.
Say it as: "event-sourced where it matters — the money; conventional rows elsewhere, with every change streamed to write-once archive storage."

Targets and alerts

Submission p99 under 1s: the eligibility call dominates it (own 2s timeout); the transaction itself is single-digit ms.
99.9% of payouts terminal within 5 business days. Over-withdrawals: zero, proven by the invariants.
Alerts: oldest-PENDING age, dead-letter arrivals, processor error rate, reconciliation mismatches (any nonzero pages), escrow drift, eligibility-service latency (the only external call on the submission path).

FLASHCARDS

The five hardest probes

show all (interview mode)

Two concurrent submissions, one balance. Where exactly does the double-spend die?

Inside the escrow debit: the guard "balance minus robux stays at or above zero" is part of the UPDATE itself, under the row lock. The second transaction waits, re-evaluates against the already-debited balance, matches zero rows, and is rejected. There is no separate check step to race.

The processor paid, then you crashed before recording it.

The row is still PROCESSING with an expiring lease. The next attempt presents the SAME idempotency key — the request id — so the processor replays its recorded success instead of paying again, and we record COMPLETED late. If even that replay is lost, the daily settlement reconciliation catches the report line with no matching row and repairs from processor truth.

Why can the rate never drift between submission and processing?

Because "which rate" is not a policy, it is data: rates are immutable versioned rows, and the submission transaction writes the rate id AND the computed USD onto the request. Processing reads the request row. There is no code path that could consult a different rate.

Walk the request state machine and what each transition writes.

PENDING to PROCESSING when a worker claims it. PROCESSING to COMPLETED, PARTIAL, or FAILED on a definitive processor answer. PENDING to CANCELLED on user cancel, guarded so it cannot race a claim. PROCESSING to ON_HOLD on an ambiguous answer (DLQ, human decides). Lease expiry loops PROCESSING back to PENDING. REJECTED is born terminal at submit. Every transition is an atomic compare-and-set on the status column, committed in the same transaction as its escrow settlement legs — burn, refund, or split — so no ending can skip the money.

Would you event-source this?

It already is, where it matters: the double-entry legs and the immutable attempts table are an append-only event log, and every balance derives from them. Rebuilding ALL state only by replay would add operational cost without adding auditability. Compliance gets every change streamed to write-once archive storage with retention — satisfying the audit ask without event-sourcing the write path.

NEXT ACTION, 3 MINUTES

Read the Step 4 walkthrough aloud once, timing yourself. If it runs past 90 seconds, cut words until it fits.