Design an Event Ticketing System (Ticketmaster)

Functional Requirements

1. Users should be able to find events by search and by browsing.
2. Users should be able to see which seats are available for an event.
3. Users should be able to hold specific seats and then complete a purchase.
4. The system should survive an on-sale spike for a hot event.

Below the line (out of scope):

• Dynamic pricing. It changes the number on the row, not the machinery that protects the row.
• The resale market and refunds. Both are new flows over the same seat and order model.
• Bot detection (fingerprinting, ML scoring, fraud review).
  • What stays in scope is the one layer the fairness argument actually rests on: a queue position bound to a verified account with payment history, one position per account per event, and a challenge at join.
    • Say that split out loud as a product decision, because "bots are the real adversary" cannot be both the reason for the queue and out of scope.

Non-Functional Requirements

1. A seat is sold at most once. Double-selling is the disaster case and the one thing that must never happen, because you cannot un-sell a seat to a person standing at the gate.
2. Holds must feel instant. The hold is the thing the user is racing for, so it is a single fast write, not a workflow.
3. The seat map may be slightly stale, but staleness must never cause a double sell. Stale reads cost a retry, never correctness.
4. The system must degrade under on-sale load rather than collapse. Serving coarser data or a longer wait is success; timing out and retry-storming is failure.
5. Payment must be at-most-once. Charging a card twice for one order is the second disaster case.

Defining the Core Entities

• Event is one performance: venue, performer, datetime, on_sale_time, and status. The on_sale_time is not decoration, it is what the waiting room schedules itself around.
• Venue owns the seat map: sections, rows, and seat labels. It is shared across many events and is pure catalog data, so it caches forever.
• Seat is one row per seat per event, and it is the center of the whole design. It carries status (AVAILABLE, HELD, PENDING_PAYMENT, SOLD), hold_id, hold_expires, order_id, and a price tier.
  • Every hard question in this interview is answered by an UPDATE against that one seat row.
• Order is user, event, seat ids, total, status, and the idempotency key from the request, under a unique index on (user_id, idempotency_key). It is created inside the same transaction that flips the seats.
  • order.status is the money truth: the seat row says who owns the seat, the order says whether it has been paid for, and the gate scanner trusts the order.
• PaymentAttempt is order, processor reference, and status. It exists so that an ambiguous processor answer has somewhere honest to live instead of being guessed at.

API or System Interface

GET    /v1/events?q=&city=&date=              // search; served from the index, cacheable
GET    /v1/events/{id}                        // the event page; behind the CDN
GET    /v1/events/{id}/seats                  // → { seats[], asOf } the map plus its own staleness
POST   /v1/events/{id}/holds  { seatIds[] }   // → 201 { holdId, expiresAt } from RETURNING
                                              // → 409 SEAT_UNAVAILABLE { lostSeatIds[] } = requested minus RETURNING
POST   /v1/orders  { holdId, paymentToken }   // header: Idempotency-Key
                                              // → 202 { orderId }, or 409 HOLD_EXPIRED
                                              // → replayed 202 if that key already wrote an order
GET    /v1/orders/{id}                        // poll for CONFIRMED / FAILED
DELETE /v1/holds/{holdId}                     // release early; guarded on status='HELD' AND hold_id
SSE    /v1/events/{id}/updates                // seat state deltas, fed by CDC (change data capture)
POST   /v1/events/{id}/waiting-room           // → { position, queueToken, eta }

• The order returns 202, not 201, because the charge happens AFTER the commit.
  • The transaction that sells the seat also writes an outbox row, and a worker charges the card; the client polls or gets pushed the result.
  • Explaining that 202 is explaining the whole payment design in one line.
• The seat map carries an as-of timestamp (asOf): the wall-clock instant the snapshot was read out of the seats table, not when it was cached and not when the client got it. Staleness is declared rather than hidden.
  • A client that knows its map is 800ms old can say so; a client that believes a stale map is live will blame you for the 409.
• The hold endpoint returns 409 SEAT_UNAVAILABLE rather than a 200 with a failure body, because losing a seat race is a normal, expected outcome at an on-sale, and the client retries against fresh state.
• Be exact about who does what for a double-click: the seat CAS is what makes it harmless, the Idempotency-Key is what makes the RESPONSE correct.
  • Store the key under a unique index on (user_id, idempotency_key); on conflict, look up the order already written for that key and replay the same 202 with its orderId.
  • Return 409 HOLD_EXPIRED only when the seat guard failed AND no order exists for that key.
  • Skip that and the second click of a successful purchase gets told its hold expired, which is the API lying to a buyer who just bought.
• The hold response uses RETURNING on the acquire, so expiresAt is the database's value rather than the app server's arithmetic on the app server's clock.
  • RETURNING also lets a partial multi-seat failure name WHICH seats were lost (requested ids minus the ids RETURNING gave back) instead of making the client refetch the whole map.
  • Dedupe seatIds server-side before the statement, or a duplicate id makes the rows-affected check fail spuriously.

1) Users should be able to find events

• Search runs over Elasticsearch holding denormalized event documents: name, performer, venue, city, date, price range, all in one doc so a query is one hop.
• Event pages sit behind a CDN. The page for a hot event is requested millions of times and changes rarely, which is the definition of cacheable.
• Postgres holds the catalog as truth, and CDC streams changes into the index. The index is derived and rebuildable, so nothing catastrophic happens if it is seconds behind or has to be rebuilt from scratch.
• The point of this section is not that search is hard. It is that browsing is cacheable and boring, so it must be cleanly separated from the booking path.
  • Every read served from the CDN or the index is a read that never touches the primary you are about to fight over.

2) Users should be able to see which seats are available

• Start by admitting the constraint: you cannot serve a perfectly live seat map to 100K concurrent viewers.
  • At 100K QPS against the booking primary, at exactly the moment it is committing sales, the read path would take down the write path.
• So serve a cached snapshot with an explicit asOf, and push deltas over SSE fed by CDC on the seat table.
  • One read of the truth fans out to everyone; the deltas keep the picture moving.
• State the contract honestly: the map is a hint, the CAS at booking time is the referee.
  • A stale map costs the user one retry and can never cost correctness, because nothing is decided by what the map said.
• Let the fan-out number pick the granularity. 330 deltas per second times 100K subscribers is about 33M messages per second, so per-seat streaming to everyone is not a thing you build and then degrade away from. Four consequences:
  • Section-level counts ("Section 112: 43 seats left") are the DEFAULT.
  • Per-seat detail is streamed only for the section or viewport a client is actually looking at.
  • Deltas are coalesced on a fixed tick of about one second.
  • Only AVAILABLE to SOLD transitions are published, never HELD churn, which is most of the volume and also hands scrapers a live inventory feed.
• Give the stream a watermark, or a client cannot tell it missed anything.
  • The snapshot carries the CDC LSN (log sequence number, Postgres's position in its write-ahead log) it was built at, and deltas carry monotonic sequence numbers.
  • A client discards anything at or below its watermark and resnapshots only on a gap.
  • Reconnects use jittered backoff against a single-flight snapshot endpoint, because "everyone re-GETs the snapshot on reconnect" is a 1M-client stampede at the worst possible moment.

3) Users should be able to hold seats and then buy them

• This is the heart of the design.
• There are three transitions on the happy path, and all three are guarded UPDATEs. Compensation adds a fourth, guarded the same way (dive 4).
• Say the isolation level out loud before the first statement: READ COMMITTED, because "rows affected is the verdict" is only true there.

Acquire the hold. This statement also reclaims expired holds, so no reaper sits on the WRITE path:

UPDATE seats SET status='HELD', hold_id=:new_hold, order_id=NULL,
       hold_expires = now() + interval '10 minutes'
 WHERE event_id=:e AND seat_id=:s
   AND (status='AVAILABLE'
        OR (status='HELD' AND hold_expires < clock_timestamp()))
RETURNING seat_id, hold_expires;
-- rows affected = 1 means the seat is yours; 0 means 409 SEAT_UNAVAILABLE

Confirm the sale. hold_id here is a fencing token, and the SET list has to clear it:

UPDATE seats SET status='SOLD', order_id=:o,
       hold_id=NULL, hold_expires=NULL
 WHERE event_id=:e AND seat_id=:s
   AND status='HELD'
   AND hold_id=:my_hold          -- THE FENCE
   AND hold_expires > clock_timestamp() - interval '5 seconds';

• Sketch this as SOLD.
  • Once payments are on the table it writes PENDING_PAYMENT instead, and the payment worker flips it to SOLD on a definitive success.
    • Why: SOLD otherwise means three different things at once.

Release on cancel. Write it out; this is where the bug lives if you only gesture at "the same shape":

UPDATE seats SET status='AVAILABLE', hold_id=NULL,
       hold_expires=NULL, order_id=NULL
 WHERE event_id=:e AND seat_id=:s
   AND status='HELD'              -- pins the STATE
   AND hold_id=:h;                -- pins the IDENTITY

• The confirm IS one transaction, and it bundles three things in this order:
  • the guarded seat flip FIRST (rows affected must equal the number of seats or ROLLBACK)
  • then the order insert
  • then an outbox row for charge-and-deliver
• Putting the contended seat rows first is the right inter-table lock order: you take the hot lock, decide, and either bail immediately or hold it for the short remainder.
• Multi-seat, and the fix that actually works. All the seats go in one transaction with a rows-affected check equal to the seat count.
  • Sorting your seat id array does nothing: lock order inside a multi-row UPDATE is the plan's row order, not your array order.
    • So two customers can both send ascending lists and still deadlock when one gets an Index Scan and the other a Bitmap Heap Scan visiting the same rows in the opposite direction.
• Take the locks explicitly first, then run the guarded UPDATE:

SELECT seat_id FROM seats
 WHERE event_id=:e AND seat_id = ANY(:ids)
 ORDER BY seat_id FOR UPDATE;   -- LockRows above the scan, order guaranteed
-- then the guarded UPDATE over the same ids

• The alternative is one UPDATE per seat issued in sorted order, which gives the same guarantee at the cost of a round trip per seat.
• Either way, the confirm is also a multi-row UPDATE over the same rows and needs the same discipline, so it takes the same ordered lock step.
• Have a deadlock policy, since you raised deadlocks. On SQLSTATE 40P01 the whole transaction is aborted by Postgres.
  • So retry it from the top, at most twice, with a few milliseconds of jitter; past that, return 409 and let the client pick again.
  • A deadlock is never a partial write, so retry is always safe here.

4) The system should survive the on-sale spike

• Put a virtual waiting room in front of the event.
  • Users get a queue position and a queue token before the sale opens, so the crowd is measured and ordered before it is served. The admission token comes later, when the line reaches them (A4).
• The admitter is a concurrency limiter, not a rate limiter. Admit to hold N sessions in flight inside the protected zone, meaning everything behind the queue: the booking path, its connection pool, and the primary it writes to. It is a bulkhead.
  • N comes from connection pool size and measured p99 by Little's law, seeded from a pre-sale capacity test.
  • Commit rate and 409 rate are secondary signals that push N up or down.
  • Pacing on commit rate alone cannot even start: at t=0 the commit rate is zero because nobody has been admitted yet.
• Why plain rate limiting is not enough: it sheds load at random, so it destroys fairness, gives every rejected user a reason to retry immediately, and hands the win to whoever retries fastest, which is a bot.
  • A queue makes the wait explicit, ordered, and honest.
• The queue is also the backpressure. Everything behind it is protected from the herd, so the connection pool, the seat map cache, and the booking primary all see a load you chose rather than a load that arrived.
• Cap what an admitted session can take. Nothing above stops one admitted session from holding the whole stadium.
  • A bot farm admitted in perfect order can hold 50K seats in seconds while every CAS returns exactly one row and the map correctly shows sold out. Three caps:
    • a per-identity hold cap at acquire time, about 8 tickets
    • a shorter TTL during the on-sale, about 2 minutes, back to 10 afterwards
    • best-available seats handed out instead of free picking for hot on-sales

• If they just nod and wait: "the riskiest part is the hold-then-confirm boundary, shall I open how a seat is never sold twice?"
• Running behind: protect dives 1 and 2. Dive 5 is the goalpost-move answer, do not open it unprompted.

1) How is a seat never sold twice?

• The race is two buyers reaching for the same AVAILABLE seat in the same millisecond.
• Plus a second, sneakier race: your hold expiring while somebody else takes the seat, and you confirming anyway.

2) What happens at the on-sale moment?

• The load is not just large, it is concentrated: one event, one shard, one set of 50K rows, one minute.

3) How fresh is the seat map, and what does staleness cost?

• 100K viewers want a picture of state that changes about 330 times a second.
  • Perfect freshness is not merely expensive here, it is the thing that would break the write path.

4) What if payment fails or the processor times out ambiguously?

• The charge is keyed by orderId, presented as the processor's idempotency key on every attempt, so a retry across crashes, restarts, and sweeps can never become a second charge.
• Never guess on a timeout. A timeout after sending means the charge may or may not have happened.
  • There are exactly two honest moves: retry with the same key, or query the processor's status endpoint for that key.
  • Only a definitive answer moves the order to CONFIRMED or FAILED.
• The outbox sweeper finishes the job after a crash. The transaction that sold the seat also wrote the outbox row, so a worker dying between commit and charge leaves work that is picked up, not work that is lost.
• Give the seat a PENDING_PAYMENT state, because SOLD currently means three different things: paid, charge in flight, and charge ambiguous.
  • The seat flips to PENDING_PAYMENT at commit and to SOLD on a definitive success.
  • order.status is the money truth and the gate scanner trusts it, not the seat row.
  • Bound how long a seat may sit unpaid during ambiguity (minutes, not hours), then force a resolution rather than leaving it parked.
• On a definitive failure, compensate, and use a different guard. Reusing the hold_id guard matches zero rows, because the seat is no longer HELD and its hold lapsed long ago. Compensation is keyed by the order:

UPDATE seats SET status='AVAILABLE', order_id=NULL,
       hold_id=NULL, hold_expires=NULL
 WHERE event_id=:e AND order_id=:o AND status IN ('SOLD','PENDING_PAYMENT');
-- then mark the order FAILED and notify

• The good news worth saying: that guard is safe against a concurrent re-hold and is idempotent.
  • No acquire predicate matches a SOLD or PENDING_PAYMENT row, so nobody can be mid-acquire on it, and a second run of the compensation matches zero rows and does nothing.
• Clearing order_id matters too, or a FAILED order stays stamped on a seat the next buyer is about to take.
• Say the uncomfortable part explicitly: the seat is already committed before the money exists.
  • That is the price of not holding a lock across the processor call, and it makes compensation a designed path rather than an assumption.
• The seat map handles a sold-then-released seat the way it handles everything else, as a delta.
• Name the genuinely nasty case rather than hiding it: the processor settles the original charge after you declared failure, released the seat, and resold it.
  • The second sale is real and cannot be un-resold, so this resolves as money, not as inventory: auto-refund the first charge, detect it from the nightly settlement reconciliation, and issue a make-good.
  • Bounding the ambiguity window is what keeps this rare.

5) Scale, and what melts first

• Shard by event_id. All the seats for one event live together, so the guarded CAS and the multi-seat transaction stay single-shard, and a hot event's contention is contained inside one shard instead of smeared across all of them.
• Browsing is served from the CDN and Elasticsearch, never from the booking primary. Catalog reads go to read replicas. The hot event's seat map is served from cache.
• What melts first is not the database. It is the seat map read path, and specifically its fan-out.
  • 100K QPS of map requests arrives before the sale even opens, while the write rate is still zero.
  • Once seats start moving, 330 deltas per second to 100K subscribers is 33M messages per second if you stream per-seat to everyone.
    • That number is why section counts are the default and per-seat detail is scoped to the section in view.
• The second thing to melt is the connection pool, because every admitted user holds a connection for the length of their request and a spike turns queueing into timeouts, which turn into retries.
  • That is exactly what the waiting room protects, and it is worth saying in those terms: the queue exists because the two things that break first are both a function of concurrent users, not of seats sold.
• Sizing sanity check: 50K seats sold over a few minutes is only a few hundred commits per second on one shard, which a single primary handles comfortably.
  • The design is not fighting write volume, it is fighting arrival concurrency.
• State the contention story precisely, because the averages hide it. Write contention is spread across 50K rows at a few hundred commits per second, which is nothing.
  • But popularity inside one event is heavily skewed, so front-section rows take thousands of failed CASes and near-100% 409 rates while the upper deck sees essentially none.
  • The hot key is a few hundred seats inside a hot event, not the event as a whole, and that is what the queue and best-available assignment are really smoothing.
• General admission is the real single hot key, and it is the one case where "a single hot key" is literally true.
  • GA has no seat rows, just a quantity, so the CAS collapses to one counter row that every buyer serializes on.
  • Fix it by bucketing: split the inventory into K counter rows, hash each buyer to a bucket and fall back to scanning the others when theirs is empty.
    • Reconcile the last few units under a single-bucket fallback, where contention is trivially low because almost nothing is left.
• Synchronous commit to at least one standby on the seats shards, RPO=0. The whole correctness story is one Postgres row.
  • With async replication a failover promotes a standby that is missing committed sales: seats resurrect as AVAILABLE and get sold a second time, which is the one NFR that must never break.
  • The extra commit latency is affordable precisely because this design was built not to care about write throughput.
• Instrument the invariant, not just the boxes. NFR one is "a seat is sold at most once", so:
  • run a continuous query that alarms on any seat with more than one confirmed order
  • reconcile nightly against the processor's settlement file in BOTH directions (charges with no order, orders with no charge)
  • page on outbox depth and oldest-unclaimed-age as the payment SLO
  • track 409 rate and hold-to-purchase conversion as the fairness signals
    • A conversion rate falling toward zero while holds climb is a hold storm in progress.

Final Design

• Browsing is cached and derived: CDN in front of event pages, Elasticsearch for search, catalog truth in Postgres with CDC feeding the index.
• The seat map is a cached snapshot carrying an asOf and the CDC LSN it was built at, kept moving by sequence-numbered SSE deltas, section counts by default and per-seat detail only for the section in view.
• The waiting room holds ordered positions and admits by concurrency, holding N sessions in flight in the protected zone, which makes it backpressure rather than a UI feature. Per-identity hold caps and a short on-sale TTL stop an admitted bot farm from holding the stadium.
• The booking path is two transactions on purpose. First a guarded acquire that also reclaims expired holds on the write path.
• Then a guarded confirm fenced by hold_id that flips the seats, clears hold_id, inserts the order, and writes the outbox row in one commit, seats first.
• A read-path sweeper expires abandoned holds so the map gets a delta, since an expiry by itself writes nothing. The write path survives a dead sweeper; the read path does not.
• The charge happens after the commit, keyed by orderId, with the seat in PENDING_PAYMENT until the answer is definitive, a sweeper for crashes, and a compensation path guarded on order_id.
• Seats are sharded by event_id, so a multi-seat purchase stays one transaction on one shard, and the seats shards commit synchronously to a standby so a failover cannot resurrect sold seats.
• The invariant is monitored, not assumed: a continuous query alarms on any seat with more than one confirmed order, and settlement reconciles both ways nightly.

What is Expected at Each Level

• Mid-level candidates are expected to produce the seat, hold, and order model, and to know that the write must be guarded rather than decided in application code after a read.
• Senior candidates are expected to:
  • nail the hold_id fencing token AND clear it on the confirm
  • guard every transition on state as well as identity
  • name the isolation level that makes rows-affected a verdict
  • state the transaction boundary explicitly (with pooled connections it is not even expressible, and it would hold back vacuum if it were)
  • use an outbox so the charge happens after the commit, at-most-once in effect (the outbox delivers at least once, the idempotency key at the processor collapses the retries)
• Staff candidates are expected to:
  • open with the concentration math that reframes the problem from throughput to skewed contention inside one hot event
  • compute the SSE fan-out and let that number pick the seat map design rather than noting it as an aside
  • treat the waiting room as a concurrency limiter rather than a UI feature, and see the hold storm it does not prevent
  • monitor the at-most-once invariant instead of asserting it
  • name honestly what the design does not solve: the processor that settles a charge after the seat was resold, and bots as the actual adversary behind every fairness mechanism

A1) The mechanism

• The Admitter does not walk the queue. It moves one number.
  • admitted_through is a single integer per event, and everyone whose position is at or below it may enter.
  • Admitting 5,000 fans is therefore one INCRBY, not 5,000 messages.
  • Clients hold an SSE stream or poll, and ask one question: has the line passed my position yet.
  • The alternative is popping a million entries and notifying each one, which rebuilds the thundering herd inside the thing that exists to prevent it.
• Step 3 draws the crowd shuffling forward, which is the right mental model to say out loud.
  • The implementation inverts it: the fans hold still and the line sweeps past them.
• The tick, every 500ms. It has to be one round trip, for a reason in A5.

local in_flight = redis.call('ZCOUNT', inflight, now, '+inf')  -- the lease set, see A3
local slots = N - in_flight                               -- how much room the zone has
if slots <= 0 then return 0 end

local adv   = math.floor(slots / claim_rate)              -- see A2
return redis.call('INCRBY', admitted_through, adv)      -- the whole admission

A2) Why the division by claim_rate

• A large share of queue positions are dead: closed tabs, people who gave up, duplicate joins.
  • Advance by exactly slots and a 40% no-show rate runs the protected zone at 60% of N, so the sale takes almost twice as long for no reason.
• So measure it rather than guessing it.
  • Of the positions the line passed in the last 30 seconds, what share actually exchanged a token for a session inside the grace window (defined two bullets down).
  • That is claim_rate, and dividing by it is a feedback loop that self-corrects when the crowd's behaviour changes mid-sale.
  • Too high starves the zone, too low overfills it, which is the same failure as getting N wrong.
• The grace window is the other half.
  • Once the line passes you, you have about 60 seconds to actually show up and trade the position for a session.
  • Miss it and the admission is void and you rejoin at the back, because without expiry a ghost holds capacity forever.

A3) How a slot ever frees, which is the part usually gotten wrong

• A session ends three ways, and only one of them can tell you about itself.
  • Clean exit: checkout finishes or the user leaves, so an explicit release is possible.
  • Abandoned tab: nothing is sent, ever.
  • Your own server dies while holding the count.
• So in_flight cannot be a counter you increment and decrement. Each admitted session is a lease.
  • One sorted set per event, scored by when that session's lease expires:
    • admit is ZADD inflight <now+120s> <sessionId>
    • heartbeat is the same ZADD with a fresh score, sent by the live tab
    • in_flight is ZCOUNT inflight <now> +inf, one cheap read
    • the sweep is ZREMRANGEBYSCORE inflight -inf <now>
  • A dead tab stops heartbeating, its lease expires, and the slot returns with nobody telling you.
  • ZADD is an upsert, which is the only reason the heartbeat can be the same command.
    • A sorted set holds unique members, so an existing member cannot be added twice.
    • Member already there: its score is overwritten in place.
    • Member not there: it is added.
    • The reply counts NEW members only, so a successful heartbeat replies 0.
• Start with three sessions, member on the left and expiry score on the right:

before            sess_A  120
                  sess_B  125
                  sess_C  130

run               ZADD inflight 180 sess_A     sess_A is already a member
reply             0                            = zero NEW members added

after             sess_B  125
                  sess_C  130
                  sess_A  180                  score 120 -> 180, so it re-sorted to the end
                                               still 3 members, no duplicate


before            sess_A  120
                  sess_B  125
                  sess_C  130

run               ZADD inflight 200 sess_D     sess_D is not a member yet
reply             1                            = one NEW member added

after             sess_A  120
                  sess_B  125
                  sess_C  130
                  sess_D  200                  appended, now 4 members

• Say out loud that this is the seat hold one level up.
  • Same expiry-and-reclaim shape as hold_expires on the seat row, applied to admission instead of inventory.
  • Two instances of one pattern reads far better than two unrelated tricks.

A4) Who calls what, and where the token comes from

• Redis is never reachable from the browser. Every command in A3 is server side.
• BookingAPI writes. It makes both ZADD calls.
  • On admission: the browser POSTs /enter with the token, BookingAPI verifies it, then writes the lease.
  • On heartbeat: the browser POSTs /heartbeat every 30s, BookingAPI writes the same lease with a later score.
  • The browser's whole job is to keep pinging. It never speaks to Redis.
• The Admitter reads and sweeps. It never writes a lease.
  • ZCOUNT on each 500ms tick, which is where in_flight comes from.
  • ZREMRANGEBYSCORE on the same tick, housekeeping only.
  • The one thing it writes is admitted_through.
• There are two different tokens, and they are easy to conflate.
  • Queue token: handed out at join time, before the sale. It proves "I hold position 412,908".
  • Admission token: handed out when the line reaches that position. It proves "I may enter now".
• How a fan actually gets the admission token:
  • joins before the sale, and WaitingRoom returns a position plus a queue token
  • the browser holds an SSE stream asking whether the line has reached that position
  • the Admitter advances admitted_through past it
  • WaitingRoom signs an admission token and sends it down that stream
  • the browser POSTs /enter with it, and BookingAPI writes the first ZADD
• The admission token carries event id, position, session id, and an expiry of about 60s, which IS the grace window from A2.
  • It is bound to one account, so it cannot be resold or handed around a bot farm.
  • HMAC signed by WaitingRoom and verified at the edge against a shared secret, so there is no lookup per request.

A5) Two admitters must not both see the same free capacity

• If two replicas each read in_flight = 4,200 against N = 5,000, they both admit 800 and the zone overshoots by 800 sessions.
• Do the read and the advance in one Lua script so the count and the INCRBY cannot interleave.
  • That beats electing a leader per event: no election to run, and an admitter dying mid-tick costs one tick instead of a failover.
• Same instinct as the signed token in A4: keep the per-event hot key from coming back, whether the pressure is one tick or one request.

A6) The failure nobody notices, and what the fan sees

• If the tick loop stops, nothing errors.
  • Postgres is healthy, the pool is empty, every dashboard is green, and the sale is dead because nobody is being let in.
  • So alarm on the invariant: positions admitted in the last 30 seconds is zero while queue depth is above zero. Zero is the alarm, not the quiet.
• If Redis dies, admission falls back to a fixed conservative N with no position display, because a missing wait estimate is survivable and an unpaced crowd is not.
• The ETA is arithmetic, so give the formula rather than a shrug: (my_position - admitted_through) / admission rate, where the rate is an EWMA over the last 30 seconds.
  • Label it approximate, and never let it jump upward on screen if you can avoid it, because an ETA that grows is a support ticket.
  • The position counter is itself the hot key from dive 2, so blocks of positions are drawn per app server and exact ordering is not promised.