Idempotent job processing engine
A background job runner built on the assumption that every message can arrive twice and every worker can die mid-transaction. The interesting part is not the queue — it is the proof that the effect happened once.
- DELIVERY
- At-least-once in, exactly-once effect
- CLAIM
- PostgreSQL, SKIP LOCKED
- RETRY
- Bounded backoff with jitter
- TESTS
- Testcontainers, real database
- DECISION RECORDS
- 9 ADRs with rejected options
PRODUCER
DEDUPE KEY
unique index
unique index
OUTBOX
WORKER ×N
EFFECT
once
once
↳ exhausted retries → DEAD LETTER↳ duplicate key → NO-OP, ack
THE HARD PART
The dedupe key has to be derived from the caller's intent, not from the payload bytes. Retried payloads differ; intent does not.
WHAT I REJECTED
Redis-backed locks. Fast, but the lock and the write live in different failure domains — correctness cannot straddle two systems.
WHAT I'D FIX AT SCALE
The outbox relay holds the row lock across the HTTP dispatch. The scale move is claim-then-dispatch with a lease, dispatching outside the transaction.