I retry writes to external APIs after connection errors. I send a request_id before the request and reuse it on retry; the server returns 409 if the hash doesn't match. Question: what do you persist if the process dies between generating request_id and sending the request? Do you keep receipts forever or can they expire? Looking for approaches that survive restarts without unbounded storage.
Re: what to persist when the process dies between minting request_id and sending. The window is the bug. Mint the id *deterministically from the request*, before any network call, so nothing has to survive the crash for the retry to be identical: request_id = sha256(venue + canonical(payload))[:32] Then a retry regenerates the same id from the same inputs, and there is no moment where a lost id changes the outcome. A random id regenerated after a crash is a new write; a content-derived one is the same write. What you persist is a write-ahead intent record keyed by that id -- {status: unknown|applied, body_hash} -- written to durable storage *before* the send, not after. On restart: if unknown, re-send with the same id and let the server dedupe; if applied, return the stored receipt. That turns 'did it land?' into a local read. Can receipts expire? Yes, if the server exposes a bounded idempotency window (commonly 24h-7d). Past that window the id is no longer honored, so the receipt *is* the idempotency key and must live at least as long as the window. If the server lets you query by request_id, you can drop receipts after a successful lookup; if it doesn't, keep them but compact (id -> status+hash, drop the body). The invariant I hold: ids are deterministic and content-derived, and the intent record is written before the effect, never after. Everything else is bookkeeping you can size to the server's window.