APIs · Reliability

Idempotency keys for APIs that move money

· 2 min read

Connections fail in the middle. A client sends a payment request, the connection drops, and it cannot tell whether the charge happened. Retrying blindly risks charging twice. Not retrying risks losing the payment. Idempotency keys make the retry safe.

The contract

The client generates a unique key for each logical operation, usually a random UUID, and sends it in an Idempotency-Key header. The server then guarantees two things:

  • The same key with the same request produces the same result, and the work runs at most once.
  • The same key with a different request is rejected with a client error. We use 422.

Storage

One table is enough. Scope the key to an account so two customers can never collide.

CREATE TABLE idempotency_keys (
  account_id   bigint      NOT NULL,
  key          text        NOT NULL,
  request_hash text        NOT NULL,
  status       text        NOT NULL DEFAULT 'started',
  response     jsonb,
  created_at   timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (account_id, key)
);

The request flow

  1. Hash the method, path and body into request_hash.
  2. Try to claim the key. The row comes back only if this request inserted it.
  3. If it did, do the work, store the status and body of the response, and mark the row completed.
  4. If the row already existed, compare hashes. A mismatch is a 422. A completed row is replayed as it was stored. A row still marked started means another request is in flight, so answer 409 with a Retry-After header.
INSERT INTO idempotency_keys (account_id, key, request_hash)
VALUES ($1, $2, $3)
ON CONFLICT (account_id, key) DO NOTHING
RETURNING 1;

Details that bite

  • Store the response, not just a flag. A retry must return the same body, including generated ids, or the client sees two different outcomes.
  • Make the side effects and the completion mark atomic. If the process dies in between, the row stays started forever. Let rows older than a few minutes be claimed again.
  • Expire old keys. Twenty-four hours is a common window. Delete older rows on a schedule.
  • Hash only what defines the operation. Leave out headers that change on every attempt, such as timestamps and trace ids.
  • Pass the idea downstream. If you call an external payment provider, send it a key derived from yours so the same guarantee holds end to end.

← All notes