Why Every Backend Engineer Should Understand Idempotency

"I Only Clicked Pay Once..."

Imagine you’re ordering a new phone online. You complete checkout, enter your payment details, and click Pay Now. The button spins for a few seconds. No confirmation appears. Thinking the payment didn’t go through, you click again.

A few moments later, you receive two payment confirmations. Your bank account has been charged twice.

You immediately think: “The website has a bug.” And you are absolutely right. But here is the interesting part – the bug wasn’t caused because you clicked twice. It happened because the backend wasn’t designed for something that occurs every single day in distributed systems: the same request being received more than once.

This is precisely the problem that idempotency solves. Understanding it deeply is what separates backend engineers who build APIs that just work from engineers who build APIs that are reliable under real-world conditions.

Why Does the Same Request Arrive More Than Once?

The instinct is to blame the user. But user error is only one of several reasons a request might be processed multiple times – and it’s not even the most common one.

The double-payment timeline: the user did everything correctly. The network failed after success.

In the diagram above, the user clicked once. The payment succeeded. But the network dropped after the server processed the transaction – before the confirmation reached the client. From the client’s perspective, the request timed out. The safest course of action is to retry. The server, receiving an identical request, has no idea it already processed it.

This scenario is not unusual. Every one of these failure modes produces the same outcome – a valid duplicate request reaching your server:

  • Network Timeouts - response lost in transit after the server already processed the request
  • Client-side Retries - mobile apps, browser fetch APIs, and HTTP clients often retry automatically
  • Load Balancer Retries - some LBs retry failed requests to another backend instance
  • Message Queue Redelivery - Kafka, RabbitMQ, SQS can all redeliver messages after processing failures
  • Cloud SDK retries - AWS , GCP, and Azure SDKs have built-in retry logic with exponential backoff
  • Impatient users - yes, this too, but it's the least interesting case

Retries are not a bug. Retries are a feature – the correct response to transient failures in distributed systems. The challenge isn’t stopping retries. It’s making retries safe.

What Exactly Is Idempotency?

Idempotency (noun)

An operation is idempotent if performing it multiple times produces exactly the same result as performing it once. The second, third, and hundredth invocations are indistinguishable from the first in terms of outcome.

In mathematics, the classic example is multiplication by one: 5 × 1 × 1 × 1 = 5. The operation can be applied any number of times and the result is unchanged. In software, the concept is identical:

// ✗  Non-idempotent - each call changes state
POST /payments  →  deducts ₹5000 from account
POST /payments  →  deducts ₹5000 again
POST /payments  →  deducts ₹5000 again
// Three calls, three charges. Disaster.
 
// ✓  Idempotent - same request, same outcome
POST /payments  +  Idempotency-Key: abc-123  →  deducts ₹5000
POST /payments  +  Idempotency-Key: abc-123  →  returns cached result
POST /payments  +  Idempotency-Key: abc-123  →  returns cached result
// Three calls, one charge. Correct.

The customer sees one payment. The bank sees one transaction. The database stores one record. The operation was performed once – the server’s awareness of the duplicate requests is what prevents the logic from executing again.

The Secret Ingredient: Idempotency Keys

The mechanism that makes this work is elegantly simple. Before sending a state-modifying request, the client generates a unique identifier – the idempotency key – and includes it in every retry of that specific operation. The server uses this key to recognise when it has already processed a request.

The idempotency key decision flow: new key → process; existing key → return cached result.

The server's logic is straightforward:
  1.     Receive the request. Extract the idempotency key from the header (or body).
  2.     Check the key store. Look up the key in a fast store (Redis is ideal for its TTL support).
  3.     New key: Process the payment. Store the key alongside the response. Return the result.
  4.     Existing key: Do not execute the business logic. Return the previously stored response immediately.

The key must be generated by the client (not the server) and must remain stable across retries of the same logical operation. A UUID4 per checkout attempt is a common pattern. The same UUID accompanies every retry of that checkout – a different checkout gets a different UUID.

// Client generates the key once per logical operation
const idempotencyKey = crypto.randomUUID();  // e.g. 3fd8b95a-cc67-46f1-a682
 
// Include it in every retry of this specific payment
const response = await fetch('/api/payments', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Idempotency-Key': idempotencyKey,  // ← same key on every retry
  },
  body: JSON.stringify({ amount: 5000, currency: 'INR', ... }),
});

The Complete Flow: Safe Retry in Action

Here is how the same network-drop scenario from the introduction plays out when the API is implemented with idempotency keys. Same failure, completely different outcome.

Same network drop. Same retry. One charge. The idempotency key makes the difference.

Notice the key difference on the retry: the server checks the key store before touching any business logic. The database is never consulted on the retry path. The stored response is returned immediately. The customer’s bank account reflects exactly one deduction, the order system has exactly one transaction record, and the user’s experience is indistinguishable from a perfectly reliable network.

The Race Condition Trap: Why 'Check First' Isn't Enough

A common beginner approach to preventing duplicates is to simply check whether the operation has already been performed:

// ✗  Naive approach - vulnerable to race conditions
public PaymentResult processPayment(PaymentRequest req) {
	if (transactionRepo.existsByReference(req.getReference())) {
    	return transactionRepo.findByReference(req.getReference());
	}
	// ... proceed with payment
}

Two retries arriving simultaneously both pass the naive existence check – the race condition produces a double insert.

The problem is a classic TOCTOU (Time of Check to Time of Use) race condition. Two retries arriving within milliseconds of each other can both execute the SELECT before either has performed the INSERT. Both get ‘not found.’ Both proceed. Both insert. Double payment.

The correct solution combines an idempotency key with either a unique database constraint or an atomic compare-and-set operation:

-- Database: unique constraint on the key makes the race impossible
ALTER TABLE idempotency_keys
ADD CONSTRAINT unique_key UNIQUE (key_hash, user_id, endpoint);
 
-- Application: use INSERT ... ON CONFLICT to make it atomic
INSERT INTO idempotency_keys (key_hash, user_id, endpoint, response)
VALUES (:key, :userId, '/payments', :response)
ON CONFLICT (key_hash, user_id, endpoint)
DO UPDATE SET accessed_at = NOW()  -- idempotent update
RETURNING response;
 
-- If INSERT succeeds: new request → process payment
-- If CONFLICT: duplicate → return stored response

The unique constraint is your safety net. The application-level check is an optimisation to avoid unnecessary DB round trips on obvious retries.
Never rely on the application check alone. The database constraint is what actually prevents duplicates in concurrent scenarios.

HTTP Methods and Built-In Idempotency

Some HTTP methods are idempotent by definition in the RFC specification. Understanding which ones – and why – helps you design APIs that behave predictably:

HTTP Methods and Idempotency

which methods are safe to retry by default – and which need extra care?

Method Idempotent? Why
GET ✓ YES Read-only. No state changed. Safe to retry infinitely.
PUT ✓ YES Replaces the resource completely. Same payload → same state.
DELETE ✓ YES Deleting an already-deleted resource still leaves it deleted.
PATCH X NO* Depends on implementation. Append ops (add $10) are NOT idempotent.
POST X NO* Usually creates a new resource each call. Needs explicit idempotency keys.

* POST can be made idempotent by including an Idempotency-Key header — exactly how Stripe and PayPal implement their payment APIs.

The key nuance is the word ‘by design’ for GET, PUT, and DELETE. The HTTP specification says these methods should be idempotent – but a careless implementation can violate that contract. A DELETE endpoint that records an audit log on each call and sends a Slack notification on each invocation is technically non-idempotent in its side effects, even if the resource state ends up the same.

POST is particularly important: it creates a new resource on each call by default, making it non-idempotent unless you explicitly add an idempotency mechanism. Stripe, PayPal, Razorpay, and virtually every major payment gateway make idempotency keys a first-class feature of their POST endpoints precisely for this reason.

It's Not Just Payments - Where Idempotency Matters

Payments are the highest-stakes example, but idempotency is relevant in any scenario where executing the same operation twice produces incorrect or unwanted state:

Six domains where idempotency prevents real-world production bugs.

A useful mental model: any API that creates, modifies, or deletes real-world state – especially one that communicates with external systems, sends notifications, moves money, or consumes limited resources – deserves an idempotency strategy.

One nuance for notification APIs: sometimes tolerating duplicates is the right call. Receiving a duplicate ‘your package shipped’ email is annoying but harmless. Receiving a duplicate charge on your credit card is a customer service crisis. Match your idempotency investment to the actual cost of duplication.

Implementing Idempotency: A Practical Guide

Step 1 - Key Design

The idempotency key must uniquely identify one specific logical operation. Its scope determines what ‘same operation’ means:

// Scope: user + checkout session + endpoint
// Generated once when checkout begins; same key on all retries
String key = UUID.randomUUID().toString();
 
// Stripe-style: include in header
POST /v1/payment_intents
Idempotency-Key: 7f3a9c12-f8e1-4b2d-9001-abc123
 
// Alternative: include in body (when headers aren't controllable)
{ "idempotency_key": "7f3a9c12-f8e1-4b2d-9001-abc123", ... }

Step 2 - Key Storage

Store processed keys with a TTL that covers your maximum retry window. Redis is well-suited: fast lookups, native TTL support, and atomic SET NX operations.

// Spring Boot + Redis example
@Service
public class IdempotencyService {
 
	private final RedisTemplate<String, String> redis;
 
	public Optional<PaymentResponse> getStoredResult(String key) {
    	String stored = redis.opsForValue().get("idem:" + key);
    	return Optional.ofNullable(stored)
                   	.map(json -> deserialize(json, PaymentResponse.class));
	}
 
	public void storeResult(String key, PaymentResponse resp) {
    	String json = serialize(resp);
        redis.opsForValue().set("idem:" + key, json, Duration.ofHours(24));
	}
}

Step 3 - Scope the Key Correctly

The same UUID might be used by different users or against different endpoints. Always scope the lookup to prevent key collisions between users:

// Key lookup must be scoped to prevent cross-user collisions
String lookupKey = String.format("%s:%s:%s",
	userId,    	// prevents user A's key matching user B
	endpoint,  	// prevents /payments key matching /refunds
	idempotencyKey // the UUID from the client
);
 
// Full key: 'user:4821:/payments:3fd8b95a-cc67-46f1-a682'

Step 4 - Wire It into Your Controller 

@PostMapping("/payments")
public ResponseEntity<PaymentResponse> createPayment(
	@RequestHeader("Idempotency-Key") String idempKey,
	@RequestBody PaymentRequest req,
	@AuthenticationPrincipal User user) {
 
	String scopedKey = user.getId() + ":/payments:" + idempKey;
 
	// 1. Check cache first
	Optional<PaymentResponse> cached = idempotencyService.getStoredResult(scopedKey);
	if (cached.isPresent()) {
    	return ResponseEntity.ok(cached.get());  // return immediately
	}
 
	// 2. Process (only runs once per unique key)
	PaymentResponse result = paymentService.process(req);
 
	// 3. Cache for the retry window
	idempotencyService.storeResult(scopedKey, result);
 
	return ResponseEntity.status(201).body(result);
}

Three Misconceptions That Lead to Bugs

❌   The Misconception ✅   The Reality

“Just disable the button after the first click”

“We already check if the transaction exists”

“Retries are bad – we should prevent them”

UI disabling improves UX but doesn’t prevent network retries, load balancer retries, SDK retries, or message redelivery. The backend must be correct regardless of the client.

A naive existence check is vulnerable to the race condition shown earlier. Two concurrent requests can both pass the check before either writes. Only a unique constraint + atomic upsert is truly safe.

Retries are the correct response to transient failures. Eliminating retries trades correctness (eventual delivery) for brittleness (lost operations). The answer is safe retries, not no retries.

The Four Questions Every API Designer Should Ask

Before marking any state-modifying endpoint as complete, run through these four questions. If the answer to any of them makes you uncomfortable, your endpoint needs an idempotency strategy:  
  1.     What happens if the client retries this request? Can the business logic execute twice without harm?
  2.     What happens if two identical requests arrive concurrently? Is there a race condition in your existence check?
  3.     What does the user experience if the response is lost after processing? Do they have to retry? Is that retry safe?
  4.     What is the cost of processing this request twice? Financial cost? Data corruption? Duplicate notification? User frustration?

Rule of thumb:  If the operation creates, moves, or deletes something of value – money, inventory, a seat, a subscription – it needs idempotency. Not eventually. From the start.

Try It Yourself - CodeKerdos Challenge

Scenario: Your API creates a coupon code for a user when they complete a referral. The client’s network drops after the coupon is generated but before the response arrives. The client retries.

Question: How would you prevent two coupon codes being generated for the same referral?

Think through: key design, storage, scope, and the race condition. What’s your unique constraint? What’s your TTL? What do you return on the retry?

Hint: the answer involves three things working together – a client-generated key, a server-side key store, and a database constraint. Each one solves a different part of the problem.

Conclusion: Build APIs That Are Ready for Reality

One of the biggest mindset shifts in backend engineering is accepting that distributed systems are unreliable by nature. Networks drop. Connections time out. Mobile devices lose signal mid-request. Services restart. Load balancers retry. Message queues redeliver.

These are not edge cases to be handled eventually. They are the normal operating conditions of any system with more than one component.

Experienced backend engineers don’t build APIs assuming every request is unique. They build APIs assuming the same request might arrive multiple times – and they design accordingly. Idempotency is what makes those retries safe, Redis what makes those retry-heavy distributed systems correct, and what keeps angry customers from calling support about double charges.

The next time you build a state-modifying endpoint, don’t just ask: “Does this API work?”

Ask the more important question: “What happens if this exact request is sent again?”

If your system behaves correctly regardless of how many times that request is retried, you haven’t just written a working API. You’ve written a reliable one.

“Reliability isn’t about what your system does when everything goes right. It’s about what it does when everything goes wrong – and still delivers the correct outcome.

Scroll to Top