Retry from the failed node n8n, prevent duplicates

When you retry from the failed node n8n, the node will re-run and can repeat external side effects. The fastest way to stop duplicates is to make each external action idempotent or gate it with a dedupe store: use an idempotency key header for APIs, insert an idempotency row in a database first, or ack webhooks immediately and process asynchronously.

Why this happens

n8n retries a node by re-executing the node logic and any connected nodes downstream. If that node does an HTTP call, sends an email, or triggers a payment, those external systems see a second request unless you put a guard in place. This problem shows up when a network glitch, API error or transient failure causes you to use the UI button Retry from the failed node n8n or the retry logic in the execution history. Forums recently discuss orchestration of external agents and how retries cause duplicates, see the community thread on Retry from the failed node and reliable orchestration patterns in agent orchestration.

Step by step fix checklist

Identify side effect nodes, for example nodes that use the HTTP Request node, SMTP nodes, payment nodes, or webhooks.
Create a stable idempotency key per business operation. Use a deterministic hash of natural keys, for example order id or agent task id. Compute it in a Code node and pass it downstream.
Persist or check the idempotency key in an authoritative store before making the external call. Use a database with a unique constraint or a key/value store. If the key exists, skip the external call.
Use API idempotency headers where supported. Many APIs accept an Idempotency-Key header (Stripe, some payment gateways). Set the header on the HTTP Request node.
Acknowledge webhooks early with the Respond to Webhook node so the external sender does not retry, and process the task from an internal queue or outbox.
Log outcomes with a single source of truth row (outbox table). Mark sent=true only after you confirm success from the external system.
Test by forcing a failure and using the UI Retry from the failed node n8n, then confirm no duplicate external effects occurred.

n8n recipes you can paste into your workflow

Below are concrete, copyable patterns. Replace fields and credentials where needed.

1. Generate an idempotency key in a Code node

// Code node (JavaScript)
const crypto = require('crypto');
const payload = $json.order_id ? { order_id: $json.order_id } : $json;
const idempotencyKey = crypto.createHash('sha256').update(JSON.stringify(payload)).digest('hex');
return [{ json: { idempotencyKey, payload } }];

2. Outbox pattern using Postgres

Create a table with a unique constraint on the key. Run a SQL insert before the external call and only proceed when a new row was created.

-- SQL to run once in Postgres
CREATE TABLE outbox (
  id SERIAL PRIMARY KEY,
  idempotency_key text UNIQUE,
  payload jsonb,
  sent boolean DEFAULT false,
  created_at timestamptz DEFAULT now()
);

-- Insert attempt from n8n Postgres node (Query operation)
INSERT INTO outbox (idempotency_key, payload) 
VALUES ('{{$json["idempotencyKey"]}}', $${{JSON.stringify($json.payload)}}$) 
ON CONFLICT (idempotency_key) DO NOTHING 
RETURNING id, sent;

Check the returned rows from the Postgres node. If no row was returned, skip the HTTP Request or email node. If a row was created, run the external action and on success update sent to true.

3. Use Idempotency-Key header on HTTP requests

In the HTTP Request node add an HTTP header named Idempotency-Key with the value {{$json["idempotencyKey"]}}. Many payment or order APIs will treat repeated requests with the same key as a single operation.

4. Respond to Webhook quickly, then process

Put a Respond to Webhook node immediately after the trigger. Send a 202 Accepted and continue processing downstream. This prevents the sender from retrying and lets you apply outbox/dedupe logic safely.

Patterns for external dedupe stores and idempotent actions

Choose a store based on latency, cost and operations skills. Common choices:

Redis: low latency, supports SET with NX and expiry for fast dedupe. If you run Redis, use a small microservice or the Redis node if available to run SET key value NX EX 300 and only proceed when it returns OK.
Postgres or MySQL: easiest with n8n built in nodes. Use unique constraints and ON CONFLICT DO NOTHING inserts as shown above.
Cloud KV: DynamoDB with a conditional put, or a managed Redis. Use the database SDK in a small service or a Code node if you have credentials available.

When you cannot change the external API, make the client side idempotent by storing the operation as pending in your outbox, then having a separate delivery worker mark it complete. This avoids repeated API calls when you Retry from the failed node n8n.

How to stop it happening again

Standardise on computing idempotency keys from business IDs and store them centrally.
Instrument your workflow to log the idempotency key with execution id and timestamp. That makes incident triage simple.
Prefer Respond to Webhook early, then use outbox delivery. This reduces the chance the external system will retry the sender.
Use the HTTP Request node headers to let the provider dedupe where supported. For payment gateways check provider docs on Idempotency.
Automate tests that simulate a failed node and then use Retry from the failed node n8n to verify no duplicates are created.

When to get help

If you have a distributed agent system, competing workers, or a third party that has no idempotency support, the choreography gets harder. Read the forum thread about reliable orchestration and heartbeats for external agents for patterns that involve claims and late callbacks at reliable orchestration. If you are stuck with a particular API or node error, include the node name, the exact error text, and sample payload when you ask for help. Our post on the n8n polling trigger watchdog may also help if your triggers stop: N8n polling trigger stops.

If you want hands on examples and guided practice building these patterns, learn to build automations like this with Automate Naija.

Frequently asked questions

Will clicking Retry from the failed node n8n always duplicate my external side effect?

Not always, but usually yes unless you have a guard. Retry re-executes the node logic and any HTTP calls or payments will run again. To avoid duplicates, make external calls idempotent, persist an idempotency key before the call, or use an outbox pattern.

How do I create an idempotency key in n8n?

Compute a deterministic key using a Code node from natural identifiers like order id or task id. A SHA256 of the business payload works. Pass that key to your Postgres insert or to your HTTP Request node as an Idempotency-Key header.

Can I stop duplicates without an external database?

You can for some APIs by using provider idempotency headers, or by storing a short lived dedupe token in a fast cache like Redis. If neither is available, respond to webhooks early and queue processing so your internal outbox controls delivery and dedupe.

What is the fastest pattern to implement now?

If you already have Postgres or MySQL, add an outbox table with a unique idempotency_key and insert first. Only perform the external action when the insert returns a new row. This uses the Postgres node and raw SQL in n8n and is quick to test and reliable.

Build it with us. The community is free and the build track teaches this kind of automation week by week: see pricing or join the community.