Skip to content

Restart a Backfill

This guide is for platform support. It walks through stopping a backfill that is doing harm in the target, and starting a new one once the cause is dealt with.

A backfill is usually stopped because the target reacted to the creates. A typical case is an automation in the target workspace that sends a welcome mail for every new contact, so the backfill sent one mail per imported customer. Starting again repeats that unless the cause has been dealt with first.

You need the delivery.backfill permission, which SuperAdmin and PlatformSupport hold, and access to the rule as for any other rule action. Without both, the two buttons below do not appear.

A consent rule never shows Start backfill. Its consents are written onto contacts rather than created, and they reach the target once they change (ADR-0046).

Step 1: Abort the running backfill

  1. Open the rule's page. A rule with a backfill running shows a Backfill in progress fact under Delivery, with how far it has come and who confirmed it, if somebody did.
  2. Choose Abort backfill in the page header and confirm.

The abort takes effect at once for every later batch. The batch that is running at that moment still finishes. If the target must stop receiving anything immediately, use Stop delivery instead; it halts the delta as well, and Resume delivery ends it.

After an abort, delta keeps running. The records the backfill had not reached are delivered once they change. The abort is written to the log with your user and the rule, and nowhere else.

If the backfill finished while the dialog was open, confirming does nothing and the dialog stays open. Close it. The page header now shows Start backfill instead of Abort backfill.

Step 2: Check what the target does on a new record

A backfill sends a create for every record the target holds no reference for, unless delivery has set that record aside. Nothing in the request can hold back what the target does with it, because that is configured in the target.

Open the target workspace, or ask whoever owns it, and look for everything that runs when a record is created:

  • automations or workflows triggered by a new contact, order or product
  • welcome or confirmation mails and other messages to people
  • imports or syncs that pass new records on to further systems
  • limits on how many records the workspace takes in a day

If any of these would fire for every record, switch it off or narrow it in the target before you go on. Where you cannot tell, ask the owner. The confirmation in step 3 says that you checked.

Step 3: Start the backfill

  1. On the rule's page, choose Start backfill in the header. The button shows only for an active push-out rule of a root domain that has no backfill running, so never for a consent rule. A stopped rule may start one, and it begins when the rule runs again.
  2. Read the dialog. It says how many records lie in the backfill's window, and how many creates the backfill would send for them. Both are counted when the dialog opens, and they do not change while it is open. On a rule that propagates deletions, the window includes the deletions the source reported. The backfill withdraws those the target holds and skips the rest.
  3. Tick I have checked what the target does when a record is created.
  4. Type the number of creates into the field and confirm.

Records that arrive while the dialog is open do not change the numbers. They go out by the regular delivery, as any new record does.

If the number you typed is not the one shown, the dialog stays open. Type the number it shows.

If the backfill would now send more creates than you confirmed, the dialog stays open and shows the new number. This is rare. It happens where a data restore brought records back, where a record the target refused counts again, or where somebody changed delivery records of the rule while the dialog was open. Type the new number to go on.

If somebody else started a backfill on the rule while the dialog was open, confirming does nothing and the dialog stays open. Close it. The page shows the running backfill, and the header shows Abort backfill instead of Start backfill.

What happens to records the first run delivered

Nothing is sent for them again. The new backfill writes an unchanged record it already delivered as suppressed, and sends it to the target only if it changed since. The creates counted in the dialog are the records the target has no reference for: those the first run never reached, and those whose create failed. A record whose create failed and whose retry is still ahead is created by the regular delivery when the retry is due, not by the backfill. It is counted all the same. A record whose ledger row was deleted, or whose delivery gave up (exhausted or remote_gone), is not created, so it is not counted. A record refused as undeliverable is counted. If its source data was corrected since, the backfill creates it, because the regular delivery does not offer it again. Delivery cannot tell whether it was corrected, so the number can be higher than what the backfill sends.

Check the result

  • The rule page shows Backfill in progress with "confirmed by" and your name.
  • The target connection's page shows Last backfill confirmation with the rule, the number of creates, your name and the time. It links to the rule, for everybody who may open the rule.
  • The Delivery page lists the backfill under "Backfills in progress" with its progress.

If the target reacts again, abort at once as in step 1 and look at what ran.

See also