Skip to content

0037. Deliver the existing dataset on a rule's first activation

Date: 2026-09-28 Status: Partially superseded by 0042 — a dependent rule opens its cursor without a backfill; the rest remains in effect

Context

A rule's delivery cursor used to be created on the rule's first delivery run and positioned at that moment, in delta mode. The scan then read only records whose content changed after it. The reason was safety: SyncRuleBlueprintService creates default rules as active on every connection activation, and a pass over an existing dataset would have been an unthrottled bulk send into somebody else's live system.

It meant that connecting a shop to an existing User workspace delivered part of the shop's dataset, depending on whether the initial pull-in or the first delivery run came first, and connecting a new workspace to an existing shop delivered none of it. Inactive customers, the core audience for win-back campaigns, never arrived. CFX-200 promises a full historical backfill on connection.

The guard rails that made the old default necessary now exist: a per-target throttle, a kill switch at three levels and a volume breaker (ADR-0035, ADR-0036).

Decision

A push-out rule's first activation opens its cursor with two positions: delta from the moment of activation (T0), and a backfill over everything the rule already carries, bounded by T0.

  • First activation is the cursor's existence. A saved listener on SyncRule opens the cursor when a push-out rule is created active or switched on and has none. A reactivation, a restore after a disconnect, and every other save find the cursor and change nothing.
  • One row, two positions. The cursor holds the delta position, backfill_until (T0) and the backfill position. A set backfill_until means a backfill is in progress; the stored mode is gone.
  • The backfill walks a set that no longer grows. It reads records whose last change lies before T0, strictly after its own position, with no overlap and no ledger join. Everything written from T0 on carries a later change signal and belongs to delta, so no record is read by both passes except in delta's overlap just before T0, where the backfill records it as suppressed.
  • Delta is never held up. A chain runs the drain and delta first, then one backfill batch, then delta again. A consent withdrawal waits at most one backfill batch.
  • Same path, same guard rails. Every backfilled record goes through the ordinary per-record path, the throttle, the kill switch and the volume breaker. A stop leaves the backfill position where it stood.
  • A missing cursor opens a backfill too. DeliveryCursor::forRule() falls back to the same backfill cursor. save() is not transactional, so the listener's insert can fail after the rule row is written; the first delivery step then opens the backfill with a slightly later bound, and nothing is lost. A lost cursor heals the same way instead of silently skipping what changed meanwhile.
  • No sign-off for the initial backfill. Connecting a shop to User is the merchant asking for their data to be there. A manually started backfill still needs the side-effect guard CFX-252 adds.

Consequences

  • Every existing record of a newly connected shop, or already in the mirror for a newly connected workspace, reaches the target once, whatever order the pull-in and the first delivery take.
  • An activated push-out rule holds delivery state from its activation on, so a force delete is refused from the start rather than after the first run.
  • A first activation against a target that runs automations on every change sets them off for the whole dataset. The throttle bounds the rate, not the total; the kill switch is the answer when a target's owner asks to stop.
  • The earlier open question whether a backfill and a delta scan share one cursor is settled: they share a row and hold two positions.
  • The day a system first declares push_out, every existing active rule into it has no cursor, so each starts its backfill at its first delivery step. All those workspaces receive their existing dataset at once, paced per target by the throttle. This is the rollout of CFX-200's promise for existing connections, not an anomaly, and it logs at info.