Skip to content

ADR-007: Leases and Idempotency

Status: PROPOSED.

Context

CURRENT: Redis BRPop removes a scan message before execution completes. There is no durable lease, visibility timeout, or attempt fencing.

Existing persistence is partially idempotent:

  • assets use upsert by (organization_id, type, normalized_value);
  • threats with non-empty external IDs use upsert by (organization_id, asset_id, external_id);
  • threat detections use UNIQUE (threat_id, scan_job_id);
  • files and some exposure changes are not fully retry-idempotent.

Decision

Use explicit task attempts, renewable leases, fencing tokens, and durable result-batch acceptance.

Core records:

  • worker_task_attempts for lease ownership and attempt history;
  • worker_result_batches for idempotent result acceptance;
  • optional artifact upload session records for retry-safe artifact registration.

Lease semantics

  • A Worker receives a lease token when acquiring a task.
  • Every mutating RPC must include task ID, attempt ID, and lease token.
  • Gateway rejects expired, superseded, or cancelled attempts.
  • Lease expiration and execution deadline are separate.
  • Stale attempts must never overwrite newer attempts.

Result idempotency

Each result batch includes:

  • sequence number;
  • idempotency key;
  • payload checksum.

Duplicate submission with the same idempotency key and checksum returns the previous acknowledgement. Duplicate submission with the same key and different checksum is rejected.

Alternatives

  • Rely only on existing asset/threat upserts.
  • Accept one final result without batch records.
  • Use Redis Streams pending entries as the only idempotency mechanism.

Consequences

  • Gateway can recover from crash-after-persist-before-response.
  • Workers can reconnect and retry safely.
  • File/artifact duplication risk is reduced.
  • Implementation requires additional tables and repository methods.

Unresolved

  • Default lease duration.
  • Default retry backoff.
  • Maximum result batch size.
  • Whether out-of-order batches are accepted or rejected in v1.