ADR-0044 — Live-ingest long-gap remediation and explicit-gap state v1¶
Status: ACCEPTED Date: 2026-09-25
Context¶
SCOPE.md ("Live Ingest Server Production Readiness v1") Active Path step 3
requires a design gate closing findings P2 (long-gap remediation path),
P3 (source authority for missing data) and P4 (gap state machine and
repair queue) before any implementation slice may touch long-gap handling
(issue #125, blocking issue #127).
Issue #110 (2026-09-24) disposed the only known candidate repair source --
the public Bybit historical archive at
https://public.bybit.com/trading/BTCUSDT/BTCUSDTYYYY-MM-DD.csv.gz -- as
NO_AUTHORITATIVE_REPAIR_PATH_PROVEN: at observation time the archive
stopped at 2026-09-22 (2026-09-23 returned 404), one day short of the
live/recent-trade window, so no archive/live overlap could be compared, and
no manifest/checksum/per-interval completeness attestation existed for any
requested interval.
Per this ADR's own governing issue, that disposition may only be revisited with newly observed, attributable evidence -- so this ADR re-audits the source before designing the state machine around it, rather than assuming
110's evidence is still current.¶
Re-audit performed 2026-09-25 (new observed evidence)¶
Direct HEAD requests (not directory-listing summarization, which proved
unreliable against this bucket's size) against the archive:
BTCUSDT2026-09-22.csv.gz 200 OK Last-Modified: 2026-09-23 01:12:08 GMT
BTCUSDT2026-09-23.csv.gz 200 OK Last-Modified: 2026-09-24 01:13:58 GMT
BTCUSDT2026-09-24.csv.gz 200 OK Last-Modified: 2026-09-25 01:13:18 GMT
BTCUSDT2026-09-25.csv.gz 404 Not Found (NoSuchKey)
This shows the archive is not stalled: it publishes each day's file at
approximately 01:1x UTC the next day, and by 2026-09-25 covers through
2026-09-24. No manifest, checksum or per-interval completeness attestation
file was found alongside the daily objects (only the .csv.gz object itself,
consistent with #110's finding).
The live recent-trades endpoint (GET /v5/market/recent-trade, ADR-0040 §7's
bounded reconciliation source) was also re-checked: it returns only the
most-recent handful of trades (by time/seq), i.e. a window of seconds at
BTCUSDT's trade rate, not a window reaching back a full day.
New finding beyond #110: the gap between the two sources is not an observation-time accident that might close on its own -- it is structural. The archive's freshest complete day is always approximately 24 hours behind "now" (publication lag), while the bounded recent-trades endpoint ADR-0040 §7 authorizes for reconciliation only reaches back seconds to low minutes. Under current, observed provider behavior, the archive's coverage and the recent-trades bounded reconciliation window structurally never overlap -- there is always an unobserved band between "yesterday's finalized archive" and "the last few minutes of live trades" that neither source covers. This is a stronger, more precise finding than #110's "one day short at observation time," and it does not depend on exactly which day is checked.
Disposition: NO_AUTHORITATIVE_REPAIR_PATH_PROVEN is reconfirmed, not
reopened, for two independent reasons: (1) no completeness attestation exists
at the archive source, and (2) the archive and the live bounded-reconciliation
window do not structurally overlap under currently observed provider
behavior. Per issue #125's own stop condition, this is "a valid design-gate
result, not a failure" -- it does not block this ADR from defining the
accepted state machine and source-authority criteria below, which apply
whether or not a source is proven today.
Decision¶
1. Long-gap is a derived classification, not a new store¶
The gap/repair state machine below is a read model derived from existing
durable records -- Declared Coverage (B04/ADR-0029) segments, A10
repair-intent/candidate/cutover records (ADR-0033), and the K10 checkpoint
(ADR-0042) -- not a new persisted table or a second source of truth for
coverage. This follows the same rule the rest of the platform already
applies (no second live-only storage/catalog/identity system).
2. Accepted state machine¶
Two levels are distinguished, because they answer different operator
questions and were conflated in SCOPE.md's original flat P4 list.
Session-level state (one value at a time, surfaced in ADR-0043's
HEALTH_SNAPSHOT signal):
CONTINUOUS no open (unrepaired) gap interval exists anywhere
in this dataset's history
RESTART_RECONCILING ADR-0042 S5 bounded reconciliation is in progress
(transient, entered on start/reconnect)
RESUMED_WITH_EXPLICIT_GAP
the live segment is running normally, but at least
one interval is in GAP_RECORDED_NON_COMPLETE or a
REPAIR_* sub-state below
RESTART_RECONCILING exits to CONTINUOUS when ADR-0042 S5 proves
continuity, or to RESUMED_WITH_EXPLICIT_GAP when it cannot (a new
GAP_RECORDED_NON_COMPLETE interval is recorded per below). This is not a
new invariant: it is the observable classification of the exact
RestartOutcome values resume_live_ingest already returns
(RESUMED -> CONTINUOUS, GAP_DETECTED -> RESUMED_WITH_EXPLICIT_GAP once
the new segment resumes, NO_CHECKPOINT -> CONTINUOUS for a first
acquisition).
Per-interval state (one value per recorded non-complete interval; an installation may have zero, one or many over its lifetime):
GAP_DETECTED transient: interruption observed, exact
[last_durable_key, new_segment_start_key) bound
being computed
GAP_RECORDED_NON_COMPLETE durable: explicit non-complete coverage persisted
via the existing Declared Coverage mechanism; the
entry state before any source evaluation has been
attempted or completed for this interval
REPAIR_SOURCE_EVALUATION_INCONCLUSIVE
durable but retryable: an evaluation against S3
was attempted for this exact interval but could
not be completed (query failure, ambiguous
evidence, evaluator crash); this is not a
judgement about the source and must not be
treated as a rejection
REPAIR_SOURCE_UNPROVEN durable, terminal absent new evidence: an
evaluation against the source-authority criteria
(S3 below) *completed* for this exact interval
and the source failed one or more criteria --
a genuine negative result, not a missing one
REPAIR_CANDIDATE_PENDING the source met S3's criteria for this exact
interval; an A10 repair candidate has been
produced and awaits cutover validation
REPAIR_CUTOVER_COMPLETE A10's existing atomic cutover has run and
re-verified the interval; coverage now reads
complete under existing coverage/publication
contracts
Fail-closed transition rules:
- the only transition into
REPAIR_CUTOVER_COMPLETEis fromREPAIR_CANDIDATE_PENDINGafter A10 cutover re-verification succeeds; no other state may be reclassified as complete; - an interval whose repair-source evaluation cannot be completed (query
failure, ambiguous evidence, evaluator crash) classifies as
REPAIR_SOURCE_EVALUATION_INCONCLUSIVE, never as proven and never collapsed intoREPAIR_SOURCE_UNPROVEN's durable-negative meaning -- the same fail-closed direction K03'sHealthStatealready uses for unavailable observations (UNKNOWN, neverHEALTHY), but kept observably distinct from an actual negative result so it remains retryable rather than durably rejected; - only an evaluation that completes against S3 for the exact interval and
fails one or more criteria may move the interval to
REPAIR_SOURCE_UNPROVEN; REPAIR_SOURCE_EVALUATION_INCONCLUSIVEis retried on the next scheduled or operator-triggered evaluation attempt without any new evidence requirement, since nothing about the source was actually decided;REPAIR_SOURCE_UNPROVEN, by contrast, requires new observed evidence before re-evaluation is attempted again (the same discipline issue #110's own closing comment already established: "a future issue could reopen this once ... evidence exists");GAP_RECORDED_NON_COMPLETE,REPAIR_SOURCE_EVALUATION_INCONCLUSIVEandREPAIR_SOURCE_UNPROVENare all stable and may persist indefinitely; the session-level state remainsRESUMED_WITH_EXPLICIT_GAPfor as long as any interval is notREPAIR_CUTOVER_COMPLETE, and normal live operation is unaffected. Today's re-audit (§ above) is a completed evaluation with a definitive negative result, so the archive candidate for any interval discovered under current provider behavior lands inREPAIR_SOURCE_UNPROVEN, notREPAIR_SOURCE_EVALUATION_INCONCLUSIVE-- this distinction is exactly why the two states must not be collapsed.
3. Source-authority criteria (S3)¶
Before any candidate source may move an interval into REPAIR_CANDIDATE_PENDING,
an evaluation must complete against all of the following, as observed and
attributable evidence -- never inferred from missing seq, elapsed wall
time, absence of trades, local buffer contents or operator convenience:
- exact interval support for the missing interval (not merely file
existence -- a completeness attestation, checksum, provider manifest, or
equally strong proof the requested
[start, end)is fully represented); - source identity and provenance (who publishes it, and why it is attributable to the same economic venue/instrument);
- overlap, or other equally strong evidence, that the candidate source's coverage and the existing canonical history's boundary actually converge (today's re-audit shows this fails structurally for the archive + recent-trades pairing -- see Context);
- a mapping to canonical semantics per S4 below without inventing sequence continuity.
Two distinct failure outcomes follow from this, and must not be conflated:
- the evaluation completes and one or more criteria are actually
unmet for the exact requested interval ->
REPAIR_SOURCE_UNPROVEN(durable negative result for that interval); - the evaluation cannot complete at all (the check itself failed, timed
out, or returned ambiguous evidence) ->
REPAIR_SOURCE_EVALUATION_INCONCLUSIVE(no judgement was made; must be retried, not treated as a rejection).
4. Canonical binding (S4) -- defined now for whenever a source is proven¶
Even though no source is proven today, the binding rule is decided now so that proof of a future source requires no further semantic decision:
- a repair-candidate row maps to
TradeKeyV1/trade-v1through the exact same canonical construction ADR-0040 §2 already defines for live records (venue,instrumentfixed to the selected vertical;exchange_tsandtrade_idtaken directly from the source's equivalent fields --timestamp/trdMatchIDfor the Bybit archive's observed schema); historical/repair rows never get a separate mapping function from live rows; - absent
seqin a candidate source (as observed in the Bybit archive schema) is handled exactly as ADR-0040 already handles it for live records: diagnostic evidence only, never required, never synthesized; - canonical total order remains
(exchange_ts, trade_id)uniformly; repair rows must sort into existing canonical history without reordering already -published records.
5. A10 composition (S5)¶
- A repair candidate for one
GAP_RECORDED_NON_COMPLETEinterval is represented through A10's existing repair-intent mechanism (ADR-0033), scoped exactly to that interval's[start_key, end_key)-- never open-ended or applied to un-recorded intervals; - A10's existing isolated-candidate-attempt and atomic compare-and-cutover transaction (reusing the A16/S14 seams) is reused unchanged; this ADR does not add a second cutover mechanism;
- ADR-0033's own accepted limitation ("coverage-trigger re-verification is
architecturally unsatisfiable inside A10's own package boundary") means
A10 cannot self-verify S3/S4 compliance. The long-gap orchestration
introduced by this ADR is the caller that performs S3/S4 verification
before invoking A10's candidate/cutover step, discharging the
attributable-evidence caller-discipline obligation
OPEN_DECISIONS.mdalready assigns to E06/F02/I04-style callers, applied here to this new caller.
6. Behavior when no source is proven (the actual current path)¶
This is not a fallback for an edge case -- per today's re-audit, it is the expected steady-state outcome for any interval discovered under current provider behavior:
- the interval remains
GAP_RECORDED_NON_COMPLETE(orREPAIR_SOURCE_UNPROVENonce a candidate has actually been evaluated and rejected) indefinitely; - A11/K10 continue with a new governed live segment per ADR-0040 §7/§9, unaffected;
- the session-level state stays
RESUMED_WITH_EXPLICIT_GAP, and every open interval must remain enumerable through the operator evidence ADR-0043 §4 wires (P6) -- an operator must never have to infer an open gap's existence from absence of alerting.
Excluded¶
This ADR does not define or authorize:
- production long-gap repair implementation (that is Active Path step 5, issue #127, blocked on this ADR);
- a second acquisition path/endpoint beyond the recent-trades bounded window and the archive already investigated (adding one, e.g. a deeper historical REST endpoint, would itself need source-authority proof under S3 and is explicitly not pre-authorized here);
- reopening or weakening issue #110's disposition -- this ADR reconfirms it with additional, more precise evidence;
- a new coverage/catalog/identity system (per §1, the state machine is a derived classification over existing B04/A10/K10 records);
- any change to ADR-0033's A10 semantics beyond specifying, as its required caller, who performs the re-verification A10 itself cannot perform.
Consequences¶
- Active Path step 5 (issue #127) can implement gap detection/recording and
the derived-state classification described here without further semantic
decisions. For the one candidate evaluated so far (the public archive),
the reachable durable state is
REPAIR_SOURCE_UNPROVEN-- a completed evaluation with a negative result, per today's re-audit. Issue #127 must still implementREPAIR_SOURCE_EVALUATION_INCONCLUSIVEas a distinct, retryable outcome for any future evaluation attempt that cannot complete (its own or a different candidate's); it must not treat every non-proven interval as if it had received a completed negative evaluation. - If a future re-audit finds a source that closes the structural overlap gap identified here (for example, a provider capability that narrows or removes the ~24h archive-publication lag, or a deeper bounded-history endpoint), S3/S4/S5 already state exactly how it would be evaluated and bound -- no new ADR is required solely to admit a proven source, only to authorize a genuinely new acquisition path if one becomes necessary.
- ADR-0043's
HEALTH_SNAPSHOTsignal must include the session-level state from §2 as one of its diagnostics dimensions, and must enumerate any interval not inREPAIR_CUTOVER_COMPLETE, closing P6 for this design gate. - Issue #110 remains closed and its disposition stands; this ADR is the "future issue" its own closing comment anticipated ("a future issue could reopen this once the archive/live overlap gap closes and completeness evidence exists") -- the overlap gap has not closed, so the disposition is reconfirmed rather than reversed.