feat(exclusion): reclaim re-applies + re-proves the excluded set (INV-E) #16
+347
−6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is the piece that actually delivers the byte savings end-to-end. The
repository backup re-ships a target when reclaim reports it DRIFTED; before this,
an exclusion archive dropped metadata, so reclaim's live walk (which still sees
the metadata) counted MORE files than were archived -> perpetual false DRIFT ->
the target kept getting re-shipped, the exact waste exclusion exists to kill,
reintroduced from the reclaim side.
INV-E (docs/EXCLUSION_SPEC.md):
- archive.py: evaluate_exclusions now records repo_root + the coverage config
in the run-log exclusion block, so the log is SELF-CONTAINED for re-gating
(no dependency on the exclude_spec file at reclaim time).
- reclaim.recheck_exclusions(log, source_folder): re-runs the git coverage gate
against the CURRENT repo for each recorded exclusion. still_covered -> subtract
from the live walk; no_longer_covered (edited/regenerated since archive, bytes
not on tape) -> left in the walk. FAIL-SAFE: if it cannot be re-proven now (no
repo_root, git unavailable, preconditions fail) NOTHING is subtracted -> the
target reads DRIFTED, never wrongly SAFE.
- reclaim.scan_source gains exclude_arcnames; verify() subtracts still_covered
and raises a clear DRIFTED reason for any no_longer_covered exclusion.
- Logs with no exclusion block behave exactly as before (fully back-compatible).
Net: a metadata-only change now reads SAFE (backup skips the re-ship); an excluded
file whose current bytes are not on tape reads DRIFTED (never a silent --delete).
Tests: tests/test_exclusion_reclaim.py (+9) — regenerated-metadata-still-covered
-> SAFE (the money case) with a no-INV-E contrast that DRIFTs; excluded file
committed-not-pushed -> DRIFTED; new metadata since archive -> DRIFTED; gate
unprovable (dirty subtree) -> conservatively DRIFTED; no-exclusion-block
back-compat -> SAFE; recheck_exclusions unit behavior. Updated the opt-in test's
scan_source stub for the new kwarg. Suite 119 -> 128 green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why this is the load-bearing piece
The repository backup re-ships a target when
reclaim.pyreports itDRIFTED. Until now, an exclusion archive dropped metadata, so reclaim's live walk (which still sees the metadata) counted more files than were archived → perpetual falseDRIFTED→ the target kept getting re-shipped. That's the exact 2.35 TB waste exclusion exists to kill, reintroduced from the reclaim side. So exclusion doesn't actually save anything end-to-end until this lands. (Builds on #13/#14/#15.)INV-E (docs/EXCLUSION_SPEC.md)
archive.py:evaluate_exclusionsnow recordsrepo_root+ the coverage config in the run-logexclusionblock, so the log is self-contained for re-gating — reclaim needs no access to the originalexclude_specfile.reclaim.recheck_exclusions(log, source_folder): re-runs the git coverage gate against the current repo for each recorded exclusion.still_covered→ subtracted from the live walk;no_longer_covered(edited/regenerated since the archive → current bytes not on tape) → left in the walk so it surfaces as drift. Fail-safe: if exclusions can't be re-proven now (norepo_root,gitunavailable, preconditions fail) → nothing is subtracted → the target readsDRIFTED, never wronglySAFE.reclaim.scan_sourcegainsexclude_arcnames;verify()subtractsstill_coveredand raises a clearDRIFTEDreason for anyno_longer_coveredexclusion.exclusionblock behave exactly as before (fully back-compatible).Net: a metadata-only change now reads
SAFE→ the backup skips the re-ship (the byte savings); an excluded file whose current bytes aren't on tape readsDRIFTED→ never a silent--delete.Tests —
tests/test_exclusion_reclaim.py(+9)Real repo + bare remote, hsi stubbed:
COVERED→SAFE, with a no-INV-E contrast proving the same live stateDRIFTs without it.DRIFTED(no longer git-COVERED); new metadata since archive →DRIFTED; gate unprovable (dirty subtree) → conservativelyDRIFTED; no-exclusion-block back-compat →SAFE;recheck_exclusionsunit behavior.Updated the opt-in test's
scan_sourcestub for the new kwarg. Suite 119 → 128 green.Remaining exclusion work
Resume-under-exclusion (archive currently forces a fresh zip when
exclude_specis set) + the standalone post-hoc re-verifier. Then Phase 2 (size-routing) and the RFC rewrite.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
🤖 Generated with Claude Code