Skip to content

Conversation

@jdoucett
Copy link
Contributor

What

The two remaining exclusion items — finishes Phase 1 end-to-end. Builds on #13#16.

1. Resume-under-exclusion (replaces the #15 force-fresh interim)

#15 always built a fresh zip when an exclude_spec was set (so the staleness guard never compared an exclusion-filtered zip against a non-exclusion source walk). This restores resume for exclusion archives:

  • archive.enumerate_source_excluded(...) — exclusion-aware sibling of enumerate_source returning (survivor_count, newest_survivor_mtime, exclusion_report). Reads the spec where it runs (so --globus works), never raises.
  • __main__ resume — the staleness guard now compares survivors against the exclusion-filtered zip (no more spurious [STALE]), and the resumed run records the re-derived exclusion block so reclaim's INV-E stays correct on resumed archives. A changed/added/removed exclude_spec shifts the survivor count → [STALE] abort → re-zip with --fresh (message says so). Computed even under --force-resume so the log is right.

2. Post-hoc re-verifier — verify_exclusions.py

The audit tool that makes "prove nothing real was dropped" answerable months later, by a different operator, without trusting the original run. Given run-log JSON(s), it re-confirms every recorded exclusion is still git-COVERED:

  • OK — still covered at the same blob.
  • CHANGED — covered but regenerated + re-committed/pushed since (still redundant in git; informational, not a failure).
  • LOST — no longer covered (untracked/dirty/unpushed/LFS/error): current bytes may not be anywhere durable → failure.

Uses the coverage config recorded in the log (INV-E's repo_root), so it needs no access to the original exclude_spec. Exits non-zero on any LOST.

Tests — tests/test_exclusion_polish.py (+8)

enumerate_source_excluded (no spec / survivors-only / unreadable-spec-counts-all) and verify_log (OK / CHANGED / LOST-when-edited-not-pushed / SKIP / gate-off-is-LOST). CLAUDE.md updated (drops the "pending" notes; adds the new script to the layout). Suite 128 → 136 green.

Phase 1 (exclusion) is now complete

PR Phase
#13 / #14 primitives + --globus robustness
#15 archive wiring
#16 reclaim INV-E
this resume-under-exclusion + re-verifier

Next: Phase 2 — size-routing (solo + content-addressed shards) and the RFC rewrite (staged hybrid).

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

🤖 Generated with Claude Code

…h Phase 1)

The two remaining exclusion polish items — restores the resume optimization for
exclusion archives and adds the audit tool.

Resume-under-exclusion (replaces the Phase 1b-1 force-fresh interim):
  - archive.enumerate_source_excluded(source, pattern, exclude_spec, exclude_optional):
    exclusion-aware sibling of enumerate_source returning
    (survivor_count, newest_survivor_mtime, exclusion_report). Reads the spec
    where it runs (so --globus works) and never raises.
  - __main__ resume: the staleness guard now compares SURVIVORS against the
    exclusion-filtered zip (no more spurious [STALE]), and the RESUMED run records
    the re-derived exclusion block — so reclaim's INV-E stays correct on resumed
    archives. A changed/added/removed exclude_spec shifts the survivor count ->
    [STALE] abort -> re-zip with --fresh (message updated to say so). Computed
    even under --force-resume so the log is right.

Post-hoc re-verifier:
  - verify_exclusions.py: given run-log JSON(s), re-confirm every recorded
    exclusion is still git-COVERED. Per file: OK (same blob), CHANGED (covered but
    regenerated+recommitted since — informational), LOST (no longer covered —
    failure). Exits non-zero on any LOST. Uses the coverage config recorded in the
    log (INV-E's repo_root), so it needs no access to the exclude_spec.

Tests: tests/test_exclusion_polish.py (+8) — enumerate_source_excluded (no spec /
survivors-only / unreadable-spec-counts-all) and verify_log (OK / CHANGED /
LOST-when-edited-not-pushed / SKIP / gate-off-is-LOST). CLAUDE.md updated. Suite
128 -> 136 green. Phase 1 (exclusion) is complete end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign in to join this conversation on GitHub.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant