feat(incremental): catalog + BLONDE primitives (PR 1) #8
+260
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nge)
First, behavior-neutral phase of the leveled-incremental plan
(docs/RFC_incremental_v2.md). Adds pure, additive helpers that nothing in the
archive flow calls yet:
- enumerate_source_catalog(): per-file {arcname: (mtime_ns, size)} catalog,
walking/matching identically to enumerate_source/make_zip_files so arcname
keys line up with file_checksums + the round-trip verify (invariant #2).
REMOTE-eligible; imports inside the body (dill / invariant #1).
- badge_of(): stable 40-bit base32 target identity (shake_128 of NUL-joined
project/source_folder/file_pattern) — the future manifest key.
- make_blonde()/decode_blonde(): BLONDE id 'badge.quaver.detail.genus' with a
lossless base36-epoch quaver; round-trips exactly.
- floor_s(): ns -> whole-second flooring so catalog mtimes compare cleanly
against second-precision zip timestamps.
Tests (tests/test_incremental_primitives.py, 17 cases): badge determinism /
format / per-field distinctness / NUL-join safety; blonde round-trip + delta
levels + quaver ordering; base36 zero/roundtrip/negative; floor_s + sub-second
jitter; catalog arcname keys, value shape, enumerate_source parity, pattern
scoping, top-level-only pattern, empty match. Full suite: 54 passing (was 37).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jun 30, 2026
…no behavior change) First slice of docs/EXCLUSION_SPEC.md. Pure, additive helpers — NOTHING in the archive flow calls them yet (behavior-neutral, mirrors PR #8's primitives slice): - compile_exclude_spec / classify_tier — the 3-tier identity classifier (DATA allowlist first + un-overridable, then EXCLUDE, then OPTIONAL first-match, else DATA-default). Rejects bare-extension rules in the subtractive tiers at compile time (the spec's "never classify by extension" rule). DATA precedence + the greenhouse_climate lookahead are enforced. - git_coverage_preconditions / git_file_verdict — the fail-safe git gate. Per-run preconditions (repo+origin present, origin off the protected mount, durable-remote attestation, no git-LFS, fresh fetch, clean subtree) and the per-file proof (L/H/C/P/B/A) returning COVERED|UNTRACKED|DIRTY|STALE|ERROR. ONLY COVERED permits exclusion; every uncertain/error verdict -> back up. - spec_sha256 / _parse_duration helpers. Imports stay inside function bodies (Globus Compute / dill, invariant #1); the compiled spec (regex objects) is local-only, the raw dict crosses the boundary. Tests: tests/test_exclusion_primitives.py (+39) — classifier precedence incl. the greenhouse data-vs-_metadata-twin counterexample, the bare-extension guard, and the full git-gate matrix against real throwaway repos with a local bare remote. Suite 60 -> 99 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 8, 2026
…uting/append) Rewrites docs/RFC_incremental_v2.md from the leveled-only RFC into the umbrella staged-hybrid RFC that matches what actually shipped and what is being built next: - Phase 1 (exclusion): DONE/merged; recap + pointer to docs/EXCLUSION_SPEC.md. - Phase 2 (size-routing/APPEND): the new build target, full design — solo objects >= T_small (256 MiB, floor 100 MB), content-addressed shards (hash(relpath) mod K, K frozen at baseline), skip-unchanged (the append win, doubles as resume), per-target AGGREGATE reclaim (verify_target — REQUIRED same slice or every routed target false-DRIFTs), reship-monitor (the Phase-3 trigger), local-mode only (refuses --globus). 4 reviewable slices with tests. - Phase 3 (leveled L0/L1/L2): DEFERRED-DORMANT banner; the prior leveled RFC content (Path A/B, restore, chain-aware reclaim, Negishi spike) preserved intact as the Phase-3 reference. - Appendix: the PR #8 primitives (badge_of/make_blonde/enumerate_source_catalog/ floor_s) reused across phases, with current line refs. Design driver = the 2026-07-08 source_files measurement: standoff imagery is stable (exclusion covers it); the data modalities GROW by appending -> append is the dominant Phase-2 win. Review gate before any engine code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 8, 2026
…uting/append) Rewrites docs/RFC_incremental_v2.md from the leveled-only RFC into the umbrella staged-hybrid RFC that matches what actually shipped and what is being built next: - Phase 1 (exclusion): DONE/merged; recap + pointer to docs/EXCLUSION_SPEC.md. - Phase 2 (size-routing/APPEND): the new build target, full design — solo objects >= T_small (256 MiB, floor 100 MB), content-addressed shards (hash(relpath) mod K, K frozen at baseline), skip-unchanged (the append win, doubles as resume), per-target AGGREGATE reclaim (verify_target — REQUIRED same slice or every routed target false-DRIFTs), reship-monitor (the Phase-3 trigger), local-mode only (refuses --globus). 4 reviewable slices with tests. - Phase 3 (leveled L0/L1/L2): DEFERRED-DORMANT banner; the prior leveled RFC content (Path A/B, restore, chain-aware reclaim, Negishi spike) preserved intact as the Phase-3 reference. - Appendix: the PR #8 primitives (badge_of/make_blonde/enumerate_source_catalog/ floor_s) reused across phases, with current line refs. Design driver = the 2026-07-08 source_files measurement: standoff imagery is stable (exclusion covers it); the data modalities GROW by appending -> append is the dominant Phase-2 win. Review gate before any engine code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR 1 of the leveled-incremental plan (
docs/RFC_incremental_v2.md§10). Behavior-neutral foundation — adds pure helpers that nothing in the archive flow calls yet, so the existing path is untouched.Adds to
archive.py:enumerate_source_catalog()— per-file{arcname: (mtime_ns, size)}catalog, walking/matching identically toenumerate_source/make_zip_filesso arcname keys line up withfile_checksums+ the round-trip verify (invariant Canonical-root invariant: paths reclaim can never delete #2). REMOTE-eligible; imports inside the body (dill / invariant chore(reclaim): hold AAPF landing-zone raw until promote loop verified #1).badge_of()— stable 40-bit base32 target identity (shake_128of NUL-joinedproject/source_folder/file_pattern); the future manifest key.make_blonde()/decode_blonde()— BLONDE idbadge.quaver.detail.genus, lossless base36-epochquaver, exact round-trip.floor_s()— ns → whole-second flooring (reconciles catalog mtimes with second-precision zip timestamps).Tests:
tests/test_incremental_primitives.py(17 cases). Full suite 54 passing (was 37); existing 37 unchanged.Next: PR 2 wires these into the level decision (
decide_level,only_arcnames, manifest, lock).🤖 Generated with Claude Code