Canonical-root invariant: paths reclaim can never delete #2
+192
−12
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…eted
Add a capability-level backstop to the advisory holds list. A "canonical
root" (declared via --canonical-root, repeatable, or FORTRESS_CANONICAL_ROOTS)
is enforced at two independent points with NO flag to override:
* verify() returns a new CANONICAL verdict first, so such a source never
enters the SAFE/deletable set, whatever the archive/hold/drift state.
* safe_to_rm() re-checks at the actual rm site, so even a direct
delete_source() with --yes --allow-no-snapshot cannot remove it.
is_canonical() is a path-boundary match on the normalized absolute path (not a
substring), so a root protects itself and all descendants but never a sibling.
The holds list says "shouldn't"; a canonical root says "can't".
Tests: helper merge/normalize/dedupe + path-boundary matching; verify() marks
CANONICAL over a would-be-SAFE source and over a hold; safe_to_rm refuses;
delete_source leaves the data untouched under the strongest delete invocation.
All 26 tests pass (7 new, 19 existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jun 26, 2026
…nge)
First, behavior-neutral phase of the leveled-incremental plan
(docs/RFC_incremental_v2.md). Adds pure, additive helpers that nothing in the
archive flow calls yet:
- enumerate_source_catalog(): per-file {arcname: (mtime_ns, size)} catalog,
walking/matching identically to enumerate_source/make_zip_files so arcname
keys line up with file_checksums + the round-trip verify (invariant #2).
REMOTE-eligible; imports inside the body (dill / invariant #1).
- badge_of(): stable 40-bit base32 target identity (shake_128 of NUL-joined
project/source_folder/file_pattern) — the future manifest key.
- make_blonde()/decode_blonde(): BLONDE id 'badge.quaver.detail.genus' with a
lossless base36-epoch quaver; round-trips exactly.
- floor_s(): ns -> whole-second flooring so catalog mtimes compare cleanly
against second-precision zip timestamps.
Tests (tests/test_incremental_primitives.py, 17 cases): badge determinism /
format / per-field distinctness / NUL-join safety; blonde round-trip + delta
levels + quaver ordering; base36 zero/roundtrip/negative; floor_s + sub-second
jitter; catalog arcname keys, value shape, enumerate_source parity, pattern
scoping, top-level-only pattern, empty match. Full suite: 54 passing (was 37).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 8, 2026
…tral)
Adds the pure, additive size-routing helpers to archive.py — nothing wires them
into the archive or reclaim flow yet (behavior-neutral; full suite 136 -> 172 green):
- route(catalog, t_small, shard_count) -> [{slot, kind, arcnames}] partition:
files >= T_small become solo objects; smaller files bucket into content-addressed
shards k = shake_128(arcname) mod K (no repack cascade -> cheap APPEND).
- shard_of / solo_slot / shard_slot / derive_shard_count (K = ceil(small_bytes /
shard_target), frozen per badge at baseline).
- object_fingerprint (solo {mtime_s,size}; shard {members,byte_sum,digest}) +
object_unchanged (the skip/append signal, also the resume mechanism).
- clamp_t_small (100 MB floor), budget_warnings (warn-only max_solo/max_objects/
oversize-shard), vault_dir_for/manifest_path/load_manifest/write_manifest
(atomic tmp+os.replace, best-effort; _vault child dodges reclaim's glob).
- make_zip_files gains an optional only_arcnames filter (no caller yet) so a later
slice can ship exactly one routed object; keys stay arcname-relpath (invariant #2).
Tests: tests/test_routing_primitives.py (36 cases) — boundary partition, shard
stability, K frozen, fingerprint change detection incl. reshuffle, skip vs
shard-append, only_arcnames subset with matching checksum keys, manifest round-trip.
See docs/RFC_incremental_v2.md §2 (slice 1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 8, 2026
…tral)
Adds the pure, additive size-routing helpers to archive.py — nothing wires them
into the archive or reclaim flow yet (behavior-neutral; full suite 136 -> 172 green):
- route(catalog, t_small, shard_count) -> [{slot, kind, arcnames}] partition:
files >= T_small become solo objects; smaller files bucket into content-addressed
shards k = shake_128(arcname) mod K (no repack cascade -> cheap APPEND).
- shard_of / solo_slot / shard_slot / derive_shard_count (K = ceil(small_bytes /
shard_target), frozen per badge at baseline).
- object_fingerprint (solo {mtime_s,size}; shard {members,byte_sum,digest}) +
object_unchanged (the skip/append signal, also the resume mechanism).
- clamp_t_small (100 MB floor), budget_warnings (warn-only max_solo/max_objects/
oversize-shard), vault_dir_for/manifest_path/load_manifest/write_manifest
(atomic tmp+os.replace, best-effort; _vault child dodges reclaim's glob).
- make_zip_files gains an optional only_arcnames filter (no caller yet) so a later
slice can ship exactly one routed object; keys stay arcname-relpath (invariant #2).
Tests: tests/test_routing_primitives.py (36 cases) — boundary partition, shard
stability, K frozen, fingerprint change detection incl. reshuffle, skip vs
shard-append, only_arcnames subset with matching checksum keys, manifest round-trip.
See docs/RFC_incremental_v2.md §2 (slice 1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 10, 2026
Disjoint-union reassembly of a size-routed target from its _vault manifest (docs/RFC_incremental_v2.md §2.9): - completeness gate BEFORE any transfer: every selected object tar must exist on Fortress (reclaim.hsi_exists) or ABORT — never a partial restore - empty-dest guard: restore only into a new/empty directory - per-object extract by recorded transport: htar -xvf -L memberlist (random-access indexed) / hsi get + tar xf for htar_large (invariant #6) - per-file MD5 verify against each object's own run-log source_files (arcname-keyed, invariant #2) - MANDATORY final reconciliation on a full restore: set(walk(dest)) == U objects' arcnames, else ABORT - --slot solo:<hex> / --slot shard:<k> / --member <relpath> selective restore (reconciliation is full-restore-only) LOCAL only (hsi/htar at /opt/hsi/bin, prepended like ship_object). Driven by either the target's archive config (badge -> manifest) or --manifest directly. Tests: tests/test_restore.py (17) — tape mocked via reclaim.hsi_exists + an injected extract fn (mirrors ship_one). Full suite 238 -> 255 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Adds a canonical-root invariant to
reclaim.py: a capability-level guard that sits above the advisory holds list. A path declared canonical (via repeatable--canonical-root, orFORTRESS_CANONICAL_ROOTS) can never be deleted, and there is deliberately no flag to override it.Why
Today the only thing protecting the curated repository from
reclaim.pyis a substring match in a holds file — advisory config. If that file is wrong, missing, or a different invocation drops it, the protection is gone. The deleter also currently runs as a group member with full delete rights. This adds a guard that says "can't", not just "shouldn't."How — two independent enforcement points
verify()returns a newCANONICALverdict first, so a canonical source never enters the SAFE/deletable set — regardless of archive, hold, or drift state.safe_to_rm()(re-checked at the actualrmtreesite) refuses the path, so even a directdelete_source()call with--yes --allow-no-snapshotleaves the data untouched.is_canonical()is a path-boundary match on the normalized absolute path (not a substring), so a root protects itself and all descendants but never a sibling like..._repository-staging.Tests
7 new tests (helper merge/normalize/dedupe, path-boundary matching,
verify()marking CANONICAL over a would-be-SAFE source and over a hold,safe_to_rmrefusal, anddelete_sourceno-bypass under the strongest delete invocation). All 26 tests pass (7 new, 19 existing).Companion change in florasense-tools wires
fs-reclaim-reportto pass the curated repository as a canonical root.🤖 Generated with Claude Code