revert(skip-unchanged): drop in-engine incremental Phase 1a #6
+0
−341
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reverts 5df25c6. The in-engine incremental approach (skip-unchanged Phase 1a + the PR #4 RFC) is superseded by Fortress-native incremental/ versioned backup, so the skip_unchanged path and its tests are removed rather than carried as dead, default-off code. No change to the default daily flow (skip_unchanged was default false). Removes archive.py's skippable_prior_log / source_matches_log helpers and the skip-decision block, plus tests/test_skip_unchanged.py. Remaining suite: 37 tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 26, 2026
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 9, 2026
…Negishi-verified)
Replaces the routed zip->htar_large chain with the transport Jarrod chose and I
verified live on Negishi (HPSS 10.3.0.p3, 2026-07-09):
- Tar SOURCE files directly (drop the zip for routed objects — it was only local
staging; also kills the full-size scratch copy). Every object is a self-describing
{stem}.tar; htar -t shows real source paths (better context than the opaque zip).
- Tool per object by max member size: htar (with -P -L memberlist) when every member
< 2^36 bytes (64 GiB) -> gets .idx -> fast list + RANDOM-ACCESS single-member extract;
htar_large only when a member >= 2^36 (rare huge solo). Cutoff locked empirically
(2^36-1 accepted, 2^36 refused); htar fail-fasts on oversize with NO partial on tape.
- Manifest: per-object transport + idx fields replace zip_checksum; restore uses
htar -x random access for indexed objects (cheap --slot/--member selective restore).
- §2.1 transport note, §2.4 schema, §2.5 orchestration (ship_object), §2.8 inv #6,
§2.9 restore, §2.10 slices 2/5 all updated. Phase-3 (deferred) sections untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Jarrod S Doucette (jdoucett)
added a commit
that referenced
this pull request
Jul 10, 2026
Disjoint-union reassembly of a size-routed target from its _vault manifest (docs/RFC_incremental_v2.md §2.9): - completeness gate BEFORE any transfer: every selected object tar must exist on Fortress (reclaim.hsi_exists) or ABORT — never a partial restore - empty-dest guard: restore only into a new/empty directory - per-object extract by recorded transport: htar -xvf -L memberlist (random-access indexed) / hsi get + tar xf for htar_large (invariant #6) - per-file MD5 verify against each object's own run-log source_files (arcname-keyed, invariant #2) - MANDATORY final reconciliation on a full restore: set(walk(dest)) == U objects' arcnames, else ABORT - --slot solo:<hex> / --slot shard:<k> / --member <relpath> selective restore (reconciliation is full-restore-only) LOCAL only (hsi/htar at /opt/hsi/bin, prepended like ship_object). Driven by either the target's archive config (badge -> manifest) or --manifest directly. Tests: tests/test_restore.py (17) — tape mocked via reclaim.hsi_exists + an injected extract fn (mirrors ship_one). Full suite 238 -> 255 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverts the skip-unchanged feature (#5, commit 5df25c6).
Why: the in-engine incremental approach (skip-unchanged Phase 1a + the #4 RFC) is superseded by Fortress-native incremental/versioned backup. Rather than carry
skip_unchangedas dead, default-off code, this removes it.What:
archive.pyskippable_prior_log/source_matches_log+ the skip-decision block (−134)tests/test_skip_unchanged.py(−207)skip_unchangeddefaulted tofalse)Verification: remaining suite green — 37 tests pass.
Companion cleanup: #4 (RFC) closed as superseded; stale
feat/skip-unchangedbranch deleted.🤖 Generated with Claude Code