feat(incremental): Phase 3 leveled incremental backup (L0/L1/L2, RFC §3)

CLAUDE.md

@@ -324,6 +324,9 @@ Personal auth (endpoint owner) never needs `allowed_principals`. The local
 | `t_small` | No | 256 MiB | Solo/shard cutoff in bytes (hard floor 100 MB). Files ≥ `t_small` ship as their own object. |
 | `shard_count` | No | `null` | Shard count K. `null` = derive from `shard_target` at baseline. **Frozen in the manifest** once a baseline exists — changing it needs `--fresh`. |
 | `shard_target` | No | 256 GiB | Target shard size (bytes) used to derive K at baseline. |
+| `incremental` | No | `false` | Phase-3 leveled incrementals (L0 baseline + delta levels — see the Leveled-incremental section and `docs/RFC_incremental_v2.md` §3). **Local mode only** (refuses `--globus`); mutually exclusive with `size_routing`; requires a SHARED, DURABLE `log_dir` (same guard as `size_routing`). Manifest lives at `{log_dir}/_vault/{badge}.leveled.manifest.json`. |
+
+`--full` (CLI flag, not a config key): force a fresh L0 baseline on an `incremental` target regardless of the (mtime,size) diff — the operational safety valve for the §3.1 same-size/same-mtime blind spot (see the Leveled-incremental section).
 
 See `config.example.json` for an annotated template.
 
@@ -447,6 +450,86 @@ archives byte-for-byte as before. Authoritative design: `docs/RFC_incremental_v2
 
 ---
 
+## Leveled incremental (Phase 3) — L0 baseline + delta levels
+
+Opt-in per config (`"incremental": true`) and **default-off**, mutually exclusive
+with `size_routing` (a badge wears exactly one manifest schema). Authoritative
+design: `docs/RFC_incremental_v2.md` §3 (all 3 build slices — engine, reclaim,
+restore — merged).
+
+- Ships an **L0 baseline** (all post-exclusion survivors) once, then only the
+  **(mtime,size)-diffed delta** on every later run: added ∪ modified files ship
+  as a new **delta level**; removed files are recorded as metadata-only
+  `deleted` entries (no tar). A run with nothing added/removed/modified is a
+  **true NO-OP** — nothing ships, no new level node.
+- **HARD INVARIANT (§3.1)**: a same-size, same-mtime **in-place content
+  rewrite is invisible** to the cheap diff and will NOT be captured in any
+  delta — re-hashing would fix this but defeats the point of the feature. A
+  true NO-OP always emails loudly with the exact wording *"detected zero
+  metadata change; same-size in-place content edits are NOT captured. Use
+  `--full` to force a fresh L0 if you rewrote files preserving mtime+size."*
+  This email is intentionally **not** paired with the T1 Teams bell
+  (`notify_bell`) — a NO-OP is a normal, potentially daily outcome, not a
+  failure; the bell stays reserved for `[FAILED]`/`[STALE]`/
+  `[EXCLUSION-ABORT]`. `--full` forces a fresh L0 baseline (a new,
+  independent chain — the old L0 + its deltas remain in the manifest,
+  superseded but not deleted) regardless of the diff, as the periodic safety
+  valve.
+- **Per-badge advisory lock** (`badge_lock`, `fcntl.flock`) is held across the
+  ENTIRE decide+ship+manifest-write critical section — not just the manifest
+  write — closing the read-modify-write race a narrower lock would leave open.
+  A failed ship writes no manifest state; a retry re-runs the same decision
+  from scratch and re-zips fresh (a delta's member set is small, so this is
+  cheap by construction).
+- **Level-aware resume (§3.5.1)**: only an **L0** ship attempts
+  `find_existing_zip`'s resume (a delta's zip is never a complete source
+  snapshot, so it always re-zips fresh, never resuming) — the L0 zip is
+  trusted only if its member count still matches the live post-exclusion
+  survivor set AND no survivor's mtime is newer than the zip's own timestamp
+  (`archive.l0_zip_is_stale`), mirroring the whole-target guard's exact
+  semantics without re-walking the source a second time.
+- **Per-target manifest** at `{log_dir}/_vault/{badge}.leveled.manifest.json`
+  (kept apart from a Phase-2 `{badge}.manifest.json` for the same badge by the
+  `kind=` filename split — a badge mid-conversion can have both on disk at
+  once without either clobbering the other). Each level node records
+  `blonde`/`level`/`parent_blonde`/`fortress_tar`/`tar_bytes`/`n_files`/
+  `source_bytes`/`catalog_ref`/`deleted`. `catalog_ref` points at a sibling
+  JSON file (not inlined) holding that level's own `{arcname: [mtime_ns,
+  size]}` subset — kept lean even for a 36k-file L0.
+- **`archive.effective_catalog(manifest, log_dir)`** folds the current chain
+  (the L0 node matching `current_l0_blonde`, plus deltas whose
+  `parent_blonde` points at it) into one `{arcname: (mtime_ns, size)}` view —
+  the single shared implementation `decide_level`, `reclaim.verify_chain`, and
+  `restore.select_chain`/`fold_expected` all replay, so all three always agree
+  on "current state." An OLDER, superseded chain left behind by a `--full`
+  rebaseline is deliberately excluded from this fold (and from reclaim's
+  tape-completeness check) — it is not part of what gets restored.
+- **Reclaim** verifies a leveled target as an AGGREGATE:
+  `reclaim.verify_chain()` replays `effective_catalog`, requires every
+  **current-chain** tar present on Fortress (superseded-chain tars are exempt
+  — see above), and re-runs the INV-E exclusion re-gate. `--scan` picks up
+  leveled manifests automatically (leveled per-level/NO-OP run-logs are
+  skipped by the legacy per-log `verify()`); the single-config invocation
+  (`python reclaim.py config.json`) routes an `"incremental": true` config the
+  same way.
+- **Restore** (`restore.py`, LOCAL, operator-invoked): full-chain or
+  point-in-time (`--to-timestamp`) restore into an **empty** dest, replaying
+  levels in strict run-time order as a file OVERLAY plus per-level deletion
+  replay. Gates, in order: chain completeness (every level tar on tape BEFORE
+  any transfer), empty dest, per-level extract + per-member MD5 vs that
+  level's own run-log, and a mandatory final reconciliation against
+  `effective_catalog`. No `--slot`/`--member` selective restore for leveled
+  targets (only full or `--to-timestamp`).
+
+  ```bash
+  python3 archive.py config.json                    # normal incremental run (L0 or delta)
+  python3 archive.py --full config.json              # force a fresh L0 rebaseline
+  python3 restore.py config.json --dest DIR          # full chain restore
+  python3 restore.py config.json --dest DIR --to-timestamp 20260601_000000  # point-in-time
+  ```
+
+---
+
 ## Known issues / history
 
 - **Unbounded subprocess hang (32h+ on Negishi):** `repository_X1D_3_metabolomics_rawspectra`