diff --git a/CLAUDE.md b/CLAUDE.md index 1c90697..6234944 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -386,6 +386,30 @@ archives byte-for-byte as before. Authoritative design: `docs/RFC_incremental_v2 confirmed rc=0 on Negishi) tar's own `Cannot open: ` under `htar_large`, since that wrapper's exit code reflects `hsi put`, not the `tar` process that hit the error. Invariant #6 still applies to retrieval. +- **This isn't theoretical for the two areas that matter — audited 2026-07-12.** + The two areas actually required to be backed up are `data/repository` and + `data/landing-zone` (legacy per-experiment archives like the old flat X0A–F + layout are being consolidated out of scope separately). A direct relpath-length + scan of both, done to settle whether they're exposed rather than waiting for a + run to fail: + - `data/repository` (46 TB, ~19 categories): **max relpath 237 chars** — + `X0F/4_omics-analysis/.../10_KEGG_enrichment/.../gmx04141...absMaxFC.png` — + only **3 characters under `HTAR_ARCNAME_LEN_LIMIT` (240)**. 13,408 files are + already ≥200 chars. This is real, in-scope production data, not the + deprecated X0A raw-delivery tree that originally broke — same + KEGG/pathway-analysis naming family, just reorganized slightly shorter by + repository's ingest. Nothing has crossed 240 yet, so nothing has silently + dropped here, but the margin is thin enough that the next omics-analysis + pipeline run with a slightly longer comparison name plausibly could without + this fix. + - `data/landing-zone` (Metabolomics + inboxes): max relpath 218 chars, 46,870 + files ≥200 chars. Same long-imaging-pipeline-name style, wider (22-char) + margin than repository's. + - Cross-checked against every historical run log: only the legacy + `X0A_Task4_Pendleton_Transcriptomics` target has ever actually hit "name too + long" — repository and landing-zone runs have not silently lost files to + date. The margin found here is why that's true, not evidence it's not a + real risk. - **Per-target manifest** at `{log_dir}/_vault/{badge}.manifest.json` — the durable cross-run state reclaim and restore rebuild the aggregate from. This is why routing requires a shared, durable `log_dir` (guard in `load_config`). diff --git a/docs/RFC_incremental_v2.md b/docs/RFC_incremental_v2.md index 8f01547..cca862d 100644 --- a/docs/RFC_incremental_v2.md +++ b/docs/RFC_incremental_v2.md @@ -198,6 +198,20 @@ htar's `Not readable: ` (unreadable source file, also silently skipped) strings but haven't been reproduced live, so are documented as a gap rather than guessed at: a symlink whose target path is too long, and a source file modified mid-create. +**Exposure audit (2026-07-12), against the areas actually in scope — `data/repository` and +`data/landing-zone`, not the legacy per-experiment X0A–F layout being consolidated +separately.** A direct relpath-length scan (not waiting for a real run to fail) found +`data/repository` (46 TB, ~19 categories) has a max relpath of **237 chars** — 3 characters +under `HTAR_ARCNAME_LEN_LIMIT` — in `X0F/4_omics-analysis/.../10_KEGG_enrichment/...`, the +same pathway-analysis naming family that broke the legacy X0A archive, just reorganized +slightly shorter by repository's ingest; 13,408 files are already ≥200 chars. +`data/landing-zone` maxes at 218 chars (46,870 files ≥200), a wider margin. Neither has +crossed 240 yet — consistent with every historical run log showing only the legacy +`X0A_Task4_Pendleton_Transcriptomics` target ever actually hitting "name too long" — but the +3-char margin on repository means this was close to being a live incident, not a +hypothetical one, and the next omics pipeline run with slightly longer comparison names +could cross it without this fix in place. + ### 2.2 Thresholds, K, and budgets (measurement-tuned; all warn-only) | Knob | Default | Basis |