fix(routing): route + detect htar's silent member-drop modes #29
+351
−19
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t size Indexed htar -L silently drops members whose relpath is >= 250 chars (rc=70, "name too long for tar archive- file omitted") - reproduced live on Negishi and the root cause of X0A_Task4_Pendleton_Transcriptomics repeatedly failing on the same overlong Novogene GSEA/KEGG path. htar_large has no such ceiling (confirmed live with a 431-char relpath), so choose_transport now also routes on arcname length (HTAR_ARCNAME_LEN_LIMIT = 240), not just size. As defense in depth, ship_object scans create stdout for htar's/tar's own unambiguous member-omission messages and fails fast with every dropped file named, instead of falling through to the rc=70-tolerant path (PR #28) or waiting on the round-trip verify to surface a bare FileNotFoundError. Generalized beyond the name-too-long case after live-testing showed two more silent-drop modes: htar's "Not readable" skip, and - the one case rc gives zero signal for, confirmed rc=0 on Negishi - tar's own "Cannot open" error under htar_large, whose exit code reflects hsi put, not the tar process that actually failed. Checked for a native htar "abort on skip" setting (no man page; pulled the full -H option list from the binary) and a structural exceptions-list option (-H exfile=) - neither exists/works, confirmed by live testing rather than assumed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Jul 12, 2026
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
htar -Lsilently drops members whose relpath is >= 250 chars (rc=70) - reproduced live on Negishi, and the root cause ofX0A_Task4_Pendleton_Transcriptomics/shard-0of8repeatedly failing on an overlong Novogene GSEA/KEGG path.htar_largehas no such ceiling (confirmed live with a 431-char relpath), sochoose_transportnow also routes on arcname length (HTAR_ARCNAME_LEN_LIMIT = 240), not just size.ship_objectnow scans create stdout for htar's/tar's own member-omission messages and fails fast, naming every dropped file, instead of falling through to the rc=70-tolerant path from PR fix(routing): tolerate transient nonzero htar/htar_large rc in ship_object #28 or waiting on the round-trip verify. Generalized beyond "name too long" after live-testing surfaced two more silent-drop modes: htar's "Not readable" skip, and the one case rc gives zero signal for (confirmed rc=0 live) - tar's own "Cannot open" error underhtar_large.-H exfile=); neither exists/works - confirmed by testing, documented in CLAUDE.md/RFC rather than assumed.Test plan
python3 -m unittest discover -s tests), including 6 new regression tests covering length-based routing and the generalized omission scan (name-too-long, not-readable, tar-cannot-open-with-rc-zero).htar_largecreate/retrieve/verify round-trip with a 431-char relpath;htar -Lsilent drop reproduced at the 250-char boundary;-H exfile=tested and confirmed not to capture these skips;htar_large's rc=0-despite-drop behavior confirmed with achmod 000file.X0A_Task4_Pendleton_Transcriptomics(config on Negishi) to confirm shard-0 (and ideally all 8 shards) ships clean.Generated with Claude Code