Fix single-file backward compatibility with original UC2 Pro

Root cause: the original UC2 Pro expects csize=0 in the cdir COMPRESS
record (it ignores the field entirely).  UC2 v3 was writing the actual
compressed size, which confused the original's archive reader.

Additional changes:
- Use default Huffman tree for all blocks (ensures tree encoding compat)
- Write method=compression_level in cdir COMPRESS (was hardcoded to 1)
- Add tests/scripts/bitdump.py for bit-level bitstream analysis

Single-file UC2 v3 archives are now fully readable by the original UC2
Pro (listing and extraction verified in DOSBox-X).  Multi-file archives
still hang — the cdir bitstream decodes correctly in our Python analyzer
but fails in the original's ASM decompressor kernel.  Investigation
continues; the bitdump.py tool enables targeted comparison.
This commit is contained in:
Eremey Valetov
2026-03-29 09:58:36 -04:00
parent be7085c4d3
commit c736b19bae
6 changed files with 423 additions and 42 deletions
+11 -15
View File
@@ -2,13 +2,10 @@
# Cross-tool round-trip test: original UC2 Pro -> UC2 v3 via DOSBox-X
#
# Tests that archives created by the original 1992 UC2 Pro can be correctly
# extracted by UC2 v3 (Direction 2).
#
# Direction 1 (UC2 v3 -> original) is not yet implemented — the original
# UC2 Pro hangs reading UC2 v3 archives. Root cause is under investigation:
# tree generation and data encoding match the original, but some bitstream-
# level difference remains (likely in the ASM decompressor's expectations
# around tree serialization or block structure).
# extracted by UC2 v3 (Direction 2). Single-file UC2 v3 archives are also
# readable by the original, but multi-file archives still hang — the exact
# cause is under investigation (treegen + cdir encoding match the original
# for single-file cdirs but diverge for multi-file cdirs).
#
# Usage: roundtrip_dosbox.sh <uc2-cli> <uc2pro.exe> <corpus-dir>
@@ -52,12 +49,11 @@ uc2pro UC2DIST
exit
DOSBOXCFG
timeout 180 flatpak run com.dosbox_x.DOSBox-X \
timeout 300 flatpak run com.dosbox_x.DOSBox-X \
-conf "$WORK/dosbox.conf" -nopromptfolder 2>/dev/null || true
if [ ! -f "$WORK/UC2DIST/UC.EXE" ]; then
echo "FAIL: UC2 Pro SFX extraction did not produce UC.EXE"
ls -la "$WORK/UC2DIST/" 2>/dev/null || echo "UC2DIST directory missing"
exit 1
fi
echo " UC.EXE extracted ($(wc -c < "$WORK/UC2DIST/UC.EXE") bytes)"
@@ -86,25 +82,25 @@ timeout 300 flatpak run com.dosbox_x.DOSBox-X \
-conf "$WORK/dosbox.conf" -nopromptfolder 2>/dev/null || true
if [ ! -f "$WORK/MARKER.TXT" ]; then
echo "FAIL: DOSBox session did not complete (archive creation may have timed out)"
echo "FAIL: DOSBox session did not complete"
exit 1
fi
# --- Verify Direction 2 (original -> UC2 v3) ---
echo "--- Verifying Direction 2 ---"
FAIL=0
DOS_ARCHIVE=""
for candidate in "$WORK/out/DOSTEST.UC2" "$WORK/out/dostest.uc2"; do
[ -f "$candidate" ] && DOS_ARCHIVE="$candidate" && break
done
if [ -z "$DOS_ARCHIVE" ]; then
echo "FAIL: UC2 Pro did not create DOSTEST.UC2"
ls -la "$WORK/out/" 2>/dev/null
exit 1
fi
echo " Archive created: $(wc -c < "$DOS_ARCHIVE") bytes"
# --- Extract with UC2 v3 and verify ---
echo "=== Extracting with UC2 v3 ==="
"$UC2_CLI" -d "$WORK/output" "$DOS_ARCHIVE"
FAIL=0
for f in "${FILES[@]}"; do
upper=$(echo "$f" | tr '[:lower:]' '[:upper:]')
extracted=""
@@ -112,7 +108,7 @@ for f in "${FILES[@]}"; do
[ -f "$candidate" ] && extracted="$candidate" && break
done
if [ -z "$extracted" ]; then
echo " FAIL: $f not extracted by UC2 v3"
echo " FAIL: $f not extracted"
FAIL=1
continue
fi