Compared raw cdir bytes between original UC2 Pro and UC2 v3 archives
(using fixed bitdump.py match decoder). Key finding: the original
NEVER uses SuperMaster (masterPrefix=0) in file COMPRESS records —
it always assigns custom master indices (>= 2). UC2 v3 uses
SuperMaster for ungrouped files.
The original's ExtractFiles() processes SuperMaster files through
ToToWalk(TTefl, SUPERMASTER, ...) which hangs. Custom master files
go through the masroot chain walk, which works correctly.
Fix: always assign custom master indices to all files, generating at
least one custom master block even for archives without dedup groups.
This matches the original's behavior.
Other differences found: original uses hidden=0xDE (sentinel), tag=0
(no EXTMETA), method=3, and csize=8 (much smaller due to custom
master compression).
Single-file UC2 v3 archives are now fully backward compatible with the
original UC2 Pro — listing and extraction verified in automated DOSBox-X
test. SFX extraction timeout increased to 600s with 22-file completeness
check (incomplete extraction caused false test results throughout the
earlier investigation). Direction 1 (UC2 v3 -> original) test added.
Located the nuke1 ASM decompressor source at NUKE.CPP (305 lines of
C++ with inline x86 assembly). Analysis of the half-buffer flush logic
in FlushIt (ULTRACMP.CPP) reveals the extraction hang root cause:
For a 49152-byte SuperMaster, the original adjusts the output position:
wTOE = 49152 - 32768 = 16384 (lower buffer half)
wComp = 0 (flush when output enters lower half)
FlushIt triggers immediately after the first decompressed byte (because
wTOE is already in the lower half with wComp=0). It reads decompressed
output from position 32768 + wSkip = 49152, but the actual output was
written at position 16384. This produces garbage and triggers early
termination.
UC2 v3's compressor starts the LZ77 window at position 49152 (raw
SuperMaster size), while the original's decompressor expects the window
at position 16384 (49152 mod 32768). Fixing this requires aligning
the compressor's starting position to match the decompressor's adjusted
wTOE.
Also corrected: the earlier "single-file extraction works" finding was
a false positive from incomplete UC2DIST extraction (15/22 files).
With complete UC2DIST, listing works but extraction hangs for all files.
Testing revealed that custom Huffman trees from our treegen cause the
original UC2 Pro to hang even for single-file archives. The original's
ASM decompressor kernel (nuke1) has undocumented assumptions about tree
shapes that our treegen doesn't match.
Default tree is the only working option for backward compatibility.
Multi-file extraction remains a separate open issue (hangs even with
default tree, while listing works).
Investigation of multi-file extraction hang:
- LocMacNtx(0) returns VNULL for SuperMaster, but V(VNULL) is safe
in the original's virtual memory system (not a null dereference)
- 0xDEDEDEDE is a legacy sentinel for old archives, not the standard
value (original uses masterPrefix=0)
- The hang is in the file data decompression phase, not the cdir
parsing (listing of multi-file archives works correctly)
Multi-file backward compat remains under investigation.
The original UC2 Pro handles archives differently based on versionMadeBy:
300 causes multi-file listing to fail, while 203 (the original's own
version) works correctly. Also write method=compression_level in file
COMPRESS records instead of hardcoded 1.
Combined with the earlier csize=0 and default-tree fixes, single-file
UC2 v3 archives are now fully backward compatible with the original
UC2 Pro (listing + extraction verified). Multi-file archives can be
listed but extraction still hangs — under investigation.
Root cause: the original UC2 Pro expects csize=0 in the cdir COMPRESS
record (it ignores the field entirely). UC2 v3 was writing the actual
compressed size, which confused the original's archive reader.
Additional changes:
- Use default Huffman tree for all blocks (ensures tree encoding compat)
- Write method=compression_level in cdir COMPRESS (was hardcoded to 1)
- Add tests/scripts/bitdump.py for bit-level bitstream analysis
Single-file UC2 v3 archives are now fully readable by the original UC2
Pro (listing and extraction verified in DOSBox-X). Multi-file archives
still hang — the cdir bitstream decodes correctly in our Python analyzer
but fails in the original's ASM decompressor kernel. Investigation
continues; the bitdump.py tool enables targeted comparison.
Port the original TreeGen/RepairLengths/CodeGen algorithms faithfully
from TREEGEN.CPP for bitstream compatibility with the 1992 UC2 Pro:
- treegen() now accepts max_code_bits parameter (13 for main trees,
7 for tree-encoding meta-tree)
- Heap uses >= for child comparison (prefer right child on ties),
matching original Reheap()
- BuildCodeTree uses extract-one-then-combine pattern
- RepairLengths uses sorted linked lists with cascading space-fill
- Single/zero symbol cases assign length 1 to two symbols
- tree_enc RLE: trigger at run > 6 (not >= 6), max 20 per chunk,
single RepeatCode per run
- First block uses default tree (tree-changed=0) matching original
behavior for small blocks
Full backward compatibility with original UC2 Pro archives (Direction 2)
is maintained. Forward compatibility (UC2 v3 -> original, Direction 1)
remains in progress — the original still hangs, likely due to residual
bitstream-level differences in the ASM decompressor kernel.
Automated test that runs the original 1992 UC2 Pro (UC.EXE) in DOSBox-X
headlessly to create archives from the test corpus, then extracts with
UC2 v3 and verifies byte-for-byte file identity.
Key findings during development:
- uc2pro.exe is a UCEXE self-extracting archive, not the tool itself;
the actual archiver is UC.EXE inside the distribution
- UC.EXE must be run from its own directory (needs DOS.SEA overlay)
- DOSBox-X flatpak requires work dirs under $HOME (filesystem=home)
- The reverse direction (UC2 v3 → original) does not work: the original
UC2 Pro hangs reading UC2 v3 archives due to compression bitstream
differences (added as a roadmap item)
Also fixes create_archives.sh to use the same two-session DOSBox pattern
(extract SFX first, then use UC.EXE).
Recursive directory scanning with parent/child ID tracking, directory
entries in the central directory (OSMETA + DIRMETA + EXTMETA long name
tags), and a CLI round-trip test verifying nested directory hierarchies.
Content-fingerprint grouping via FNV-1a hash of file headers: files
sharing identical first 4096 bytes are assigned a custom master block
built from the largest file in the group. Masters are compressed with
SuperMaster and written as MASMETA records in the central directory.
Files below 1 KB or without a group continue using the SuperMaster.
Includes CLI integration test and documentation updates (format spec,
usage, roadmap).
Sphinx docs with Furo theme covering CLI usage, library API reference,
archive format specification, build instructions, history, and roadmap.
GitHub Actions workflow deploys to Pages on push to main.
Implement a compressor that produces bitstreams compatible with the
existing Bobrowski decompressor. The engine uses LZ77 sliding-window
match finding with hash chains, Huffman entropy coding, and delta-coded
tree serialization matching the original UC2 format exactly.
New files:
- lib/src/compress.c: LZ77+Huffman compressor (~950 lines)
- lib/src/uc2_internal.h: shared constants, types, checksums
- lib/src/uc2_tables.c: vval/ivval delta tables, default Huffman tree
- tests/src/test_roundtrip.c: compress→archive→decompress→verify tests
Key details:
- 4 compression levels (Fast/Normal/Tight/Ultra) with tunable search
- Lazy evaluation for better match selection at higher levels
- Delta-coded Huffman tree serialization with RLE
- Fletcher/XOR checksum computation
- Round-trip test covers 8 patterns × 4 levels (32 test cases)
Fixed 28 errors in the hand-computed ivval inverse delta table (rows
9-13) that caused the decompressor to reconstruct wrong Huffman trees
from compressor output.
Test corpus (empty, text, binary, compressible, incompressible) with
reference archives created by original UC2 v2.3 in DOSBox. Two CTest
tests: test_identify (magic detection) and test_extract (full
extraction pipeline verified byte-for-byte against corpus).
Restructure compat layer: #include_next headers moved to posix/ for
MinGW, new standalone headers in msvc/ for MSVC (unistd.h, utime.h,
getopt.h). Add getopt() implementation, chmod/unlink/chdir compat
functions, MSVC CRT initializer for UTF-8 console, _pgmptr fix.
Add a DJGPP CMake toolchain file and DOS compatibility layer (err.h,
fnmatch, getprogname/setprogname) so UC2 builds as a native DOS
executable via cross-compilation from Linux. The toolchain works
around a baked-in /usr/include in the DJGPP GCC binary by using -I
instead of -isystem to ensure DJGPP headers take precedence.
Add GitHub Actions CI workflow that builds and smoke-tests on both
ubuntu-latest and macos-latest.
UC2 should have warm, quirky status messages as its default voice,
continuing the original's tradition. Not a --fun flag, just how
the tool talks. Quiet mode for scripting.
Decompression MVP based on Jan Bobrowski's portable unuc2/libunuc2.
CMake build system targeting Linux (GCC/Clang) with MSVC fallback.
Includes original UC2 source by Nico de Vries and unuc2-0.6 for reference.
Eremey Valetov