Eremey Valetov
38c0898bc2
New library (uc2_preprocess.h / uc2_preprocess.c) for Phase 4:
BCJ (Branch/Call/Jump) filter:
- E8/E9 x86 address normalization (relative → absolute)
- Makes calls to the same function from different locations produce
identical byte sequences, improving LZ77 matching
- Round-trip verified; address normalization confirmed
BWT (Burrows-Wheeler Transform):
- Suffix-array-based forward transform
- LF-mapping inverse with reverse reconstruction
- Groups similar contexts for better entropy coding
- Round-trip verified for text ("banana") and binary data
Delta filter:
- Byte-wise delta encoding with configurable stride
- Stride 1 for sequential data, stride 2+ for interleaved channels
- Constant-delta sequences (arithmetic progressions) reduce to
repeated single values
Content detection:
- Automatic content type identification (text/x86/structured/binary)
- MZ/PE and ELF header recognition for x86
- Printable ASCII ratio for text detection
11 unit tests covering all filters and detection.
43 lines
1.6 KiB
ReStructuredText
43 lines
1.6 KiB
ReStructuredText
Roadmap
|
|
=======
|
|
|
|
The development roadmap is maintained in ``ROADMAP.md`` at the project
|
|
root. Key phases:
|
|
|
|
1. **Decompression MVP** — Done. Portable decompressor, CLI tool,
|
|
CMake build system.
|
|
|
|
2. **Original Compression Engine** — Done. LZ77+Huffman compressor
|
|
with custom Huffman trees, full backward compatibility, and UC2
|
|
personality (``-q`` for scripting). Automated DOSBox-X round-trip
|
|
validates 4+5 files in both directions.
|
|
|
|
3. **Modernized Master-Block Deduplication** — Done.
|
|
CDC with Gear hash, Merkle DAG with content addressing,
|
|
cross-archive block store, SimHash near-duplicate detection,
|
|
and delta compression. All Phase 3 items complete.
|
|
|
|
4. **Modern Compression Backends** — In progress. rANS entropy coder,
|
|
zstd-style dictionary metadata, content-aware preprocessing (BCJ,
|
|
BWT, delta filter with auto-detection). Remaining: LZ4 ultra-fast
|
|
mode, benchmarking mode, preprocessing integration into CLI.
|
|
|
|
5. **Quantum-Resistant Encryption** — CRYSTALS-Kyber + AES-256-GCM.
|
|
|
|
6. **DOS / FreeDOS / Retro-Computing** — DJGPP toolchain, vintage
|
|
hardware support, self-extracting archives.
|
|
|
|
7. **Cryptographic Integrity & Timestamping** — BLAKE3 hashing,
|
|
OpenTimestamps.
|
|
|
|
8. **Decentralized & Cloud Integration** — IPFS pinning,
|
|
content-addressable dedup, cloud archiving.
|
|
|
|
9. **Zero-Knowledge Proofs** — Privacy-preserving archive verification.
|
|
|
|
10. **Ecosystem Integrations** — libarchive plugin, streaming dedup
|
|
ingestion, file manager plugins.
|
|
|
|
See the full roadmap: `ROADMAP.md on GitHub
|
|
<https://github.com/evvaletov/uc2/blob/main/ROADMAP.md>`_.
|