Add SimHash near-duplicate detection and delta compression
Completes Phase 3 (Modernized Master-Block Deduplication).
SimHash (uc2_simhash.h): 64-bit locality-sensitive fingerprint using
4-byte shingles. Similar files produce fingerprints with small Hamming
distance. Detects patched executables (16 bytes changed in 8KB: dist<=8),
slightly edited documents, and minor file revisions. 6 unit tests.
Delta compression (uc2_delta.h): binary diff with COPY (from source)
and INSERT (new data) instructions. Hash-based source matching for
fast encoding. 16KB file with 96 patched bytes: >50% delta size
savings. Full round-trip verified for identical, different, patched,
appended, and empty inputs. 6 unit tests.
All Phase 3 items now complete:
- [x] Content-fingerprint grouping (FNV-1a)
- [x] Custom master-block generation
- [x] MASMETA cdir records
- [x] SuperMaster-compressed masters
- [x] CDC with Gear rolling hash
- [x] Merkle DAG content addressing
- [x] Cross-archive block store
- [x] Near-duplicate detection (SimHash)
- [x] Delta compression
Eremey Valetov
7b1833a94c