Conversation
|
Closing after evaluation — the code works (I merged main in, resolved the 16-month go.mod conflict, and verified byte-exact round-trips including through the stdin pipeline), but the value case does not hold up: The algorithm is superseded. FPC is from 2007. The lineage since: Gorilla (2015) → Chimp/Chimp128 (2022) → Elf/Elf+ (VLDB 2023) → ALP (SIGMOD 2024), which is ~49% better than Gorilla and ~43% better than Chimp. None of those have a usable Go implementation: ALP is C++ (cgo would break our A dependency-free alternative beats it. The classic byte-shuffle filter (transpose the 8 byte-planes of each float64 — what Blosc/HDF5/Parquet do) feeding our existing compressors, on 160 KB samples:
Shuffle+xz wins on both, needs no new dependency, is ~20 lines of pure Go, composes with every compressor ccat already has, and generalises to float32/int64/any fixed-width record via a So: not merging |
No description provided.