LZ4 Extreme Speed Compression

LZ4 is a lossless compression algorithm, providing compression speed greater than 500 MB/s per core, with an additional decompression speed that is multiple times faster, measured at over 2 GB/s per core. Created by Yann Collet in 2011, LZ4 has over 10,000 stars as of 2026 and is used in production by Linux kernel (Btrfs, ZFS, SquashFS), Hadoop, Kafka, MySQL, MongoDB, PostgreSQL (TOAST compression), and Redis. Key features include: extreme compression speed (over 500 MB/s per core on modern CPUs, faster than memcpy on some architectures), extreme decompression speed (over 2 GB/s per core, the fastest decompressor available), LZ4 HC mode (High Compression variant trading speed for better ratios, still at 50-100 MB/s compression but achieving 30-50 percent better compression than fast mode), block and frame format (LZ4 Block for raw compression within applications, and LZ4 Frame format with magic numbers, content size, checksums, and dictionary support for interoperable compressed files), streaming API (incremental compression and decompression of data streams of arbitrary size without needing the entire input in memory), dictionary compression (pre-loading a shared dictionary for improved ratio on small inputs, used in Zstandard and high-frequency data patterns), zero-allocation path (decompression without dynamic memory allocation for embedded and real-time systems), branchless decode (the decompressor uses minimal branches per byte, enabling CPU pipeline efficiency), multi-threaded compression (frame format supporting multi-threaded compression with independent blocks), memory efficiency (minimal memory overhead with configurable table sizes from 4 KB to 64 KB), and cross-platform (C89 compatible, running on x86, ARM, MIPS, WebAssembly, and all major operating systems with bindings for Python, Java, Go, Rust, and Node.js).