Skip to content
forked from inikep/lizard

Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.

License

Notifications You must be signed in to change notification settings

mayhemheroes/lizard

 
 

Repository files navigation

Lizard - efficient compression with very fast decompression

Lizard (formerly LZ5) is a lossless compression algorithm which contains 4 compression methods:

  • fastLZ4 : compression levels -10...-19 are designed to give fast decompression speed
  • LIZv1 : compression levels -20...-29 are designed to give better compression ratio keeping 75% of decompression speed
  • fastLZ4 + Huffman : compression levels -30...-39 add Huffman coding to fastLZ4
  • LIZv1 + Huffman : compression levels -40...-49 give the best ratio (comparable to zlib and low levels of zstd/brotli)

Lizard library is based on frequently used LZ4 library by Yann Collet but the Lizard compression format is not compatible with LZ4. Lizard library is provided as open-source software using BSD 2-Clause license. The high compression/decompression speed is achieved without any SSE and AVX extensions.

Status
Build Status

2025 Update

Back in 2017, Lizard 1.0 filled a niche between LZ4 and zstd in terms of compression ratio, as well as compression and decompression speed. However, based on benchmarks conducted in 2025 (shared below), both LZ4 and zstd have made substantial advancements, particularly in compression and decompression speed, leaving Lizard outdated and effectively obsolete.

Benchmarks

The following results were obtained using lzbench 2.0, built with gcc 14.2.0 and executed with the options -t16,16 -o1c4 -elz4/lz4hc/lizard/zstd_fast/zstd/zlib/brotli. The tests were run on a single core of an AMD EPYC 9554 processor at 3.10 GHz, with the CPU governor set to performance and turbo boost disabled for stability. The operating system was Ubuntu 24.04.1, and the benchmark made use of silesia.tar, which contains tarred files from the Silesia compression corpus.

Compressor name Compression Decompress. Compr. size Ratio
memcpy 16253 MB/s 16151 MB/s 211947520 100.00
lz4 1.10.0 579 MB/s 3728 MB/s 100880800 47.60
lz4hc 1.10.0 -1 265 MB/s 3225 MB/s 89135429 42.06
lz4hc 1.10.0 -2 265 MB/s 3227 MB/s 89135429 42.06
lz4hc 1.10.0 -3 92.7 MB/s 3373 MB/s 81342421 38.38
lz4hc 1.10.0 -4 76.6 MB/s 3429 MB/s 79807909 37.65
lz4hc 1.10.0 -5 62.6 MB/s 3474 MB/s 78895329 37.22
lz4hc 1.10.0 -6 51.4 MB/s 3494 MB/s 78385708 36.98
lz4hc 1.10.0 -7 42.7 MB/s 3511 MB/s 78102452 36.85
lz4hc 1.10.0 -8 36.1 MB/s 3525 MB/s 77957479 36.78
lz4hc 1.10.0 -9 31.0 MB/s 3531 MB/s 77884448 36.75
lz4hc 1.10.0 -10 21.4 MB/s 3604 MB/s 77596766 36.61
lz4hc 1.10.0 -11 12.5 MB/s 3630 MB/s 77315480 36.48
lz4hc 1.10.0 -12 10.5 MB/s 3622 MB/s 77262620 36.45
lizard 2.1 -10 484 MB/s 2161 MB/s 103402971 48.79
lizard 2.1 -11 316 MB/s 1935 MB/s 93861621 44.29
lizard 2.1 -12 166 MB/s 2005 MB/s 86232422 40.69
lizard 2.1 -13 102 MB/s 2040 MB/s 83773119 39.53
lizard 2.1 -14 90.3 MB/s 2082 MB/s 82205976 38.79
lizard 2.1 -15 77.8 MB/s 2106 MB/s 81187330 38.31
lizard 2.1 -16 56.0 MB/s 1951 MB/s 79372512 37.45
lizard 2.1 -17 26.5 MB/s 1986 MB/s 78041714 36.82
lizard 2.1 -18 6.67 MB/s 1978 MB/s 77586984 36.61
lizard 2.1 -19 4.81 MB/s 1989 MB/s 77416400 36.53
lizard 2.1 -20 388 MB/s 1674 MB/s 96924204 45.73
lizard 2.1 -21 229 MB/s 1748 MB/s 89239174 42.10
lizard 2.1 -22 162 MB/s 1709 MB/s 84866725 40.04
lizard 2.1 -23 64.5 MB/s 1769 MB/s 81052209 38.24
lizard 2.1 -24 36.8 MB/s 1792 MB/s 78170875 36.88
lizard 2.1 -25 22.3 MB/s 1748 MB/s 75131286 35.45
lizard 2.1 -26 6.33 MB/s 1767 MB/s 72459161 34.19
lizard 2.1 -27 4.46 MB/s 1813 MB/s 70447615 33.24
lizard 2.1 -28 2.43 MB/s 1821 MB/s 69762972 32.92
lizard 2.1 -29 2.13 MB/s 1818 MB/s 68694227 32.41
lizard 2.1 -30 363 MB/s 1185 MB/s 85727429 40.45
lizard 2.1 -31 263 MB/s 1325 MB/s 81688522 38.54
lizard 2.1 -32 164 MB/s 1274 MB/s 78652654 37.11
lizard 2.1 -33 151 MB/s 1410 MB/s 76929454 36.30
lizard 2.1 -34 96.6 MB/s 1477 MB/s 75427930 35.59
lizard 2.1 -35 85.7 MB/s 1542 MB/s 74563583 35.18
lizard 2.1 -36 74.7 MB/s 1573 MB/s 73850400 34.84
lizard 2.1 -37 54.5 MB/s 1521 MB/s 71854743 33.90
lizard 2.1 -38 26.2 MB/s 1565 MB/s 70968686 33.48
lizard 2.1 -39 4.66 MB/s 1509 MB/s 69807522 32.94
lizard 2.1 -40 297 MB/s 1141 MB/s 80843049 38.14
lizard 2.1 -41 192 MB/s 1202 MB/s 76100661 35.91
lizard 2.1 -42 142 MB/s 1231 MB/s 73350988 34.61
lizard 2.1 -43 59.7 MB/s 1325 MB/s 70927858 33.46
lizard 2.1 -44 34.4 MB/s 1353 MB/s 68763512 32.44
lizard 2.1 -45 21.7 MB/s 1345 MB/s 66676653 31.46
lizard 2.1 -46 11.4 MB/s 1201 MB/s 65413061 30.86
lizard 2.1 -47 6.07 MB/s 1271 MB/s 63704146 30.06
lizard 2.1 -48 4.31 MB/s 1297 MB/s 62097965 29.30
lizard 2.1 -49 2.08 MB/s 1316 MB/s 60679215 28.63
zstd_fast 1.5.6 --5 573 MB/s 1948 MB/s 103093752 48.64
zstd_fast 1.5.6 --4 542 MB/s 1905 MB/s 98825267 46.63
zstd_fast 1.5.6 --3 518 MB/s 1818 MB/s 94674672 44.67
zstd_fast 1.5.6 --2 489 MB/s 1773 MB/s 90474358 42.69
zstd_fast 1.5.6 --1 459 MB/s 1716 MB/s 86984009 41.04
zstd 1.5.6 -1 421 MB/s 1342 MB/s 73421914 34.64
zstd 1.5.6 -2 344 MB/s 1241 MB/s 69503444 32.79
zstd 1.5.6 -3 250 MB/s 1210 MB/s 66523842 31.39
zstd 1.5.6 -4 214 MB/s 1192 MB/s 65323637 30.82
zstd 1.5.6 -5 125 MB/s 1194 MB/s 63040310 29.74
zstd 1.5.6 -6 91.7 MB/s 1272 MB/s 61540426 29.04
zstd 1.5.6 -7 80.7 MB/s 1283 MB/s 60546558 28.57
zstd 1.5.6 -8 64.1 MB/s 1316 MB/s 60015064 28.32
zstd 1.5.6 -9 61.9 MB/s 1306 MB/s 59375344 28.01
zstd 1.5.6 -10 48.0 MB/s 1323 MB/s 58675464 27.68
zstd 1.5.6 -11 34.5 MB/s 1339 MB/s 58262299 27.49
zstd 1.5.6 -12 31.8 MB/s 1339 MB/s 58206982 27.46
zstd 1.5.6 -13 14.2 MB/s 1343 MB/s 57986550 27.36
zstd 1.5.6 -14 11.3 MB/s 1360 MB/s 57575723 27.17
zstd 1.5.6 -15 8.39 MB/s 1368 MB/s 57168834 26.97
zstd 1.5.6 -16 6.13 MB/s 1255 MB/s 55304789 26.09
zstd 1.5.6 -17 4.76 MB/s 1226 MB/s 54243339 25.59
zstd 1.5.6 -18 3.76 MB/s 1164 MB/s 53329873 25.16
zstd 1.5.6 -19 2.98 MB/s 1147 MB/s 52875116 24.95
zstd 1.5.6 -20 2.73 MB/s 1079 MB/s 52456743 24.75
zstd 1.5.6 -21 2.40 MB/s 1075 MB/s 52344203 24.70
zstd 1.5.6 -22 2.09 MB/s 1071 MB/s 52333880 24.69
zlib 1.3.1 -1 93.2 MB/s 323 MB/s 77259029 36.45
zlib 1.3.1 -2 83.4 MB/s 333 MB/s 75002277 35.39
zlib 1.3.1 -3 60.3 MB/s 344 MB/s 72967040 34.43
zlib 1.3.1 -4 58.4 MB/s 334 MB/s 71002817 33.50
zlib 1.3.1 -5 39.2 MB/s 337 MB/s 69161685 32.63
zlib 1.3.1 -6 25.3 MB/s 344 MB/s 68228431 32.19
zlib 1.3.1 -7 20.3 MB/s 345 MB/s 67939647 32.05
zlib 1.3.1 -8 13.1 MB/s 347 MB/s 67693427 31.94
zlib 1.3.1 -9 10.3 MB/s 348 MB/s 67644548 31.92
brotli 1.1.0 -0 341 MB/s 352 MB/s 78433298 37.01
brotli 1.1.0 -1 234 MB/s 379 MB/s 73499222 34.68
brotli 1.1.0 -2 139 MB/s 415 MB/s 68069489 32.12
brotli 1.1.0 -3 116 MB/s 429 MB/s 67392492 31.80
brotli 1.1.0 -4 80.3 MB/s 452 MB/s 64122989 30.25
brotli 1.1.0 -5 36.9 MB/s 450 MB/s 59555446 28.10
brotli 1.1.0 -6 26.9 MB/s 457 MB/s 58470465 27.59
brotli 1.1.0 -7 17.5 MB/s 472 MB/s 57688191 27.22
brotli 1.1.0 -8 12.4 MB/s 476 MB/s 57148304 26.96
brotli 1.1.0 -9 9.19 MB/s 480 MB/s 56703946 26.75
brotli 1.1.0 -10 1.25 MB/s 345 MB/s 51720800 24.40
brotli 1.1.0 -11 0.58 MB/s 387 MB/s 50407795 23.78

Automatic CI/CD testing

lizard undergoes automated testing using Azure Pipelines with the following compilers:

  • Ubuntu: gcc (versions 7.5 to 14.2) and clang (versions 6.0 to 19), gcc 14.2 (32-bit)
  • MacOS: Apple LLVM version 15.0.0
  • Windows: mingw32-gcc 14.2.0 (32-bit) and mingw64-gcc 14.2.0 (64-bit)
  • Cross-compilation: gcc for ARM (32-bit and 64-bit) and PowerPC (32-bit and 64-bit)
  • Visual Studio 2019 / 2022 (32-bit and 64-bit)

Documentation

The raw Lizard block compression format is detailed within lizard_Block_format.

To compress an arbitrarily long file or data stream, multiple blocks are required. Organizing these blocks and providing a common header format to handle their content is the purpose of the Frame format, defined into lizard_Frame_format. Interoperable versions of Lizard must respect this frame format.

About

Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages

  • C 94.7%
  • Makefile 4.1%
  • Other 1.2%