|
Goals
Test the new fast compressors Brotli
(Google) and Zstandard (Facebook) at different compression settings to
determine what compression level and compression / decompression speed
the two
standards are capable to offer as general purpose file compressors.
- Find out what is the
more efficient
algorithm between Brotli and Zstandard in terms of speed and
compression
ratio, at different compression settings ranging from minimum to
maximum. Which one is the fastest compressor / decompressor? Which ones
compresses better? Which algorithm scales better at different
compression levels?
- Compare Brotli and
Zstandard performances
with classic reference
compression formats as 7Z, RAR and ZIP at standard compression levels.
|
Software
settings
Benchmarks
are conducted
on Windows 10 2009 64 bit.
PeaZip 7.2.0 64 bit is tested for compression and decompression in
following formats:
- 7z 19.00 (64 bit)
- Brotli 1.0.7 (64 bit)
- Zstd 1.4.4 (64 bit)
The reference application for ZIP and RAR formats performances in this
test is WinRar 5.8 64 bit. |
Hardware
settings
Notebook with Intel Core i7-8565U CPU, 4
physical cores with hyper-threading (8 logical cores), 8 GB RAM
System disk 512 GB PCIe NVMe SSD, NTFS filesystem
|
|
|
|
Compression
formats compared in this benchmark
- 7Z file
format
Open Source archive format
by 7-Zip, providing high compression ratio, tested at default
compression level, default LZMA2 algorithm, from PeaZip
- Br (Brotli) compression format
created by Google for providing very fast
compression and decompression, tested from fastest to maximum
compression level from PeaZip
- RAR file
format
(RarLabs RAR5 revision) proprietary archive format providing
better
compression that ZIP using PPMd algorithm, tested
at default compression level from WinRar
- ZIP file
format
widely used archive format, tested at default compression setting (with
Deflate algorithm) from
WinRar, which qualified as the fastest zip compressor in previous batch
of PeaZip's compression
benchmarks
- Zst (Zstandard)
compression format created by Facebook, multithreaded Zstd
compressor,
providing very fast
compression and decompression, tested from fastest to maximum
compression level from PeaZip
|
Input data
Benchmark input contains 43 files in 4 directories for total 1.22 GB
(1,318,000,857 bytes), composed by well known reference files
representative of different data structures, widely used for
compression
benchmarks:
|
|
|
Benchmark methods
Benchmark
input data is saved to system disk (PCIe SSD) and compressed to system
disk, same partition, separate directory; the resulting archives are
then extracted to
separate directory on same (system) disk/partition. The use of a fast
disk support is meant to minimize the impact of disk speed on Brotli
and Zstandard performances.
Each compression and extraction test is repeated 10 times to get an
average value; size is expressed in MB, time in seconds.
Brotli and Zstandard requires to
consolidate the multiple input files of the benchmark into a single TAR
file, the overhead of tar / untar operations is about one second (about
one second performance penalty to tar, about one second to untar) and
needs to be taken in account when comparing with results of classic
compressors in 7Z, RAR and ZIP formats.
Following compression levels were tested:
- fastest, minimum compression supported by the
format, defined as 0 in Brotli, 1 in Zstd
- default, defined as 3 for both compressors
- high and very high compression levels, intermediate
settings between default and maximum, chosen as 5 and 7 for Brotli,
and as 9 and 15 for Zstd
- maximum compression level, defined as 9 for Brotli
and 19 for Zstd
As current
implementation of Zstandard is multithreaded while Brotli is
not, the same test were conducted also on Zstd (Zstandard) compressor
with -T1 option which disables multithreading, to provide also a
meaningful comparison on the single thread. PeaZip by default uses
the -T0 option enabling automatic multithreading using all suitable CPU
resources.
As Zstandard window size is
variable, depending on the compression
level, while Brotli uses fixed window size, a second test batch was
conducted in order to compare the two algorithms in closer conditions -
while the first batch of test aims to a simple out-of-the-box
performances comparison.
In this second test, window size
was fixed to 128 MB (2^27 bytes) for
both
algorithms, with --large_window=27 option for Brotli, and --long=27 for
Zstd, which also runs this test series single threaded with -T1
parameter.
Having the two algorithms running with same windows size and same
number of threads allows to better understand efficiency of Brotli and
Zstandard on the basis of reasonably similar cost of computing
resources, and choosing a such large window size challenges the ability
of the algorithms in preserving speed performances and efficiently
scale the compression ratio.
No extra
compression settings were used to fine tune the compression,
neither for Brotli nor for Zstandard, compression was defined
exclusively setting the compression level parameter. Both algorithms
features plenty fine-tuning options, which are out of the scope of this
benchmark.
For classic file archiving formats ZIP, RAR, and 7Z, it was tested only
the default compression level, to highlight typical compression / speed
tradeoff commonly offered out of the box. Testing the entire
performance spectrum of those algorithms against Brotli and Zstandard
is out of the scope of the present analysis, but is a possible future
topic of interest.
Brotli vs
Zstandard
benchmark results
tables
Out-of-the-box compression and extraction times in
seconds, size of
resulting archive in MB, the lower the better for all columns.
|
Brotli
|
Zstandard
|
Compression level
|
Compression
time (sec)
|
Extraction
time (sec)
|
Archive
size (MB) |
Compression
-T0 (sec)
|
Compression
single thread -T1 (sec)
|
Extraction
time (sec)
|
Archive
size (MB) |
Minimum
Brotli -0 / Zstd -1
|
5.4
|
5.0
|
473
|
1.9
|
4.6
|
2.6
|
451
|
Default
Brotli -3 / Zstd -3
|
15.0
|
4.5
|
408
|
3.6
|
7.5
|
3.1
|
398
|
High
Brotli -5 / Zstd -9
|
46.9
|
4.6
|
353
|
38.4
|
60.0
|
3.2
|
355
|
Very
high
Brotli -7 / Zstd -15
|
175.0
|
4.7
|
336
|
157.0
|
461.0
|
3.3
|
330
|
Maximum
Brotli -9 / Zstd -19
|
677.0
|
5.7
|
322
|
359.0
|
1030.0
|
4.0
|
302
|
Fixed 128 MB window size
for both algorithms:
|
Brotli
--large_window=27
|
Zstandard
-T1 --long=27
|
Compression level
|
Compression
time (sec)
|
Extraction
time (sec)
|
Archive
size (MB) |
Compression
-T0 (sec)
|
Extraction
time (sec)
|
Archive
size (MB) |
Minimum
Brotli -0 / Zstd -1
|
5.7
|
5.0
|
473
|
8.7
|
2.8
|
407
|
Default
Brotli -3 / Zstd -3
|
16.8
|
4.1
|
373
|
12.3
|
3.1
|
362
|
High
Brotli -5 / Zstd -9
|
52.3
|
4.2
|
322
|
63.0
|
3.2
|
321
|
Very
high
Brotli -7 / Zstd -15
|
162.0
|
4.4
|
307
|
461.0
|
3.7
|
295
|
Maximum
Brotli -9 / Zstd -19
|
721.0
|
5.4
|
293
|
1037.0
|
3.5
|
273
|
For reference, results of traditional compressors group, at default
compression settings:
Compression format
|
Compression
time (sec)
|
Extraction
time (sec)
|
Archive
size (MB) |
ZIP
WinRar, default
|
15.6
|
6.0
|
407
|
RAR
WinRar, default
|
100.0
|
4.0
|
318
|
7Z
PeaZip, default
|
241
|
3.2
|
293
|
Compression
speed benchmark
results
The
curve of the performances for the two algorithms shows that, out of the
box, Zstandard is
generally faster and provides better compression ratio than Brotli.
Only in the point of high compression level (-5 for Brotli, -9 for
Zstd) Brotli provides a marginally better compression ratio (353 MB
versus 355 MB) and the speed advantage of Zstandard is reduced.
For minimum and default compression settings,
Zstandard is both faster
and compresses better than Brotli, this hold true running Zstd
single-threaded (-T1
parameter).
For very
high and maximum compression settings,
Zstandard again
provides better compression ratio in less time than Brotli, but at
top compression levels multi-threading is clearly a key factor for
Zstd performance. Single threaded compression times, comparatively, are
significantly worse than Brotli ones.
Maximum compression ratio reached with Zstandard is significantly
better than Brotli one, with 302 MB (24.04%) vs 322 MB (25.64%).
For reference, fastest implementation of fastest traditional
compression format (ZIP in
WinRar, at default compression level)
completed the test with a result strikingly similar to Brotli at
default level
(-3): 15.6 sec for zip compression vs 15 seconds, 16 taking in account
adding files to tar beforehand, for Brotli, with very similar output
size (408 for
Brotli vs 407 for ZIP).
However, at default compression level Zstd outperformed Brotli and Zip,
providing a better compression (398 MB) nearly four time as fast - or
nearly twice as fast when launched single-threaded.
RAR and 7Z, at default
compression level, reached compression ratios comparable or better than
top ones provided by Brotli and Zstandard ar maximum compression level
- RAR slightly better than Brotli but worse than Zstandard (318 MB vs
respectively 322 and 302 MB), 7Z reaching best compression with 293 MB
otput archive.
Also, notwithstanding the similar compression ratio provided,
compression times are shorter for Rar and 7z at default compression
level than Brotli / Zstandard at maximum compression level, proving the
algorithms in use being less efficient that PPMd and LZMA2 for working
at those levels of compression.
Compression
speed with same window size
The second batch of tests, running with window size fixed to 2^17 (128
MB) for both algorithms, shows a sizeable improvement of compression
ratio for both Brotli and Zstd.
The decrease in speed is moderate for both algorithms, except for
Zstandard at fast compression levels which is significantly slower with
128 MB wide window than with automatically size window size.
Comparing Brotli and Zstd with same window size, and both running
single-threaded, shows Zstandard providing a better compression at each
compression level, being faster than Brotli al default, but not at
minimum, compression level and slower at higher compression
levels.
Performance differences are more evident at extremes (faster and slower
compression levels), while at an intermediate point of the graph (
Brotli -5 / Zstd -9) both speed and compression ratio differences are
less pronounced - similarly that in the previous batch of tests.
With a 128 MB window size both Brotli and Zstandard fares better in
comparison of classic compression formats
Zstd -1 provides same compression of ZIP (default) in half the time,
and Brotli -5 and Zstd -9 come close to RAR (default) compression in
slightly more than half the time: those numbers shows both algorithms
have excellent potential for fast to intermediate compression.
Zstd -15 and Brotli -9 rivals 7Z at default compression level, and Zstd
-19 compresses the output to 20 MB smaller size, but both algorithms
are significantly slower confirming not being optimal choices for high
compression tasks.
Brotli / Zstandard
comparative at fastest
compression settings
Focusing
on fastest and default compression levels, clearly the most
interesting settings for projects targeted to high performances, the
benchmark shows a sizeable advantage for Zstandard in terms of
compression ratio and speed, over Brotli.
With fastest compression settings, multithreading (while still
important) is
not a critical factor for Zstd speed performances, as even single
threaded Zstandard completes the test faster than Brotli.
This is the
opposite of what seen at higher compression levels in
previous chart, where Zstd still provides better
compression but at cost of more computing power - being significantly
slower than Brotli when running single threaded.
Using
the same window size for both algorithms, 128 MB, and running them
single threaded, Zstandard at minimum compression level provides a
consistent advantage over Brotli (407 MB output vs 473 MB) but is 3
second slower.
At default compression level, Zstandard is both faster and compresses
better than Brotli.
At the first high compression setting (Brotli -5 vs Zstd -9) the
compression ratio performances are very close (with a slight output
size advantage for Zstd), but Zstandard is slower than Brotli, with the
trend continuing on this side of the graph with single threaded Zstd
being slower than Brotli with same window size, but providing higher
compression.
Extraction speed
comparison
Extraction speed
slightly slows as compression level increase (with the exception of
Brotli at minimum compression taking half a second more than at default
level) but generally - unlike the compression speed - it remains
remarkably
constant from lower to highest compression levels.
Extraction from classic archive formats highlights 7Z as the format
allowing fastest decompression, followed by RAR and then ZIP: and both
Zstandard and Brotli always remains
competitive for decompression speeds with the ones of mainstream
archiving formats, even taking in
account the one second overhead penalty to untar the content after
decompressing the .br / .zst archives.
Comparing
Brotli and Zstandard extraction speed, Zstandard
provides the
best decompression performances for every compression level,
being
uniformly about 40% faster than Brotli- except for extracting the file
compressed
at minimum compression level, where Zstd is about twice as fast as
Brotli.
For Brotli, decompression of archives created with fixed 128 MB window
size is constantly slightly faster for each compression level
(excluding for the
minimum), than decompression of standard archives.
For Zstd, extraction times remains quite similar for the two series of
tests, standard archives and ones created with 128 MB fixed window
size,
without one type showing a clear advantage, unlike for Brotli. A slight
trend can be inferred of extraction times of second group being
comparatively slower for lower compression and faster for higher
compression levels.
|
Brotli, Zstandard
performances comparison
Zstandard outperforms
Brotli at minimum and standard compression levels, which are
likely the most useful range for fast compressors aiming to operate in
real-time. Both speed and compression ratio results are better for
Zstandard in this range of the benchmark.
The speed advantage is reduced, but still very noticeable, when
comparing the two current implementations of Brotli and Zstd running
single threaded.
Increasing the
compression level, Zstandard algorithm exhibits more
flexibility allowing to reach better compression ratio than Brotli, but
this come with severe performance penalty that makes single-threaded
Zstd slower than Brotli.
Running Zstd with multi threading option enabled more than makes up for
the difference, allowing Zstd to run faster than Brotli in this range
of the benchmark, this of course means allocating more computing
resources to Zstandard: while on a desktop system this is a positive
factor, on a server needing to allocate resources for multiple
concurrent instances, the advantage of Brotli in completing the high
compression tests faster per thread may be preferable.
The lack of a multi-threaded implementation of Brotli for this
benchmark
limits the ability to compare the two algorithms more closely.
Running
a second batch of the test single
threaded and with same window
size for both algorithms (128 MB in current benchmark) confirms
the trend already seen in the previous test batch.
Using same window size Zstandard provides
better compression results at lower compression levels, operating
faster at comparable compression ratio - even if Brotli is faster at
minimum compression setting at cost of consistently reduced compression
ratio.
The performances levels
out at an average level (Brotli -5, Zstd -9), and then, at higher
compression levels, Brotli is faster than single-threaded Zstandard
which in turn provides better compression.
Decompression is always
faster for Zstandard than for Brotli, with a consistent
advantage (around 40%) in the entire range of benchmarks.
Both algorithms exhibit the capability to keep the extraction time
reasonably constant decompressing files created ranging from minimum to
maximum compression levels.
ADDENDUM: in the Maximum
compression benchmark it is now available a direct
comparison of Brotli and Zstandard at maximum compression settings, in terms of compression ratio and
compression / decompression speed.
Conclusions in
brief:
Which is the fastest
compressor, Brotli or Zstd?
Zstd is overall faster, but
it is due to its multi-threading
capabilities that Brotli lacks.
Running in a single thread, Zstandard is faster for low compression
levels, slower at high compression levels.
Which is the faster
decompressor between Brotli and Zstd?
Zstandard decompression is
always faster than Brotli ones.
Which format
provides best compression, Brotli BR or Zstandard ZST?
Zstandard compression ratio
is consistently better than Brotli ones,
looking at the curves of speed / ratio performances tradeoff the two
algorithms provides quite overlapping results only at an intermediate
point
(Brotli -5 / Zstd -9) otherwise ZST-compressed files are significantly
smaller than BR-compressed files in all remaining points.
With a large window size, fixed at 128 MB for both algorithms,
compression is clearly improved for both algorithms, but the curves of
speed / compression ratio shows the same dynamics.
Which
algorithm scales
better at different compression levels, Brotli or Zstd?
Zstandard scales better on
the entire spectrum of results, providing
either better and faster compression
at minimum / default settings, and better maximum compression ratio at
maximum settings, but the latter advantage comes at expense of more a
more computing intensive data compression strategy.
With both algorithms using same window size (128 MB) and running in a
single thread, Brotli provides a minimum compression faster that
Zstandard (but inferior for compression ratio) and Zstandard provides a
better maximum compression (but slower).
Syntetic test results
by compression level:
Zstd
fastest compression
settings compared with Brotli
This
is a clear win for Zstd, providing 22 MB smaller output in less than
half the time, speed advantage is consistently reduced in single
thread mode.
With same window size (128 MB), Brotli surpasses Zstandard in
single-threaded speed, but output size advantage for Zstd increases to
66 MB.
Zstd default
compression settings compared with Brotli
This
use case shows a consistent advantage for Zstandard, creating a 10 MB
smaller output in a quarter of the time needed by Brotly to complete
same test, and still twice faster running in a single thread. Using
same window size for the two algorithms, both size and speed advantages
for Zstandard are reduced.
Zstd maximum
compression settings compared with Brotli
Zstandard
beats Brotli for maximum compression by a 20 MB margin, comparing the
speed of the two algorithms in this case Brotli (which is single
threaded) is slower if compared with multi-threaded Zstd result, but
faster if compared with single threaded result.
With a fixed 128 window size for both algorithms, Zstd preserve its 20
MB difference on output size, and while Brotli remains faster its speed
advantage is proportionally reduced over Zstandard.
Brotli and
Zstandard
compared to 7Z LZMA2, RAR PPMd, ZIP Deflate
Compared with
classic Zip format,
Brotli at standard level operated at compression / extraction speed,
and compression ratio very similar to fastest Zip implementation,
taking in account tar/untar overhead to operate as general purpose
archiver.
Zstandard at
same compression level shows a clear advantage in all
those aspects over Brotli and Zip / Deflate.
Comparison with Rar format
shows neither Brotli nor Zstandard are specialized for operating at
such high compression ratio: RAR operating with PPMd at default
compression level completed the test in times that are intermediate to
ones of high and very high level for both algorithms, but results are
comparable with those at the much slower maximum compression setting.
Zstandard at maximum
compression level is still capable of beating Rar
PPMd compression ratio at default level, but being 3.5 times slower,
showing it is not optimized for this usage.
7z format at
default level outperforms maximum compression attainable with both
Brotli and Zstandard at maximum level, while remaining significantly
faster, showing its LZMA2 algorithm it is more fit for high compression
tasks. LZMA2 scores the smallest output size in this test, and the
result is not matched even by other classic compressors.
Brotli and Zstandard are not a match for RAR PPMd or 7Z LZMA2 in the
range
of strong data compression, even increasing window size to 128 MB,
providing a poor tradeoff between
compression gain and speed decrease when running at highest compression
settings: even if very flexible and efficient, quick data compression
remains the primary field of application for both algorithms.
Bottom line of the comparison with classic file archiving formats is
that
Brotli and Zstandard are excellent choices for fast / very fast
compression,
and as general purpose archiving format are viable alternatives to Zip
in terms of speed / compression tradeoff.
Of the two, Zstandard clearly
stands off for this case of use, surpassing Deflate and Brotli in
efficiency even when single-threaded.
Increasing windows size to 128 MB, fixed size, quite large for those
algorithms,
does not excessively impact on speed and in turn offers welcome
improvements in compression ratio for both functions, with Brotli -5
and Zstd -9 providing results almost comparable to RAR (default)
compression in significantly faster times, showing excellent potential
for tasks requiring intermediate
compression - but still not competitive for high compression
tasks.
Read more about how to create
Brotli compressed files, how to create Zstandard compressed
files, how to use Brotli and
Zstandard compression in 7z archives, and how to optimize file
compression speed.
Synopsis: Fast compression
benchmark. Comparison of Brotli versus Zstandard formats
performances at different compression settings. BR vs ZST compression
and extraction speed comparative
test. Comparison of Brotli and Zstandard speed with 7Z LZMA2, RAR PPMd,
and ZIP Deflate algorithms for fastest compressor and decompressor.
Topics: fast compression
benchmark, Brotli vs Zstandard benchmarks results, decompression speed
test
PeaZip > Compression
benchmark > Fast compression: Brotli, Zstandard comparative speed,
performances test
|
|
