ckolivas/lrzip
1
0
Fork 0
mirror of https://github.com/ckolivas/lrzip.git synced 2026-04-04 13:57:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Updated benchmark results.

Browse source 
More tidying up.
This commit is contained in:
Con Kolivas 2010-11-05 14:52:14 +11:00
parent 296534921a
commit a66dafe66a
5 changed files with 73 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

74
doc/README.benchmarks
View file

@ -1,5 +1,3 @@
These are benchmarks performed on a 3GHz quad core Intel Core2 with 8GB ram
using lrzip v0.42.
The first comparison is that of a linux kernel tarball (2.6.31). In all cases
the default options were used. 3 other common compression apps were used for
@ -7,22 +5,25 @@ comparison, 7z which is an excellent all-round lzma based compression app,
gzip which is the benchmark fast standard that has good compression, and bzip2
which is the most common linux used compression.
In the following tables, lrzip means lrzip default options, lrzip(lzo) means
lrzip using the lzo backend, lrzip(gzip) means using the gzip backend,
lrzip(bzip2) means using the bzip2 backend and lrzip(zpaq) means using the zpaq
In the following tables, lrzip means lrzip default options, lrzip -l means
lrzip using the lzo backend, lrzip -g means using the gzip backend,
lrzip -b means using the bzip2 backend and lrzip -z means using the zpaq
backend.
linux-2.6.31.tar
These are benchmarks performed on a 3GHz quad core Intel Core2 with 8GB ram
using lrzip v0.42.
Compression Size Percentage Compress Decompress
None 365711360 100
7z 53315279 14.6 2m4.770s 0m5.360s
lrzip 52372722 14.3 2m48.477s 0m8.336s
lrzip(zpaq) 43455498 11.9 10m11.335 10m14.296
lrzip(lzo) 112151676 30.7 0m14.913s 0m5.063s
lrzip(gzip) 73476127 20.1 0m29.628s 0m5.591s
lrzip(bzip2) 60851152 16.6 0m43.539s 0m12.244s
lrzip -z 43455498 11.9 10m11.335 10m14.296
lrzip -l 112151676 30.7 0m14.913s 0m5.063s
lrzip -g 73476127 20.1 0m29.628s 0m5.591s
lrzip -b 60851152 16.6 0m43.539s 0m12.244s
bzip2 62416571 17.1 0m44.493s 0m9.819s
gzip 80563601 22.0 0m14.343s 0m2.781s
@ -37,29 +38,34 @@ What lrzip offers at this end of the spectrum is extreme compression if
desired.
Let's take two kernel trees one version apart as a tarball, linux-2.6.31 and
linux-2.6.32-rc8. These will show lots of redundant information, but hundreds
Let's take six kernel trees one version apart as a tarball, linux-2.6.31 to
linux-2.6.36. These will show lots of redundant information, but hundreds
of megabytes apart, which lrzip will be very good at compressing. For
simplicity, only 7z will be compared since that's by far the best general
purpose compressor at the moment:
These are benchmarks performed on a 2.53Ghz dual core Intel Core2 with 4GB ram
using lrzip v0.5.1. Note that it was running with a 32 bit userspace so only
2GB addressing was posible. However the benchmark was run with the -U option
allowing the whole file to be treated as one large compression window.
Tarball of two kernel trees, one version apart.
Tarball of 6 consecutive kernel trees.
Compression Size Percentage Compress Decompress
None 749066240 100
7z 108710624 14.5 4m4.260s 0m11.133s
lrzip 57943094 7.7 3m08.788s 0m10.747s
lrzip(lzo) 124029899 16.6 0m18.997s 0m7.107s
None 2373713920 100
7z 344088002 14.5 17m26s 1m22s
lrzip -U 73356070 3.1 08m53s 43s
lrzip -Ul 158851141 6.7 04m31s 35s
Things start getting very interesting now when lrzip is really starting to
shine. Note how it's not that much larger for 2 kernel trees than it was for
shine. Note how it's not that much larger for 6 kernel trees than it was for
one. That's because all the similar data in both kernel trees is being
compressed as one copy and only the differences really make up the extra size.
All compression software does this, but not over such large distances. If you
copy the same data over multiple times, the resulting lrzip archive doesn't
get much larger at all.
Using the first example (linux-2.6.31.tar) and simply copying the data multiple
times over gives these results with lrzip(lzo):
@ -70,7 +76,7 @@ Copies Size Compressed Compress Decompress
I had the amusing thought that this compression software could be used as a
bullshit detector if you were to compress peoples' speeches because if their
bullshit detector if you were to compress people's speeches because if their
talks were full of catchphrases and not much actual content, it would all be
compressed down. So the larger the final archive, the less bullshit =)
@ -83,31 +89,31 @@ system and some basic working software on it. The default options on the
10GB Virtual image:
These benchmarks were done on the quad core with version 0.5.1
Compression Size Percentage Compress Time Decompress Time
None 10737418240 100.0
gzip 2772899756 25.8 05m47.35s 2m46.77s
bzip2 2704781700 25.2 20m34.269s 7m51.362s
xz 2272322208 21.2 58m26.829s 4m46.154s
7z 2242897134 20.9 29m28.152s 6m35.952s
lrzip* 1354237684 12.6 29m13.402s 6m55.441s
lrzip M* 1079528708 10.1 23m44.226s 4m05.461s
lrzip(lzo)* 1793312108 16.7 05m13.246s 3m12.886s
lrzip(lzo)M* 1413268368 13.2 04m18.338s 2m54.650s
lrzip(zpaq)* 1299844906 12.1 04h32m14s 04h33m
lrzip(zpaq)M* 1066902006 9.9 04h07m14s 04h08m
lrzip 1354237684 12.6 29m13.402s 6m55.441s
lrzip -M 1079528708 10.1 23m44.226s 4m05.461s
lrzip -l 1793312108 16.7 05m13.246s 3m12.886s
lrzip -lM 1413268368 13.2 04m18.338s 2m54.650s
lrzip -z 1299844906 12.1 04h32m14s 04h33m
lrzip -zM 1066902006 9.9 04h07m14s 04h08m
(The benchmarks with * were done with version 0.5.1)
At this end of the spectrum things really start to heat up. The compression
advantage is massive, with the lzo backend even giving much better results than
7z, and over a ridiculously short time. Note that it's not much longer than it
takes to just *read* a 10GB file. What appears to be a big disappointment is
actually zpaq here which takes more than 8 times longer than lzma for a measly
.2% improvement. The reason is that most of the advantage here is achieved by
the rzip first stage since there's a lot of redundant space over huge distances
on a virtual image. The -M option which works the memory subsystem rather hard
making noticeable impact on the rest of the machine also does further wonders
for the compression and times.
7z, and over a ridiculously short time. What appears to be a big disappointment
is actually zpaq here which takes more than 8 times longer than lzma for a
measly .2% improvement. The reason is that most of the advantage here is
achieved by the rzip first stage since there's a lot of redundant space over
huge distances on a virtual image. The -M option which works the memory
subsystem rather hard making noticeable impact on the rest of the machine also
does further wonders for the compression and times.
This should help govern what compression you choose. Small files are nicely
compressed with zpaq. Intermediate files are nicely compressed with lzma.
@ -117,4 +123,4 @@ Or, to make things easier, just use the default settings all the time and be
happy as lzma gives good results. :D
Con Kolivas
Tue, 4th Nov 2010
Tue, 5th Nov 2010