ckolivas/lrzip
1
0
Fork 0
mirror of https://github.com/ckolivas/lrzip.git synced 2026-04-04 13:57:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Initial import

Browse source
This commit is contained in:
Con Kolivas 2010-03-29 10:07:08 +11:00
parent 725e478e19
commit 6dcceb0b1b
69 changed files with 26485 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

83
man/lrunzip.1.pod Normal file
View file

@ -0,0 +1,83 @@
# Copyright
#
# Copyright (C) 2004-2009 Jari Aalto
#
# License
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Description
#
# To learn what TOP LEVEL section to use in manual pages,
# see POSIX/Susv standard and "tility Description Defaults" at
# http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap01.html#tag_01_11
#
# This is manual page in Perl POD format. Read more at
# http://perldoc.perl.org/perlpod.html or run command:
#
# perldoc perlpod | less
#
# To check the syntax:
#
# podchecker *.pod
#
# Create manual page with command:
#
# pod2man PAGE.N.pod > PAGE.N
=pod
=head1 NAME
lrunzip - Uncompress LRZ files
=head1 SYNOPSIS
lrztar [options] FILE [... FILE]
=head1 DESCRIPTION
lrunzip is identical to C<lrzip -d> used to decompress files.
=head1 OPTIONS
See lrzip(1).
=head1 ENVIRONMENT
None.
=head1 FILES
None.
=head1 SEE ALSO
bzip2(1),
gzip(1),
lzop(1),
lrzip(1),
rzip(1),
zip(1)
=head1 AUTHORS
Program was written by Con Kolivas.
This manual page was written by Jari Aalto <jari.aalto@cante.net>, for
the Debian GNU system (but may be used by others). Released under license
GNU GPL version 2or (at your option) any later version. For more
information about license, visit <http://www.gnu.org/copyleft/gpl.html>.
=cut

276
man/lrzip.1 Normal file
View file

@ -0,0 +1,276 @@
.TH "lrzip" "1" "December 2009" "" ""
.SH "NAME"
lrzip \- a large-file compression program
.SH "SYNOPSIS"
.PP
lrzip [OPTIONS] <file>
.br
lrzip \-d [OPTIONS] <file>
.br
lrunzip [OPTIONS] <file>
.br
lrztar [lrzip options] <directory>
.br
lrztar \-d [lrzip options] <directory>
.br
LRZIP=NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>
.PP
.SH "DESCRIPTION"
.PP
LRZIP is a file compression program designed to do particularly
well on very large files containing long distance redundancy\&.
lrztar is a wrapper for LRZIP to simplify compression and decompression
of directories.
.PP
.SH "OPTIONS SUMMARY"
.PP
Here is a summary of the options to lrzip\&.
.nf
\-w size compression window in hundreds of MB
default chosen by heuristic dependent on ram and chosen compression
\-d decompress
\-o filename specify the output file name and/or path
\-O directory specify the output directory when \-o is not used
\-S suffix specify compressed suffix (default '.lrz')
\-f force overwrite of any existing files
\-D delete existing files
\-P don't set permissions on output file. It may leave it world-readable
\-q don't show compression progress
\-L level set lzma/bzip2/gzip compression level (1\-9, default 7)
\-n no backend compression. Prepare for other compressor
\-l lzo compression (ultra fast)
\-b bzip2 compression
\-g gzip compression using zlib
\-z zpaq compression (best, extreme compression, extremely slow)
\-M Maximum window and level - (all available ram and level 9)
\-T value Compression threshold with LZO test. (0 (nil) - 10 (high), default 1)
\-N value Set nice value to value (default 19)
\-v[v] Increase verbosity
\-V show version
\-t test compressed file integrity
\-i show compressed file information
If no filenames or "-" is specified, stdin/out will be used.
.fi
.PP
.SH "OPTIONS"
.PP
.IP "\fB-h\fP"
Print an options summary page
.IP
.IP "\fB-V\fP"
Print the lrzip version number
.IP
.IP "\fB-v[v]\fP"
Increases verbosity. \-vv will print more messages than \-v.
.IP
.IP "\fB-w n\fP"
Set the compression window size to n in hundreds of megabytes. This is the amount
of memory lrzip will search during its first stage of pre-compression and is
the main thing that will determine how much benefit lrzip will provide over
ordinary compression with the 2nd stage algorithm. Because of buffers and
compression overheads, the value chosen must be significantly smaller than
your available ram or lrzip will induce a massive swap load. If not set
(recommended), the value chosen will be determined by internal heuristic in
lrzip which uses the most memory that is reasonable. It is limited to 2GB on
32bit machines.
.IP
.IP "\fB-L 1\&.\&.9\fP"
Set the compression level from 1 to 9. The default is
to use level 7, which is a reasonable compromise between speed and
compression. The compression level is also strongly related to how much
memory lrzip uses. See the \-w option for details.
.IP
.IP "\fB-M \fP"
Maximum compression\&. If this option is set, then lrzip ignores the heuristic
mentioned for the default window and tries to set it to all available ram,
and sets the compression level to maximum. This will cause a significant swap
load on most machines, and may even fail without enough swap space allocated.
Be prepared to walk away if you use this option. It is not recommended to use
this as it hardly ever improves compression.
.IP
.IP "\fB-T 0\&.\&.10\fP"
Sets the LZO compression threshold when testing a data chunk when slower
compression is used. The threshold level can be from 0 to 10.
This option is used to speed up compression by avoiding doing the slow
compression pass. The reasoning is that if it is completely incompressible
by LZO then it will also be incompressible by them, thereby saving time.
The default is 1.
.IP
.IP "\fB-d\fP"
Decompress. If this option is not used then lrzip looks at
the name used to launch the program. If it contains the string
"lrunzip" then the \-d option is automatically set.
.IP
.IP "\fB-l\fP"
LZO Compression. If this option is set then lrzip will use the ultra
fast lzo compression algorithm for the 2nd stage. This mode of compression
gives bzip2 like compression at the speed it would normally take to simply
copy the file, giving excellent compression/time value]&.
.IP
.IP "\fB-n\fP"
No 2nd stage compression. If this option is set then lrzip will only
perform the long distance redundancy 1st stage compression. While this does
not compress any faster than LZO compression, it produces a smaller file
that then responds better to further compression (by eg another application),
also reducing the compression time substantially.
.IP
.IP "\fB-b\fP"
Bzip2 compression. Uses bzip2 compression for the 2nd stage, much like
the original rzip does.
.IP "\fB-g\fP"
Gzip compression. Uses gzip compression for the 2nd stage, much like
the original rzip does. Uses libz compress and uncompress functions.
.IP
.IP "\fB-z\fP"
ZPAQ compression. Uses ZPAQ compression which is from the PAQ family of
compressors known for having some of the highest compression ratios possible
but at the cost of being extremely slow on both compress and decompress.
.IP
.IP "\fB-o\fP"
Set the output file name. If this option is not set then
the output file name is chosen based on the input name and the
suffix. The \-o option cannot be used if more than one file name is
specified on the command line.
.IP
.IP "\fB-O\fP"
Set the output directory for the default filename. This option
cannot be combined with \-o.
.IP
.IP "\fB-S\fP"
Set the compression suffix. The default is '.lrz'.
.IP
.IP "\fB-f\fP"
If this option is not specified (Default) then lrzip will not
overwrite any existing files. If you set this option then rzip will
silently overwrite any files as needed.
.IP
.IP "\fB-D\fP"
If this option is specified then lrzip will delete the
source file after successful compression or decompression. When this
option is not specified then the source files are not deleted.
.IP
.IP "\fB-P\fP"
If this option is specified then lrzip will not try to set the file
permissions on writing the file. This helps when writing to a brain
damaged filesystem like fat32 on windows.
.IP
.IP "\fB-q\fP"
If this option is specified then lrzip will not show the
percentage progress while compressing. Note that compression happens in
bursts with lzma compression which is the default compression. This means
that it will progress very rapidly for short periods and then stop for
long periods.
.IP "\fB-N value\fP"
The default nice value is 19. This option can be used to set the priority
scheduling for the lrzip backup or decompression. Valid nice values are
from \-20 to 19.
.IP
.IP "\fB-t\fP"
This tests the compressed file integrity. It does this by decompressing it
to a temporary file and then deleting it.
.IP
.IP "\fB-i\fP"
This shows information about a compressed file. It shows the compressed size,
the decompressed size, the compression ratio and what compression was used.
Note that the compression mode is detected from the first block only and
it will show no compression used if the first block was incompressible, even
if later blocks were compressible.
.IP
.PP
.SH "INSTALLATION"
.PP
"make install" or just install lrzip somewhere in your search path.
.PP
.SH "COMPRESSION ALGORITHM"
.PP
LRZIP operates in two stages. The first stage finds and encodes large
chunks of duplicated data over potentially very long distances (limited
only by your available ram) in the input file. The second stage is to
use a compression algorithm to compress the output of the
first stage. The compression algorithm can be chosen to be optimised
for size (lzma - default), speed (lzo), legacy (bzip2) or (gzip)
or can be omitted entirely doing only the first stage. A one stage only
compressed file can almost always improve both the compression size and
speed done by a subsequent compression program.
.PP
The key difference between lrzip and other well known compression
algorithms is its ability to take advantage of very long distance
redundancy. The well known deflate algorithm used in gzip uses a
maximum history buffer of 32k. The block sorting algorithm used in
bzip2 is limited to 900k of history. The history buffer in lrzip can be
any size long, limited only by available ram.
.
.PP
It is quite common these days to need to compress files that contain
long distance redundancies. For example, when compressing a set of
home directories several users might have copies of the same file, or
of quite similar files. It is also common to have a single file that
contains large duplicated chunks over long distances, such as pdf
files containing repeated copies of the same image. Most compression
programs won't be able to take advantage of this redundancy, and thus
might achieve a much lower compression ratio than lrzip can achieve.
.IP
.PP
.SH "FILES"
.PP
LRZIP now recognizes a configuration file that contains default settings.
This configuration is searched for in the current directory, /etc/lrzip,
and $HOME/.lrzip. The configuration filename must be \fBlrzip.conf\fP.
.PP
.SH "ENVIRONMENT"
By default, lrzip will search for and use a configuration file, lrzip.conf.
If the user wishes to bypass the file, a startup ENV variable may be set.
.br
.B LRZIP =
.I "NOCONFIG "
.B "[lrzip|lrunzip]"
[OPTIONS] <file>
.br
which will force lrzip to ignore the configuration file.
.PP
.SH "HISTORY - Notes on rzip by Andrew Tridgell"
.PP
The ideas behind rzip were first implemented in 1998 while I was
working on rsync. That version was too slow to be practical, and was
replaced by this version in 2003.
LRZIP was created by the desire to have better compression and/or speed
by Con Kolivas on blending the lzma and lzo compression algorithms with
the rzip first stage, and extending the compression windows to scale
with increasing ram sizes.
.PP
.SH "BUGS"
.PP
Probably lots.
.PP
.SH "SEE ALSO"
lrzip.conf(5)
.PP
.SH "AUTHOR and CREDITS"
.br
rzip was written by Andrew Tridgell.
.br
lzma was written by Igor Pavlov.
.br
lzo was written by Markus Oberhumer.
.br
zpaq was written by Matt Mahoney.
.br
lrzip was bastardised from rzip by Con Kolivas.
.br
Peter Hyman added informational output, updated LZMA SDK,
and aded multi-threading capabilities.
.PP
If you wish to report a problem or make a suggestion then please email Con at
kernel@kolivas.org
.PP
lrzip is released under the GNU General Public License version 2.
Please see the file COPYING for license details.

59
man/lrzip.conf.5 Normal file
View file

@ -0,0 +1,59 @@
.TH "lrzip.conf" "5" "January 2009" "" ""
.SH "NAME"
lrzip.conf \- Configuration File for lrzip
.SH "DESCRIPTION"
.PP
This file if used, will be read by the lrzip program\&, parsed\&,
and options passed to the program\&. Some options may be overridden
on the command line\&. Others are fixed\&.
.PP
The configuration file must be called \fBlrzip\&.conf\fP\&.
The lrzip program will search for the file automatically in one of
three places\&:
.nf
$PWD \- Current Directory
/etc/lrzip
$HOME/\&./lrzip
.PP
Parameters are set in \fBPARAMETER\&=VALUE\fP fashion where any line
beginning with a \fB#\fP or that is blank will be ignored\&.
.PP
.SH "CONFIG FILE EXAMPLE"
.nf
# This is a comment.
# Compression Window size in 100MB. Normally selected by program.
WINDOW = 5
# Compression Level 1-9 (7 Default).
COMPRESSIONLEVEL = 7
# Compression Method, rzip, gzip, bzip2, lzo, or lzma (default).
COMPRESSIONMETHOD = lzma
# Test Threshold value 1-10 (2 Default).
TESTTHRESHOLD = 2
# Default output directory
OUTPUTDIRECTORY = location
# Verbosity, true or 1, or max or 2
VERBOSITY = max
# Show Progress as file is parsed, true or 1, false or 0
SHOWPROGRESS = true
# Set Niceness. 19 is default. \-20 to 19 is the allowable range
NICE = 19
# Delete source file after compression
# this parameter and value are case sensitive
# value must be YES to activate
# DELETEFILES = NO
# Replace existing lrzip file when compressing
# this parameter and value are case sensitive
# value must be YES to activate
# REPLACEFILE = NO
.fi
.PP
.SH "NOTES"
.PP
Be careful when using \fBDELETEFILES\fP or \fBREPLACEFILE\fP as
no warning will be given and lrzip will simply delete the source
or replace the output file!
.PP
.SH "SEE ALSO"
lrzip(1)

85
man/lrztar.1.pod Normal file
View file

@ -0,0 +1,85 @@
# Copyright
#
# Copyright (C) 2004-2010 Jari Aalto
#
# License
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Description
#
# To learn what TOP LEVEL section to use in manual pages,
# see POSIX/Susv standard and "tility Description Defaults" at
# http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap01.html#tag_01_11
#
# This is manual page in Perl POD format. Read more at
# http://perldoc.perl.org/perlpod.html or run command:
#
# perldoc perlpod | less
#
# To check the syntax:
#
# podchecker *.pod
#
# Create manual page with command:
#
# pod2man PAGE.N.pod > PAGE.N
=pod
=head1 NAME
lrztar - Directory wrapper for lrzip
=head1 SYNOPSIS
lrztar [options] DIRECTORY
lrztar -d [options] DIRECTORY.tar.lrz
=head1 DESCRIPTION
lrztar is a wrapper for compressing and decompressing while directories with lrzip(1)
to corresponding file C<DIRECTORY.tar.lrz>.
=head1 OPTIONS
See lrzip(1).
=head1 ENVIRONMENT
None.
=head1 FILES
None.
=head1 SEE ALSO
bzip2(1),
gzip(1),
lzop(1),
lrzip(1),
rzip(1),
zip(1)
=head1 AUTHORS
Program was written by Con Kolivas.
This manual page was written by Jari Aalto <jari.aalto@cante.net>, for
the Debian GNU system (but may be used by others). Released under license
GNU GPL version 2or (at your option) any later version. For more
information about license, visit <http://www.gnu.org/copyleft/gpl.html>.
=cut