ckolivas/lrzip
1
0
Fork 0
mirror of https://github.com/ckolivas/lrzip.git synced 2025-12-06 07:12:00 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
lrzip/util.c

197 lines
5.9 KiB
C
Raw Normal View History

Update copyright notices courtesy of Jari Aalto.
2010-12-15 23:45:21 +01:00
/*
Copyright (C) 2006-2010 Con Kolivas
Copyright (C) 1998 Andrew Tridgell
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
Initial import
2010-03-29 01:07:08 +02:00
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
Update copyright notices courtesy of Jari Aalto.
2010-12-15 23:45:21 +01:00
Initial import
2010-03-29 01:07:08 +02:00
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Update copyright notices courtesy of Jari Aalto.
2010-12-15 23:45:21 +01:00
Initial import
2010-03-29 01:07:08 +02:00
You should have received a copy of the GNU General Public License
Update copyright notices courtesy of Jari Aalto.
2010-12-15 23:45:21 +01:00
along with this program. If not, see <http://www.gnu.org/licenses/>.
Initial import
2010-03-29 01:07:08 +02:00
*/
/*
Utilities used in rzip
tridge, June 1996
*/
/*
* Realloc removed
* Functions added
* read_config()
* Peter Hyman, December 2008
*/
#include "rzip.h"
void fatal(const char *format, ...)
{
va_list ap;
if (format) {
va_start(ap, format);
vfprintf(stderr, format, ap);
va_end(ap);
}
/* Delete temporary files generated for testing or faking stdio */
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
if (TEST_ONLY || STDOUT)
Initial import
2010-03-29 01:07:08 +02:00
unlink(control.outfile);
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
if (DECOMPRESS && STDIN)
Initial import
2010-03-29 01:07:08 +02:00
unlink(control.infile);
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
perror(NULL);
print_output("Fatal error - exiting\n");
Initial import
2010-03-29 01:07:08 +02:00
exit(1);
}
void sighandler()
{
/* Delete temporary files generated for testing or faking stdio */
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
if (TEST_ONLY || STDOUT)
Initial import
2010-03-29 01:07:08 +02:00
unlink(control.outfile);
Huge rewrite of buffer reading in rzip.c. We use a wrapper instead of accessing the buffer directly, thus allowing us to have window sizes larger than available ram. This is implemented through the use of a "sliding mmap" implementation. Sliding mmap uses two mmapped buffers, one large one as previously, and one page sized smaller one. When an attempt is made to read beyond the end of the large buffer, the small buffer is remapped to the file area that's being accessed. While this implementation is 100x slower than direct mmapping, it allows us to implement unlimited sized compression windows. Implement the -U option with unlimited sized windows. Rework the selection of compression windows. Instead of trying to guess how much ram the machine might be able to access, we try to safely buffer as much ram as we can, and then use that to determine the file buffer size. Do not choose an arbitrary upper window limit unless -w is specified. Rework the -M option to try to buffer the entire file, reducing the buffer size until we succeed. Align buffer sizes to page size. Clean up lots of unneeded variables. Fix lots of minor logic issues to do with window sizes accepted/passed to rzip and the compression backends. More error handling. Change -L to affect rzip compression level directly as well as backend compression level and use 9 by default now. More cleanups of information output. Use 3 point release numbering in case one minor version has many subversions. Numerous minor cleanups and tidying. Updated docs and manpages.
2010-11-04 11:14:55 +01:00
if (DECOMPRESS && STDIN)
Initial import
2010-03-29 01:07:08 +02:00
unlink(control.infile);
exit(0);
}
void read_config( struct rzip_control *control )
{
/* check for lrzip.conf in ., $HOME/.lrzip and /etc/lrzip */
FILE *fp;
char *parameter;
char *parametervalue;
char *line, *s;
char *HOME, *homeconf;
line = malloc(255);
homeconf = malloc(255);
if (line == NULL || homeconf == NULL)
fatal("Fatal Memory Error in read_config");
fp = fopen("lrzip.conf", "r");
if (fp)
Make messages come via stdout instead of stderror courtesy of Alexander Saprykin
2010-04-25 08:26:00 +02:00
fprintf(control->msgout, "Using configuration file ./lrzip.conf\n");
Initial import
2010-03-29 01:07:08 +02:00
if (fp == NULL) {
fp = fopen("/etc/lrzip/lrzip.conf", "r");
if (fp)
Make messages come via stdout instead of stderror courtesy of Alexander Saprykin
2010-04-25 08:26:00 +02:00
fprintf(control->msgout, "Using configuration file /etc/lrzip/lrzip.conf\n");
Initial import
2010-03-29 01:07:08 +02:00
}
if (fp == NULL) {
HOME=getenv("HOME");
if (HOME) {
strcpy(homeconf, HOME);
strcat(homeconf,"/.lrzip/lrzip.conf");
fp = fopen(homeconf, "r");
if (fp)
Make messages come via stdout instead of stderror courtesy of Alexander Saprykin
2010-04-25 08:26:00 +02:00
fprintf(control->msgout, "Using configuration file %s\n", homeconf);
Initial import
2010-03-29 01:07:08 +02:00
}
}
if (fp == NULL)
return;
/* if we get here, we have a file. read until no more. */
while ((s = fgets(line, 255, fp)) != NULL) {
if (strlen(line))
line[strlen(line) - 1] = '\0';
parameter = strtok(line, " =");
if (parameter == NULL)
continue;
/* skip if whitespace or # */
if (isspace(*parameter))
continue;
if (*parameter == '#')
continue;
parametervalue = strtok(NULL, " =");
if (parametervalue == NULL)
continue;
/* have valid parameter line, now assign to control */
if (!strcasecmp(parameter, "window"))
control->window = atoi(parametervalue);
else if (!strcasecmp(parameter, "compressionlevel")) {
control->compression_level = atoi(parametervalue);
if ( control->compression_level < 1 || control->compression_level > 9 )
fatal("CONF.FILE error. Compression Level must between 1 and 9");
} else if (!strcasecmp(parameter, "compressionmethod")) {
/* valid are rzip, gzip, bzip2, lzo, lzma (default) */
if (control->flags & FLAG_NOT_LZMA)
fatal("CONF.FILE error. Can only specify one compression method");
if (!strcasecmp(parametervalue, "bzip2"))
control->flags |= FLAG_BZIP2_COMPRESS;
else if (!strcasecmp(parametervalue, "gzip"))
control->flags |= FLAG_ZLIB_COMPRESS;
else if (!strcasecmp(parametervalue, "lzo"))
control->flags |= FLAG_LZO_COMPRESS;
else if (!strcasecmp(parametervalue, "rzip"))
control->flags |= FLAG_NO_COMPRESS;
else if (!strcasecmp(parametervalue, "zpaq"))
control->flags |= FLAG_ZPAQ_COMPRESS;
else if (strcasecmp(parametervalue, "lzma"))
fatal("CONF.FILE error. Invalid compression method %s specified",parametervalue);
} else if (!strcasecmp(parameter, "testthreshold")) {
control->threshold = atoi(parametervalue);
if (control->threshold < 1 || control->threshold > 10)
fatal("CONF.FILE error. Threshold value out of range %d", parametervalue);
control->threshold = 1.05-control->threshold / 20;
} else if (!strcasecmp(parameter, "outputdirectory")) {
control->outdir = malloc(strlen(parametervalue) + 2);
if (!control->outdir)
fatal("Fatal Memory Error in read_config");
strcpy(control->outdir, parametervalue);
if (strcmp(parametervalue + strlen(parametervalue) - 1, "/"))
strcat(control->outdir, "/");
} else if (!strcasecmp(parameter,"verbosity")) {
if (control->flags & FLAG_VERBOSE)
fatal("CONF.FILE error. Verbosity already defined.");
if (!strcasecmp(parametervalue, "true") || !strcasecmp(parametervalue, "1"))
control->flags |= FLAG_VERBOSITY;
else if (!strcasecmp(parametervalue,"max") || !strcasecmp(parametervalue, "2"))
control->flags |= FLAG_VERBOSITY_MAX;
} else if (!strcasecmp(parameter,"nice")) {
control->nice_val = atoi(parametervalue);
if (control->nice_val < -20 || control->nice_val > 19)
fatal("CONF.FILE error. Nice must be between -20 and 19");
} else if (!strcasecmp(parameter, "showprogress")) {
/* true by default */
if (!strcasecmp(parametervalue, "false") || !strcasecmp(parametervalue," 0"))
control->flags &= ~FLAG_SHOW_PROGRESS;
} else if (!strcmp(parameter, "DELETEFILES")) {
/* delete files must be case sensitive */
if (!strcmp(parametervalue, "YES"))
control->flags &= ~FLAG_KEEP_FILES;
} else if (!strcmp(parameter, "REPLACEFILE")) {
/* replace lrzip file must be case sensitive */
if (!strcmp(parametervalue, "YES"))
control->flags |= FLAG_FORCE_REPLACE;
}
}
/* clean up */
free(line);
free(homeconf);
/* fprintf(stderr, "\nWindow = %d \
\nCompression Level = %d \
\nThreshold = %1.2f \
\nOutput Directory = %s \
\nFlags = %d\n", control->window,control->compression_level, control->threshold, control->outdir, control->flags);
*/
}
Reference in a new issue Copy permalink