ckolivas/lrzip
1
0
Fork 0
mirror of https://github.com/ckolivas/lrzip.git synced 2025-12-06 07:12:00 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
175 commits 5 branches 43 tags 2.8 MiB
Commit graph

175 commits

This branch
This branch
All branches
Author SHA1 Message Date
Con Kolivas bb33f7571c Multi-threading speed ups. 
Add one more thread on compression and decompression to account for the staggered nature of thread recruitment.
Make the initial buffer slightly smaller and make it progressively larger, thus recruiting threads sooner and more evenly.
This also speeds up decompression for the same reason.
Check the amount of memory being used by each thread on decompression to ensure we don't try to recruit too much ram.
 2011-02-22 00:49:50 +11:00
Con Kolivas 88e3df6af1 Print perror before unlinking files. 
Join common parts of fatal errors.
Update copyright notices.
Small improvement to visual output.
 2011-02-21 16:11:59 +11:00
Con Kolivas a7b4708bd2 Use a different failure mode for when perror is unlikely to be set. 
Add 2 unlikely wrappers.
 2011-02-21 14:51:20 +11:00
Con Kolivas 9f3256bda8 Do all file unlinking from the same function. 
Delete temporary files generated when testing from stdin.
 2011-02-21 14:17:05 +11:00
Con Kolivas 5c80a0957d Add usage() to failure to read stdin/out. 2011-02-21 13:29:45 +11:00
Con Kolivas b06287cce5 Make it not read stdin from a terminal or write stdout to a terminal unless forced with -f. 2011-02-21 13:26:51 +11:00
Con Kolivas 29dfb16c7a Clean up the messy help output. 2011-02-21 12:49:44 +11:00
Con Kolivas a85b17a463 Minor help change. 2011-02-21 12:06:49 +11:00
Con Kolivas 74df2b5973 Minor updates to man pages, lrzip.conf example file. 
Update main help screen to include environment settings.
Update to respect $TMP environment variable for TMP files.
Updated control structure to include tmpdir pointer.
Update lrzip.conf parser to respect -U -M options.
Update lrzip.conf example to include new parameters.
Reorder main Switch loop in main.c for readability.
Have MAXRAM and control.window be exclusive. MAXRAM wins.
Have UNLIMITED and control.window be exclusive. UNLIMITED wins.
Have UNLIMITED and MAXRAM be exclusive. UNLIMITED wins.
Corrects heuristic computation in rzip.c which would override
MAXRAM or UNLIMITED if control.window set
Show heuristically computed control.window when computed.
Remove display compression level from control.window verbose output.
Update print_verbose format for Testing for incompressible data in stream.c
to omit extra \n.

Changes by Peter Hyman <pete@peterhyman.com>
 2011-02-21 12:03:08 +11:00
Con Kolivas 98d264fac8 Update more documentation. 2011-02-21 00:33:23 +11:00
Con Kolivas 3119986634 Update lrztar to take new options in version 0.560 2011-02-21 00:20:41 +11:00
Con Kolivas 0b81d38259 Update version to 0.560 2011-02-21 00:14:03 +11:00
Con Kolivas f056000f7c Update documentation. 2011-02-21 00:02:14 +11:00
Con Kolivas d4fb4d025b Update manpage. 2011-02-20 23:22:45 +11:00
Con Kolivas 57e25da244 Update copyright yeah in updated files. 2011-02-20 23:04:44 +11:00
Con Kolivas c9863e0e60 Change default behaviour to deleting broken or damaged files that occur by interrupting lrzip or that fail integrity testing. 
Implement the -k option to optionally keep broken or damaged files.
 2011-02-20 23:02:15 +11:00
Con Kolivas 7b073160a3 Can't always open fd_out in runzip for integrity testing, so use fd_hist. 2011-02-20 22:44:10 +11:00
Con Kolivas 9b264959f5 Implement the ability to test the integrity of the file written to disk on decompression. 2011-02-20 22:29:49 +11:00
Con Kolivas a9881db04a Revert "Remove unused md5_stream function." 
This reverts commit b94f68ba41.

Reinstate the md5_stream function enabling file verification.
 2011-02-20 21:39:30 +11:00
Con Kolivas 3fa7e75296 Update copyright notices for md5.* 2011-02-20 20:20:21 +11:00
Con Kolivas 8a27dc5057 Changes to make md5 be used for integrity testing. 
Add the md5 value to the end of each archive.
This can then be used for integrity testing instead of crc32.
Keep crc in new archives to maintain compatibility with version 0.5 files.
Use md5 integrity testing on decompression when available in preference, and disable calculation of crc32.
Display the choice of integrity testing in verbose output and when -i is used.
Display the md5 and crc values when max verbosity, file info, or display hash is enabled.
Store a new flag in the magic header to show that the md5 value is stored at the end of the file.
Update the magic header information document.
 2011-02-20 18:01:19 +11:00
Con Kolivas 44a279579e Add option to display hash information without enabling verbose mode. 2011-02-20 12:20:05 +11:00
Con Kolivas 4036125f94 Make the buffer passed to hash checking page sized which gives a minor speedup. 2011-02-20 12:16:45 +11:00
Con Kolivas b94f68ba41 Remove unused md5_stream function. 2011-02-19 22:14:47 +11:00
Con Kolivas 744202a47f Remove unused variable. 2011-02-19 10:39:07 +11:00
Con Kolivas 9e589b3d2e Revert "Forgot the md5.x changes to support md5 on decompression." 
This reverts commit 006691ca8e.

md5_midstream no longer needed with decompression using existing buffers too now.
 2011-02-19 10:35:24 +11:00
Con Kolivas 7287ab8a66 Fix md5 process bytes to occur at the same time as crc with the same buffer, saving time. 2011-02-19 10:34:45 +11:00
Con Kolivas d1a70bbb90 Remove the sb.low_top idea by trimming the size of sb.size_low when it gets to the end of the file instead. 2011-02-18 21:14:16 +11:00
Con Kolivas 006691ca8e Forgot the md5.x changes to support md5 on decompression. 2011-02-18 15:21:34 +11:00
Con Kolivas fb2a12744a Implement md5 checking on decompression. 
Implement hash check flag to determine whether to show md5sum on compression/decompression or not.
 2011-02-18 15:16:13 +11:00
Con Kolivas c2417877bc Being implementing md5 hash checking by importing the md5 code from coreutils. 
Implement md5 hash checking on compression by doing the md5 hash check as each sb low buffer has been allocated to avoid going over the file again where possible.
 2011-02-18 09:09:40 +11:00
Con Kolivas cd8b086bf2 Minimise the number of mallocs in unzip_match. 2011-02-17 09:32:01 +11:00
Con Kolivas 626e0be281 Convert semaphore primitives to pthread_mutexes making them more portable, thus allowing multithreading to work on OSX. 2011-02-17 00:24:28 +11:00
Con Kolivas 05c5326df3 Revert "OSX doesn't support unnamed semaphores so to make it work, fake the threading by just creating the threads and waiting for them to finish." 
This reverts commit b81542cea4.

Revert the change bypassing semaphores in OSX in preparation for changing the semaphores to mutexes.
 2011-02-16 17:40:50 +11:00
Con Kolivas ea9b00c839 Update version number to 0.552 and update documentation. 2011-02-11 13:22:29 +11:00
Con Kolivas b81542cea4 OSX doesn't support unnamed semaphores so to make it work, fake the threading by just creating the threads and waiting for them to finish. 
This is done by making the semaphore wrappers null functions on osx and closing the thread in the creation wrapper.
Move the wrappers to rzip.h to make this change clean.
 2011-02-11 12:22:09 +11:00
Con Kolivas f2d33c00f8 Cast the mallocs to their variable type. 
Check that read and write actually return greater than zero.
 2011-02-11 11:46:58 +11:00
Con Kolivas 3879807865 Try limiting stream_read in unzip_literal and just returning how much was read. 2011-02-10 16:57:22 +11:00
Con Kolivas 9a3bfe33d1 Revert "Make sure to read the full length asked of unzip_literal." 
This reverts commit 499ae18cef.

Wrong fix, revert it.
 2011-02-10 16:46:35 +11:00
Con Kolivas 499ae18cef Make sure to read the full length asked of unzip_literal. 2011-02-10 15:30:31 +11:00
Con Kolivas 0a32b5f72d Convert mmaps to malloc in runzip as they may fail if not a multiple of page size! 2011-02-10 13:53:42 +11:00
Con Kolivas bece82a593 Trivial documentation fixes courtesy of Laszlo Ersek. 2011-02-10 13:14:36 +11:00
Con Kolivas 2a0553bc54 Revert "Decompress more than one stream at a time if there are threads free and the end of one stream is reached." 
This reverts commit 8ee9ef64f5.

This change is unreliable. Hence revert it and all dependent patches.
 2011-02-09 12:39:15 +11:00
Con Kolivas 8239635038 Revert "Limit the maximum number of threads on stream 0 to 1 again as stream 1 data always appear after a chunk of stream 0 data." 
This reverts commit 0b0f6db606.
 2011-02-09 12:39:02 +11:00
Con Kolivas 44399d88ac Revert "Check we dont go over control.threads when trying to fill the other stream." 
This reverts commit 0bded0a7d9.
 2011-02-09 12:38:32 +11:00
Con Kolivas 0bded0a7d9 Check we dont go over control.threads when trying to fill the other stream. 
Update comments to reflect when we can go over control.threads.
 2011-02-09 11:07:38 +11:00
Con Kolivas 0b0f6db606 Limit the maximum number of threads on stream 0 to 1 again as stream 1 data always appear after a chunk of stream 0 data. 2011-02-09 10:45:03 +11:00
Con Kolivas 8ee9ef64f5 Decompress more than one stream at a time if there are threads free and the end of one stream is reached. 
Still limit total threads running to control.threads.
This affords a small speedup on decompression.
 2011-02-08 11:58:01 +11:00
Con Kolivas aa00c29fba The free semaphore is now only updated from the main process on decompression so there are no synchronisation concerns. 
Remove the free semaphore and the fragile use of sem_trywait and replace it with a simple busy flag for threads on decompression.
 2011-02-08 08:55:36 +11:00
Con Kolivas 9c2b86fec6 We are flushing the wrong file on decompression. Make sure we flush the file out. 2011-02-08 08:27:22 +11:00