Con Kolivas
6ac74aa9f0
Create a flag to know when the temporary output buffer is in use, in preparation for when we use it on decompression.
2011-03-13 08:34:06 +11:00
Con Kolivas
d067a6ea9e
Implement the real workings of writing to a temporary buffer before flushing to stdout.
2011-03-12 22:46:57 +11:00
Con Kolivas
7fbec0a783
Prepare to write compressed output by flushing stdout after each chunk is compressed.
2011-03-12 19:56:08 +11:00
Con Kolivas
d87df4f44e
Store the file size as zero when the magic header will be written before the size is known.
2011-03-11 23:45:22 +11:00
Con Kolivas
81ab444971
Remove unused flags by fake mremap.
2011-03-09 13:13:33 +11:00
Con Kolivas
dc92d477d1
Remove unused best_h.
2011-03-09 13:10:00 +11:00
Con Kolivas
1a7c409e10
header-mangling-part-X-move-all-headers-defines-into
2011-03-09 08:37:26 +11:00
Con Kolivas
99c3ea2ab9
header-mangling-part-3-remove-ugly-hacks-for-liblrzi
2011-03-09 08:36:07 +11:00
Con Kolivas
1511c27aad
header-mangling-part-2-move-all-function-prototypes
2011-03-09 08:34:44 +11:00
Con Kolivas
f6f0a25ef6
rebase-of-function-split-and-control-additions-to-fu
2011-03-09 08:32:14 +11:00
ckolivas
59e5bc0492
Remove all open coded strerrors when they're going to be shown by fatal().
2011-03-08 12:05:03 +11:00
ckolivas
82a3bd3028
Clean up unused variables.
2011-03-08 11:43:02 +11:00
Con Kolivas
44ed5727b3
Clean up and fix times displayed.
...
Merge fixes.
2011-03-07 21:05:20 +11:00
Con Kolivas
9fc551c159
Merge branch 'temp'
...
Conflicts:
main.c
2011-03-07 17:25:10 +11:00
Con Kolivas
32e182c95a
Check for free space in the right place for compression and give the right message.
2011-03-07 17:00:28 +11:00
Con Kolivas
38eca38743
Unify maxram allocation and limit threads when there isn't enough ram.
2011-03-07 13:23:14 +11:00
Con Kolivas
a4ac1b8221
Display the progress percentage and ETA when it's known on STDIN compress.
2011-03-07 13:03:43 +11:00
ckolivas
63566a2992
Control.window was broken comparing to zero max_chunk.
2011-02-28 14:10:46 +11:00
ckolivas
ab656aa98a
Wrong check in rzip.c will leave a small chunk with unlimited mode.
2011-02-28 14:05:33 +11:00
Con Kolivas
8bdd5688c8
Check for free space before compression/decompression and abort if -f option is not enabled.
2011-02-26 23:10:28 +11:00
Con Kolivas
db4f9f5fea
Only retry mmapping if it's due to a memory allocation error.
2011-02-26 20:00:26 +11:00
Con Kolivas
7acd389382
Trivial cast to uchar *
2011-02-26 19:45:00 +11:00
Con Kolivas
8bf60c3114
Don't round to page size if the entire buffer will fit, otherwise this generates one extra pass of a tiny chunk.
2011-02-24 11:38:31 +11:00
Con Kolivas
9c8b1ee795
Deprecate the -M option as it is no longer of any significant utility compared to the -U option.
2011-02-22 20:38:39 +11:00
Con Kolivas
bb33f7571c
Multi-threading speed ups.
...
Add one more thread on compression and decompression to account for the staggered nature of thread recruitment.
Make the initial buffer slightly smaller and make it progressively larger, thus recruiting threads sooner and more evenly.
This also speeds up decompression for the same reason.
Check the amount of memory being used by each thread on decompression to ensure we don't try to recruit too much ram.
2011-02-22 00:49:50 +11:00
Con Kolivas
a7b4708bd2
Use a different failure mode for when perror is unlikely to be set.
...
Add 2 unlikely wrappers.
2011-02-21 14:51:20 +11:00
Con Kolivas
57e25da244
Update copyright yeah in updated files.
2011-02-20 23:04:44 +11:00
Con Kolivas
8a27dc5057
Changes to make md5 be used for integrity testing.
...
Add the md5 value to the end of each archive.
This can then be used for integrity testing instead of crc32.
Keep crc in new archives to maintain compatibility with version 0.5 files.
Use md5 integrity testing on decompression when available in preference, and disable calculation of crc32.
Display the choice of integrity testing in verbose output and when -i is used.
Display the md5 and crc values when max verbosity, file info, or display hash is enabled.
Store a new flag in the magic header to show that the md5 value is stored at the end of the file.
Update the magic header information document.
2011-02-20 18:01:19 +11:00
Con Kolivas
44a279579e
Add option to display hash information without enabling verbose mode.
2011-02-20 12:20:05 +11:00
Con Kolivas
4036125f94
Make the buffer passed to hash checking page sized which gives a minor speedup.
2011-02-20 12:16:45 +11:00
Con Kolivas
7287ab8a66
Fix md5 process bytes to occur at the same time as crc with the same buffer, saving time.
2011-02-19 10:34:45 +11:00
Con Kolivas
d1a70bbb90
Remove the sb.low_top idea by trimming the size of sb.size_low when it gets to the end of the file instead.
2011-02-18 21:14:16 +11:00
Con Kolivas
fb2a12744a
Implement md5 checking on decompression.
...
Implement hash check flag to determine whether to show md5sum on compression/decompression or not.
2011-02-18 15:16:13 +11:00
Con Kolivas
c2417877bc
Being implementing md5 hash checking by importing the md5 code from coreutils.
...
Implement md5 hash checking on compression by doing the md5 hash check as each sb low buffer has been allocated to avoid going over the file again where possible.
2011-02-18 09:09:40 +11:00
Con Kolivas
2cabb335cb
Update copyright notices courtesy of Jari Aalto.
2010-12-16 09:45:21 +11:00
Con Kolivas
427df4cd4a
Fix stdin failing due to getting sizes all wrong.
...
Fix stdin compression values not being shown at end.
Fix inappropriate failure when lzma doesn't compress block.
2010-12-12 17:40:58 +11:00
Con Kolivas
4331ca4058
Move message to after size is finalised.
2010-12-11 12:25:43 +11:00
Con Kolivas
6c33071118
Rationalise the testing now that the default lzma settings use a lot less ram by default.
2010-12-11 01:07:43 +11:00
Con Kolivas
50437a8447
Move the threading on compression to higher up in the code, allowing the next stream to start using compression threads before the previous stream has finished.
...
This overlapping of compressing streams means that when files are large enough to be split into multiple blocks, all CPUs will be used more effectively throughout the compression, affording a nice speedup.
Move the writing of the chunk byte size and initial headers into the compthread to prevent any races occurring.
Fix a few dodgy callocs that may have been overflowing!
The previous commit reverts were done because the changes designed to speed it up actually slowed it down instead.
2010-12-10 23:51:59 +11:00
Con Kolivas
8dd9b00496
Revert "Make threads spawn at regular intervals along chunk size thus speeding up compression."
...
This reverts commit 688aa55c79
2010-12-08 21:25:00 +11:00
Con Kolivas
5be0f6c5c9
Revert "Make sure chunk to compress is of a minimum size."
...
This reverts commit 8dd1b6ea58
2010-12-08 21:23:41 +11:00
Con Kolivas
dc42add0cf
Since 32 bit seems to fall over so easily, get ruthless with limits on it, while still maintaining a large window with sliding mmap.
2010-12-04 08:39:52 +11:00
Con Kolivas
8dd1b6ea58
Make sure chunk to compress is of a minimum size.
2010-12-03 20:02:39 +11:00
Con Kolivas
688aa55c79
Make threads spawn at regular intervals along chunk size thus speeding up compression.
...
One more fix for unnecessary argument passing.
2010-12-03 19:33:56 +11:00
Con Kolivas
2a4892afa1
Streams passed are constant to some functions, don't pass stream argument to them.
2010-12-03 19:30:56 +11:00
Con Kolivas
f2b5852872
Fix divide by zero error when a file is smaller than the page size and compressed with -U.
2010-11-30 11:22:02 +11:00
Con Kolivas
f04b8aa15b
Tweak defaults some more.
...
Set maximum size of compression window to chunk size in the case of -M mode and mmap size normally.
Set maximum window to 1.5x ram in -M mode.
2010-11-20 18:29:28 +11:00
Con Kolivas
6929492874
Fix the sliding mmap buffer not working on 2nd pass onwards which was causing a slowdon of death.
2010-11-20 09:44:54 +11:00
Con Kolivas
13ca377279
More tidying up and ensure control window is obeyed.
2010-11-20 09:22:47 +11:00
Con Kolivas
75e675e6dd
Bump version number to 0.542.
...
Choose sane defaults for memory usage since linux ludicriously overcommits.
Use sliding mmap for any compression windows greater than 2/3 ram.
Consolidate and simplify testing of allocatable ram.
Minor tweaks to output.
Round up the size of the high buffer in sliding mmap to one page.
Squeeze a little more out of 32 bit compression windows.
2010-11-20 01:23:08 +11:00
