ckolivas/lrzip
1
0
Fork 0
mirror of https://github.com/ckolivas/lrzip.git synced 2026-04-21 06:03:54 +00:00
Code Issues Projects Releases Packages Wiki Activity

Reworked the multithreading massively.

Browse source 
Place the data from each stream into a buffer that then is handed over to one thread which is allowed to begin doing the backend compression while the main rzip stream continues operating.
Fork up to as many threads as CPUs and feed data to them in a ring fashion, parallelising the workload as much as possible.
This causes a big speed up on the compression side on SMP machines.
Thread compression is limited to a minimum of 10MB compressed per thread to minimise the compromise to compression of smaller windows.
Alter the progress output to match some of the changes in verbose modes.
This commit is contained in:
Con Kolivas 2010-11-13 01:26:09 +11:00
parent 5505097b2f
commit 02de002c58
4 changed files with 183 additions and 147 deletions
Download patch file Download diff file Expand all files Collapse all files

2
zpipe.cpp
View file

@ -1738,7 +1738,7 @@ static void compress(FILE *in, FILE *out, FILE *msgout, long long int buf_len, i
len++;
pct = (len * 100 / buf_len);
if (pct != last_pct) {
fprintf(msgout, "\r\tZPAQ Chunk %i of 2 compress: %i%% \r", (chunk + 1), pct);
fprintf(msgout, "\r\t\t\tZPAQ Chunk %i of 2 compress: %i%% \r", (chunk + 1), pct);
fflush(msgout);
last_pct = pct;
}