Documentation.

ChangeLog

@@ -1,6 +1,45 @@
 lrzip ChangeLog
 MARCH 2011, version 0.600 Con Kolivas
-* Massive rewrite
+* Massive rewrite with new file format to accomodate new features.
+* Allocate temporary buffers of safely sized ram that can act as temporary
+storage for performing de/compression to/from STDIN/STDOUT without requiring
+temporary physical files. Files compressed on machines with much larger ram
+being decompressed on smaller ram machines may still require temporary files,
+but as much as possible is done using in-ram decompression, and minimally
+sized temporary files. Information displayed is more verbose and accurate in
+STDIN/STDOUT mode.
+* The temporary buffers created for decompressing to STDOUT are also used
+for decompressing regular files now avoiding multiple write/reads when
+re-constructing the file on decompression. This can dramatically speed up
+the rzip phase of decompression on complicated files with many small matches.
+* Compress block headers as well now since we know how many bytes can be used
+to describe the length of the block, decreasing overall file size.
+* Store the rzip chunk size per chunk to make it possible to check total file
+size by summating each rzip chunk size when it's not known till the end (as
+happens when compressing from STDIN).
+* Implement password protected encryption. Import the polarssl code for
+sha512 and aes128 routines. Read password without echoing to screen by
+disabling echo via terminfo. Take the password then multiply hash it
+according to the date it was generated. Inrease the number of hashes according
+to Moore's law so it always takes approximately 1 second per password on the
+most modern hardware when first encrypted. Hash the password against 8 bytes
+of salt which is a combination of the 2 byte encoded loop counter (for how 
+many times to hash the password) and 6 random bytes. Take random from
+/dev/urandom if it's available and fall back to random() if not. Encrypt each
+block of compressed data with 8 extra bytes of random salt. Once the headers
+are written, go back and encrypt the headers as well. Then encrypt the md5
+hash value as well. Anything beyond the initial lrzip magic header should
+apppear as random data and no two successive encryptions of the same data with
+the same password should generate the same data.
+* New build system should be more robust and portable.
+* Abstract out functions better into separate files and headers, and remove
+all use of global variables. This will make the generation of an lrzip
+library possible in the future.
+* Prevent testmalloc from coming up with a negative number when determining
+how big a block of memory to allocate by decreasing the number of threads to
+be used and then aborting to a minimum value should it still be too much
+apparent ram.
+* Numerous other fixes, documentation and cleanups.
 
 MARCH 2011, version 0.571 Con Kolivas
 * Only retry mmaping if it's a memory error, otherwise it may give spurious

WHATS-NEW

@@ -1,6 +1,32 @@
+lrzip-0.600
+
+Compressing/decompressing to/from STDIN/STDOUT now works without generating
+any temporary files. Very large files compressed in this way will be less
+efficiently compressed than if the whole solid file is presented to lrzip,
+but it is guaranteed not to generate temporary files on compression. 
+Decompressing files on a machine with the same amount of ram will also not
+generate temporary files, but if a file was generated on a larger ram machine,
+lrzip might employ temporary files, but they will not be the full size of the
+final file.
+Decompression should now be faster as the rzip reconstruction stage is mostly
+performed in ram before being written to disk.
+Final file sizes should be slightly smaller as block headers are now also
+compressed.
+Heavy grade encryption is now provided with the -e option. A combination of
+a time scaled multiply hashed sha512 password with random salt followed by
+aes128 block encryption of all data, including the data headers, provides for
+extremely secure encryption. Passwords up to 500 characters in length are
+supported, and the same file encrypted with the same password is virtually
+guaranteed to never produce the same data twice. All data beyond the basic
+lrzip opening header is completely obscured. Don't lose your password!
+Lrzip will not try to malloc a negative amount of ram on smaller ram machines,
+preferring to decrease the number of threads used when compressing, and then
+aborting to a nominal minimum.
+A new build configuration system which should be more robust and provides
+neater output during compilation.
+
 lrzip-0.571
 
-A new build configuration system.
 Avoid spurious errors on failing to mmap a file.
 Fee space will now be checked to ensure there is enough room for the
 compressed or decompressed file and lrzip will abort unless the -f option is