lrzip/doc/magic.header.txt

lrzip-0.6x file format
March 2011
Con Kolivas

Byte	Content
0-38	Magic
---
39->98	Rzip chunk data
99+	Data blocks
--- repeat
(end-MD5_DIGEST_SIZE)->(end) md5 hash

Magic data:
0->3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6->14	Source File Size or 0 if unknown
16->20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21	Flag that md5sum hash is stored at the end of the archive
22	Flag that the data is encrypted
23->27	Time in seconds (first 5 bytes of salt)
28->29	Encoded number of hash loops (bytes 6-7 of salt)
30->38	Random data (last 9 bytes of salt)

Rzip chunk data:
0	Data offsets byte width
1	Flag that there is another chunk beyond this
2->9	Chunk decompressed size
10->34	Stream 0 header data
35->59	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
1->8	Compressed data length
9->16	Uncompressed data length
17->24	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data


lrzip-0.5x file format
March 2011
Con Kolivas

Byte	Content
0->22	Magic
--
23->74	Rzip chunk data
75+	Data blocks
-- repeat
(end-MD5_DIGEST_SIZE)->(end) md5 hash

Magic data:
0->3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6->14	Source File Size
16->20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21	Flag that md5sum hash is stored at the end of the archive
22	not used

Rzip chunk data:
0	Data offsets byte width
1-25	Stream 0 header data
26-50	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
1-8	Compressed data length
9-16	Uncompressed data length
17-24	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data


lrzip-0.40+ file header format
November 2009
Con Kolivas

Byte	Content
0-3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6-14	Source File Size
16-20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21-22	not used
23-48	Stream 1 header data
49-74	Stream 2 header data

Block Data:
Byte:
0	Compressed data type
1-8	Compressed data length
9-16	Uncompressed data length
17-24	Next block head
25+	Data

End:
0-1	crc data


lrzip-0.24+ file header format
January 2009
Peter Hyman, pete@peterhyman.com

Byte	Content
0-3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6-9	Source File Size (no HAVE_LARGE_FILES)
6-14	Source File Size
16-20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21-22	not used
23-36	Stream 1 header data
37-50	Stream 2 header data
51	Compressed data type
Create tentative design for new file format. 2011-03-11 12:34:26 +01:00			`lrzip-0.6x file format`
			`March 2011`
			`Con Kolivas`

			`Byte Content`
			`0-38 Magic`
			`---`
Update magic header info. It turns out I can't add. Fix fileinfo to be able to read version 0.6x files. 2011-03-12 01:40:51 +01:00			`39->98 Rzip chunk data`
			`99+ Data blocks`
Create tentative design for new file format. 2011-03-11 12:34:26 +01:00			`--- repeat`
			`(end-MD5_DIGEST_SIZE)->(end) md5 hash`

			`Magic data:`
			`0->3 LRZI`
			`4 LRZIP Major Version Number`
			`5 LRZIP Minor Version Number`
			`6->14 Source File Size or 0 if unknown`
			`16->20 LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)`
			`21 Flag that md5sum hash is stored at the end of the archive`
			`22 Flag that the data is encrypted`
Get the seconds, salt and encryption loop data before compressing. Store seconds in only 5 bytes which is enough for 400 years, leaving more room for random data. 2011-03-15 04:41:47 +01:00			`23->27 Time in seconds (first 5 bytes of salt)`
			`28->29 Encoded number of hash loops (bytes 6-7 of salt)`
			`30->38 Random data (last 9 bytes of salt)`
Create tentative design for new file format. 2011-03-11 12:34:26 +01:00
			`Rzip chunk data:`
			`0 Data offsets byte width`
			`1 Flag that there is another chunk beyond this`
			`2->9 Chunk decompressed size`
			`10->34 Stream 0 header data`
			`35->59 Stream 1 header data`

			`Stream Header Data:`
			`Byte:`
			`0 Compressed data type`
			`1->8 Compressed data length`
			`9->16 Uncompressed data length`
			`17->24 Next block head`

			`Data blocks:`
			`0->(end-2) data`
			`(end-1)->end crc data`


			`lrzip-0.5x file format`
Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00			`March 2011`
Update magic header info. Move offset check to after reading chunk width. Realloc instead of free and malloc. 2010-11-01 05:27:35 +01:00			`Con Kolivas`

			`Byte Content`
Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00			`0->22 Magic`
			`--`
			`23->74 Rzip chunk data`
			`75+ Data blocks`
			`-- repeat`
			`(end-MD5_DIGEST_SIZE)->(end) md5 hash`

			`Magic data:`
			`0->3 LRZI`
Update magic header info. Move offset check to after reading chunk width. Realloc instead of free and malloc. 2010-11-01 05:27:35 +01:00			`4 LRZIP Major Version Number`
			`5 LRZIP Minor Version Number`
Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00			`6->14 Source File Size`
			`16->20 LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)`
Changes to make md5 be used for integrity testing. Add the md5 value to the end of each archive. This can then be used for integrity testing instead of crc32. Keep crc in new archives to maintain compatibility with version 0.5 files. Use md5 integrity testing on decompression when available in preference, and disable calculation of crc32. Display the choice of integrity testing in verbose output and when -i is used. Display the md5 and crc values when max verbosity, file info, or display hash is enabled. Store a new flag in the magic header to show that the md5 value is stored at the end of the file. Update the magic header information document. 2011-02-20 08:01:19 +01:00			`21 Flag that md5sum hash is stored at the end of the archive`
			`22 not used`
Update magic header info. Move offset check to after reading chunk width. Realloc instead of free and malloc. 2010-11-01 05:27:35 +01:00
Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00			`Rzip chunk data:`
			`0 Data offsets byte width`
			`1-25 Stream 0 header data`
			`26-50 Stream 1 header data`

			`Stream Header Data:`
Update magic header info. Move offset check to after reading chunk width. Realloc instead of free and malloc. 2010-11-01 05:27:35 +01:00			`Byte:`
			`0 Compressed data type`
			`1-8 Compressed data length`
			`9-16 Uncompressed data length`
			`17-24 Next block head`

Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00			`Data blocks:`
Create tentative design for new file format. 2011-03-11 12:34:26 +01:00			`0->(end-2) data`
			`(end-1)->end crc data`
Correct magic data header info and be more verbose about file format. 2011-03-11 12:17:26 +01:00
Update magic header info. Move offset check to after reading chunk width. Realloc instead of free and malloc. 2010-11-01 05:27:35 +01:00
Initial import 2010-03-29 01:07:08 +02:00			`lrzip-0.40+ file header format`
			`November 2009`
			`Con Kolivas`

			`Byte Content`
			`0-3 LRZI`
			`4 LRZIP Major Version Number`
			`5 LRZIP Minor Version Number`
			`6-14 Source File Size`
			`16-20 LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)`
			`21-22 not used`
			`23-48 Stream 1 header data`
			`49-74 Stream 2 header data`

			`Block Data:`
			`Byte:`
			`0 Compressed data type`
			`1-8 Compressed data length`
			`9-16 Uncompressed data length`
			`17-24 Next block head`
			`25+ Data`

			`End:`
			`0-1 crc data`


			`lrzip-0.24+ file header format`
			`January 2009`
			`Peter Hyman, pete@peterhyman.com`

			`Byte Content`
			`0-3 LRZI`
			`4 LRZIP Major Version Number`
			`5 LRZIP Minor Version Number`
			`6-9 Source File Size (no HAVE_LARGE_FILES)`
			`6-14 Source File Size`
			`16-20 LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)`
			`21-22 not used`
			`23-36 Stream 1 header data`
			`37-50 Stream 2 header data`
			`51 Compressed data type`