Hello!

I have a gzipped byte-stream (cStr) that I'd like to decompress with XbZlib.

On gzip.org it is said that zlib is capable of doing that.

I tried the XbZ_UnCompress(@cStr, @nLen, @nError) function but it 
returned -3 (Z_DATA_ERROR) and its return value equals Space(nLen) even 
if I provide the original decompressed length of the stream. (Another 
problem that in the real use-case I won't know that number in advance.)

The forum topic below discusses a similar / same problem and tells that 
inflateinit2() should be called to tell zlib the format of the stream to 
decompress. 
https://stackoverflow.com/questions/1838699/how-can-i-decompress-a-gzip-stream-with-zlib

Could anyone help me with this?

Thanks very much!

Best Regards,
Otto

Otto,

>I have a gzipped byte-stream (cStr) that I'd like to decompress with XbZlib.
>On gzip.org it is said that zlib is capable of doing that.

Yes, it is.

>I tried the XbZ_UnCompress(@cStr, @nLen, @nError) function but it 
>returned -3 (Z_DATA_ERROR) and its return value equals Space(nLen) even 
>if I provide the original decompressed length of the stream. (Another 
>problem that in the real use-case I won't know that number in advance.)

XbZ_UnCompress() can only compress strings in memory, and is therefore 
limited by the available RAM, which for Xbase++ is about 800 MB.

>The forum topic below discusses a similar / same problem and tells that 
>inflateinit2() should be called to tell zlib the format of the stream to 
>decompress. 

Yes, for processing stream data, you need to use the following functions 
from XbZLib:

XbZ_InflateInit(pStream, cZLibVer, nStreamSize)
XbZ_InflateStream(pStream, nFlush)
XbZ_InflateEnd(xStream)

This can be done by calling the XbZ_Inflate() function in XbZLib, like this:

XbZ_Inflate(nHandleIn, nHandleOut, nAtPos, nToRead, nCRC, cError, ; 
            nChunkSize, lDisplay, lConsole, bDisplay)

For an example of how to use this function, see the XbZLibZip:ExtractEntry() 
method in "XbZLibCl.prg":

nError := XbZ_Inflate(::nHandle, nHandle, nStart, ; 
          UnSigned32(oCDRec:CompSize), @nCRC, @cError, ::StreamSize, ;
          ::ShowProgress, ::IsConsole, ::OnProgress)

As always, the latest version of XbZLib can be found here:

	http://www.AbsoluteSoftwareLLC.com/Downloads/

Hope that helps,

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas@AbsoluteSoftwareLLC.com
web:   http://www.AbsoluteSoftwareLLC.com
[L]:   https://www.LinkedIn.com/in/AndreasGehrsPahl
[F]:   https://www.FaceBook.com/AbsoluteSoftwareLLC

Andreas,

Thanks very much for your reply.

> XbZ_UnCompress() can only compress strings in memory, and is therefore
> limited by the available RAM, which for Xbase++ is about 800 MB.

I think it'd be (more than) enough for the task I plan.

I've prepared a small test with XbZ_Inflate() and XbZ_UnCompress() to 
show my issue. After build you can call the 'unzip' executable with a 2 
character long parameter:

unzip di   OK         - defalted data        -> inflate
unzip du   OK         - defalted data        -> uncompress
unzip zi   OK         - zlib compressed data -> inflate
unzip zu   OK         - zlib compressed data -> uncompress
unzip gi   ERROR -3   - gzipped data         -> inflate
unzip gu   ERROR -3   - gzipped data         -> uncompress

My question is how to tell XbZ_Inflate() and XbZ_UnCompress() that the 
data is compressed with gzip.

In my example the compressed data is almost the same in all three cases 
(deflated, zlib compressed, gzipped) but some leading and trailing bytes 
are different (gzipped byte sequence has some header and footer bytes. 
Probably it's easy to cut those bytes off to get the DEFLATE-compressed 
payload but it might not be that safe. (Optional extra headers may 
appear, see https://en.wikipedia.org/wiki/Gzip).

> As always, the latest version of XbZLib can be found here:
> 	http://www.AbsoluteSoftwareLLC.com/Downloads/

My browser gives "403 - Forbidden: Access is denied" when I open the link.

Best Regards,
Otto

Gzip file format is described in RFC 1952 
(https://tools.ietf.org/html/rfc1952) and as I studied it along with a 
sample file I guess that if I can process the header and thus cut out 
the DEFLATE compressed content then it can be decompressed / inflated 
with XbZ_UnCompress() or XbZ_Inflate(). Even the original (uncompressed 
file size) is included in the format, see the final four blocks ISIZE in 
the specification which is a needed parameter for XbZ_UnCompress().

I hope I can do that (I'll report it back), if an easier solution is 
possible, please tell me.

Best Regards,
Otto

Andreas,

Finally I've written functions XbZ_GzUnCompress() and XbZ_GzCompress(), 
They manage the header and footer data, I hope they are suitable, in my 
tests they succeeded. See the attached zip.

Thanks for the hints and of course for your superb library!

Best Regards,
Otto

After more tests I changed the line

nCRC32:= Bin2U(Left(cFooter, 4))
   to
nCRC32:= Bin2L(Left(cFooter, 4))

because nCrc32() returns a signed value.

Best Regards,
Otto

Otto,

>>http://www.AbsoluteSoftwareLLC.com/Downloads/

>My browser gives "403 - Forbidden: Access is denied" when I open the link.

Sorry, that should have been:

	http://www.AbsoluteSoftwareLLC.com/Downloads.htm

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas@AbsoluteSoftwareLLC.com
web:   http://www.AbsoluteSoftwareLLC.com
[L]:   https://www.LinkedIn.com/in/AndreasGehrsPahl
[F]:   https://www.FaceBook.com/AbsoluteSoftwareLLC