Open Radar

 
Summary:
The caller of libcompression compression_decode_buffer() is required to take a best guess at the data's expected uncompressed size.

Due to this limitation the user of the buffer API either must know with a high degree of certainty what the uncompressed size will be, or they must alloc a (possibly extremely) large dst buffer to ensure it is capable of holding the uncompressed data without truncation.

It would be highly beneficial if an API was provided to determine the expected uncompressed buffer size. Perhaps a function like the following:

extern size_t 
compression_decode_uncompressed_size(const uint8_t * __restrict src_buffer, size_t src_size, compression_algorithm algorithm);

Short of that, a consideration of changing compression_decode_buffer() to add an out parameter (size_t** uncompressed_size) that would hold the total uncompressed size so if the result is truncated the caller can choose to recall with the new size if needed.

Steps to Reproduce:
Utilize compression_decode_buffer()

Expected Results:
Ability to decode buffer with full, un-truncated, results.

Actual Results:
Must alloc a large buffer to ensure contents will fit.

Version:
10.11 Beta 15A216g

Notes:
In lieu of the ability to get at the uncompressed data size directly, I think a standard way of accessing any available headers would be beneficial. This would keep the burden of dealing with the sizes at the higher level caller, while at the same time providing the higher level caller a standard way to get at any headers they might know about and be able to use (such as the uncompressed size).

http://blog.shiftybit.net/tinkering-with-compressionlib/

Configuration:


Attachments:


Resolution:
libcompression is a low level API and this kind of meta-data is expected to be provided at a higher level. 
Furthermore, some archive formats, like zlib and lzma, don't provide this kind of meta-data in their headers.

For more details see my series here:
http://blog.shiftybit.net/series/tinkering-with-compressionlib/