Paket: golang-github-buengese-sgzip-dev (0.0~git20220517.9bca1b6-4)

experiments for a seekable gzip for use in rclone based on pgzip (library)

This is an experimental implementation of gzip that allows seeking in the compressed file. In normal gzip files that can only be achieved by decompressing from the start and discarding all data until the selected offset. This gzip implementation works around this by creating a special metadata file that maps uncompressed blocks to compressed blocks allowing it to only read the compressed blocks required.

Due to necessity of being able to start decompression from any block the dictionary is reset after every block. This somewhat negatively effects compression ratio but is a necessary tradeof in our use case. The gzip files created by this library are valid normal gzip files and can be decompressed by any other gzip implementation.

Warning

This library was purpose build for the rclone compression backend. If you are looking for a multithreaded golang gzip implementation you should be using klauspost/pgzip (https://github.com/klauspost/pgzip) which is the base for this library.

This page is also available in the following languages (How to set the default document language):