Dear python developers, As a bioinformatician I work a lot with gzip-compressed data. Recently I discovered Intel's Storage Acceleration Libraries at https://github.com/intel/isa-l. These people implemented the DEFLATE and INFLATE algorithms in assembly language. As a result it is much faster than zlib.
I have posted a few benchmarks in this python bug https://bugs.python.org/issue41566. (I just discovered bugs.python.org is the wrong place for feature requests. I am sorry, I am still learning about the proper way of doing this things, as this is my first feature proposal). The TLDR is that it can speed up compression by 5x while speeding up compression by 3x compared to standard gzip. Isa-l is bsd-3-clause licensed and as such I see no licensing issues when using it in CPython. It is packaged in linux distros already, so I also see no problems in availability. Furthermore the non-Assembly parts are written in C so including from CPython should not pose very big problems. I am willing to write the PEP if more people think it is a good idea to do this. Best regards, Ruben Vorderman _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7GQXPL7VYQOHFOUL72XZQ4KD76U6PBNE/ Code of Conduct: http://python.org/psf/codeofconduct/
