On Wed, Sep 23, 2020 at 3:10 AM David Mertz <me...@gnosis.cx> wrote: > On Tue, Sep 22, 2020 at 11:55 PM Paul Moore <p.f.mo...@gmail.com> wrote: > >> The point of this request is that Python's packaging infrastructure is >> looking at what compression we use for wheels - the current >> compression is suboptimal for huge binaries like tensorflow. > > > There are definitely some intermediate compression levels where both > brotli and zstd are significantly faster [than lzma], but not at the higher > levels where lzma does as well or better. >
I'd assume that only decompression speed matters for packages, and on that metric both brotli and zstd beat lzma by a mile regardless of the compression level. But I think that lzma gets exceptionally good ratios on x86/x64 machine code. Even after all these years it seems to be the state of the art for "best ratio that isn't painfully slow to decompress". On Wed, Sep 23, 2020 at 3:10 AM David Mertz <me...@gnosis.cx> wrote: > On Tue, Sep 22, 2020 at 11:55 PM Paul Moore <p.f.mo...@gmail.com> wrote: > >> The point of this request is that Python's packaging infrastructure is >> looking at what compression we use for wheels - the current >> compression is suboptimal for huge binaries like tensorflow. Packaging >> is in a unique situation, because it *cannot* use external libraries > > > It's hard to see where packaging would have any advantage with brotli or > zstd over lzma. XZ is more widely used, and package size seems to dominate > speed. There are definitely some intermediate compression levels where > both brotli and zstd are significantly faster, but not at the higher levels > where lzma does as well or better. > > Is there a concrete need here, or just an abstract point that compression > of packages shouldn't be outside the stdlib? > > Honestly, if you really want compression size over everything else, PPM is > going to beat the LZ based approaches. But being ungodly slow and using > tons of memory. > > -- > The dead increasingly dominate and strangle both the living and the > not-yet born. Vampiric capital and undead corporate persons abuse > the lives and control the thoughts of homo faber. Ideas, once born, > become abortifacients against new conceptions. > _______________________________________________ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/SC3SQKFK4GHASRNQJXFYDMNRB25P7SJ4/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IIRUPDNBCPIGUM5USU7WSWGW5AMBTRMB/ Code of Conduct: http://python.org/psf/codeofconduct/
