[Python-ideas] Re: Minor contributions to PEPs (Was: PEP 637 - support for indexing with keyword arguments)

2020-09-23 Thread 2QdxY4RzWzUUiLuE
On 2020-09-23 at 23:28:11 +0100,
Henk-Jaap Wagenaar  wrote:

> >>> a[3]   # returns the fourth element of a
> 
> has the comment unfinished. I guess it should say list or something
> similar.]

Yes, I agree:  it looks like it's broken, but it's okay.  a[3] returns
the fourth element of a sequence named "a."  If the sequence were named
something_descriptive, then something_descriptive[3] returns the fourth
element of a sequence named "something_descriptive."
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Z5JPXAIDPBMTNREXDNEX54N7J3PZ7OLJ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Minor contributions to PEPs (Was: PEP 637 - support for indexing with keyword arguments)

2020-09-23 Thread Christopher Barker
And here:

https://github.com/python/peps/blob/master/pep-0637.txt

is the source to look at -- and the repo to do a PR against.

-CHB


On Wed, Sep 23, 2020 at 3:34 PM Chris Angelico  wrote:

> On Thu, Sep 24, 2020 at 8:29 AM Henk-Jaap Wagenaar
>  wrote:
> >
> > I noticed a sentence that was not completed in PEP 637. Though I have
> made (pretty minor) contributions to CPython and some other things, it
> isn't entirely clear to me whether it would be appropriate for me to submit
> an issue or pull request for this, and what the general policy is?
> >
> > https://github.com/python/peps/blob/master/CONTRIBUTING.rst does not
> make it any clearer to me, I guess it is kinda the Wild West and depends on
> the PEP/what the change is? Would it be worth clarifying this?
> >
> >
> > Secondly, I wasn't 100% sure this make was a rendering mistake or in the
> source. Would it be possible/an idea to include a link to the source (like
> the Python documentation does)?
> >
> > [I am sure people will be curious what the mistake was:
> >
> > >>> a[3]   # returns the fourth element of a
> >
> > has the comment unfinished. I guess it should say list or something
> similar.]
> >
>
> Actually it's returning the fourth element of "the thing in the
> variable named 'a'", so it's not the English article. If you feel
> that's unclear, you could propose a change that renames the variable,
> perhaps.
>
> Pull requests are absolutely appropriate for simple changes,
> copyediting, etc. You'll be asked to sign the licensing agreement, but
> for extremely tiny changes (say, just fixing a missed bit of
> punctuation), we can override the CLA bot and merge the change anyway.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/IXZOMRI45NLGUR25K2NNHNVP4YIJMYS4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QC7GEXP63L2VIR3BWPZSOPFCZNSFRQXY/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Minor contributions to PEPs (Was: PEP 637 - support for indexing with keyword arguments)

2020-09-23 Thread Chris Angelico
On Thu, Sep 24, 2020 at 8:29 AM Henk-Jaap Wagenaar
 wrote:
>
> I noticed a sentence that was not completed in PEP 637. Though I have made 
> (pretty minor) contributions to CPython and some other things, it isn't 
> entirely clear to me whether it would be appropriate for me to submit an 
> issue or pull request for this, and what the general policy is?
>
> https://github.com/python/peps/blob/master/CONTRIBUTING.rst does not make it 
> any clearer to me, I guess it is kinda the Wild West and depends on the 
> PEP/what the change is? Would it be worth clarifying this?
>
>
> Secondly, I wasn't 100% sure this make was a rendering mistake or in the 
> source. Would it be possible/an idea to include a link to the source (like 
> the Python documentation does)?
>
> [I am sure people will be curious what the mistake was:
>
> >>> a[3]   # returns the fourth element of a
>
> has the comment unfinished. I guess it should say list or something similar.]
>

Actually it's returning the fourth element of "the thing in the
variable named 'a'", so it's not the English article. If you feel
that's unclear, you could propose a change that renames the variable,
perhaps.

Pull requests are absolutely appropriate for simple changes,
copyediting, etc. You'll be asked to sign the licensing agreement, but
for extremely tiny changes (say, just fixing a missed bit of
punctuation), we can override the CLA bot and merge the change anyway.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IXZOMRI45NLGUR25K2NNHNVP4YIJMYS4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Minor contributions to PEPs (Was: PEP 637 - support for indexing with keyword arguments)

2020-09-23 Thread Henk-Jaap Wagenaar
I noticed a sentence that was not completed in PEP 637. Though I have made
(pretty minor) contributions to CPython and some other things, it isn't
entirely clear to me whether it would be appropriate for me to submit an
issue or pull request for this, and what the general policy is?

https://github.com/python/peps/blob/master/CONTRIBUTING.rst does not make
it any clearer to me, I guess it is kinda the Wild West and depends on the
PEP/what the change is? Would it be worth clarifying this?

Secondly, I wasn't 100% sure this make was a rendering mistake or in the
source. Would it be possible/an idea to include a link to the source (like
the Python documentation does)?

[I am sure people will be curious what the mistake was:

>>> a[3]   # returns the fourth element of a

has the comment unfinished. I guess it should say list or something
similar.]

On Wed, 23 Sep 2020 at 21:59, Stefano Borini 
wrote:

> Dear all,
>
> "Support for indexing with keyword arguments" has now been merged with
> the assigned PEP number 637.
>
> For future reference, this email will be added to the Post-History of the
> PEP.
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZLWTL5QXTSFG3ZWWAOXEL3WAQWZZIPCR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

2020-09-23 Thread Todd
It is working for me now.

On Wed, Sep 23, 2020, 17:33 Ricky Teachey  wrote:

> Hmmm, getting a 404 at:
>
> https://www.python.org/dev/peps/pep-0637
>
> Is this just a temporary condition or a bug?
>
> ---
> Ricky.
>
> "I've never met a Kentucky man who wasn't either thinking about going home
> or actually going home." - Happy Chandler
>
>
> On Wed, Sep 23, 2020 at 4:56 PM Stefano Borini 
> wrote:
>
>> Dear all,
>>
>> "Support for indexing with keyword arguments" has now been merged with
>> the assigned PEP number 637.
>>
>> For future reference, this email will be added to the Post-History of the
>> PEP.
>>
>>
>> On Mon, 21 Sep 2020 at 21:34, Stefano Borini 
>> wrote:
>> >
>> > PEP for support for indexing with keyword arguments is now submitted as
>> PR.
>> >
>> > https://github.com/python/peps/pull/1612
>> >
>> > Thanks to everybody involved in the development of the PEP and the
>> > interesting discussion. All your contributions have been received and
>> > often added to the document.
>> > If the PEP is approved, I would like to attempt an implementation, but
>> > I am not particularly skilled in the python internals. If a core
>> > developer is willing to teach me the ropes (especially in the new
>> > parser, I barely understood the syntax of the old one, but have no
>> > idea about the new one) and review my code I can give it a try. I
>> > would not mind refreshing my C a bit.
>>
>> --
>> Kind regards,
>>
>> Stefano Borini
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/25ZPXADTQNJRMCSG4BJO4QJ7OQAL67N6/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/IEXUPAAWAHQFHD5F5S3D33IDO3XAXESS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I37DI7352524DYM6FT7OKORIWVHP3LVI/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

2020-09-23 Thread Chris Angelico
On Thu, Sep 24, 2020 at 7:33 AM Ricky Teachey  wrote:
>
> Hmmm, getting a 404 at:
>
> https://www.python.org/dev/peps/pep-0637
>
> Is this just a temporary condition or a bug?
>

Everything seems to be happy, so I'm going to guess that it's just
taking a bit of time to propagate out. Give it a few minutes, maybe an
hour tops, and see if it's live; if it still isn't, there might be an
infrastructure issue somewhere.

For now, you can read the text of the PEP here:

https://github.com/python/peps/blob/master/pep-0637.txt

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NZM6YGU26VNQN7XIN45QMF4C3EKZTG6C/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

2020-09-23 Thread Ricky Teachey
Hmmm, getting a 404 at:

https://www.python.org/dev/peps/pep-0637

Is this just a temporary condition or a bug?

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler


On Wed, Sep 23, 2020 at 4:56 PM Stefano Borini 
wrote:

> Dear all,
>
> "Support for indexing with keyword arguments" has now been merged with
> the assigned PEP number 637.
>
> For future reference, this email will be added to the Post-History of the
> PEP.
>
>
> On Mon, 21 Sep 2020 at 21:34, Stefano Borini 
> wrote:
> >
> > PEP for support for indexing with keyword arguments is now submitted as
> PR.
> >
> > https://github.com/python/peps/pull/1612
> >
> > Thanks to everybody involved in the development of the PEP and the
> > interesting discussion. All your contributions have been received and
> > often added to the document.
> > If the PEP is approved, I would like to attempt an implementation, but
> > I am not particularly skilled in the python internals. If a core
> > developer is willing to teach me the ropes (especially in the new
> > parser, I barely understood the syntax of the old one, but have no
> > idea about the new one) and review my code I can give it a try. I
> > would not mind refreshing my C a bit.
>
> --
> Kind regards,
>
> Stefano Borini
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/25ZPXADTQNJRMCSG4BJO4QJ7OQAL67N6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IEXUPAAWAHQFHD5F5S3D33IDO3XAXESS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)

2020-09-23 Thread Stefano Borini
Dear all,

"Support for indexing with keyword arguments" has now been merged with
the assigned PEP number 637.

For future reference, this email will be added to the Post-History of the PEP.


On Mon, 21 Sep 2020 at 21:34, Stefano Borini  wrote:
>
> PEP for support for indexing with keyword arguments is now submitted as PR.
>
> https://github.com/python/peps/pull/1612
>
> Thanks to everybody involved in the development of the PEP and the
> interesting discussion. All your contributions have been received and
> often added to the document.
> If the PEP is approved, I would like to attempt an implementation, but
> I am not particularly skilled in the python internals. If a core
> developer is willing to teach me the ropes (especially in the new
> parser, I barely understood the syntax of the old one, but have no
> idea about the new one) and review my code I can give it a try. I
> would not mind refreshing my C a bit.

-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/25ZPXADTQNJRMCSG4BJO4QJ7OQAL67N6/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Ben Rudiak-Gould
On Wed, Sep 23, 2020 at 3:10 AM David Mertz  wrote:

> On Tue, Sep 22, 2020 at 11:55 PM Paul Moore  wrote:
>
>> The point of this request is that Python's packaging infrastructure is
>> looking at what compression we use for wheels - the current
>> compression is suboptimal for huge binaries like tensorflow.
>
>
> There are definitely some intermediate compression levels where both
> brotli and zstd are significantly faster [than lzma], but not at the higher
> levels where lzma does as well or better.
>

I'd assume that only decompression speed matters for packages, and on that
metric both brotli and zstd beat lzma by a mile regardless of the
compression level.

But I think that lzma gets exceptionally good ratios on x86/x64 machine
code. Even after all these years it seems to be the state of the art for
"best ratio that isn't painfully slow to decompress".


On Wed, Sep 23, 2020 at 3:10 AM David Mertz  wrote:

> On Tue, Sep 22, 2020 at 11:55 PM Paul Moore  wrote:
>
>> The point of this request is that Python's packaging infrastructure is
>> looking at what compression we use for wheels - the current
>> compression is suboptimal for huge binaries like tensorflow. Packaging
>> is in a unique situation, because it *cannot* use external libraries
>
>
> It's hard to see where packaging would have any advantage with brotli or
> zstd over lzma.  XZ is more widely used, and package size seems to dominate
> speed.  There are definitely some intermediate compression levels where
> both brotli and zstd are significantly faster, but not at the higher levels
> where lzma does as well or better.
>
> Is there a concrete need here, or just an abstract point that compression
> of packages shouldn't be outside the stdlib?
>
> Honestly, if you really want compression size over everything else, PPM is
> going to beat the LZ based approaches.  But being ungodly slow and using
> tons of memory.
>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born.  Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SC3SQKFK4GHASRNQJXFYDMNRB25P7SJ4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IIRUPDNBCPIGUM5USU7WSWGW5AMBTRMB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Antoine Pitrou
On Wed, 23 Sep 2020 13:26:13 +0300
Omer Katz  wrote:
> I pointed out a use case for Brotli & HTTP2 as a concrete example for why
> it'd be more convenient to include brotli as a module.
> I'm sure there are other cases I haven't thought about.
> 
> I don't understand why LZMA should be included while zstd or brotli
> shouldn't.
> What's the actual policy here?

The main policy requirement is a maintainer willing to shepherd
inclusion and then maintain the module in the stdlib.  One person
volunteered at some point to do that for LZMA, and it had desirabled
characteristics as a compression algorithm, so it happened.

(and when LZMA was included in 2011, ZStandard didn't even exist AFAIK)

Regards

Antoine.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/42AHQUBS6YB2GD2WUNQW3TMT3EAI6FR5/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread David Mertz
Let's put it this way. If you can only support 3 compression algorithms in
the stdlib, which there would you choose? If only 4? If only 10?

Each one is concrete maintenance work. There's nothing *wrong* with any of
them, and someone uses each of the top 10 or 50. But some kind of cut-off
of usefulness vs burden is necessary.

Paul's case of packaging needing stdlib support is plausible. I'm not quite
convinced even by that though. Just like we have ensurepip, it would be
perfectly possible to make whizbang-compression the first dependency of
every other package, but package whizbang itself using plain tar.

However, HTTP2 is absolutely something that's fine for PyPI. That really
doesn't need to be stdlib.

On Wed, Sep 23, 2020, 12:26 AM Omer Katz  wrote:

> I pointed out a use case for Brotli & HTTP2 as a concrete example for why
> it'd be more convenient to include brotli as a module.
> I'm sure there are other cases I haven't thought about.
>
> I don't understand why LZMA should be included while zstd or brotli
> shouldn't.
> What's the actual policy here?
>
> ‫בתאריך יום ד׳, 23 בספט׳ 2020 ב-13:09 מאת ‪David Mertz‬‏ <‪me...@gnosis.cx
> ‬‏>:‬
>
>> On Tue, Sep 22, 2020 at 11:55 PM Paul Moore  wrote:
>>
>>> The point of this request is that Python's packaging infrastructure is
>>> looking at what compression we use for wheels - the current
>>> compression is suboptimal for huge binaries like tensorflow. Packaging
>>> is in a unique situation, because it *cannot* use external libraries
>>
>>
>> It's hard to see where packaging would have any advantage with brotli or
>> zstd over lzma.  XZ is more widely used, and package size seems to dominate
>> speed.  There are definitely some intermediate compression levels where
>> both brotli and zstd are significantly faster, but not at the higher levels
>> where lzma does as well or better.
>>
>> Is there a concrete need here, or just an abstract point that compression
>> of packages shouldn't be outside the stdlib?
>>
>> Honestly, if you really want compression size over everything else, PPM
>> is going to beat the LZ based approaches.  But being ungodly slow and using
>> tons of memory.
>>
>> --
>> The dead increasingly dominate and strangle both the living and the
>> not-yet born.  Vampiric capital and undead corporate persons abuse
>> the lives and control the thoughts of homo faber. Ideas, once born,
>> become abortifacients against new conceptions.
>>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NCPRPEO5SSCIEQSDPTA5REHARYQPSLJW/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Paul Moore
On Wed, 23 Sep 2020 at 11:09, David Mertz  wrote:

> It's hard to see where packaging would have any advantage with brotli or zstd 
> over lzma.  XZ is more widely used, and package size seems to dominate speed. 
>  There are definitely some intermediate compression levels where both brotli 
> and zstd are significantly faster, but not at the higher levels where lzma 
> does as well or better.
>
> Is there a concrete need here, or just an abstract point that compression of 
> packages shouldn't be outside the stdlib?
>
> Honestly, if you really want compression size over everything else, PPM is 
> going to beat the LZ based approaches.  But being ungodly slow and using tons 
> of memory.

The discussion over on the Packaging discourse channel is ongoing, and
hasn't reached any conclusion yet. But yes, there have been lots of
debates over what's the best compression method. That debate triggered
this request, but it doesn't indicate a specific need for these
compression methods. I can't speak directly for Omer, but I assume the
intention was to explore what's possible/likely, to inform the
discussion of wheel compression with some concrete information about
what we can reasonably assume might be in the stdlib.

I don't have a personal stake here - I'd likely never use brotli or
zstd unless a file format that I needed to process used them, and for
any use case *other* than packaging, I don't currently care if the
support is in the stdlib or not. If either format becomes commonly
used, then I'd argue for them being in the stdlib, but right now they
seem relatively uncommon in my personal experience.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y7NKIMU7AKMCZSV2KCWIVUJKVO4JZAUO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Omer Katz
I pointed out a use case for Brotli & HTTP2 as a concrete example for why
it'd be more convenient to include brotli as a module.
I'm sure there are other cases I haven't thought about.

I don't understand why LZMA should be included while zstd or brotli
shouldn't.
What's the actual policy here?

‫בתאריך יום ד׳, 23 בספט׳ 2020 ב-13:09 מאת ‪David Mertz‬‏ <‪me...@gnosis.cx
‬‏>:‬

> On Tue, Sep 22, 2020 at 11:55 PM Paul Moore  wrote:
>
>> The point of this request is that Python's packaging infrastructure is
>> looking at what compression we use for wheels - the current
>> compression is suboptimal for huge binaries like tensorflow. Packaging
>> is in a unique situation, because it *cannot* use external libraries
>
>
> It's hard to see where packaging would have any advantage with brotli or
> zstd over lzma.  XZ is more widely used, and package size seems to dominate
> speed.  There are definitely some intermediate compression levels where
> both brotli and zstd are significantly faster, but not at the higher levels
> where lzma does as well or better.
>
> Is there a concrete need here, or just an abstract point that compression
> of packages shouldn't be outside the stdlib?
>
> Honestly, if you really want compression size over everything else, PPM is
> going to beat the LZ based approaches.  But being ungodly slow and using
> tons of memory.
>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born.  Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QXFFEBJTBJLA24ZVPCIWA37TCDPJXVFE/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread David Mertz
On Tue, Sep 22, 2020 at 11:55 PM Paul Moore  wrote:

> The point of this request is that Python's packaging infrastructure is
> looking at what compression we use for wheels - the current
> compression is suboptimal for huge binaries like tensorflow. Packaging
> is in a unique situation, because it *cannot* use external libraries


It's hard to see where packaging would have any advantage with brotli or
zstd over lzma.  XZ is more widely used, and package size seems to dominate
speed.  There are definitely some intermediate compression levels where
both brotli and zstd are significantly faster, but not at the higher levels
where lzma does as well or better.

Is there a concrete need here, or just an abstract point that compression
of packages shouldn't be outside the stdlib?

Honestly, if you really want compression size over everything else, PPM is
going to beat the LZ based approaches.  But being ungodly slow and using
tons of memory.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SC3SQKFK4GHASRNQJXFYDMNRB25P7SJ4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Paul Moore
On Wed, 23 Sep 2020 at 08:08, Paul Sokolovsky  wrote:
> In the meantime, why can't you use modules on PyPI/github/wherever
> else?

There are significant use cases where 3rd party modules are not easy
to use. But let's not get sucked into that digression here.

The point of this request is that Python's packaging infrastructure is
looking at what compression we use for wheels - the current
compression is suboptimal for huge binaries like tensorflow. Packaging
is in a unique situation, because it *cannot* use external libraries -
you can't have the wheel format depend on something that you need to
install a wheel to use. If the decision is not to include these, we
just have to stick to the existing compression methods, and accept the
cost. But it's certainly worth looking at the possibility.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6YBOIHWDIYFLQL6XTZUYXPLIOY2ROM2G/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Omer Katz
Mainly because we previously explored creating wheels with better
compression.
But I think that if LZMA was included, then other new algorithms should be
included as well.

בתאריך יום ד׳, 23 בספט׳ 2020, 10:08, מאת Paul Sokolovsky ‏:

> Hello,
>
> On Wed, 23 Sep 2020 06:11:27 -
> "Omer Katz"  wrote:
>
> > zstd is used when you need fast compression speed, not the best ratio.
> >
> > Maybe we can ask google and facebook to contribute their
> > implementations?
>
> And $$$ to support maintaining it over the years.
>
> In the meantime, why can't you use modules on PyPI/github/wherever
> else? Why CPython stdlib should become an ugly ever-growing monster,
> siphoning, like a black hole, everything around?
>
>
> --
> Best regards,
>  Paul  mailto:pmis...@gmail.com
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W5U6IWU5RAEFGKTJMBYODGYY5YRE77WH/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Paul Sokolovsky
Hello,

On Wed, 23 Sep 2020 06:11:27 -
"Omer Katz"  wrote:

> zstd is used when you need fast compression speed, not the best ratio.
> 
> Maybe we can ask google and facebook to contribute their
> implementations?

And $$$ to support maintaining it over the years.

In the meantime, why can't you use modules on PyPI/github/wherever
else? Why CPython stdlib should become an ugly ever-growing monster,
siphoning, like a black hole, everything around?


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PWQNU6L5EPNQE2LRALOQQGGVQXPPGZNR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules

2020-09-23 Thread Omer Katz
zstd is used when you need fast compression speed, not the best ratio.

Maybe we can ask google and facebook to contribute their implementations?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5LUJ4CC25OQX54K6DU35TP2AODGCW3BX/
Code of Conduct: http://python.org/psf/codeofconduct/