[Python-Dev] Re: Switching to Discourse

[Alex Krupp via Python-Dev]

> This does not solve the problem of engaging actively in a discussion, of
course

I just submitted a proposal to create a Discourse plugin to improve the
accuracy of their inbound email parsing, which is something that several
people have complained about in this thread. This would enable two things:

   - Folks who prefer to live in their inbox could continue to do so and
   contribute by just replying to emails. Discourse currently has
   reply-by-email, but it often mangles formatting and/or entirely deletes
   text. Once these issues are fixed, folks who like the current experience
   would be able to just pretend the forum doesn't exist and continue having
   the same experience as they currently have with GNU Mailman.
   - Right now importing the archives from GNU Mailman into Discourse isn't
   realistic for the same reasons; some messages will import correctly, but
   others will be mangled or missing text. This means you would still need to
   maintain the Malman archive as the canonical source of truth. Once fixed,
   not only would the [Python-Dev] archives be searchable within Discourse,
   but they should also rank better in search than they do in their current
   archive.

If this is something you care about (positively or negatively), here is the
exploratory proposal:

https://meta.discourse.org/t/proposed-plugin-to-improve-reply-by-email-accuracy/252944

Any feedback and/or testing would be much appreciated! Right now Discourse
recognizes that this is a problem and is interested in solving it, but
getting it prioritized will require folks to A) speak up saying they want
it done B) test the underlying API to verify that it actually solves the
problem.

Alex

On Sun, Dec 11, 2022 at 1:54 PM Tiziano Zito  wrote:

>
> On Sat 10 Dec, 17:47 +0100, Baptiste Carvello <
> devel2...@baptiste-carvello.net> wrote:
> >There is a small catch though: unless I'm mistaken, Discourse won't let
> >you subscribe to just a set of categories, so any filtering has to
> >happen on the Mailman side.
>
> Well, it is actually possible to achieve what you want.
>
> I have set up Discourse in mailing-list mode [1].
>
> By default muted categories are not included in the emails you get in
> mailing list mode.
>
> So, you just need to mute all categories you don't care about. It is a bit
> of work, but it needs to be done only once. To have an almost complete
> equivalent of the topics that were once discussed on python-dev, you can
> just mute every thing except the "Core Development" category. This is the
> setting I am using since a while and I am quite happy with it. You may want
> to unmute the "PEPs" category as well.
>
> Threading info is kept quite nicely, so I read the discourse mail
> notifications as if it were a mailing list and I almost do not see any
> difference. Text is sometimes a bit messy if people heavily use the
> discourse formatting capabilities, but this kind of posts are quite rare in
> my experience.
>
> This does not solve the problem of engaging actively in a discussion, of
> course, but at least for me it is OK to login to discourse if I have to
> post, given that 99.99% of the time I just want to read posts in my mail
> client.
>
> Ciao!
> Tiziano
>
> [1] You can do this while editing your profile preferences, under the
> "Emails" menu
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7ZJWPADSL7BGBZ5Y6BRHP2LDTHQFZ7UV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 

Alex Krupp
Cell: (607) 351 2671

Read my Email: www.fwdeveryone.com/u/alex3917
Subscribe to my blog: https://alexkrupp.typepad.com/
My homepage: www.alexkrupp.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y3BO5TBIM2YQXWCQ4D4RPOBBIDVHNXAL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Alex Waygood]

There's also the "Experts index" in the devguide: 
https://devguide.python.org/experts/#expertsBest, Alex
 Original message From: Skip Montanaro 
 Date: 29/03/2022  22:36  (GMT+00:00) To: "Eric V. 
Smith"  Cc: Python Dev  Subject: 
[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now 
time to deprecate the stdlib urllib module] There's the CODEOWNERS file: 
https://github.com/python/cpython/blob/main/.github/CODEOWNERSThanks. Never 
would have thought there was such a thing. I was looking for files with 
"maintain" in them. Skimming it, it would seem that most of the stuff in Lib or 
Modules isn't really associated with a particular person or small group.Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CN7SXNP2D4ML4U2OBIUC4DGQOKEYNSKE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Restrict the type of __slots__

[Alex Waygood]

I'm +1 on disallowing strings specifically. The unexpected behaviour that 
ensues if you try to iterate over a class's slots is a footgun that I've 
trodden on a few times, and it's always been annoying.

But I don't see any particular reason to disallow any other kind of iterable, 
and I'm definitely -1 on disallowing using a dictionary for __slots__. Using a 
dictionary for __slots__ as a way to document attributes is a feature I very 
much like.

Best,
Alex

> On 18 Mar 2022, at 20:59, Guido van Rossum  wrote:
> 
> 
>> On Fri, Mar 18, 2022 at 9:40 AM Paul Bryan  wrote:
>>> On Fri, 2022-03-18 at 09:35 -0700, Guido van Rossum wrote:
>>> The motivation has been explained already.
>> 
>> In this thread?
> 
> Yes, Eric's message.
>  
>>> What on earth did your test do that got a speedup by using sets? Was it 
>>> repeatedly checking whether a variable was in a slot? The other thing this 
>>> does is rearrange the order in which slots appear from run to run (since 
>>> set order is affected by hash randomization) and you might have gotten 
>>> lucky with a popular attribute being moved to the front, where it's more 
>>> likely to be in the memory cache already (cache lines being 64 bytes and 
>>> pointers being 8 bytes nowadays).
>> 
>> I created objects in a tight loop, populating attributes, noting the elapsed 
>> time.
> 
> It does not make sense that that is correlated to the type of __slots__, 
> since __slots__ is not used for instance creation at all (it is only used to 
> create the class). I stick to my likely explanation.
> 
> Regarding Serhiy's proposal, I'm +0 on disallowing strings, and +0 on 
> disallowing things that can't be reiterated (but it's not a problem in 
> practice). Given other responses the status quo is likely best.
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/EYZ32GBU4QUTRPU26BDCYYMKG4GX633N/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PO32SWLW27Y2MUWROXD7UUPTUTXTDEEE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Increase of Spammy PRs and PR reviews

[Alex Waygood]

Nikita found a very real (and slightly embarrassing!) bug in a patch I wrote 
for the enum module a few months back, due to his efforts to improve test 
coverage. And there is an entire section of the DevGuide devoted to "Improving 
test coverage", stating that PRs such as the ones Nikita has been filing are 
valuable.I have seen multiple PRs by other authors closed or criticised because 
they tried to change things in too many modules at once, or make "broad and 
sweeping" changes. Nikita, however, has always taken care to make his PRs easy 
to review, by keeping them small and focused.His mentor has already stated that 
he specifically asked Nikita to cc him in whenever he filed PRs (and I 
certainly didn't get a "bad impression about his intentions" from the fact he 
was cc'ing in his mentor, which seems like a very reasonable thing for a new 
member of the triage team to do).I'm only a triager (like Nikita), but I really 
don't see any problem here, personally.Best wishes, Alex
 Original message From: Nikita Sobolev  Date: 
01/02/2022  04:14  (GMT+00:00) To: python-dev@python.org Subject: [Python-Dev] 
Re: Increase of Spammy PRs and PR reviews Hi, my name is Nikita and I think 
that I am the person behind these spammy PRs.Link: 
https://github.com/python/cpython/pulls/sobolevnFirst of all, Lrupert, sorry 
for all the noise and inconvenience I caused to you personally and to other 
community members! This totally was **not** my intention. You could have 
reached out to me earlier via email or directly in some PR to share your 
concerns, maybe we could have this whole situation avoided.Secondly, thanks for 
letting me know about how others might feel about my work. Feedback is always 
important. I hope I can learn a lot from this case.But, I can tell you honestly 
that I've never been to a situation like this before and I don't know exactly 
how to handle it and what to improve in this specific case.Third, I agree that 
almost all my PRs were about some trivial things. Mostly because I was reading 
through typeshed code (which requires looking through multiple CPython modules 
at a very high level) and test code for these modules where I've spotted quite 
a lot of details to fix.I cannot obviously judge the quality of my work (except 
for being ok-ish), so I will leave this part out of scope. The only thing I can 
say here is that it's a bit sad thread to read on python-dev mailing list, but 
I will keep my optimism for the future :)So, to sum things up:- I am open to 
any feedback: if others also think that my work does not bring any value - this 
is fine! I can totally improve or work on something simpler / some other 
project. Anyone interested can write me a direct email: m...@sobolevn.me- I am 
sorry for causing this thread. It is far from being a pleasant or technical 
readBest Regards,Nikita 
Sobolevhttps://github.com/sobolevn___Python-Dev
 mailing list -- python-dev@python.orgTo unsubscribe send an email to 
python-dev-leave@python.orghttps://mail.python.org/mailman3/lists/python-dev.python.org/Message
 archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LF7V3ZASXK6DGD5MBBXP3YKHGOLW36D5/Code
 of Conduct: http://python.org/psf/codeofconduct/___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S772DSPACD6346E7TIG5EL47CBQ4JWU6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Alex Martelli via Python-Dev]

Indeed, normative annex https://www.unicode.org/reports/tr31/tr31-35.html
section 5 says: "if the programming language has case-sensitive
identifiers, then Normalization Form C is appropriate" (vs NFKC for a
language with case-insensitive identifiers) so to follow the standard we
should have used NFC rather than NFKC. Not sure if it's too late to fix
this "oops" in future Python versions.

Alex

On Sun, Nov 14, 2021 at 9:17 AM Christopher Barker 
wrote:

> On Sat, Nov 13, 2021 at 2:03 PM  wrote:
>
>> def 횑퓮햑풍표():
>>
>> try:
>>
>> 픥e헅핝횘︴ = "Hello"
>>
>> 함픬r퓵ᵈ﹎ = "World"
>>
>> ᵖ햗퐢혯퓽(f"{헵ｅ퓵픩º_}, {햜ₒ풓lⅆ︴}!")
>>
>> except 퓣핪ᵖｅ햤헿ᵣ햔횛 as ⅇ헑c:
>>
>> 풑rℹₙₜ("failed: {}".핗헼ʳᵐªｔ(ᵉ퐱퓬))
>>
>
> Wow. Just Wow.
>
> So why does Python apply  NFKC normalization to variable names?? I can't
> for the life of me figure out why that would be helpful at all.
>
> The string methods, sure, but names?
>
> And, in fact, the normalization is not used for string comparisons or
> hashes as far as I can tell.
>
> In [36]: weird
> Out[36]: 'ᵖ햗퐢혯퓽'
>
> In [37]: normal
> Out[37]: 'print'
>
> In [38]: eval(weird + "('yup, that worked')")
> yup, that worked
>
> In [39]: weird == normal
> Out[39]: False
>
> In [40]: weird[0] in normal
> Out[40]: False
>
> This seems very odd (and dangerous) to me.
>
> Is there a good reason? and is it too late to change it?
>
> -CHB
>
>
>
>
>
>
>
>
>
>>
>>
>> if _︴ⁿ퓪푚핖__ == "__main__":
>>
>> 풉eℓˡ허()
>>
>>
>>
>>
>>
>> # snippet from unittest/util.py
>>
>> _퓟Ⅼ햠홲험ℋ풪Lᴰ푬핽﹏핷피헡 = 12
>>
>> def _픰ʰ퓸ʳ핥홚푛(픰, p푟픢fi햝핝횎푛, ｓᵤ푓헳헂푥헹ₑ횗):
>>
>> ˢ헸ｉ헽 = 퐥ｅ혯(햘) - ｐr횎햋퐢x헅ᵉ퓷 - 풔홪ffｉ혅헹홚ₙ
>>
>> if sｋi혱 > _퐏헟햠혊홴H핺Ｌ핯홀혙﹏L픈풩:
>>
>> 혴 = '%s[%d chars]%s' % (홨[:혱퐫핖푓핚ｘℓ풆핟], ₛ횔풊p, 퓼[퓁풆햓
>> (횜) - 홨횞풇fix홡ᵉ혯:])
>>
>> return ₛ
>>
>>
>>
>>
>>
>> You should able to paste these into your local UTF-8-aware editor or IDE
>> and execute them as-is.
>>
>>
>>
>> (If this doesn’t come through, you can also see this as a GitHub gist at 
>> Hello,
>> World rendered in a variety of Unicode characters (github.com)
>> <https://gist.github.com/ptmcg/bf35d5ada416080d481d789988b6b466>. I have
>> a second gist containing the transformer, but it is still a private gist
>> atm.)
>>
>>
>>
>>
>>
>> Some other discoveries:
>>
>> “·” (ASCII 183) is a valid identifier body character, making “_···” a
>> valid Python identifier. This could actually be another security attack
>> point, in which “s·join(‘x’)” could be easily misread as “s.join(‘x’)”, but
>> would actually be a call to potentially malicious method “s·join”.
>>
>> “_” seems to be a special case for normalization. Only the ASCII “_”
>> character is valid as a leading identifier character; the Unicode
>> characters that normalize to “_” (any of the characters in “︳︴﹍﹎﹏＿”) can
>> only be used as identifier body characters. “︳” especially could be
>> misread as “|” followed by a space, when it actually normalizes to “_”.
>>
>>
>>
>>
>>
>> Potential beneficial uses:
>>
>> I am considering taking my transformer code and experimenting with an
>> orthogonal approach to syntax highlighting, using Unicode groups instead of
>> colors. Module names using characters from one group, builtins from
>> another, program variables from another, maybe distinguish local from
>> global variables. Colorizing has always been an obvious syntax highlight
>> feature, but is an accessibility issue for those with difficulty
>> distinguishing colors. Unlike the “ransom note” code above, code
>> highlighted in this way might even be quite pleasing to the eye.
>>
>>
>>
>>
>>
>> -- Paul McGuire
>>
>>
>>
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/GBLXJ2ZTIMLBD2MJQ4VDNUKFFTPPIIMO/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U3DJOQKMREWY35SHCRSD6V6FQA2T3SW7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Alex Waygood]

I'll join Christopher Barker and Chris Angelico in voicing scepticism --
personally, I feel like I'm yet to be persuaded that the use case is strong
enough. sortedcontainers is a wonderful package, and I would completely
support a prominent link to the library in the Python documentation. But
the use case seems too niche and specific, in my opinion, for the library
to be added to the standard library. I see the standard library as a
collection of basic building blocks that can be used to create a multitude
of other things. sortedcontainers feel like a specialised solution to a
small collection of problems, rather than a general solution to a large
collection of problems. PyPI feels like the right place for this library.

Best,
Alex

On Thu, Nov 11, 2021 at 12:33 PM Paul Moore  wrote:

> On Thu, 11 Nov 2021 at 11:51, Antoine Pitrou  wrote:
> >
> > On Wed, 10 Nov 2021 21:12:17 -0600
> > Tim Peters  wrote:
> > > [Bob Fang ]
> > > > This is a modest proposal to consider having sorted containers
> > > > (http://www.grantjenks.com/docs/sortedcontainers/) in standard
> library.
> > >
> > > +1 from me, but if and only if Grant Jenks (its author) wants that too.
> > >
> > > It's first-rate code in all respects, including that it's a fine
> > > example _of_ Python programming (it's not written in C - in Python).
> >
> > Agreed with Tim.  This is a perfect example of some basic and perennial
> > facility that would fit very well in the stdlib.
>
> I agree as well. Is anyone interested enough to ask the library author
> if he supports doing this? That seems to be the main unanswered
> question here.
>
> But if anyone wants to argue the "the stdlib should be shrinking, not
> growing" position, I suggest they do so *before* someone reaches out
> to the module author. No point in us making the suggestion and then
> being forced to withdraw it.
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5SURNB4C5FGJ6LSXUPVW2EFP22ERKSGB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EM3CCN2YNKVUUQIGNXE5XIIR5TJKSTCZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Relaxing the annotation syntax

[Alex Hall]

What about creating a new syntax for annotating metadata? For example, 
`type_hint :: metadata` could be equivalent to `Annotated[type_hint, 
"metadata"]`, and if we wanted type guards to look like TypeScript they could 
look like this:

```
def is_str_list(val: List[object]) -> bool :: is List[str]:
```

This might not be the best syntax, it's just one idea, but my point is that 
achieving all of these goals simultaneously seems quite doable:

- Showing the actual return type
- Showing metadata
- Putting arbitrary non-Python syntax in metadata
- Retrieving the type part of the annotation at runtime as an actual type 
value, not a string
- Retrieving the metadata at runtime as a string
- Avoiding the cumbersome `Annotated[]` syntax

In addition, if someone wants annotations only for other metadata and not for 
types at all [1] then it's easy to just omit the type part, e.g

```
def foo(bar :: bar metadata) :: foo metadata:
```

Again, the `::` may be a bad way to do this, but the same principle of 
non-type-meta-only-annotations could probably be applied to other similar 
syntax proposals.

I'm sure I'm not the first to suggest something like this, but I couldn't see 
anything in PEP 593. I was particularly expecting something like "New syntax" 
as a section under "Rejected ideas". The closest thing is "Using (Type, Ann1, 
Ann2, ...) instead of Annotated[Type, Ann1, Ann2, ...]" which felt a bit weak.

[1] e.g. as I have done in https://github.com/alexmojaki/friendly_states which 
looks lovely but completely breaks mypy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LJMKGAXQGN2TARLJ2AJI6IUJ2JMJT56E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: In support of PEP 649

[Alex Hall]

I'd like to chime in with an example of how PEP 563 breaks code that uses 
dataclasses.

I've written a library instant_api (https://github.com/alexmojaki/instant_api) 
that is heavily inspired by FastAPI but uses dataclasses for complex types 
instead of pydantic. The example at the beginning of the README is short and 
demonstrates it nicely. Basically it lets you write code on both the client and 
server sides that work seamlessly with standard dataclasses, type hints, and 
type checkers without any plugins, instead of untyped dicts parsed from the 
JSON that is communicated behind the scenes.

`from __future__ import annotations` breaks that README example, even though 
there are no locally defined types, because as mentioned the dataclass field 
now contains a string instead of a type.

Going a bit deeper, instant_api is powered by 
https://github.com/alexmojaki/datafunctions, which is more generic than 
instant_api so that others can build similar tools. Again, the idea is that you 
can write code with nice dataclasses and type hints, but call it with basic 
JSON serializable types like dicts. For example:

```
from dataclasses import dataclass
from datafunctions import datafunction

@dataclass
class Point:
x: int
y: int

@datafunction
def translate(p: Point, dx: int, dy: int) -> Point:
return Point(p.x + dx, p.y + dy)

assert translate({"x": 1, "y": 2}, 3, 4) == {"x": 4, "y": 6}

# This is equivalent to the following without @datafunction
# assert translate(Point(1, 2), 3, 4) == Point(4, 6)
```

In the same way as before, `from __future__ import annotations` breaks this 
code. The reason is that datafunctions itself is powered by 
https://github.com/lovasoa/marshmallow_dataclass. Here's an example:

```
from dataclasses import dataclass
from marshmallow_dataclass import class_schema

@dataclass
class Point:
x: int
y: int

schema = class_schema(Point)()
assert schema.load({"x": 1, "y": 2}) == Point(1, 2)
```

Again, in the same way as before, `from __future__ import annotations` breaks 
this code. Specifically `class_schema(Point)` breaks trying to deal with the 
string `'int'` instead of a type.

This problem was raised in 
https://github.com/lovasoa/marshmallow_dataclass/issues/13 two years ago. It's 
by far the oldest open issue in the repo. It was clear from the beginning that 
it's a difficult problem to solve. Little progress has been made, there's one 
PR that's not in good shape, and it seems there's been no activity there for a 
while. A couple of other issues have been closed as duplicates. One of those 
issues is about being unable to use recursive types at all.

marshmallow_dataclass has 266 stars. It builds on 
https://github.com/marshmallow-code/marshmallow, an extremely popular and 
important data (de)serialization and validation library. Here's a little 
timeline:

- 2013: marshmallow 0.1.0 first released in 2013
- 2014: marshmallow 1.0.0 released
- 2015: attrs (precursor to dataclasses) first released
- 2016: Python 3.6.0 final released, allowing the variable annotations which 
make pydantic and dataclasses possible.
- 2017: First version of pydantic released
- 2018: Python 3.7.0 final released, introducing dataclasses

Nowadays pydantic is the natural successor/alternative to marshmallow - Google 
autocompletes "pydantic vs " with marshmallow as the first option, and vice 
versa. But marshmallow is clearly well established and entrenched, and thanks 
to marshmallow_dataclass it was the better fit for my particular use case just 
last year when I made instant_api.

If someone wants to keep combining dataclasses and marshmallow, but without 
marshmallow_dataclass (e.g. if PEP 563 goes through before 
marshmallow_dataclass is ready) then they need to revert to the raw marshmallow 
API which doesn't use type hints. The previous example becomes much uglier:

```
from dataclasses import dataclass
from marshmallow import Schema, fields, post_load

@dataclass
class Point:
x: int
y: int

class PointSchema(Schema):
x = fields.Int()
y = fields.Int()

@post_load
def make_point(self, data, **kwargs):
return Point(**data)

schema = PointSchema()
assert schema.load({"x": 1, "y": 2}) == Point(1, 2)
```

This post turned out longer than I initially planned! In summary,  my point is 
that type hints and dataclasses as they work right now make it possible to 
write some really nice code - nice for humans to both write and read, nice for 
type checkers and other static analysis, and providing very nice features using 
annotations at runtime. And despite clear demand and benefits and ample time, 
people haven't managed to make this code continue working with stringified type 
annotations. Clearly doing so is not easy. So there's a good case for the 
dataclasses module to resolve these 

[Python-Dev] Re: Allow custom headers in `http.server`

[Alex Yursha]

I thought about adding it as a command line option when invoked as `python3
-m http.server`.

On Sat, Oct 26, 2019, 18:02 Guido van Rossum  wrote:

> Is this not something you can do yourself by calling send_header() after
> calling send_response()?
>
> On Sat, Oct 26, 2019 at 7:33 AM Alex Yursha  wrote:
>
>> Hi CPython maintainers,
>>
>> I need to test my CORS setup and looking for a possibility to set a
>> custom *Access-Control-Allow-Origin *header in http.server. As of now,
>> there is no such feature. Are you interested in me writing a patch to
>> contribute a feature of setting custom headers directly to `http.server`?
>>
>> Best,
>> - Alex
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/TB3HCDXFED3G6VGSKSYLHYJICEDJ7FCA/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SGUIPISTQXY526TIKWAKIKCQN3ECKTBP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Allow custom headers in `http.server`

[Alex Yursha]

Hi CPython maintainers,

I need to test my CORS setup and looking for a possibility to set a
custom *Access-Control-Allow-Origin
*header in http.server. As of now, there is no such feature. Are you
interested in me writing a patch to contribute a feature of setting custom
headers directly to `http.server`?

Best,
- Alex
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TB3HCDXFED3G6VGSKSYLHYJICEDJ7FCA/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Alex Walters]

I've watched the entire thread and its taken me a few days to put a finger
on what bothers me about it.

In my opinion, this shouldn't be a pep describing the list of modules that
need to go as "dead batteries", but should be a process pep describing how
dead batteries should be removed, and the individual modules should be given
their own pep.  I think reactions to individual module peps will give a
better indication of if it's a used module or not.

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Christian Heimes
> Sent: Monday, May 20, 2019 4:15 PM
> To: Python Dev 
> Subject: [Python-Dev] PEP 594: Removing dead batteries from the standard
> library
> 
> Hi,
> 
> here is the first version of my PEP 594 to deprecate and eventually remove
> modules from the standard library. The PEP started last year with talk
during
> the Python Language Summit 2018, https://lwn.net/Articles/755229/.
> 
> The PEP can be confirmed in two stages. I'm not planning any code changes
> for 3.8. Instead I only like to document a bunch of modules as deprecated.
> Active deprecation is planned for 3.9 and removal for 3.10. The long
> deprecation phase gives us 3 years to change our minds or handle edge
> cases, too.
> 
> Regards,
> Christian
> 
> 
>

> PEP: 594
> Title: Removing dead batteries from the standard library
> Author: Christian Heimes 
> Status: Active
> Type: Process
> Content-Type: text/x-rst
> Created: 20-May-2019
> Post-History:
> 
> 
> Abstract
> 
> 
> This PEP proposed a list of standard library modules to be removed from
the
> standard library. The modules are mostly historic data formats and APIs
that
> have been superseded a long time ago, e.g. Mac OS 9 and Commodore.
> 
> 
> Rationale
> =
> 
> Back in the early days of Python, the interpreter came with a large set of
> useful modules. This was often refrained to as "batteries included"
> philosophy and was one of the corner stones to Python's success story.
> Users didn't have to figure out how to download and install separate
> packages in order to write a simple web server or parse email.
> 
> Times have changed. The introduction of the cheese shop (PyPI),
setuptools,
> and later pip, it became simple and straight forward to download and
install
> packages. Nowadays Python has a rich and vibrant ecosystem of third party
> packages. It's pretty much standard to either install packages from PyPI
or
> use one of the many Python or Linux distributions.
> 
> On the other hand, Python's standard library is piling up cruft,
unnecessary
> duplication of functionality, and dispensable features. This is
undesirable
> for several reasons.
> 
> * Any additional module increases the maintenance cost for the Python core
>   development team. The team has limited resources, reduced maintenance
> cost
>   frees development time for other improvements.
> * Modules in the standard library are generally favored and seen as the
>   de-facto solution for a problem. A majority of users only pick 3rd party
>   modules to replace a stdlib module, when they have a compelling reason,
> e.g.
>   lxml instead of `xml`. The removal of an unmaintained stdlib module
>   increases the chances of a community contributed module to become
> widely
>   used.
> * A lean and mean standard library benefits platforms with limited
resources
>   like devices with just a few hundred kilobyte of storage (e.g. BBC
>   Micro:bit). Python on mobile platforms like BeeWare or WebAssembly
>   (e.g. pyodide) also benefit from reduced download size.
> 
> The modules in the PEP have been selected for deprecation because their
> removal is either least controversial or most beneficial. For example
> least controversial are 30 years old multimedia formats like ``sunau``
> audio format, which was used on SPARC and NeXT workstations in the late
> 1980ties. The ``crypt`` module has fundamental flaws that are better
solved
> outside the standard library.
> 
> This PEP also designates some modules as not scheduled for removal. Some
> modules have been deprecated for several releases or seem unnecessary at
> first glance. However it is beneficial to keep the modules in the standard
> library, mostly for environments where installing a package from PyPI is
not
> an option. This can be cooperate environments or class rooms where
> external
> code is not permitted without legal approval.
> 
> * The usage of FTP is declining, but some files are still provided over
>   the FTP protocol or hosters offer FTP to upload content. Therefore
>   ``ftplib`` is going to stay.
> * The ``optparse`` and ``getopt`` module are widely used. They are mature
>   modules with very low maintenance overhead.
> * According to David Beazley [5]_ the ``wave`` module is easy to teach to
>   kids and can make crazy sounds. Making a computer generate crazy sounds
> is
>   powerful and highly motivating exercise for 

Re: [Python-Dev] PEPs from non-core devs now need a sponsor

[Alex Walters]

I'm confused about this.  Didn't you need someone with merge permissions 
already to merge a pep into the pep repo?  Isn't this just adding a layer of 
paperwork to something that was already the case for all practical purposes?

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Brett Cannon
> Sent: Monday, March 4, 2019 8:44 PM
> To: python-dev 
> Subject: [Python-Dev] PEPs from non-core devs now need a sponsor
> 
> The steering council has implemented a new idea called sponsors to the PEP
> process (added in
> https://github.com/python/peps/commit/c58d32c33bd06eb386d3f33963a14
> 34510528f68). The thinking is that to help make sure PEPs from non-core
> developers receive appropriate guidance through the PEP process, a core
> developer needs to sign on to be a sponsor of the PEP. Being a sponsor does
> not preclude the core dev from eventually becoming a co-author or BDFL-
> delegate later on (but obviously not both), but the expectation is the
> sponsor is supportive of the idea (because if a single core dev won't sign on
> to help then what chance does the PEP have of being accepted?).
> 
> 
> If this doesn't turn out well we can obviously revert this, but hopefully this
> will make things smoother for those who are new to the PEP process.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Get a running instance of the doc for a PR.

[Alex Walters]

Doesn't read the docs already do this for pull requests?  Even if it doesn't, 
don't the core maintainers of read the docs go to pycon?  I wouldn't suggest 
read the docs for primary docs hosting for python, but they are perfectly fine 
for live testing pull request documentation without having to roll our own.

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Stephane Wirtel
> Sent: Sunday, November 4, 2018 8:38 AM
> To: python-dev@python.org
> Subject: [Python-Dev] Get a running instance of the doc for a PR.
> 
> Hi all,
> 
> When we receive a PR about the documentation, I think that could be
> interesting if we could have a running instance of the doc on a sub
> domain of python.org.
> 
> For example, pr-1-doc.python.org or whatever, but by this way the
> reviewers could see the result online.
> 
> The workflow would be like that:
> 
> New PR -> build the doc (done by Travis) -> publish it to a server ->
> once published, the PR is notified by "doc is available at URL".
> 
> Once merged -> we remove the doc and the link (hello bedevere).
> 
> I am interested by this feature and if you also interested, tell me.
> I would like discuss with Julien Palard and Ernest W.  Durbin III for a
> solution as soon as possible.
> 
> Have a nice day,
> 
> Stéphane
> 
> --
> Stéphane Wirtel - https://wirtel.be - @matrixise
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Change in Python 3's "round" behavior

[Alex Walters]

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Greg Ewing
> Sent: Saturday, September 29, 2018 9:50 PM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] Change in Python 3's "round" behavior
> 
> I don't really get the statistical argument. If you're doing something
> like calculating an average and care about accuracy, why are you rounding
> the values before averaging? Why not average first and then round the
> result if you need to?
> 

Other use case is finance, where you can end up with interest calculations
that are fractional of the base unit of currency.  US$2.345 is impossible to
represent in real currency, so it has to be rounded.  With
half-towards-even, that rounds to $2.34, and $2.355 rounds to $2.36.  It
evens out in the long run.  While that is very helpful for finance
calculations, if you are doing finance with that level of precision, you
should be using decimal instead of float anyways and decimal's round has
configurable round method.

> --
> Greg
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Change in Python 3's "round" behavior

[Alex Walters]

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Steven D'Aprano
> Sent: Thursday, September 27, 2018 9:54 AM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] Change in Python 3's "round" behavior
> 
> On Thu, Sep 27, 2018 at 05:55:07PM +1200, Greg Ewing wrote:
> > j...@math.brown.edu wrote:
> > >I understand from
> > >https://github.com/cosmologicon/pywat/pull/40#discussion_r219962259
> > >that "to always round up... can theoretically skew the data"
> >
> > *Very* theoretically. If the number is even a whisker bigger than
> > 2.5 it's going to get rounded up regardless:
> >
> > >>> round(2.501)
> > 3
> >
> > That difference is on the order of the error you expect from
> > representing decimal fractions in binary, so I would be surprised
> > if anyone can actually measure this bias in a real application.
> 
> I think you may have misunderstood the nature of the bias. It's not
> about individual roundings and it definitely has nothing to do with
> binary representation.
> 
> Any one round operation will introduce a bias. You had a number, say
> 2.3, and it gets rounded down to 2.0, introducing an error of -0.3. But
> if you have lots of rounds, some will round up, and some will round
> down, and we want the rounding errors to cancel.
> 
> The errors *almost* cancel using the naive rounding algorithm as most of
> the digits pair up:
> 
> .1 rounds down, error = -0.1
> .9 rounds up, error = +0.1
> 
> .2 rounds down, error = -0.2
> .8 rounds up, error = +0.2
> 
> etc. If each digit is equally likely, then on average they'll cancel and
> we're left with *almost* no overall error.
> 
> The problem is that while there are four digits rounding down (.1
> through .4) there are FIVE which round up (.5 through .9). Two digits
> don't pair up:
> 
> .0 stays unchanged, error = 0
> .5 always rounds up, error = +0.5
> 
> Given that for many purposes, our data is recorded only to a fixed
> number of decimal places, we're dealing with numbers like 0.5 rather
> than 0.51, so this can become a real issue. Every ten rounding
> operations will introduce an average error of +0.05 instead of
> cancelling out. Rounding introduces a small but real bias.
> 
> The most common (and, in many experts' opinion, the best default
> behaviour) is Banker's Rounding, or round-to-even. All the other digits
> round as per the usual rule, but .5 rounds UP half the time and DOWN the
> rest of the time:
> 
> 0.5, 2.5, 3.5 etc round down, error = -0.5
> 1.5, 3.5, 5.5 etc round up, error = +0.5
> 
> thus on average the .5 digit introduces no error and the bias goes away.
> 
> 

...and we have a stats module that would be a great place for a round
function that needs to cancel rounding errors.  The simple case should be
the intuitive case for most users.  My experience and that of many users of
the python irc channel on freenode is that round-half-to-even is not the
intuitive, or even desired, behavior - round-half-up is.  This wouldn't be
frustrating to the human user if the round built-in let you pick the method,
instead you have to use the very complicated decimal module with a modified
context to get intuitive behavior.

I could live with `round(2.5) -> 2.0` if `round(2.5, method='up') -> 3.0`
(or some similar spelling) was an option.  As it stands, this is a wart on
the language.  "Statistically balanced rounding" is a special case, not the
default case.

> 
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

[Alex Walters]

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Victor Stinner
> Sent: Thursday, July 12, 2018 4:01 AM
> To: Serhiy Storchaka 
> Cc: python-dev 
> Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
> 
> 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka :
> >> Is there any real application which marshal.dumps() performance is
> >> critical?
> >
> > EVE Online is a well known example.
> 
> EVE Online has been created in 2003. I guess that it still uses Python
2.7.
> 
> I'm not sure that a video game would pick marshal in 2018.
> 

EVE doesn't use stock CPython, IIRC.  They use a version of stackless 2,
with their own patches.  If a company is willing to patch python itself, I
don't think their practices should be cited without more context about what
they actually modified.

> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

[Alex Walters]

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Paul Moore
 
> Why not just have a second button, "Download Python 3.7.0 (64-bit)"
> alongside or below the "Download Python 3.7.0" button? People who
> don't know the difference will just ignore it, people who do will be
> able to choose straight from the main download page.
> 

I think this is the solution.

* If you don't know your architecture, 32-bit will mostly work so should
remain the default (I say most, not all, since there are windows versions
that run on ARM, but I think you can only install software through the store
on those anyways.)

* It's not exposed in the download drop down at all that x64 editions exist
for those who know they want it.  The drop down is wide enough to support a
second download button.

> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Microsoft to acquire GitHub for $7.5 b

[Alex Walters]

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Ivan Pozdeev via Python-Dev
> Sent: Tuesday, June 5, 2018 10:01 PM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] Microsoft to acquire GitHub for $7.5 b
> 
> On 05.06.2018 17:28, Martin Gainty wrote:
> who owns the Data hosted on Github?
> 
> Github Author?
> Microsoft?
> 
> Martin
> https://help.github.com/articles/github-terms-of-service/#d-user-
> generated-content :
> 
> "You own content you create, but you allow us certain rights to it, so
that we
> can display and share the content you post. You still have control over
your
> content, and responsibility for it, and the rights you grant us are
limited to
> those we need to provide the service. We have the right to remove content
> or close Accounts if we need to."
> 
> 
And this is not likely to change under Microsoft.  CodePlex had similar
language, as does BitBucket, GitLab, and pretty much any service that hosts
creative content that isn’t a social network (and even then, some of them do
too.)

> 
> 
> From: Python-Dev  bounces+mgainty=hotmail@python.org> on behalf of
> M.-A. Lemburg 
> Sent: Tuesday, June 5, 2018 7:54 AM
> To: Antoine Pitrou; python-dev@python.org
> Subject: Re: [Python-Dev] Microsoft to acquire GitHub for
> $7.5 billion
> 
> Something that may change is the way they treat Github
> accounts, after all, MS is very much a sales driven company.
> 
> But then there's always the possibility to move to Gitlab
> as alternative (hosted or run on PSF VMs), so I would
> worry too much.
> 
> Do note, however, that the value in Github is not so much
> with
> the products they have, but with the data. Their databases
> know more about IT developer than anyone else and given
> that Github is put under the AI umbrella in MS should tell
> us something :-)
> 
> 
> On 04.06.2018 19:02, Antoine Pitrou wrote:
> >
> > That's true, but Microsoft has a lot of stakes in the
> ecosystem.
> > For example, since it has its own CI service that it tries to
> promote
> > (VSTS), is it in Microsoft's best interest to polish and
> improve
> > integrations with other CI services?
> >
> > Regards
> >
> > Antoine.
> >
> >
> > On Mon, 4 Jun 2018 09:06:28 -0700
> > Guido van Rossum  wrote:
> >> On Mon, Jun 4, 2018 at 8:40 AM, Antoine Pitrou
>  wrote:
> >>
> >>>
> >>> On Mon, 4 Jun 2018 17:03:27 +0200
> >>> Victor Stinner  wrote:
> 
>  At this point, I have no opinion about the event :-) I just
> guess that
>  it should make GitHub more sustainable since Microsoft
> is a big
>  company with money and interest in GitHub. I'm also
> confident that
>  nothing will change soon. IMHO there is no need to
> worry about
>  anything.
> >>>
> >>> It does spell uncertainty on the long term.  While there is
> no need to
> >>> worry for now, I think it gives a different colour to the
> debate about
> >>> moving issues to Github.
> >>>
> >>
> >> I don't see how this *increases* the uncertainty. Surely if
> GitHub had
> >> remained independent there would have been be similar
> concerns about how it
> >> would make enough money to stay in business.
> >>
> >
> >
> __
> _
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> Python-
> Dev Info
> Page
> mail.pyth
> on.org
> Do not
> post
> general
> Python
> questions
> to this list.
> For help
> with
> Python
> please
> see the
> Python
> help
> page.. On
> this list
> the key
> Python
> developer
> s discuss
> the future
> of the
> language
> and its
> implemen
> tation.
> 
> 
> 
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-
> dev/mal%40egenix.com
> >
> 
> --
> Marc-Andre Lemburg
> eGenix.com
> 
> Professional Python Services directly from the Experts (#1,
> Jun 05 2018)
> >>> Python Projects, Coaching and Consulting
> ...  http://www.egenix.com/
> >>> Python Database Interfaces
> ...   http://products.egenix.com/
> >>> Plone/Zope Database Interfaces
> ...   http://zope.egenix.com/
> __
> __
> 
> ::: We implement business ideas - efficiently in both time and
> costs :::
> 
>    eGenix.com Software, Skills and Services GmbH  Pastor-
> Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-
> Andre Lemburg
>    Registered at Amtsgericht Duesseldorf: HRB 46611
>    http://www.egenix.com/company/contact/
>   http://www.malemburg.com/
> 
> __
> _
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-
> dev/mgainty%40hotmail.com
> 
> 
> 
> __
> _
> Python-Dev mailing list
> Python-Dev@python.org
> 

Re: [Python-Dev] What is the rationale behind source only releases?

[Alex Walters]

> -Original Message-
> From: Paul Moore <p.f.mo...@gmail.com>
> Sent: Wednesday, May 16, 2018 4:07 AM
> To: Alex Walters <tritium-l...@sdamon.com>
> Cc: Python Dev <python-dev@python.org>
> Subject: Re: [Python-Dev] What is the rationale behind source only releases?
> 
> On 16 May 2018 at 05:35, Alex Walters <tritium-l...@sdamon.com> wrote:
> > In the spirit of learning why there is a fence across the road before I tear
> > it down out of ignorance [1], I'd like to know the rationale behind source
> > only releases of cpython.  I have an opinion on their utility and perhaps an
> > idea about changing them, but I'd like to know why they are done (as
> opposed
> > to source+binary releases or no release at all) before I head over to
> > python-ideas.  Is this documented somewhere where my google-fu can't
> find
> > it?
> >
> >
> > [1]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence
> 
> Assuming you're referring to the practice of no longer distributing
> binaries for patch releases of older versions of Python, the reason is
> basically as follows:
> 
> 1. Producing binaries (to the quality we normally deliver - I'm not
> talking about auto-built binaries produced from a CI system) is a
> chunk of extra work for the release managers.

This is actually the heart of the reason I asked the question.  CI tools are 
fairly good now.  If the CI tools could be used in such a way to make the 
building of binary artifacts less of a burden on the release managers, would 
there be interest in doing that, and in the process, releasing binary artifact 
installers for all security update releases.

My rationale for asking if its possible is... well.. security releases are 
important, and it's hard to ask Windows users to install Visual Studio and 
build python to use the most secure version of python that will run your python 
program.  Yes there are better ideal solutions (porting your code to the latest 
and greatest feature release version), but that’s not a zero burden option 
either.

If CI tools just aren't up to the task, then so be it, and this isn't something 
I would darken -ideas' door with.

> 2. The releases in question are essentially end of life, and we're
> only accepting security fixes.
> 3. Not even releasing sources means that people still using those
> releases will no longer have access to security fixes, so we'd be
> reducing the length of time we offer that level of support.
> 
> So extra binaries = more work for the release managers, no source
> release = less support for our users.
> 
> There's no reason we couldn't have a discussion on changing the
> policy, but any such discussion would probably need active support
> from the release managers if it were to stand any chance of going
> anywhere (as they are the people directly impacted by any such
> change).
> 
> Paul

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What is the rationale behind source only releases?

[Alex Walters]

This is precisely what I meant.  Before asking this question, I didn’t fully 
understand why, for example, 3.5.4 got a binary installer for windows and mac, 
but 3.5.5 did not.  This thread has cleared that up for me.

 

From: Python-Dev  On 
Behalf Of Donald Stufft
Sent: Wednesday, May 16, 2018 1:23 AM
To: Ben Finney 
Cc: python-dev@python.org
Subject: Re: [Python-Dev] What is the rationale behind source only releases?

 

 

On May 16, 2018, at 1:06 AM, Ben Finney  > wrote:

 


I'd like to know the rationale behind source only releases of cpython.


Software freedom entails the freedom to modify and build the software.
For that, one needs the source form of the software.

Portable software should be feasible to build from source, on a platform
where no builds (of that particular release) were done before. For that,
one needs the source form of the software.

 

I’m guessing the question isn’t why is it useful to have a source release of 
CPython, but why does CPython transition from having both source releases and 
binary releases to only source releases. My assumption is the rationale is to 
reduce the maintenance burden as time goes on for older release channels.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What is the rationale behind source only releases?

[Alex Walters]

Thank you, that's exactly what I needed to read.

> -Original Message-
> From: Ned Deily <n...@python.org>
> Sent: Wednesday, May 16, 2018 7:07 AM
> To: Alex Walters <tritium-l...@sdamon.com>
> Cc: Python-Dev <python-dev@python.org>
> Subject: Re: [Python-Dev] What is the rationale behind source only
releases?
> 
> On May 16, 2018, at 00:35, Alex Walters <tritium-l...@sdamon.com> wrote:
> > In the spirit of learning why there is a fence across the road before I
tear
> > it down out of ignorance [1], I'd like to know the rationale behind
source
> > only releases of cpython.  I have an opinion on their utility and
perhaps an
> > idea about changing them, but I'd like to know why they are done (as
> opposed
> > to source+binary releases or no release at all) before I head over to
> > python-ideas.  Is this documented somewhere where my google-fu can't
> find
> > it?
> 
> The Python Developer's Guide has a discussion of the lifecycle of cPython
> releases here:
> 
> https://devguide.python.org/#status-of-python-branches
> 
> The ~short answer is that we produce source+binary (Windows and macOS
> binary installers) artifacts for release branches in "bugfix" (AKA
> "maintenance") mode (currently 3.6 and 2.7) as well as during the later
> stages of the in-development phase for future feature releases
> ("prerelease" mode) (currently 3.7); we produce only source releases for
> release branches in "security" mode.
> 
> After the initial release of a new feature branch (for example, the
upcoming
> 3.7.0 release), we will continue to support the previous release branch in
> bugfix mode for some overlapping period of time.  So, for example, the
> current plan is to support both 3.7.x and 3.6.x (along with 2.7.x) in
bugfix
> mode, releasing both source and binary artifacts for about six months
after
> the 3.7.0 release.  At that point, 3.6.x will transition to
security-fix-only mode,
> where we will only produce releases on an as-needed basis and only in
> source form.  Currently, 3.5 and 3.4 are also in security-fix-only mode.
> Eventually, usually five years after its initial release, a release branch
will
> reach end-of-life: the branch will be frozen and no further issues for
that
> release branch will be accepted nor will fixes be produced by Python Dev.
> 2.7 is a special case, with a greatly extended bugfix phase; it will
proceed
> directly to end-of-life status as of 2020-01-01.
> 
> There is more information later elsewhere in the devguide:
> 
> https://devguide.python.org/devcycle/
> 
> and in the release PEPs linked in the Status of Python Branches section.
> 
> Hope that helps!
> 
> --
>   Ned Deily
>   n...@python.org -- []


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] What is the rationale behind source only releases?

[Alex Walters]

In the spirit of learning why there is a fence across the road before I tear
it down out of ignorance [1], I'd like to know the rationale behind source
only releases of cpython.  I have an opinion on their utility and perhaps an
idea about changing them, but I'd like to know why they are done (as opposed
to source+binary releases or no release at all) before I head over to
python-ideas.  Is this documented somewhere where my google-fu can't find
it?


[1]: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASE] Python 2.7.15

[Alex Walters]

I've gotten some mixed signals on the status of this release, notably from
the BDFL:

https://twitter.com/gvanrossum/status/991170064417153025
"Python 2.7.15 released -- the last 2.7 release!" (and a link to this
thread)

I was under the impression that 2.7 was being supported until 2020.  If this
is the final release of 2.7, what would "support" constitute?  My assumption
was that the final release of 2.7 would be sometime in 2020 (or much closer
to 2020 than 19 months).

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Benjamin Peterson
> Sent: Tuesday, May 1, 2018 12:10 AM
> To: python-l...@python.org; python-annou...@python.org; python-
> d...@python.org
> Subject: [Python-Dev] [RELEASE] Python 2.7.15
> 
> Greetings,
> I'm pleased to announce the immediate availability of Python 2.7.15, the
> latest bug fix release in the senescent Python 2.7 series.
> 
> Source and binary downloads may be found on python.org:
> 
>  https://www.python.org/downloads/release/python-2715/
> 
> Bugs should be reported to https://bugs.python.org/
> 
> The source tarball contains a complete changelog in the Misc/NEWS file.
The
> only change since the release candidate is a fix for undefined C behavior
that
> newer compilers (including GCC 8) have started to exploit.
> 
> Users of the macOS binaries should note that all python.org macOS
installers
> now ship with a builtin copy of OpenSSL. Additionally, there is a new
> additional installer variant for macOS 10.9+ that includes a built-in
version of
> Tcl/Tk 8.6. See the installer README for more information.
> 
> Happy May,
> Benjamin
> 2.7 release manager
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572 contradicts PEP 3099

[Alex Walters]

> -Original Message-
> From: Python-Dev <python-dev-bounces+tritium-
> list=sdamon@python.org> On Behalf Of Greg Ewing
> Sent: Saturday, April 28, 2018 10:53 PM
> To: 'Python-Dev' <python-dev@python.org>
> Subject: Re: [Python-Dev] PEP 572 contradicts PEP 3099
> 
> Alex Walters wrote:
> > PEP 3099 is the big list of things that will not happen in Python 3.
> >
> > "There will be no alternative binding operators such as :=."
> 
> The thread referenced by that is taling about a different issue,
> i.e. using a different symbol to rebind names in an outer scope.
> 

Yeah, that's not really the issue, if there is an issue - just the wording
of direct assertion.  I'm just adding to the list of things that need to be
touched if 572 is accepted, and that includes clarifying what was meant by
that.

> --
> Greg
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 572 contradicts PEP 3099

[Alex Walters]

PEP 3099 is the big list of things that will not happen in Python 3.
Everything on that list is still true, 12 years after it was posted.
However...

"There will be no alternative binding operators such as :=."

While earlier versions of PEP 572 avoided breaking this declaration, the
current version does not.  Now, this isn't a major issue or anything - if
572 is accepted, that section of 3099 would just need a note added (or just
be removed).  I don't want this message to impact the fate of 572.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3

[Alex Walters]

http://pyvideo.org/pycascades-2018/bdfl-python-3-retrospective.html link to 
Guido’s talk, for your convenience

 

From: Python-Dev  On 
Behalf Of Guido van Rossum
Sent: Thursday, April 26, 2018 6:12 PM
To: Brett Cannon 
Cc: Barry Warsaw ; Python-Dev 
Subject: Re: [Python-Dev] (Looking for) A Retrospective on the Move to Python 3

 

Also see my talk at PyCascades and Victor's upcoming talk at PyCon.

 

On Thu, Apr 26, 2018, 12:02 Brett Cannon  > wrote:

 

On Thu, 26 Apr 2018 at 10:19 Barry Warsaw  > wrote:

On Apr 26, 2018, at 09:28, Eric Snow  > wrote:
> 
> On Thu, Apr 26, 2018 at 10:25 AM, Eric Snow   > wrote:
>> In pondering our approach to future Python major releases, I found
>> myself considering the experience we've had with Python 3.  The whole
>> Py3k effort predates my involvement in the community so I missed a
>> bunch of context about the motivations, decisions, and challenges.
>> While I've pieced some of that together over the years now since I've
>> been around, I've certainly seen much of the aftermath.  For me, at
>> least, it would be helpful to have a bit more insight into the
>> history. :)

It would certainly be an interesting document, but I suspect you’ll get a bit 
of the old “ask 3 lawyers and get 5 opinions” kind of response. ;)

As I remember it, there was definitely a feeling like, this would be our only 
chance to clean up some annoying cruft, and rectify some (in hindsight) 
incorrect design decisions made over the years, couple with a healthy dose of 
“we have no idea how to do the bytes/str split in a backward compatible way".  
There was probably a sense that the Python community was just small enough to 
be able to handle such a disruptive change, but wouldn’t ever be so again.  The 
latter is definitely true today, even if the former was overly optimistic.

 

I agree with everything Barry said. There are some lessons in hindsight of how 
we could have handled bytes/str, but it was more of a decision of "really long 
transition versus a short one" -- jokes on us for what "short" became ;) -- 
which we simply won't make ever again.

___
Python-Dev mailing list
Python-Dev@python.org  
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/guido%40python.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is it useful to update cgitb module?

[Alex Walters]

Extremely useful … for CGI scripts.  Now, my sample size is quite small – 
mainly the #python irc channel on freenode, but I have been informally asking 
for a few years, and you are the first person I have encountered who admitted 
to writing and maintaining new CGI based sites.  I still say the value of 
supporting such an obsolete model is… questionable.

 

That said, I have no power to block commits.

 

From: Python-Dev <python-dev-bounces+tritium-list=sdamon@python.org> On 
Behalf Of Glenn Linderman
Sent: Sunday, April 8, 2018 2:28 AM
To: python-dev@python.org
Subject: Re: [Python-Dev] Is it useful to update cgitb module?

 

On 4/7/2018 9:45 PM, Alex Walters wrote:

Are there people still actively developing new cgi scripts in python?  I know 
some modern HTTPDs don’t even support classic cgi without some kind of fastcgi 
daemon in between.  I am aware that some parts of various wsgi tools use the 
cgi module, but is the cgitb module useful for them?


Yes. I have several web sites and applications built as Python CGI scripts.  
cgitb is extremely useful.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is it useful to update cgitb module?

[Alex Walters]

Are there people still actively developing new cgi scripts in python?  I know 
some modern HTTPDs don’t even support classic cgi without some kind of fastcgi 
daemon in between.  I am aware that some parts of various wsgi tools use the 
cgi module, but is the cgitb module useful for them?

Your suggestions might be good ideas, but I don’t know if they would be used.  
I feel like its kind of like updating the macpath module - sure you can make 
code improvements to it if you want, but its for a workflow that is very rarely 
used.

> -Original Message-
> From: Python-Dev  list=sdamon@python.org> On Behalf Of Stéphane Blondon
> Sent: Saturday, April 7, 2018 4:21 PM
> To: python-dev@python.org
> Subject: [Python-Dev] Is it useful to update cgitb module?
> 
> Hello,
> 
> I wonder if it's useful to update the cgitb module, in particular the
> html output.
> I see some possible improvements:
> 
> 1. In both text and html versions:
> 
> When a module is called, there are no parameters (displayed as '()'). I
> think they are unnecessary. Perhaps the parentheses should be removed?
> Perhaps it's better to keep them for backward compatibility?
> 
> ### example for the text version ###
> $ python3 demo.py
> [...]
>  /home/stephane/src/cgitest/demo.py in ()
> 7 def func1(a, b):
> [...]
> ### end of example ###
> 
> 2. In html version only:
>  a. If the executed code is in : in this case, it is not shown
> in the html version because the square brackets are interpreted as a
> html tag (see the picture in attachement).
>  b. Update the style of the html or/and using html5. It would be
> prettier but it will be a big change for probably too few benefits.
> 
> What do you think about them? I can report bugs and send pull-requests
> for them but I would prefer to get feedbacks before.
> 
> Regards,
> Stéphane

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Is 4.0 a major breaking changes release?

[Alex Walters]

I am still working on porting code from 2.x to 3.x.  As of late on the lists
I've seen comments about making somewhat major changes in 4.0 - now I'm
concerned that I should pause my porting effort until that is released.  Is
python 4 going to be another python 3?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[Alex Walters]

Are you aware of pypy?

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of asrp asrp
> Sent: Friday, February 2, 2018 7:02 PM
> To: python-dev@python.org
> Subject: [Python-Dev] A minimal Python interpreter written in Python for
> experimenting with language changes
> 
> Hello,
> 
> I don't know if this is the right place to post this. Please redirect as
needed.
> 
> I've made a small Python interpreter in Python with runtime AST node
> semantics and edit-and-continue. I thought it could make prototyping
> language changes more easily and visualize usage before writing them in C.
> 
> Its here: https://github.com/asrp/python_terp
> 
> So, for example, redefining the for_stmt function in the right scope
changes
> the behaviour of future for loops at runtime.
> 
> Although from discussion I've read in PEPs, actual implementation always
> look like a non-issue (which seems like magic to me) so maybe no-one here
> actually needs this.
> 
> (I really needed edit-and-continue for one of my projects but of course,
> running it in this extra interpreter is much too slow.)
> 
> asrp
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?

[Alex Walters]

I would suggest throwing this to -ideas, rather than just keeping it in -dev
as there is a much wider community of users and usecases in -ideas.

... and -ideas will shoot it down because user installs are too useful.  It
is also my understanding that it is the desire of PyPA to eventually have
pip default to --user when run outside of a virtualenv (to mitigate people
running sudo pip).  Eliminating user installs would change the situation
from one where, with pip install --user, one can recoverably break their
system to one, with sudo pip install, one can un-recoverably break their
system.

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Christian Heimes
> Sent: Saturday, January 13, 2018 12:06 PM
> To: python-dev@python.org
> Subject: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
> 
> Hi,
> 
> PEP 370 [1] was my first PEP that got accepted. I created it exactly one
> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have
> virtual environment support in Python. Ian Bicking had just started to
> create the virtualenv project a couple of months earlier.
> 
> Fast forward 10 years...
> 
> Nowadays Python has venv in the standard library. The user-specific
> site-packages directory is no longer that useful. I would even say it's
> causing more trouble than it's worth. For example it's common for system
> script to use "#!/usr/bin/python3" shebang without -s or -I option.
> 
> I propose to deprecate the feature and remove it in Python 4.0.
> 
> Regards,
> Christian
> 
> [1] https://www.python.org/dev/peps/pep-0370/
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff

[Alex Gaynor]

I also feel this decision was a mistake. If there's a consensus to revert,
I'm happy to draft a PEP.

Alex

On Nov 6, 2017 1:58 PM, "Neil Schemenauer" <nas-pyt...@arctrix.com> wrote:

> On 2017-11-06, Nick Coghlan wrote:
> > Gah, seven years on from Python 2.7's release, I still get caught by
> > that. I'm tempted to propose we reverse that decision and go back to
> > enabling them by default :P
>
> Either enable them by default or make them really easy to enable for
> development evironments.  I think some setting of the PYTHONWARNINGS
> evironment variable should do it.  It is not obvious to me how to do
> it though.  Maybe there should be an environment variable that does
> it more directly.  E.g.
>
> PYTHONWARNDEPRECATED=1
>
> Another idea is to have venv to turn them on by default or, based on
> a command-line option, do it.  Or, maybe the unit testing frameworks
> should turn on the warnings when they run.
>
> The current "disabled by default" behavior is obviously not working
> very well.  I had them turned on for a while and found quite a
> number of warnings in what are otherwise high-quality Python
> packages.  Obviously the vast majority of developers don't have them
> turned on.
>
> Regards,
>
>   Neil
> ___
> python-committers mailing list
> python-committ...@python.org
> https://mail.python.org/mailman/listinfo/python-committers
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Alex Walters]

 

 

From: Python-Dev [mailto:python-dev-bounces+tritium-list=sdamon@python.org] 
On Behalf Of Chris Barker
Sent: Thursday, October 26, 2017 12:46 PM
To: Wes Turner 
Cc: Python-Dev 
Subject: Re: [Python-Dev] iso8601 parsing

 

> No, it doesn't -- it may call them "timezones", but it only supports offsets 
> -- that is, and offset of -6 could be US Eastern Standard Time or US Central 
> Daylight TIme (or I got that backwards :-)  )

 

US Central Standard, Mountain Daylight.  (Eastern is -5/-4DST)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Alex Walters]

> -Original Message-
> From: Alexander Belopolsky [mailto:alexander.belopol...@gmail.com]
> Sent: Wednesday, October 25, 2017 4:33 PM
> To: Alex Walters <tritium-l...@sdamon.com>
> Cc: Elvis Pranskevichus <elpr...@gmail.com>; Python-Dev  d...@python.org>; Chris Barker <chris.bar...@noaa.gov>
> Subject: Re: [Python-Dev] iso8601 parsing
> 
> On Wed, Oct 25, 2017 at 3:48 PM, Alex Walters <tritium-l...@sdamon.com>
> wrote:
> >  Why make parsing ISO time special?
> 
> It's not the ISO format per se that is special, but parsing of str(x).
> For all numeric types, int, float, complex and even
> fractions.Fraction, we have a roundtrip invariant T(str(x)) == x.
> Datetime types are a special kind of numbers, but they don't follow
> this established pattern.  This is annoying when you deal with time
> series where it is common to have text files with a mix of dates,
> timestamps and numbers.  You can write generic code to deal with ints
> and floats, but have to special-case anything time related.

>>> repr(datetime.datetime.now())
'datetime.datetime(2017, 10, 25, 17, 16, 20, 973107)'

You can already roundtrip the repr of datetime objects with eval (if you care 
to do so).  You get iso formatting from a method on dt objects, I don’t see why 
it should be parsed by anything but a classmethod.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Alex Walters]

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Elvis Pranskevichus
> Sent: Tuesday, October 24, 2017 8:12 PM
> To: python-dev@python.org
> Cc: Chris Barker 
> Subject: Re: [Python-Dev] iso8601 parsing
> 
> On Tuesday, October 24, 2017 5:53:58 PM EDT Alexander Belopolsky wrote:
> > No, but the last time I suggested that that datetime types should
> > satisfy the same invariants as numbers, namely
> > T(repr(x)) == x, the idea was met will silence.  I, on the other hand,
> > am not very enthusiastic about named constructors such as
> > date.isoparse().  Compared with date(s:str), this is one more method
> > name to remember, plus the potential for abuse as an instance method.
> > What is d.isoparse('2017-11-24')?
> 
> Agreed.  datetime(s:str) seems like a far more natural and consistent
> choice.

It's inconsistent with the rest of the module.  All other constructions of
datetime objects are on classmethods.  Why make parsing ISO time special?

> 
>  Elvis
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Alex Walters]

> -Original Message-
> From: Alexander Belopolsky [mailto:alexander.belopol...@gmail.com]
> Sent: Wednesday, October 25, 2017 12:07 PM
> To: Alex Walters <tritium-l...@sdamon.com>
> Cc: Chris Barker <chris.bar...@noaa.gov>; Python-Dev  d...@python.org>
> Subject: Re: [Python-Dev] iso8601 parsing
> 
> 
> 
> > On Oct 25, 2017, at 11:45 AM, Alex Walters <tritium-l...@sdamon.com>
> wrote:
> >
> > it means
> > the type of the first argument changes the semantic meaning of
> subsequent
> > arguments, and that just adds a level of confusion to any api.
> 
> No, it does not. Passing a string a the first of three arguments will
still be a
> type error.

And that is a confusing api.  The problem has already been solved by
classmethod alternate constructors - they are already used widely in the
datetime api.  NOT using classmethod constructors is new and confusing for
the SINGLE use case of parsing iso formatted dates and times.  Why is that
special?  Why isn't ordinal time special to get into __init__?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Alex Walters]

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Alexander Belopolsky
> Sent: Tuesday, October 24, 2017 5:54 PM
> To: Chris Barker 
> Cc: Python-Dev 
> Subject: Re: [Python-Dev] iso8601 parsing
> 
> On Tue, Oct 24, 2017 at 5:26 PM, Chris Barker 
> wrote:
> > On Mon, Oct 23, 2017 at 5:33 PM, Hasan Diwan 
> wrote:
> >>
> > can anyone argue that it's not a good idea for datetime ot
> > be able to read the iso format it puts out?
> 
> No, but the last time I suggested that that datetime types should
> satisfy the same invariants as numbers, namely
> T(repr(x)) == x, the idea was met will silence.  I, on the other hand,
> am not very enthusiastic about named constructors such as
> date.isoparse().  Compared with date(s:str), this is one more method
> name to remember, plus the potential for abuse as an instance method.
> What is d.isoparse('2017-11-24')?

Datetime.datetime.fromiso() (classmethod) is much more in keeping with the
rest of the datetime api - in fact, I have tried calling that method more
than once, before remembering datetime *doesn't* have that classmethod.
Making it a classmethod solves any concerns about calling it as an instance
method (the same way d.now() and d.strptime() just create and return a new
datetime objects, not mutates the current).  In fact, looking at the docs,
most of the methods are classmethods, so an additional classmethod is
fitting.

I really do not like the idea of making the first positional argument of the
datetime constructor int or str.  What happens when you pass a string for
the first argument and ints for subsequent arguments?  You would have to
raise a typeerror or valueerror.  I don't like that API design - it means
the type of the first argument changes the semantic meaning of subsequent
arguments, and that just adds a level of confusion to any api.  You might be
able to get away with that in third party code, but this is the standard
library, and this is the time manipulation module in the standard library -
you have to assume that this is one of the first modules a new user uses, we
have to keep the api sane.

The only way I can think of keeping the api sane and still pass an iso
string to the constructor is to pass It as a keyword argument - and at that
point you have to remember the argument name anyways, so you might as well
make it a classmethod to match everything else in the library.

$0.02

> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RFC] Removing pure Python implementation of OrderedDict

[alex goretoy]

https://www.youtube.com/watch?v=pNe1wWeaHOU=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i=1


On Wed, Sep 6, 2017 at 5:49 PM, INADA Naoki  wrote:
> OK, I stop worring about thread safety and other implementation
> detail behavior on edge cases.
>
> Thanks,
>
> INADA Naoki  
>
>
> On Wed, Sep 6, 2017 at 7:40 PM, Paul Moore  wrote:
>> On 6 September 2017 at 11:09, Antoine Pitrou  wrote:
>>> On Wed, 6 Sep 2017 11:26:52 +0900
>>> INADA Naoki  wrote:

 Like that, should we say "atomic & threadsafe __setitem__ for simple
 key is implementation detail of CPython and PyPy.  We recommend
 using mutex when using OrderedDict from multiple thread."?
>>>
>>> I think you may be overstating the importance of making OrderedDict
>>> thread-safe.  It's quite rare to be able to rely on the thread safety
>>> of a single structure, since most often your state is more complex than
>>> that and you have to use a lock anyway.
>>>
>>> The statu quo is that only experts rely on the thread-safety of list
>>> and dict, and they should be ready to reconsider if some day the
>>> guarantees change.
>>
>> Agreed. I wasn't even aware that list and dict were guaranteed
>> threadsafe (in the language reference). And even if they are, there's
>> going to be a lot of provisos that mean in practice you need to know
>> what you're doing to rely on that. Simple example:
>>
>> mydict[the_value()] += 1
>>
>> isn't thread safe, no matter how thread safe dictionaries are.
>>
>> I don't have a strong opinion on making OrderedDict "guaranteed thread
>> safe" according to the language definition. But I'm pretty certain
>> that whether we do or not will make very little practical difference
>> to the vast majority of Python users.
>>
>> Paul
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/agoretoy%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 548: More Flexible Loop Control

[alex goretoy]

https://www.youtube.com/watch?v=pNe1wWeaHOU=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i=1
Thank you,
-Alex Goretoy


On Wed, Sep 6, 2017 at 7:05 PM, Ben Hoyt <benh...@gmail.com> wrote:
> I think Serhiy's response is excellent and agree with it. My gut reaction is
> "this looks like Perl" (and not in a good way), but more specifically it
> makes the control flow almost invisible. So I'm definitely -1 on this.
>
> The current while True ... break idiom is not pretty, but it's also very
> clear and obvious, and the control flow is immediately visible.
>
> One thing I do like about the proposal is the bare "while:", and I think
> that syntax is obvious and might be worth keeping (separately to the rest of
> the proposal). A bare "while:" (meaning "while True:") seems somehow less
> insulting to the intelligence, is still clear, and has precedent in Go's
> bare "for { ... }".
>
> -Ben
>
> On Wed, Sep 6, 2017 at 2:42 AM, Serhiy Storchaka <storch...@gmail.com>
> wrote:
>>
>> 06.09.17 03:11, R. David Murray пише:
>>>
>>> I've written a PEP proposing a small enhancement to the Python loop
>>> control statements.  Short version: here's what feels to me like a
>>> Pythonic way to spell "repeat until":
>>>
>>>  while:
>>>  
>>>  break if 
>>>
>>> The PEP goes into some detail on why this feels like a readability
>>> improvement in the more general case, with examples taken from
>>> the standard library:
>>>
>>>   https://www.python.org/dev/peps/pep-0548/
>>>
>>> Unlike Larry, I don't have a prototype, and in fact if this idea
>>> meets with approval I'll be looking for a volunteer to do the actual
>>> implementation.
>>>
>>> --David
>>>
>>> PS: this came to me in a dream on Sunday night, and the more I explored
>>> the idea the better I liked it.  I have no idea what I was dreaming about
>>> that resulted in this being the thing left in my mind when I woke up :)
>>
>>
>> This looks rather like Perl way than Python way.
>>
>> "There should be one-- and preferably only one --obvious way to do it."
>>
>> This proposing saves just a one line of the code. But it makes "break" and
>> "continue" statement less visually distinguishable as it is seen in your
>> example from uuid.py.
>>
>> If allow "break if" and "continue if", why not allow "return if"? Or
>> arbitrary statement before "if"? This adds PHP-like inconsistency in the
>> language.
>>
>> Current idiom is easier for modification. If you add the second condition,
>> it may be that you need to execute different code before "break".
>>
>> while True:
>> 
>> if not :
>> 
>> break
>> 
>> if not :
>> 
>> break
>>
>> It is easy to modify the code with the current syntax, but the code with
>> the proposed syntax should be totally rewritten.
>>
>> Your example from sre_parse.py demonstrates this. Please note that
>> pre-exit code is slightly different. In the first case self.error() is
>> called with one argument, and in the second case it is called with two
>> arguments. Your rewritten code is not equivalent to the existing one.
>>
>> Other concern is that the current code is highly optimized for common
>> cases. Your rewritten code checks the condition "c is None" two times in
>> common case.
>>
>> I'm -1 for this proposition.
>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/benhoyt%40gmail.com
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/agoretoy%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] Cherry picker bot deployed in CPython repo

[Alex Gaynor]

This is a great UX win for our development process. Thanks for making this
happen!

Alex

On Tue, Sep 5, 2017 at 9:10 PM, Mariatta Wijaya <mariatta.wij...@gmail.com>
wrote:

> Hi,
>
> The cherry picker bot has just been deployed to CPython repo, codenamed
> miss-islington.
>
> miss-islington made the very first backport PR for CPython and became a
> first time GitHub contributor: https://github.com/python/cpython/pull/3369
>
>
> GitHub repo: https://github.com/python/miss-islington
>
> What is this?
> ==
>
> As part of our workflow, quite often changes made on the master branch
> need to be backported to the earlier versions. (for example: from master to
> 3.6 and 2.7)
>
> Previously the backport has to be done manually by either a core developer
> or the original PR author.
>
> With the bot, the backport PR is created automatically after the PR has
> been merged. A core developer will need to review the backport PR.
>
> The issue was tracked in https://github.com/python/core-workflow/issues/8
>
> How it works
> ==
>
> 1. If a PR needs to be backported to one of the maintenance branches, a
> core developer should apply the "needs backport to X.Y" label. Do this
> **before** you merge the PR.
>
> 2. Merge the PR
>
> 3. miss-islington will leave a comment on the PR, saying it is working on
> backporting the PR.
>
> 4. If there's no merge conflict, the PR should be created momentarily.
>
> 5. Review the backport PR created by miss-islington and merge it when
> you're ready.
>
> Merge Conflicts / Problems?
> ==
>
> In case of merge conflicts, or if a backport PR was not created within 2
> minutes, it likely failed and you should do the backport manually.
>
> Manual backport can be done using cherry_picker: https://pypi.
> org/project/cherry-picker/
>
> Older merged PRs not yet backported?
> ==
>
> At the moment, those need to be backported manually.
>
> Don't want PR to be backported by a bot?
> 
>
> My recommendation is to apply the "needs backport to X.Y" **after** the PR
> has been merged. The label is still useful to remind ourselves that this PR
> still needs backporting.
>
> Who is Miss Islington?
> =
>
> I found out from Wikipedia that Miss Islington is the name of the witch in
> Monty Python and The Holy Grail.
>
> miss-islington has not signed the CLA!
> =
>
> A core dev can ignore the warning and merge the PR anyway.
>
> Thanks!
>
>
> Mariatta Wijaya
>
> ___
> python-committers mailing list
> python-committ...@python.org
> https://mail.python.org/mailman/listinfo/python-committers
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
>


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: D1B3 ADC0 E023 8CA6
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is Windows XP still supported on Python 2.7?

[Alex Walters]

The promise that PEP-11 is making is that as long as a python was released
while Microsoft still supported that OS, and that python is still supported,
there will still be a python that works for you.  So, yes, Windows XP is
long since unsupported by Microsoft, but a disturbing number of people still
run it.  (I think the NHS in the UK still runs embedded windows XP, just to
name a big one).  Yes, it's a support burden, but it's on the support burden
version of python anyways.  2.7 is a very slow moving branch so it shouldn't
be THAT big of a pain for the last 2 years of python 2 support.

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Victor Stinner
> Sent: Monday, July 24, 2017 5:05 AM
> To: Python Dev ; David Bolen
> 
> Subject: [Python-Dev] Is Windows XP still supported on Python 2.7?
> 
> Hi,
> 
> We have a Windows XP buildbot for Python 2.7, run by David Bolen:
> http://buildbot.python.org/all/builders/x86%20Windows%20XP%202.7/
> 
> test_bsddb3 fails randomly on this buildbot:
> http://bugs.python.org/issue30778
> 
> But Windows XP clearly reached its end-of-life, Microsoft doesn't
> support it anymore. So my question is if it makes sense to spend time
> on it?
> 
> We have a rule for new x.y.0 released, but not if a Microsoft Windows
> support expires during the lifetime of a Python release (2.7 here):
> https://www.python.org/dev/peps/pep-0011/#microsoft-windows
> 
> Firefox made great efforts to support Windows XP last years, but they
> decided to drop support last March with Firefox 52, last release
> supporting XP and Visa:
> https://support.mozilla.org/en-US/kb/end-support-windows-xp-and-vista
> 
> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python startup time

[Alex Walters]

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Paul Moore
> Sent: Saturday, July 22, 2017 4:14 AM
> To: David Mertz 
> Cc: Barry Warsaw ; Python-Dev  d...@python.org>
> Subject: Re: [Python-Dev] Python startup time


> It's a bit of a chicken and egg problem - Windows users avoid
> excessive command line program invocation because startup time is
> high, so no-one optimises startup time because Windows users don't use
> short-lived command line programs. But I'm seeing a trend away from
> that - more and more Windows tools these days seem to be comfortable
> spawning subprocesses. I don't know what prompted that trend.

The programs I see that are comfortable spawning processes willy-nilly on
windows are mostly .net, which has a lot of the runtime assemblies cached by
the OS in the GAC - if you are spawning a second processes of yourself, or
something that uses the same libraries as you, the compile step on those can
be skipped.  Unless you are talking about python/non-.NET programs, in which
case, I have no answer.
 
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Alex Gaynor]

I'm +1 on this, I even wrote the patch: https://bugs.python.org/issue22559
:-) If you're interested in making sure that still applies and tests still
pass, I'd be a big fan.

In addition to all the benefits you mentioned, it also substantially
reduces the diff between 2.7 and 3.x (or at least it did when I originally
wrote it).

Cheers,
Alex

On Tue, May 23, 2017 at 8:46 PM, Victor Stinner <victor.stin...@gmail.com>
wrote:

> Hi,
>
> Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python
> 2.7? I can do the backport.
>
>   https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO
>
> Cory Benfield told me that it's a blocking issue for him to implement
> his PEP 543 -- A Unified TLS API for Python 2.7:
>
>   https://www.python.org/dev/peps/pep-0543/
>
> And I expect that if a new cool TLS API happens, people will want to
> use it on Python 2.7-3.6, not only on Python 3.7. Security evolves
> more quickly that the current Python release process, and people wants
> to keep their application secure.
>
> From what I understood, he wants to first implement an abstract
> MemoryBIO API (http://sans-io.readthedocs.io/ like API? I'm not sure
> about that), and then implement a socket/FD based on top of that.
> Maybe later, some implementations might have a fast-path using
> socket/FD directly.
>
> He described me his PEP and I strongly support it (sorry, I missed it
> when he posted it on python-dev), but we decided (Guido van Rossum,
> Christian Heimes, Cory Benfield and me, see the tweet below) to not
> put this in the stdlib right now, but spend more time on testing it on
> Twisted, asyncio, requests, etc. So publishing an implementation on
> PyPI was proposed instead. It seems like we agreed on a smooth plan
> (or am I wrong, Cory?).
>
>   https://twitter.com/VictorStinner/status/865467388141027329
>
> I'm quite sure that Twisted will love MemoryBIO on Python 2.7 as well,
> to implement TLS, especially on Windows using IOCP. Currently,
> external libraries (C extensions) are required.
>
> I'm not sure if the PEP 466 should be amended for that? Is a new PEP
> really needed? MemoryBIO/SSLObject are tiny. Nick (Coghlan): what do
> you think?
>
>   https://www.python.org/dev/peps/pep-0466/
>
> Victor
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: D1B3 ADC0 E023 8CA6
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Introducing Python for CloudABI

[Alex Willmer]

Morning all, I'm writing to introduce myself and a port of CPython 3.6
to a CloudABI.

The port is reaching the point where it might be of interest to
others. Namely it ran it's first .py script yesterday during the
EuroPython scripts. Having said that it's still very early days, the
patches do horrible things - particularly to the import machinery.

I writing this to raise awareness, and open discussions. I'd love to
answer any questions/comments you might have.

Background:

# What is CloudABI?
CloudABI is a POSIX-like platform, with Capability based Security
applied. At the syscall/libc layer functions which perform IO or
acquire resources without a pre-existing file descriptor (e.g. open(),
stat(), bind() etc) are removed. All IO operations must be performed
through functions that accept a file descriptor, or a path relative to
an fd.

In this way descriptors server as capability tokens. All such tokens
are provided to a process when it is spawned. If none are provided
then the program in question is limited to just pure computation &
memory allocation. Even stdin, stdout & stderr are not provided by
default.

# Why bother with CloudABI?
It makes it possible to isolate programs from the OS, without
resorting to e.g. containers. Possibly even to run untrusted binaries.
A compromised CloudABI process can only damaged the things it has
access to e.g. a transcoding job can only read the provided input and
write to the provided output. It couldn't read /etc/passwd, or try to
brute force SSH. This kind of isolation is still possible with UNIX,
but it's not the default - which makes it rare.

Personally, I find it interesting. I like the fact that CloudABI
processes can be run by unprivileged users - unlike containers. The
no-default-global-resources nature makes it easier to write code that
can be tested. The fd provided to a webapp doesn't have to be a TCP
socket, it could be a domain socket, or just a file stream.

# What is the state of Python for CloudABI?
Python for CloudABI is a proof of concept. The port takes the form of
a number of patches to CPython 3.6.0a3. These mostly add autoconf &
#ifdef entries for POSIX API functions that CloudABI deliberately does
not support.

A few differences make their way through Python code, for instance
 - sys.path is a list of file descriptors, rather than a  list of strings
 - sys.executable is None
 - sys.argv is not present
 - The uid and gid entries of stat tuples are set to None (like on Windows)

I got print('Hello World', file=...) working about a month ago, and
executed my first .py file yesterday (commit pending).

The current TODO list is
 - Finish script support
- Module execution (python -m) support
- zipimport support for file descriptors
- ssl support
- patch cleanup
- try to run test suite

There is no Python 2.x support, and I don't plan to add any.

# What's the state of CloudABI?
CloudABI runs on FreeBSD, NetBSD, macOS and Linux. For now it requires
a patched kernel on Linux; FreeBSD 11 will include it out the box.
Various libraries/packages have been ported (e.g. curl, libpng, x265,
lua, libressl).

# What's the history of CloudABI?
The project started about 2 years ago. Ed Schouten is the project
leader & creator. I became involved this year, having seen a talk by
Ed at CCC around new year.

# Where can I get more info?
- https://nuxi.nl - CloudABI homepage, including Ed Schouten's CCC talk
- http://slides.com/alexwillmer/cloudabi-capability-security - My EP2016 talk
- https://www.youtube.com/watch?v=wlUtkBa8tK8=youtu.be=49m
- https://github.com/NuxiNL/cloudlibc
- https://github.com/NuxiNL/cloudabi-ports
- https://github.com/NuxiNL/cloudabi-ports/tree/master/packages/python
- #cloudabi on Efnet IRC

Regards, Alex
-- 
Alex Willmer <a...@moreati.org.uk>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Alex Walters]

> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Sebastian Krause
> Sent: Friday, June 10, 2016 1:01 PM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] BDFL ruling request: should we block forever
> waiting for high-quality random bits?
> 
> Guido van Rossum  wrote:
> > I just don't like the potentially blocking behavior, and experts'
opinions
> > seem to widely vary on how insecure the fallback bits really are, how
> > likely you are to find yourself in that situation, and how probable an
> > exploit would be.
> 
> This is not just a theoretical problem being discussed by security
> experts that *could* be exploited, there have already been multiple
> real-life cases of devices (mostly embedded Linux machines)
> generating predicatable SSH keys because they read from an
> uninitialized /dev/urandom at first boot. Most recently in the
> Raspbian distribution for the Raspberry Pi:
> https://www.raspberrypi.org/forums/viewtopic.php?f=66=126892
> 
> At least in 3.6 there should be obvious way to get random data that
> *always* guarantees to be secure and either fails or blocks if it
> can't guarantee that.
> 
> Sebastian

And that should live in the secrets module.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for pronouncement on PEP 493 (HTTPS verification backport guidance)

[Alex Gaynor]

Hi all,

While I appreciate the vote of confidence from everyone, I'm not interested
in being the BDFL-delegate for this. I don't think it's a good idea, and
I'm not willing to put further time into.

If he's interested, Donald Stufft would make a good choice for delegate.

Really do appreciate everyone's confidence.

Cheers,
Alex

On Mon, Nov 23, 2015 at 2:35 PM, Christian Heimes <christ...@python.org>
wrote:

> On 2015-11-17 01:00, Guido van Rossum wrote:
> > Hm, making Christian the BDFL-delegate would mean two out of three
> > authors *and* the BDFL-delegate all working for Red Hat, which clearly
> > has a stake (and IIUC has already committed to this approach ahead of
> > PEP approval). SO then it would look like this is just rubber-stamping
> > Red Hat's internal decision process (if it's a process -- sounds more
> > like an accident :-).
> >
> > So, Alex, do you want to approve this PEP?
>
> I haven't read this thread until now. Independently from your objection
> I have raised the same concern with Nick today. I'd be willing to BDFL
> the PEP but I'd rather have somebody outside of Red Hat. Alex is a great
> choice.
>
>
> In the same mail I sent Nick a quick review of the latest PEP version in
> private.
>
>
> 1) The example implementation of the function doesn't check the
> sys.flags.ignore_environment. Internally CPython has specialized getenv
> function that ignores env vars with PYTHON prefix when the flag is set.
> PYTHON* env vars aren't removed from os.environ. Modules have to check
> the flag.
>
>
> 2) The PEP is rather Linux-centric. What's the recommended path to the
> config file on other platforms like BDS (/usr/local/etc/ is preferred
> for additional dependencies on FreeBSD), OSX and Windows?
>
>
> 3) What's the interaction between the location of the config file and
> virtual envs? Would it make sense to search for the file in a venv's
> etc/ first and then dispatch to global /etc/? That way venvs can
> influence the setting, too.
>
>
> 4) It makes sense to make the cert-verification.cfg file future-proof
> and reserve it for other cert-related configuration in the future. For
> example it could be used to define new contexts, set protocols, ciphers
> or hashes for cert pinning. It should be enough to say that CPython
> reserves the right to add more sections and keys later.
>
>
> 5) I'm not particular fond of the section name [https]. For one It is
> ambiguous because it doesn't distinguish between server certs and client
> certs. It's also not correct. The default context is used for other
> protocols like imap, smtp etc. over TLS.
>
> Christian
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 (Type Hints) announcement

[Alex Grönholm]

Would you mind updating the typing package on PyPI now to contain 
something useful? Thanks.


22.05.2015, 23:51, Mark Shannon kirjoitti:

Hello all,

I am pleased to announce that I am accepting PEP 484 (Type Hints).

Given the proximity of the beta release I thought I would get this 
announcement out now, even though there are some (very) minor details 
to iron out.
(If you want to know the details, it's all at 
https://github.com/ambv/typehinting)



I hope that PEP 484 will be a benefit to all users of Python.
I think the proposed annotation semantics and accompanying module are 
technically sound and I hope that they are socially acceptable to the 
Python community.


I have long been aware that as well as a powerful, sophisticated and 
production quality language, Python is also used by many casual 
programmers, and as a language to introduce children to programming.
I also realise that this PEP does not look like it will be any help to 
the part-time programmer or beginner. However, I am convinced that it 
will enable significant improvements to IDEs (hopefully including 
IDLE), static checkers and other tools.

These tools will then help us all, beginners included.

This PEP has been a huge amount of work, involving a lot of people.
So thank you to everyone involved. If I were to list names I would 
inevitably miss someone out. You know who you are.


Finally, if you are worried that this will make Python ugly and turn 
it into some sort of inferior Java, then I share you concerns, but I 
would like to remind you of another potential ugliness; operator 
overloading.


C++, Perl and Haskell have operator overloading and it gets abused 
something rotten to produce concise (a.k.a. line noise) code.
Python also has operator overloading and it is used sensibly, as it 
should be. Why?

It's a cultural issue; readability matters.

Python is your language, please use type-hints responsibly :)

Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/alex.gronholm%40nextday.fi


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 wishes

[Alex Grönholm]

18.05.2015, 02:50, Guido van Rossum kirjoitti:
On Sun, May 17, 2015 at 3:07 PM, Alex Grönholm 
alex.gronh...@nextday.fi mailto:alex.gronh...@nextday.fi wrote:


Looking at PEP 484, I came up with two use cases that I felt were
not catered for:

 1. Specifying that a parameter should be a subclass of another
(example: Type[dict] would match dict or OrderedDict; plain
Type would equal type from builtins)


I don't understand. What is Type? Can you work this out in a full 
example? This code is already okay:


def foo(a: dict):
...

foo(OrderedDict())
This code is passing an /instance/ of OrderedDict. But how can I specify 
that foo() accepts a /subclass/ of dict, and not an instance thereof?


A full example:

def foo(a: Type[dict]):
...

foo(dict)  # ok
foo(OrderedDict)  # ok
foo({'x': 1})  # error


 1. Specifying that a callable should take at least the specified
arguments but would not be limited to them: Callable[[str,
int, ...], Any]

Case #2 works already (Callable[[str, int], Any] if the
unspecified arguments are optional, but not if they're mandatory.
Any thoughts?

For #2 we explicitly debated this and found that there aren't use 
cases known that are strong enough to need additional flexibility in 
the args of a callable. (How is the code calling the callable going to 
know what arguments are safe to pass?) If there really is a need we 
can address in a future revision.
Consider a framework where a request handler always takes a Request 
object as its first argument, but the rest of the arguments could be 
anything. If you want to only allow registration of such callables, you 
could do this:


def calculate_sum(request: Request, *values):
   return sum(values)

def register_request_handler(handler: Callable[[Request, ...], Any]):
   ...

--
--Guido van Rossum (python.org/~guido http://python.org/%7Eguido)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 wishes

[Alex Grönholm]

18.05.2015, 18:05, Guido van Rossum kirjoitti:
On Mon, May 18, 2015 at 12:14 AM, Alex Grönholm 
alex.gronh...@nextday.fi mailto:alex.gronh...@nextday.fi wrote:




18.05.2015, 02:50, Guido van Rossum kirjoitti:

On Sun, May 17, 2015 at 3:07 PM, Alex Grönholm
alex.gronh...@nextday.fi mailto:alex.gronh...@nextday.fi wrote:

Looking at PEP 484, I came up with two use cases that I felt
were not catered for:

 1. Specifying that a parameter should be a subclass of
another (example: Type[dict] would match dict or
OrderedDict; plain Type would equal type from builtins)


I don't understand. What is Type? Can you work this out in a
full example? This code is already okay:

def foo(a: dict):
...

foo(OrderedDict())

This code is passing an /instance/ of OrderedDict. But how can I
specify that foo() accepts a /subclass/ of dict, and not an
instance thereof?

A full example:

def foo(a: Type[dict]):
...

foo(dict)  # ok
foo(OrderedDict)  # ok
foo({'x': 1})  # error


You want the argument to be a *class*. We currently don't support that 
beyond using 'type' as the annotation. We may get to this in a future 
version; it is relatively uncommon. As to what notation to use, 
perhaps it would make more sense to use Class and Class[dict], since 
in the world of PEP 484, a class is a concrete thing that you can 
instantiate, while a type is an abstraction used to describe the 
possible values of a variable/argument/etc.


Also, what you gave is still not a full example, since you don't show 
what you are going to do with that type. Not every class can be easily 
instantiated (without knowing the specific signature). So if you were 
planning to instantiate it, perhaps you should use Callable[..., dict] 
as the type instead. (The ellipsis is not yet supported by mypy -- 
https://github.com/JukkaL/mypy/issues/393 -- but it is allowed by the 
PEP.)

Here's one example, straight from the code of my new framework:

@typechecked
def register_extension_type(ext_type: str, extension_class: type, 
replace: bool=False):


Adds a new extension type that can be used with a dictionary based 
configuration.


:param ext_type: the extension type identifier
:param extension_class: a class that implements IExtension
:param replace: ``True`` to replace an existing type


assert_subclass('extension_class', extension_class, IExtension)
if ext_type in extension_types and not replace:
raise ValueError('Extension type {} already 
exists'.format(ext_type))


extension_types[ext_type] = extension_class

I would like to declare the second argument as extension_class: 
Type[IExtension] (or Class[IExtension], doesn't matter to me). 
Likewise, the type hint for extension_types should be Dict[str, 
Type[IExtension]].



 1. Specifying that a callable should take at least the
specified arguments but would not be limited to them:
Callable[[str, int, ...], Any]

Case #2 works already (Callable[[str, int], Any] if the
unspecified arguments are optional, but not if they're
mandatory. Any thoughts?

For #2 we explicitly debated this and found that there aren't use
cases known that are strong enough to need additional flexibility
in the args of a callable. (How is the code calling the callable
going to know what arguments are safe to pass?) If there really
is a need we can address in a future revision.

Consider a framework where a request handler always takes a
Request object as its first argument, but the rest of the
arguments could be anything. If you want to only allow
registration of such callables, you could do this:

def calculate_sum(request: Request, *values):
   return sum(values)

def register_request_handler(handler: Callable[[Request, ...], Any]):
   ...


Hm... Yeah, you'd be stuck with using Callable[..., Any] for now. 
Maybe in a future version of the PEP. (We can't boil the ocean of 
typing in one PEP. :-)


--
--Guido van Rossum (python.org/~guido http://python.org/%7Eguido)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 484 wishes

[Alex Grönholm]

Looking at PEP 484, I came up with two use cases that I felt were not 
catered for:


1. Specifying that a parameter should be a subclass of another
   (example: Type[dict] would match dict or OrderedDict; plain Type
   would equal type from builtins)
2. Specifying that a callable should take at least the specified
   arguments but would not be limited to them: Callable[[str, int,
   ...], Any]

Case #2 works already (Callable[[str, int], Any] if the unspecified 
arguments are optional, but not if they're mandatory. Any thoughts?


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] unittest test discovery and namespace packages

[Alex Shkop]

Hello!

There's an issue considering test discovery in unittest module. Basically
it is about unittest module that doesn't find tests in namespace packages.
For more info see issue http://bugs.python.org/issue23882.

I'm willing to make a patch for this bug. But I need help to formulate how
test discovery should work.

Documentation states that all importable modules that match pattern will be
loaded. This means that test modules inside namespace packages should be
loaded too. But enabling this would change things drastically. For example
now, running

python -m unittest

inside cpython source root does nothing. If we will enable test discovery
inside namespace packages then this command will start running the whole
python test suite in Lib/test/.

So I'm looking for someone's help to clarify how test discovery should work.

Thanks,
Alex
--
Issue in bugtracker - http://bugs.python.org/issue23882
Documentation for discover() method -
https://docs.python.org/3.4/library/unittest.html#unittest.TestLoader.discover
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status on PEP-431 Timezones

[Alex Lord]

Newb question time, what's BoF

On Wed, Apr 8, 2015 at 7:31 PM, Alexander Belopolsky 
alexander.belopol...@gmail.com wrote:


 On Wed, Apr 8, 2015 at 6:52 PM, Isaac Schwabacher ischwabac...@wisc.edu
 wrote:
 
   So storing the offset and storing a flag are not two alternative
 solutions to the same problem- these
   are two solutions to two different problems.
 
  I'm viewing a time zone as a map from UTC to local time; for example,
 America/Chicago is a time zone. I'm not proposing storing the offset as an
 alternative to storing the *time zone*, I'm proposing it as an alternative
 to storing whether a given time is DST or not.


 When you are proposing to store something, you also need to specify where
 you are proposing to store it.  In the current design, local time
 information is stored in the datetime object and the rules that govern UTC
 to local  mapping (and back) are stored in the tzinfo object.  The
 implementors of concrete tzinfo subclasses are supposed to have access to
 time zone rules and implement utcoffset(dt), together with dst(dt) and
 tzname(dt) methods.

 Storing isdst in the datetime object would allow utcoffset(dt) to
 distinguish between 1:30AM before clock change and 1:30AM after.  Where do
 you propose to store the offset?  If you store it in dt, why would you need
 the tzinfo?


 
  We really don't care whether a time is DST or not;

 You may not care about it, but current tzinfo interface and
 tzinfo.fromutc(dt) method do.  Whatever new features are added to the
 standard library, they cannot break existing uses.  This means that
 whatever concrete tzinfo implementations we add to stdlib, they must
 provide an implementation of tzinfo.dst(dt) method.

   So our times would look like 2013-11-03 01:30:00-0500 America/Chicago
 and an hour later, 2013-11-03 01:30:00-0600 America/Chicago. And all of
 that information is stored in the datetime object.

 I don't think this is what most people would expect

 $ TZ=America/Chicago date
 Wed Apr  8 18:26:01 CDT 2015

 or

 $ TZ=America/Chicago date +%c %z
 Wed 08 Apr 2015 06:27:09 PM CDT -0500

 and not have location as a part of their timestamps.

 Regardless, whatever the proposal to add timezones to stdlib will end up
 being, it must include the ability to implement an equivalent of UNIX date
 command shown above.




 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/alexmlord%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition

[Alex Gaynor]

Ben Finney ben+python at benfinney.id.au writes:

 
 Rather, the claim is that *if* one's code base doesn't migrate to Python
 3, it will be decreasingly supported by the PSF and the Python community
 at large.
 

The PSF doesn't support any versions of Python. We have effectively no
involvement in the development of Python the language, or CPython. We 
certainly
don't care what version of Python you use.

Members of the python-dev list, or the CPython core development teams have
opinions probably, but that doesn't make them the opinion of the PSF.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Alex Gaynor]

On Sun Nov 30 2014 at 10:28:50 AM Brett Cannon br...@python.org wrote:

 Why specifically? Did you have a web UI for reviewing patches previously?
 Do you have CI set up for patches now and didn't before? What features did
 you specifically gain from the switch to GitHub that you didn't have
 before? IOW was it the magic of GitHub or some technical solution that
 you got as part of the GitHub package and thus could theoretically be
 replicated on python.org?

 -Brett


Previously someone looking for a review (read: any non-committer) would
export a diff from their VCS, upload it as a patch to trac, and then
reviewers could leave comments as trac comments. CPython's present process
is a clear improvement, insofar as Rietveld allows inlining commenting, but
it is otherwise basically the same.

By contrast, the Github process does not require a patch author to leave
their workflow, they simply git push to update a patch. We now also have
CI for PRs, but that's a recent addition.

It's not magic, it's a good UX :-)

Alex
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github

[Alex Gaynor]

Donald Stufft donald at stufft.io writes:


 [words words words]


I strongly support this PEP. I'd like to share two pieces of information. Both
of these are personal anecdotes:

For the past several years, I've been a contributor on two major projects using
mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching,
basically all contributors regularly make branches in the main repo for their
work, and we're very free in giving people commit rights, so almost everyone
working on PyPy in any way has this level of access. This workflow works ok. I
don't love it as much as git, but it's fine, it's not an impediment to my work.

By contrast, CPython does not have this type of workflow, there are almost no
in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg
user for years, I have no idea how to create a local-only branch, or a branch
which is pushed to a remote (to use the git term). I also don't generally
commit my own work to CPython, even though I have push privledges, 
because I
prefer to *always* get code review on my work. As a result, I use a git mirror
of CPython to do all my work, and generate patches from that.

The conclusion I draw from this is that hg's workflow is probably fine if
you're a committer on the project, or don't ever need to maintain multiple
patches concurrently (and thus can just leave everything uncommitted in the
repo). However, the hg workflow seems extremely defficient at non-committer
contributors.

The seconds experience I have is that of Django's migration to git and github.
For a long time we were on SVN, and we were very resistant to moving to 
DVCS in
general, and github in particular. Multiple times I said that I didn't see how
exporting a patch and uploading it to trac was more difficult than sending a
pull request. That was very wrong on my part.

My primary observation is not about new contributors though, it's actually
about the behavior of core developers. Before we were on github, it was fairly
rare for core developers to ask for reviews for anything besides *gigantic*
patches, we'd mostly just commit stuff to trunk. Since the switch to github,
I've seen that core developers are *far* more likely to ask for reviews of
their work before merging.

Big +1 from me, thanks for writing this up Donald,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP476: Enabling certificate validation by default

[Alex Gaynor]

Guido van Rossum guido at python.org writes:

 
 OK, I'll hold off a bit on approving the PEP, but my intention is to approve
 it. Go Alex go!
 

A patch for the environmental variable overrides on Windows has landed; thanks
Benjamin!

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP476: Enabling certificate validation by default

[Alex Gaynor]

Done and done.

Alex

On Fri, Sep 19, 2014 at 4:13 PM, Guido van Rossum gu...@python.org wrote:

 +1 on Nick's suggestion. (Might also mention that this is the reason why
 both functions should exist and have compatible signatures.)

 Also please, please, please add explicit mention of Python 2.7, 3.4 and
 3.5 in the Abstract (for example in the 3rd paragraph of the abstract).

 On Fri, Sep 19, 2014 at 3:52 PM, Nick Coghlan ncogh...@gmail.com wrote:

 On 20 September 2014 08:34, Alex Gaynor alex.gay...@gmail.com wrote:
  Pushed a new version which I believe adresses all of these. I added an
  example of opting-out with urllib.urlopen, let me know if there's any
 other
  APIs you think I should show an example with.

 It would be worth explicitly stating the process global monkeypatching
 hack:

 import ssl
 ssl._create_default_https_context = ssl._create_unverified_context

 Adding that hack to sitecustomize allows corporate sysadmins that can
 update their standard operating environment more easily than they can
 fix invalid certificate infrastructure to work around the problem on
 behalf of their users. It also helps out users that will be able to
 deal with such broken infrastructure without updating each and every
 one of their scripts.

 It's deliberately ugly because it's a genuinely bad idea that folks
 should want to avoid using, but as a matter of practical reality,
 corporate IT departments are chronically understaffed, and often fully
 committed to fighting the crisis du jour, without sufficient time
 being available for regular infrastructure maintenance tasks.

 Regards,
 Nick.

 --
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia




 --
 --Guido van Rossum (python.org/~guido)




-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP476: Enabling certificate validation by default

[Alex Gaynor]

That sounds reasonable to me -- at this point I don't expect this to make
it into 3.4.2; Nick has some working code on the ticket:
http://bugs.python.org/issue22417 it's mostly missing documentation.

Alex

On Sat, Sep 20, 2014 at 9:46 AM, Guido van Rossum gu...@python.org wrote:

 Nice. I just realized the release candidate for 3.4.2 is really close (RC1
 Monday, final Oct 6, see PEP 429). What's your schedule for 3.4? I see no
 date for 2.7.9 yet (but that could just be that PEP 373 hasn't been
 updated). What about the Apple and Microsoft issues Christian pointed out?

 Regarding the approval process, I want to get this into 2.7 and 3.4, but I
 want it done right, and I'm not convinced that the implementation is
 sufficiently worked out. I don't want you to feel rushed, and I don't want
 you to feel that you can't start coding until the PEP is approved, but I
 also feel that I want to see more working code and some beta testing before
 it goes live. Perhaps I should just approve the PEP but separately get to
 approve the code? (Others will have to review it for correctness -- but I
 want to understand and review the API.)

 On Sat, Sep 20, 2014 at 8:54 AM, Alex Gaynor alex.gay...@gmail.com
 wrote:

 Done and done.

 Alex

 On Fri, Sep 19, 2014 at 4:13 PM, Guido van Rossum gu...@python.org
 wrote:

 +1 on Nick's suggestion. (Might also mention that this is the reason why
 both functions should exist and have compatible signatures.)

 Also please, please, please add explicit mention of Python 2.7, 3.4 and
 3.5 in the Abstract (for example in the 3rd paragraph of the abstract).

 On Fri, Sep 19, 2014 at 3:52 PM, Nick Coghlan ncogh...@gmail.com
 wrote:

 On 20 September 2014 08:34, Alex Gaynor alex.gay...@gmail.com wrote:
  Pushed a new version which I believe adresses all of these. I added an
  example of opting-out with urllib.urlopen, let me know if there's any
 other
  APIs you think I should show an example with.

 It would be worth explicitly stating the process global monkeypatching
 hack:

 import ssl
 ssl._create_default_https_context = ssl._create_unverified_context

 Adding that hack to sitecustomize allows corporate sysadmins that can
 update their standard operating environment more easily than they can
 fix invalid certificate infrastructure to work around the problem on
 behalf of their users. It also helps out users that will be able to
 deal with such broken infrastructure without updating each and every
 one of their scripts.

 It's deliberately ugly because it's a genuinely bad idea that folks
 should want to avoid using, but as a matter of practical reality,
 corporate IT departments are chronically understaffed, and often fully
 committed to fighting the crisis du jour, without sufficient time
 being available for regular infrastructure maintenance tasks.

 Regards,
 Nick.

 --
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia




 --
 --Guido van Rossum (python.org/~guido)




 --
 I disapprove of what you say, but I will defend to the death your right
 to say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
 The people's good is the highest law. -- Cicero
 GPG Key fingerprint: 125F 5C67 DFE9 4084




 --
 --Guido van Rossum (python.org/~guido)




-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP476: Enabling certificate validation by default

[Alex Gaynor]

Hi all,

I've just updated the PEP to reflect the API suggestions from Nick, and the
fact that the necessary changes to urllib were landed.

I think this is ready for pronouncement, Guido?

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP476: Enabling certificate validation by default

[Alex Gaynor]

Pushed a new version which I believe adresses all of these. I added an
example of opting-out with urllib.urlopen, let me know if there's any other
APIs you think I should show an example with.

On Fri, Sep 19, 2014 at 3:06 PM, Guido van Rossum gu...@python.org wrote:

 The PEP doesn't specify any of the API changes for Python 2.7. I feel it
 is necessary for the PEP to show a few typical code snippets using urllib
 in Python 2.7 and how one would modify these in order to disable the cert
 checking.

 There are also a few typos; especially this paragraph puzzled me:

 This will be acheived by adding a new ``ssl._create_default_https_context``
 function, which is the same as ``ssl.create_default``. ``http.client`` can
 then
 replace it's usage of ``ssl._create_stdlib_context`` with the new
 ``ssl._create_default_https_context``.

 (1) spelling: it's achieved, not achieved

 (2) method name: it's ssl.create_default_context, not ssl.create_default

 (3) There's not enough whitespace (in the rendered HTML on
 legacy.python.org) before http.client -- I kept reading it as ... which
 is the same as ssl.create_default.http.client ...

 (4) There's no mention of the Python 2 equivalent of http.client.

 Finally, it's kind of non-obvious in the PEP that this affects Python
 2.7.X (I guess the one after the next) as well as 3.4 and 3.5.

 On Fri, Sep 19, 2014 at 9:53 AM, Alex Gaynor alex.gay...@gmail.com
 wrote:

 Hi all,

 I've just updated the PEP to reflect the API suggestions from Nick, and
 the
 fact that the necessary changes to urllib were landed.

 I think this is ready for pronouncement, Guido?

 Cheers,
 Alex

 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/guido%40python.org




 --
 --Guido van Rossum (python.org/~guido)




-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

Guido van Rossum guido at python.org writes:

 
 

Would you be willing to officially pronounce on PEP-476 in the context of 3.4.x,
so we can get it into the release, and then we can defer on officially approving
it for 2.7.X until we figure out all the moving pieces?

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed schedule for 3.4.2

[Alex Gaynor]

*Shifts uncomfortably* it looks like presently there's not a good way to
change anything about the SSL configuration for urllib.request.urlopen. It
does not take a `context` argument, as the http.client API does:
https://docs.python.org/3/library/urllib.request.html#module-urllib.request
and instead takes the cafile, capath, cadefault args.

This would need to be updated first, once it *did* take such an argument,
this would be accomplished by:

context = ssl.create_default_context()
context.verify_mode = CERT_OPTIONACERT_NONE
context.verify_hostname = False
urllib.request.urlopen(https://something-i-apparently-dont-care-much-about;,
context=context)

Alex


On Mon, Sep 8, 2014 at 10:35 AM, Guido van Rossum gu...@python.org wrote:

 I will pronounce for 3.4 once you point me to the documentation that
 explains how to disable cert validation for an example program that
 currently pulls down an https URL using urlopen. Without adding package
 dependencies.

 On Mon, Sep 8, 2014 at 10:25 AM, Alex Gaynor alex.gay...@gmail.com
 wrote:

 Guido van Rossum guido at python.org writes:

 
 

 Would you be willing to officially pronounce on PEP-476 in the context of
 3.4.x,
 so we can get it into the release, and then we can defer on officially
 approving
 it for 2.7.X until we figure out all the moving pieces?

 Cheers,
 Alex

 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/guido%40python.org




 --
 --Guido van Rossum (python.org/~guido)




-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Ethan Furman ethan at stoneleaf.us writes:

 
 I apologize if I missed this point, but if we have the source code then it is
 possible to go in and directly modify the application/utility to be able to
 talk over https to a router with an invalid certificate?  This is an option
 when creating the ssl_context?
 
 --
 ~Ethan~
 


Yes, it's totally possible to create (and pass to ``http.client``) an
``SSLContext`` which doesn't verify various things. My proposal is only about
changing what happens when you don't explicitly pass a context.

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Guido van Rossum guido at python.org writes:

 OK, that changes my position for 2.7 (but not for 3.5). I had assumed there
 was a way to disable the cert check by changing one parameter to the
 urlopen() call. (And I had wanted to add that there should be a clear FAQ
 about the subject.) If this isn't possible that changes the situation. (But I
 still think that once we do have that simple change option we should do it,
 in a later 2.7 upgrade.) I apologize for speaking before I had read all
 facts, and I'll await what you and Nick come up with.
 --Guido

This probably doesn't surprise anyone, but I'm more than happy to do the back-
porting work for httplib, and any other modules which need SSLContext support;
does this require an additional PEP, or does it fit under PEP466 or PEP476?

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Antoine Pitrou solipsis at pitrou.net writes:

 
 And how many people are using Twisted as an HTTPS client?
 (compared to e.g. Python's httplib, and all the third-party libraries
 building on it?)
 

I don't think anyone could give an honest estimate of these counts, however
there's two factors to bare in mind: a) It's extremely strongly recommended to
use requests to make any HTTP requests precisely because httplib is negligent
in certificate and hostname checking by default, b) We're talking about
Python3, which has fewer users than Python2.

  Furthermore, disable verification is a nonsensical thing to do with TLS.
 
 It's not. For example, if you have an expired cert, all you can do
 AFAIK is to disable verification.
 

It really is a nonsensical operation, accepting any random TLS certificate
without pinning or use of a certificate authorities makes a valid connection
completely indistinguishable from a MITM attack.

If I were the emperor of the universe (or even just Python ;-)) I wouldn't
allow this operation at all, however, I'm not and you can still disable any and
all verification. It just requires you to pass a different argument, which
doesn't seem overly burdensome.

This whole scenario seems to be predicated on a siutation where: You have a
peer whose certificate you can't change, and you have a piece of code you can't
change, and you're going to upgrade your Python installation, and you want to
talk to this peer, and you need to use an encrypted channel, but don't really
care if it's being MITM'd. It doesn't seem to me that this is reasonably
Python's responsibility to deal with the fact that you have no ability to
upgrade any of your infrastructure, except your Python version.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

The Windows certificate store is used by ``load_default_certs``:

* https://github.com/python/cpython/blob/master/Lib/ssl.py#L379-L381
* https://docs.python.org/3.4/library/ssl.html#ssl.enum_certificates

Cheers, Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Hi all,

I've just submitted PEP 476, on enabling certificate validation by default for
HTTPS clients in Python. Please have a look and let me know what you think.

PEP text follows.

Alex


---

PEP: 476
Title: Enabling certificate verification by default for stdlib http clients
Version: $Revision$
Last-Modified: $Date$
Author: Alex Gaynor alex.gay...@gmail.com
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-August-2014

Abstract


Currently when a standard library http client (the ``urllib`` and ``http``
modules) encounters an ``https://`` URL it will wrap the network HTTP traffic
in a TLS stream, as is necessary to communicate with such a server. However,
during the TLS handshake it will not actually check that the server has an X509
certificate is signed by a CA in any trust root, nor will it verify that the
Common Name (or Subject Alternate Name) on the presented certificate matches
the requested host.

The failure to do these checks means that anyone with a privileged network
position is able to trivially execute a man in the middle attack against a
Python application using either of these HTTP clients, and change traffic at
will.

This PEP proposes to enable verification of X509 certificate signatures, as
well as hostname verification for Python's HTTP clients by default, subject to
opt-out on a per-call basis.

Rationale
=

The S in HTTPS stands for secure. When Python's users type HTTPS they are
expecting a secure connection, and Python should adhere to a reasonable
standard of care in delivering this. Currently we are failing at this, and in
doing so, APIs which appear simple are misleading users.

When asked, many Python users state that they were not aware that Python failed
to perform these validations, and are shocked.

The popularity of ``requests`` (which enables these checks by default)
demonstrates that these checks are not overly burdensome in any way, and the
fact that it is widely recommended as a major security improvement over the
standard library clients demonstrates that many expect a higher standard for
security by default from their tools.

The failure of various applications to note Python's negligence in this matter
is a source of *regular* CVE assignment [#]_ [#]_ [#]_ [#]_ [#]_ [#]_ [#]_ [#]_
[#]_ [#]_ [#]_.

.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-4340
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-3533
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-5822
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-5825
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-1909
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2037
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2073
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2191
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-4111
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-6396
.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-6444

Technical Details
=

Python would use the system provided certificate database on all platforms.
Failure to locate such a database would be an error, and users would need to
explicitly specify a location to fix it.

This can be achieved by simply replacing the use of
``ssl._create_stdlib_context`` with ``ssl.create_default_context`` in
``http.client``.

Trust database
--

This PEP proposes using the system-provided certificate database. Previous
discussions have suggested bundling Mozilla's certificate database and using
that by default. This was decided against for several reasons:

* Using the platform trust database imposes a lower maintenance burden on the
  Python developers -- shipping our own trust database would require doing a
  release every time a certificate was revoked.
* Linux vendors, and other downstreams, would unbundle the Mozilla
  certificates, resulting in a more fragmented set of behaviors.
* Using the platform stores makes it easier to handle situations such as
  corporate internal CAs.

Backwards compatibility
---

This change will have the appearance of causing some HTTPS connections to
break, because they will now raise an Exception during handshake.

This is misleading however, in fact these connections are presently failing
silently, an HTTPS URL indicates an expectation of confidentiality and
authentication. The fact that Python does not actually verify that the user's
request has been made is a bug, further: Errors should never pass silently.

Nevertheless, users who have a need to access servers with self-signed or
incorrect certificates would be able to do so by providing a context with
custom trust roots or which disables validation (documentation should strongly
recommend the former where possible). Users will also be able to add necessary
certificates to system trust stores in order to trust them globally

Re: [Python-Dev] PEP 476: Enabling certificate validation by default!

[Alex Gaynor]

Thanks for the rapid feedback everyone!

I want to summarize the action items and discussion points that have come up so
far:

To add to the PEP:

* Emit a warning in 3.4.next for cases that would raise a Exception in 3.5
* Clearly state that the existing OpenSSL environment variables will be
  respected for setting the trust root

Discussion points:

* Disabling verification entirely externally to the program, through a CLI flag
  or environment variable. I'm pretty down on this idea, the problem you hit is
  that it's a pretty blunt instrument to swing, and it's almost impossible to
  imagine it not hitting things it shouldn't; it's far too likely to be used in
  applications that make two sets of outbound connections: 1) to some internal
  service which you want to disable verification on, and 2) some external
  service which needs strong validation. A global flag causes the latter to
  fail silently when subjected to a MITM attack, and that's exactly what we're
  trying to avoid. It also makes things much harder for library authors: I
  write an API client for some API, and make TLS connections to it. I want
  those to be verified by default. I can't even rely on the httplib defaults,
  because someone might disable them from the outside.


Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: Minor API improvements for bytes bytearray

[Alex Gaynor]

Donald Stufft donald at stufft.io writes:

 
 
 
 For the record I’ve had all of the problems that Nick states and I’m
 +1 on this change.
 
 
 ---
 Donald Stufft
 PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
 

I've hit basically every problem everyone here has stated, and in no uncertain
terms am I completely opposed to deprecating anything. The Python 2 to 3
migration is already hard enough, and already proceeding far too slowly for
many of our tastes. Making that migration even more complex would drive me to
the point of giving up.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!

[Alex Gaynor]

Antoine Pitrou antoine at python.org writes:
 No, IIRC there shouldn't be a cycle. It's just complicated in a 
 different way than 3.x 
 
 Regards
 
 Antoine.
 

Indeed, you're right, this is just differently convoluted so no leak (not that
I would call collected by a normal GC a leak :-)).

That said, I've hit another issue, with SNI callbacks. The first argument to an
SNI callback is the socket. The callback is set up by some C code, which right
now has access to only the _socket.socket object, not the ssl.SSLSocket object,
which is what the public API needs there.

Possible solutions are:

* Pass the SSLObject *in addition* to the _socket.socket object to the C code.
  This generates some additional divergence from the Python3 code, but is
  probably basically straightforward.
* Try to refactor the socket code in the same way as Python3 did, so we can
  pass *only* the SSLObject here. This is some nasty scope creep for PEP466,
  but would make the overall _ssl.c diff smaller.
* Some super sweet and simple thing I haven't thought of yet.

Thoughts?

By way of a general status update, the only failing tests left are this, and a
few things about SSLError's str(), so this will hopefully be ready to upload
any day now for review.

Cheers,
Alex

PS: Please review and merge http://bugs.python.org/issue22023 :-)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!

[Alex Gaynor]

Antoine Pitrou antoine at python.org writes:

 
 You mean for use with SSL_set_app_data?

Yes, if you look in ``_servername_callback``, you can see where it uses 
``SSL_get_app_data`` and then reads ``ssl-Socket``, which is supposed to be 
the same object that's returned by ``context.wrap_socket()``. 

Alex


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [PEP466] SSLSockets, and sockets, _socketobjects oh my!

[Alex Gaynor]

Hi all,

I've been happily working on the SSL module backports for Python2 (pursuant to
PEP466), and I've hit something of a snag:

In python3, the SSLSocket keeps a weak reference to the underlying socket,
rather than a strong reference, as Python2 uses.

Unfortunately, due to the way sockets work in Python2, this doesn't work:

On Python2, _socketobject composes around _real_socket from the _socket module,
whereas on Python3, it subclasses _socket.socket. Since you now have a Python-
level class, you can weak reference it.

The question is:

a) Should we backport weak referencing _socket.sockets (changing the structure
   of the module seems overly invasive, albeit completely backwards
   compatible)?
b) Does anyone know why weak references are used in the first place? The commit
   message just alludes to fixing a leak with no reference to an issue.

Anyone who's interested in the state of the branch can see it at:
github.com/alex/cpython on the backport-ssl branch. Note that many many tests
are still failing, and you'll need to apply the patch from
http://bugs.python.org/issue22023 to get it to work.

Thanks,
Alex

PS: Any help in getting http://bugs.python.org/issue22023 landed which be very
much appreciated.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Network Security Backport Status

[Alex Gaynor]

Hi all,

I wanted to bring everyone up to speed on the status of PEP 466, what's been
completed, and what's left to do.

First the completed stuff:

* hmac.compare_digest
* hashlib.pbkdf2_hmac

Are both backported, and I've added support to use them in Django, so users
should start seeing these benefits just as soon as we get a Python release into
their hands.

Now the uncompleted stuff:

* Persistent file descriptor for ``os.urandom``
* SSL module

It's the SSL module that I'll spend the rest of this email talking about.


Backporting the features from the Python3 version of this module has proven
more difficult than I had expected. This is primarily because the stdlib took a
maintenance strategy that was different from what most Python projects have
done for their 2/3 support: multiple independent codebases.

I've tried a few different strategies for the backport, none of which has
worked:

* Copying the ``ssl.py``, ``test_ssl.py``, and ``_ssl.c`` files from Python3
  and trying to port all the code.
* Coping just ``test_ssl.py`` and then copying individual chunks/functions as
  necessary to get stuff to pass.
* Manually doing stuff.

All of these proved to be a massive undertaking, and made it too easy to
accidentally introduce breaking changes.

I've come up with a new approach, which I believe is most likely to be
successful, but I'll need help to implement it.

The idea is to find the most recent commit which is a parent of both the
``2.7`` and ``default`` branches. Then take every single change to an ``ssl``
related file on the ``default`` branch, and attempt to replay it on the ``2.7``
branch. Require manual review on each commit to make sure it compiles, and to
ensure it doesn't make any backwards incompatible changes.

I think this provides the most iterative and guided approach to getting this
done.

I can do all the work of reviewing each commit, but I need some help from a
mercurial expert to automate the cherry-picking/rebasing of every single
commit.


What do folks think? Does this approach make sense? Anyone willing to help with
the mercurial scripting?

Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Remove the redundant and poorly worded warning message.

[Alex Gaynor]

Hi python-dev and Raymond,

I think this change is a considerable usability regression for the
documentation. Right now the warnings about CSPRNGs are hidden in the
introductory paragraph, which users are likely to skip. I agree that
there's no need to repeat the same advice twice, but I'd much rather we
kept the .. warning::  version, so users are more likely to actually read
it.

Also, there's a few errors with your commit message. First, we can
reasonably assert that urandom provides an acceptable CSPRNG, mostly
because it does on every platform I'm aware of. Second, urandom is still a
psuedo-random number generator, however they are cryptographically secure,
it's not more random. Wikipedia does a good job laying out the necessary
properties for a CSPRNG:
https://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator#Requirements

Cheers,
Alex


On Sat, May 10, 2014 at 2:05 PM, raymond.hettinger 
python-check...@python.org wrote:

 http://hg.python.org/cpython/rev/b466dc34b86e
 changeset:   90618:b466dc34b86e
 parent:  90616:ce070040e1a6
 user:Raymond Hettinger pyt...@rcn.com
 date:Sat May 10 14:05:28 2014 -0700
 summary:
   Remove the redundant and poorly worded warning message.

 The paragraph above already says, clearly and correctly, that
 However, being completely deterministic, it is not suitable for
 all purposes, and is completely unsuitable for cryptographic purposes.

 Also we should make any promises about SystemRandom or os.urandom()
 being cryptographically secure (they may be, but be can't validate
 that promise).  Further, those are actual random number generators
 not psuedo-random number generators.

 files:
   Doc/library/random.rst |  6 --
   1 files changed, 0 insertions(+), 6 deletions(-)


 diff --git a/Doc/library/random.rst b/Doc/library/random.rst
 --- a/Doc/library/random.rst
 +++ b/Doc/library/random.rst
 @@ -43,12 +43,6 @@
  uses the system function :func:`os.urandom` to generate random numbers
  from sources provided by the operating system.

 -.. warning::
 -
 -   The pseudo-random generators of this module should not be used for
 -   security purposes.  Use :func:`os.urandom` or :class:`SystemRandom` if
 -   you require a cryptographically secure pseudo-random number generator.
 -

  Bookkeeping functions:


 --
 Repository URL: http://hg.python.org/cpython

 ___
 Python-checkins mailing list
 python-check...@python.org
 https://mail.python.org/mailman/listinfo/python-checkins




-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466 (round 5): selected network security enhancements for Python 2.7

[Alex Gaynor]

This mostly looks good to me, however I'm not sure I understand the point of
this sentence: Rather, it is intended to send a clear signal to potential
corporate contributors that the core development team are willing to accept
offers of corporate assistance in putting this policy into effect [...]. It's
fairly evident to me that the folks most likely to actually do the work of
implementing this are myself and Donald. This PEP really has nothing to do with
corporate contribution, so I think this sentence ought to be removed.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements

[Alex Gaynor]

A casual glance at
https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/
util.py#L610
which is probably the most widely used consumer of these APIs, outside the
stdlib itself, looks to me like if these names were to suddenly show up,
everything would continue to work just fine, with the advance of being able to
explicitly specify some options.

All of which is to say: I don't think this is a real concern.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466 (round 4): Python 2.7 network security enhancements

[Alex Gaynor]

At this I think this PEP has become a little too vague and abstract, and I
think we'd probably be better served by getting more concrete:

Problem:

Some of Python 2's modules which are fundamentally necessary for interop with
the broader internet, and the security thereof, are missing really important
features.

Right now Python 2 has a policy of getting absolutely new features.

Solution:

We're going to ignore that policy for a couple of pretty important features to
that end.

Here's my proposed list of such featuers:

* hmac
* constant_time_compare
* os
* Persisant FD for os.urandom()
* ssl
* SNI
* SSLContext
* A giant suite of constants from OpenSSL
* The functions for checking a hostname against a certificate
* The functions for finding the platform's certificate store


Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 466: Proposed policy change for handling network security enhancements

[Alex Gaynor]

Thanks for putting this together Nick.

I suspect it goes without saying that I'm wildly +1 on this as a whole. I'm in
favor of leaving it somewhat implicit as to exactly which networking modules
concern the health of the internet as a whole.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GC pauses in CPython

[Alex Gaynor]

Maciej Fijalkowski fijall at gmail.com writes:

 
 HI
 
 I'm working on an incremental GC for PyPy. How do I measure GC pauses
 in CPython? (that is, the circular reference searching stuff)
 
 Cheers,
 fijal
 


For what it's worth I threw together some code that might be helpful:
http://bpaste.net/show/140334/ if someone was interested it might be a cool
idea to properly organize this up and find a place to expose VM statistics like
this. It'd also probably useful to use sane units, and maybe (it's unclear to
me) exclude some amount of finalizations (Ideally I think you'd ignore use
__del__ functions, but keep the bits of C code that decref other things and
actually call free()).

Alex

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Accelerating extension module compilation [distutils]

[Alex Leach]

Hi,

Apologies if this is the wrong place to ask, but thought this question  
would be relevant to Python core and extension module devs.. This the  
right place?


I've been using distutils to compile C++ extensions / bindings written  
with Boost.Python, and have been implementing some (often frowned-upon)  
monkey-patching magic to speed up the compilation process. I was wondering  
if other Python devs would appreciate the benefit from a  
distutils-integrated patch of the same functionality, using less-frowned  
upon programming techniques.


More specifically, I have felt it useful during development to incorporate  
the following functionality into a setupext.py file:-


  1. Parallel compilation, by monkey-patching  
distutils.ccompiler.CCompiler.compile. (A basic solution was provided on  
StackOverflow[1].)


  2. Create a unity-build[2].

  3. Only re-compile objects whose source-code / included files have  
changed.


I'll happily share my code (hacks), but before getting too technical with  
the discussion, I was just wondering whether any of these would be  
considered useful enough / easy enough to implement without breaking  
backwards-compatibility, to incorporate into core distutils. Any thoughts?


Thanks for your time.
Kind regards,
Alex


[1] -  
http://stackoverflow.com/questions/11013851/speeding-up-build-process-with-distutils
[2] -  
http://stackoverflow.com/questions/543697/include-all-cpp-files-into-a-single-compilation-unit


--
Using Opera's mail client: http://www.opera.com/mail/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] performance of {} versus dict()

[Alex Gaynor]

Stefan Behnel stefan_ml at behnel.de writes:
 Right. If that makes a difference, it's another bug.
 
 Stefan
 
 


It's fixed, with, I will note, fewer lines of code than many messages in this
thread:
https://bitbucket.org/pypy/pypy/changeset/c30cb1dcb7a9adc32548fd14274e4995

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)

[Alex Gaynor]

Brett Cannon brett at python.org writes:

 
 Time to ask the other VMs what they are currently doing (the ast module came 
into existence in Python 2.6 so all the VMs should be answer the question since 
Jython is in alpha for 2.7 compatibility).
 

As far as I know PyPy supports the ast module, and produces ASTs that are the
same as CPython's. That said I do regard this as an implementation detail,
further I'm guessing this is the context of the AST optimizer thread, and though
I have neither the time nor the inclination to wade into that, put me down as -1
a) everything proposed there is possible, b) making this a front-and-center API
makes it really easy to shoot themselves in the foot, by doing things like
breaking Python with invalid optimizations (hint: almost every optimization
proposed in that thread is invalid in the general case).

Alex


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting pronouncement on PEP 0424

[Alex Gaynor]

On Mon, Jul 30, 2012 at 9:51 AM, Guido van Rossum gu...@python.org wrote:

 Also, I have a few content quibbles:

 - Is it really worth flagging a negative return value with ValueError?
 I'd just as well clip this to zero. What's the worry? That the
 computed value is wrong? But it's only meant to be a hint, and why
 would -1 be any more wrong than e.g. 10?


This was done for consistency with len(), I'm not particularly attached to
any behavior.


 - Did you mean to define operator.length_hint()?


Of course :)


 - The default can be zero with no semantic impact, so I think there's
 no need to require the caller to specify a default.


I suppose that's fair.


 - Most importantly: calling len(obj) and catching TypeError can only
 be a substitute for the real implementation, which IMO ought to check
 for the presence of a tp_len slot. Alas, checking hasattr(obj,
 '__len__') doesn't quite cut it either, since this returns true for a
 class object that defines a __len__ method for its instances (the
 class itself doesn't have a length). Still, I worry that calling
 len(obj) and catching all TypeErrors overspecifies the desired
 behavior; what I *want* to happen is to check if there is a __len__
 method, and if so, call it and let any exceptions bubble through. It
 may be best to add a comment explaining that am implementation doesn't
 have to follow the letter of the Python code in the PEP, in
 particular, if obj *has* a __len__() method but calling it raises an
 exception, then length_hint(obj) may (ought to?) pass this exception
 on instead of calling obj.__length_hint__().


Seems reasonable, rather than try to spec that out precisely in the
pseudocode (aka Python ;)) a note like you suggest sounds good.


 --
 --Guido van Rossum (python.org/~guido)



Alex

-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting pronouncement on PEP 0424

[Alex Gaynor]

Guido van Rossum guido at python.org writes:

 
 
 Looks good to me, so accepted.But why isn't it visible on 
 python.org/dev/peps/ 
yet?

I just realized the text in the python.org repo did not match what I had 
locally.
I've pushed what I intended to be the latest text, if everyone could take a new
look at that I would be very grateful. Sorry for the mixup.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Requesting pronouncement on PEP 0424

[Alex Gaynor]

Hi all,

The discussion on PEP 0424 seems to have subsided (and I haven't gotten angry 
emails in a week!). So I would like to request a BDFL or BDFP pronouncement
on PEP 0424, text available here: 
http://hg.python.org/peps/file/tip/pep-0424.txt

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A new JIT compiler for a faster CPython?

[Alex Gaynor]

That's not, strictly speaking, true. Mozilla added a method-JIT (Jaegermonkey)
and then added another one (IonMonkey) because their tracing JIT (Tracemonkey)
was bad.  There's no fundamental reason that tracing has to only cover loops,
indeed PyPy's tracing has been generalized to compile individual functions,
recursion, etc. And any profiling JIT, in practice, needs a compile heuristic
for how many calls must occur before a unit is compiled, even the Hotspot JVM
has  one.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A new JIT compiler for a faster CPython?

[Alex Gaynor]

Victor Stinner victor.stinner at gmail.com writes:


 Example:
 
 a = GETLOCAL(0); # a
 if (a == NULL) /* error */
 b = GETLOCAL(1); # b
 if (b == NULL) /* error */
 return PyNumber_Add(a, b);
 
 
 I don't expect to run a program 10x faster, but I would be happy if I
 can run arbitrary Python code 25% faster.
 
 --
 
 Specialization / tracing JIT can be seen as another project, or at
 least added later.
 
 Victor
 

This is almost exactly what Unladen Swallow originally did.  First, LLVM will 
not
do all of the optimizations you are expecting it to do out of the box.  It will
still have all the stack accesses, and it will have all of the ref counting
operations.  You can get a small speed boost from removing the interpretation
dispatch overhead, but you also explode your memory usage, and the speedups are
tiny.

Please, learn from Unladen Swallow and other's experiences, otherwise they're
for naught, and frankly we (python-dev) waste a lot of time.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

[Alex Gaynor]

I've updated the PEP to reflect the discussion. There are two major changes:

1) NotImplemented may be used by __length_hint__ to indicate that there is no 
   finite length hint available.
2) callers of operator.length_hint() must provide their own default value, this
   is also required by the current C _PyObject_LengthHint implementation.

There are no provisions for infinite iterators, that is not within the scope of
this proposal.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 0424: A method for exposing a length hint

[Alex Gaynor]

Hi all,

I've just submitted a PEP proposing making __length_hint__ a public API for 
users to define and other VMs to implement:

PEP: 424
Title: A method for exposing a length hint
Version: $Revision$
Last-Modified: $Date
Author: Alex Gaynor alex.gay...@gmail.com
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 14-July-2012
Python-Version: 3.4

Abstract


CPython currently defines an ``__length_hint__`` method on several types, such
as various iterators. This method is then used by various other functions (such 
as
``map``) to presize lists based on the estimated returned by
``__length_hint__``. Types can then define ``__length_hint__`` which are not
sized, and thus should not define ``__len__``, but can estimate or compute a
size (such as many iterators).

Proposal


This PEP proposes formally documenting ``__length_hint__`` for other
interpreter and non-standard library Python to implement.

``__length_hint__`` must return an integer, and is not required to be accurate.
It may return a value that is either larger or smaller than the actual size of
the container. It may raise a ``TypeError`` if a specific instance cannot have
its length estimated. It may not return a negative value.

Rationale
=

Being able to pre-allocate lists based on the expected size, as estimated by 
``__length_hint__``, can be a significant optimization. CPython has been
observed to run some code faster than PyPy, purely because of this optimization
being present.

Open questions
==

There are two open questions for this PEP:

* Should ``list`` expose a kwarg in it's constructor for supplying a length
  hint.
* Should a function be added either to ``builtins`` or some other module which
  calls ``__length_hint__``, like ``builtins.len`` calls ``__len__``.

Copyright
=

This document has been placed into the public domain.

..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8




Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

[Alex Gaynor]

On Sat, Jul 14, 2012 at 4:18 PM, Benjamin Peterson benja...@python.orgwrote:

 2012/7/14 Alex Gaynor alex.gay...@gmail.com:
 
  Proposal
  
 
  This PEP proposes formally documenting ``__length_hint__`` for other
  interpreter and non-standard library Python to implement.
 
  ``__length_hint__`` must return an integer, and is not required to be
 accurate.
  It may return a value that is either larger or smaller than the actual
 size of
  the container. It may raise a ``TypeError`` if a specific instance
 cannot have
  its length estimated. It may not return a negative value.

 And what happens if you return a negative value?


ValueError, the same as with len.


 
  Rationale
  =
 
  Being able to pre-allocate lists based on the expected size, as
 estimated by
  ``__length_hint__``, can be a significant optimization. CPython has been
  observed to run some code faster than PyPy, purely because of this
 optimization
  being present.
 
  Open questions
  ==
 
  There are two open questions for this PEP:
 
  * Should ``list`` expose a kwarg in it's constructor for supplying a
 length
hint.
  * Should a function be added either to ``builtins`` or some other module
 which
calls ``__length_hint__``, like ``builtins.len`` calls ``__len__``.

 Let's try to keep this as limited as possible for a public API.


Sounds reasonable to me!  Should we just go ahead and strip those out now?



 --
 Regards,
 Benjamin


Alex

-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

[Alex Gaynor]

On Sat, Jul 14, 2012 at 10:16 PM, Nick Coghlan ncogh...@gmail.com wrote:

 On Sun, Jul 15, 2012 at 9:18 AM, Benjamin Peterson benja...@python.org
 wrote:
  Open questions
  ==
 
  There are two open questions for this PEP:
 
  * Should ``list`` expose a kwarg in it's constructor for supplying a
 length
hint.
  * Should a function be added either to ``builtins`` or some other
 module which
calls ``__length_hint__``, like ``builtins.len`` calls ``__len__``.
 
  Let's try to keep this as limited as possible for a public API.

 Length hints are very useful for *any* container implementation,
 whether those containers are in the standard library or not. Just as
 we exposed operator.index when __index__ was added, we should expose
 an operator.length_hint function with the following semantics:

 def length_hint(obj):
 Return an estimate of the number of items in obj. This is
 useful for presizing containers when building from an iterable.

 If the object supports len(), the result will be exact.
 Otherwise, it may over or underestimate by an arbitrary amount. The
 result will be an integer = 0.
 
 try:
 return len(obj)
 except TypeError:
 try:
 get_hint = obj.__length_hint__
 except AttributeError:
 return 0
 hint = get_hint()
 if not isinstance(hint, int):
 raise TypeError(Length hint must be an integer, not
 %r % type(hint))
 if hint  0:
 raise ValueError(Length hint (%r) must be = 0 % hint)
 return hint

 There's no reason to make pure Python container implementations
 reimplement all that for themselves.

 Cheers,
 Nick.

 --
 Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia


Sounds reasonable to me, the only issue with your psuedocode (err... I mean
Python ;)), is that there's no way for the __lenght_hint__ to specify that
that particular instance can't have a length hint computed.  e.g. imagine
some sort of lazy stream that cached itself, and only wanted to offer a
length hint if it had already been evaluated.  Without an exception to
raise, it has to return whatever the magic value for length_hint is (in
your impl it appears to be 0, the current _PyObject_LengthHint method in
CPython has a required `default` parameter).  The PEP proposes using
TypeError for that.

Anyways that code looks good, do you want to add it to the PEP?

Alex

-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Alex Clark]

 as source packages i.e. packages containing 
source code.


- Refer to binary dists as binary packages i.e. packages containing 
byte code and executables.



I believe this is the most human thing we can do[2].



Alex



[1] http://www.ibm.com/developerworks/linux/library/l-cppeak3/index.html

[2] http://python-for-humans.heroku.com





- I reject setup.cfg, as I believe ini-style configuration files are
not appropriate for a metadata format that needs to include file
listings and code fragments

- I reject bento.info, as I think if we accept
yet-another-custom-configuration-file-format into the standard library
instead of just using YAML, we're even crazier than is already
apparent

- I shall use dist.yaml as my proposed name for my I wish I could
define packages like this format (and yes, that means adding yaml
support to the standard library is part of the wish)

- many of the details below will be flawed, but I want to give a clear
idea for how a concept like this might work in practice

- we need to define a clear set of build phases, and then design the
dist metadata format accordingly. For example:
 - source
 - uses a source section in dist.yaml
 - source/install maps source files directly to desired
install locations
- essentially what the setup.cfg Resources section tries to do
- used for pure Python code, documentation, etc
- See below for example
 - source/files defines a list of extra files to be included
 - source/exclude defines the list of files to be excluded
 - source/run defines a Python fragment to be executed
 - serves a similar purpose to the files section in setup.cfg
 - creates a temporary directory (and sets it as the working directory)
 - dist.yaml is copied to the temporary directory
 - all files to be installed are copied to the temporary directory
 - all extra files are copied to the temporary directory
 - the Python fragment in source/run is executed (which can
thus easily add more files)
 - if sdist archive creation is requested, entire contents of
temporary directory are included
 - build
 - uses a build section in dist.yaml
 - build/install maps built files to desired install locations
- like source/install, but for build artifacts
- compiled C extensions, .pyc and .pyo files, etc would all go here
 - build/run defines a Python fragment to be executed
 - build/files defines the list of files to be included
 - build/exclude defines the list of files to be excluded
 - build/requires defines extra dependencies not needed at runtime
 - starting environment is a source directory that is either:
   - preexisting (e.g. to allow building in-place in the source tree)
   - created by running source first
   - created by unpacking an sdist archive
 - the Python fragment in build/run is executed to trigger the build
 - if the build succeeds (i.e. doesn't throw an exception)
   - create a temporary directory
   - copy dist.yaml
   - copy all specified files
   - this is the easiest way to exclude build artifacts from
the distribution, while still keeping them around to enable
incremental builds
 - if bdist_simple archive creation is requested, entire
contents of temporary directory are included
 - other bdist formats (such as bdist_rpm) will have their own
rules for getting from the bdist_simple format to the platform
specific format
 - install
 - uses an install section in dist.yaml
 - install/pre defines a Python fragment to be executed
before copying files
 - install/post defines a Python fragment to be executed
after copying files
 - starting environment is a bdist_simple directory that is either:
   - preexisting (e.g. to allow creation by system packaging tools)
   - created by running build first
   - created by unpacking a bdist_simple archive
 - end result is a fully installed and usable piece of software
 - test
 - uses a test section in dist.yaml
 - test/run defines a Python fragment to be executed to start the 
tests
 - test/requires defines extra dependencies needed to run the
test suite

- Example source/install based on
http://alexis.notmyidea.org/distutils2/setupcfg.html#complete-example
(my YAML may be a bit dodgy).
   - With this scheme, module installation is just another install category.
   - A solution for easily installing entire subtrees is desirable. I
propose the recursive glob ** syntax for that purpose.
   - Unlike setup.cfg, every category would have an -excluded
counterpart to filter unwanted files. Explicit is better than
implicit.

 source:
   install:
 modules:
   example.py
   example_pkg/*.py
   example_pkg/**/*.py

Re: [Python-Dev] Status of packaging in 3.3

[Alex Clark]

Hi,

On 6/21/12 7:56 AM, Tarek Ziadé wrote:

On 6/21/12 11:08 AM, Dag Sverre Seljebotn wrote:

...
David Cournapeau's Bento project takes the opposite approach,
everything is explicit and without any magic.

http://cournape.github.com/Bento/

It had its 0.1.0 release a week ago.

Please, I don't want to reopen any discussions about Bento here --
distutils2 vs. Bento discussions have been less than constructive in
the past -- I just wanted to make sure everybody is aware that
distutils2 isn't the only horse in this race. I don't know if there
are others too?


That's *exactly* the kind of approach that has made me not want to
continue.

People are too focused on implementations, and 'how distutils sucks'
'how setuptools sucks' etc 'I'll do better' etc

Instead of having all the folks involved in packaging sit down together
and try to fix the issues together by building PEPs describing what
would be a common set of standards, they want to create their own tools
from scratch.

That will not work.



But you can't tell someone or some group of folks that, and expect them 
to listen. Most times NIH is pejorative[1], but sometimes something 
positive comes out of it.




And I will say here again what I think we should do
imho:

1/ take all the packaging PEPs and rework them until everyone is happy
(compilation sucks in distutils ?  write a PEP !!!)

2/ once we have a consensus, write as many tools as you want, if they
rely on the same standards = interoperability = win.

But I must be naive because everytime I tried to reach people that were
building their own tools to ask them to work with us on the PEPs, all I
was getting was distutils sucks!'



And that's the best you can do: give your opinion. I understand the 
frustration, but we have to let people succeed and/or fail on their own[2].





It worked with the OS packagers guys though, we have built a great data
files managment system in packaging + the versions (386)



Are you referring to the packaging/distutils2 or something else?


Alex


[1] http://en.wikipedia.org/wiki/Not_invented_here
[2] 
http://docs.pythonpackages.com/en/latest/advanced.html#buildout-easy-install-vs-virtualenv-pip









--
Dag Sverre Seljebotn
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com





--
Alex Clark · http://pythonpackages.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Alex Clark]

Hi,

On 6/21/12 1:20 PM, Tarek Ziadé wrote:

On 6/21/12 6:44 PM, Chris McDonough wrote:




Yes. At the very least, there will be updated development snapshots
(which are what buildout uses anyway).

(Official releases are in a bit of a weird holding pattern.
distribute's versioning scheme leads to potential confusion: if I
release e.g. 0.6.1, then it sounds like it's a lesser version than
whatever distribute is up to now.  OTOH, releasing a later version
number than distribute implies that I'm supporting their feature
enhancements, and I really don't want to add new features to 0.6...  but
don't have time right now to clean up all the stuff I started in the 0.7
line either, since I've been *hoping* that the work on packaging would
make 0.7 unnecessary.  And let's not even get started on the part where
system-installed copies of distribute can prevent people from
downloading or installing setuptools in the first place.)



Welp, I don't want to get in the middle of that whole mess.  But maybe
the distribute folks would be kind enough to do a major version bump
in their next release; e.g. 1.67 instead of 0.67.  That said, I don't
think anyone would be confused by overlapping version numbers between
the two projects.

Oh yeah no problem, if Philip backports all the things we've done like
Py3 compat, and bless more people to maintain setuptools, we can even
discontinue distribute !

If not, I think you are just being joking here -- we don't want to go
back into the lock situation we've suffered for many years were PJE is
the only maintainer then suddenly disappears for a year, telling us no
one that is willing to maintain setuptools is able to do so. (according
to him)



It's known that they have been diverging for a while.

Yeah the biggest difference is Py3 compat, other than that afaik I don't
think any API has been removed or modified.


In my opinion, distribute is the only project that should go forward
since it's actively maintained and does not suffer from the bus factor.


+1. I can't help but cringe when I read this (sorry, PJ Eby!):

Official releases are in a bit of a weird holding pattern. due to 
distribute.


Weren't they in a bit of a weird holding pattern before distribute? 
Haven't they always been in a bit of a weird holding pattern?


Let's let setuptools be setuptools and distribute be distribute i.e. as 
long as distribute exists, I don't care at all about setuptools' release 
schedule (c.f. PIL/Pillow) and I like it that way :-). If one day 
setuptools or packaging/distutils2 comes along and fixes everything, 
then distribute can cease to exist.




Alex




--
Alex Clark · http://pythonpackages.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of packaging in 3.3

[Alex Clark]

Hi,

On 6/21/12 5:38 PM, Donald Stufft wrote:

On Thursday, June 21, 2012 at 4:01 PM, Paul Moore wrote:

End users should not need packaging tools on their machines.


Sort of riffing on this idea, I cannot seem to find a specification for
what a Python
package actually is.



FWIW according to distutils[1], a package is: a module or modules inside 
another module[2]. So e.g.::



  foo.py is a module


and:

  foo/__init__.py
  foo/foo.py

is a simple package containing the following modules:

  import foo, foo.foo


Alex


[1] 
http://docs.python.org/distutils/introduction.html#general-python-terminology


[2] And a distribution is a compressed archive of a package, in case 
that's not clear.





 Maybe the first effort should focus on this instead
of arguing one
implementation or another.

As a packager:
 I should not (in general) care what tool (pip, pysetup,
easy_install, buildout, whatever) is used
 to install my package, My package should just describe what to do
to install itself.

As a end user:
I should not (in general) care what tool was used to create a
package (setuptools, bento, distutils,
whatever). My tool of choice should look at the package and preform
the operations that the package
says are needed for install.

Ideally the package could have some basic primitives that are enough to
tell the package installer
tool what to do to install it, These primitives should be enough to
cover the common cases (pure python
modules at the very least, maybe additionally some C modules). Now as
others have remarked it would
be insane to attempt to do this in every case as it would involve
writing a build system that is more
advanced than anything else existing, so a required primitive would be
something that allows calling out
to a specific package decided build system (waf, make, whatever) to
handle the build configuration.

The eventual end goal here being to make a package from something that
varies from implementation
to implementation to a standardized format that any number of tools can
build on top of. It would likely
include some things defining where metadata MUST be defined.

For instance, if metadata in setuptools was compiled down to static
file, and easy_install, pip et;al
used that static file to install from instead of executing setup.py,
then the end user would not have
required setup tools installed and instead any number of tools could
have been created that utilized
that data.





--
Alex Clark · http://pythonpackages.com



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

[Alex Gaynor]

For PyPy: I'm not an expert in our import, but from looking at the source

1) imp.cache_from_source is unimplemented, it's an AttributeError.

2) sys.dont_write_bytecode is always false, we don't respect that flag (we 
really
   should IMO, but it's not a high priority for me, or anyone else apparently)

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

[Alex Gaynor]

On Tue, Jun 12, 2012 at 11:47 AM, Brett Cannon br...@python.org wrote:



 On Tue, Jun 12, 2012 at 12:38 PM, Alex Gaynor alex.gay...@gmail.comwrote:

 For PyPy: I'm not an expert in our import, but from looking at the source

 1) imp.cache_from_source is unimplemented, it's an AttributeError.


 Well, you will have it come Python 3.3 one way or another. =)



Sure, I'm not totally up to speed on the py3k effort.



 2) sys.dont_write_bytecode is always false, we don't respect that flag
 (we really
   should IMO, but it's not a high priority for me, or anyone else
 apparently)


 But doesn't PyPy read and write .pyc files (
 http://doc.pypy.org/en/latest/config/objspace.usepycfiles.html suggests
 you do)? So I would assume you are not affected by this. Jython and
 IronPython, though, would be (I think).


This is a compile time option, not a runtime option.  However, it looks
like I lied, someone did implement it correctly, so we have the same
behavior as CPython.

Alex


-- 
I disapprove of what you say, but I will defend to the death your right to
say it. -- Evelyn Beatrice Hall (summarizing Voltaire)
The people's good is the highest law. -- Cicero
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

[Alex Gaynor]

Eric Snow ericsnowcurrently at gmail.com writes:

 
 Nick's option 2 would be an improvement, but I imagine that option 3
 would have been the most effective by far.  Of course, the key thing
 is how closely the various implementors would follow the new list.
 Only they could say, though Frank Wierzbicki seemed positive about it.

 -eric
 


I'm +1 on such a list, I don't have the time to follow every single thread on
python-dev, and I'm sure I miss a lot of things, have a dedicated place for
things I know are relevant to my work would be a great help.

Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] c/ElementTree XML serialisation

[Alex Leach]

On Wednesday 09 May 2012 08:02:09 Xavier Morel wrote:
| Erm… you have them? What do you think `lt;` and `gt;` are?

I was under the impression that those (let's call them) HTML representations 
of  and  don't get interpreted correctly by Javascript engines. I'll have to 
check that though..

| 
| As to writing a loop in javascript without  and , == and != generally
| work rather well, as does Array.prototype.forEach[0]

Thanks for the tips!
Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] c/ElementTree XML serialisation

[Alex Leach]

On Wednesday 09 May 2012 08:02:09 Xavier Morel wrote:
| Erm… you have them? What do you think `lt;` and `gt;` are?

I was under the impression that those (let's call them) HTML representations 
of  and  don't get interpreted correctly by Javascript engines. I'll have to 
check that though..

| 
| As to writing a loop in javascript without  and , == and != generally
| work rather well, as does Array.prototype.forEach[0]

Thanks for the tips!
Cheers,
Alex

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com