[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Victor Stinner]

Hi,

I wrote a PEP based on the previous thread "Backport ssl.MemoryBIO on
Python 2.7?". Thanks for Cory Benfield, Alex Gaynor and Nick Coghlan
who helped me to write it!

HTML version:
https://www.python.org/dev/peps/pep-0546/

Inline verison below.

Victor


PEP: 546
Title: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner ,
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30-May-2017


Abstract


Backport the ssl.MemoryBIO and ssl.SSLObject classes from Python 3 to Python
2.7 to enhance the overall security of Python 2.7.


Rationale
=

While Python 2.7 is getting closer to its end-of-support date (scheduled for
2020), it is still used on production systems and the Python community is still
responsible for its security. This PEP will help facilitate the future adoption
of :pep:`543` across all supported Python versions, which will improve security
for both Python 2 and Python 3 users.

This PEP does NOT propose a general exception for backporting new
features to Python 2.7 - every new feature proposed for backporting will
still need to be justified independently. In particular, it will need to
be explained why relying on an independently updated backport on the
Python Package Index instead is not an acceptable solution.


PEP 543
---

:pep:`543` defines a new TLS API for Python which would enhance Python
security by giving Python applications access to the native TLS implementations
on Windows and macOS, instead of using OpenSSL. A side effect is that it gives
access to the system trust store and certificates installed
locally by system administrators, enabling Python applications to use "company
certificates" without having to modify each application and so to correctly
validate TLS certificates (instead of having to ignore or bypass TLS
certificate validation).

For practical reasons, Cory Benfield would like to first implement an
I/O-less class similar to ssl.MemoryBIO and ssl.SSLObject for
:pep:`543`, and to provide a second class based on the first one to use
sockets or file descriptors.  This design would help to structure the code
to support more backends and simplify testing and auditing, as well as
implementation. Later, optimized classes using directly sockets or file
descriptors may be added for performance.

While :pep:`543` defines an API, the PEP would only make sense if it
comes with at least one complete and good implementation. The first
implementation would ideally be based on the ``ssl`` module of the Python
standard library, as this is shipped to all users by default and can be used as
a fallback implementation in the absence of anything more targetted.

If this backport is not performed, the only baseline implementation that could
be used would be pyOpenSSL. This is problematic, however, because of the
interaction with pip, which is shipped with CPython on all supported versions.


requests, pip and ensurepip
---

There are plans afoot to look at moving Requests to a more event-loop-y
model, and doing so basically mandates a MemoryBIO. In the absence of a
Python 2.7 backport, Requests is required to basically use the same
solution that Twisted currently does: namely, a mandatory dependency on
`pyOpenSSL `_.

The `pip `_ program has to embed all its
dependencies for practical reasons: namely, that it cannot rely on any other
installation method being present. Since pip depends on requests, it means
that it would have to embed a copy of pyOpenSSL. That would imply substantial
usability pain to install pip. Currently, pip doesn't support embedding
C extensions which must be compiled on each platform and so require a C
compiler.

Since Python 2.7.9, Python embeds a copy of pip both for default
installation and for use in virtual environments via the new ``ensurepip``
module. If pip ends up bundling PyOpenSSL, then CPython will end up
bundling PyOpenSSL. Only backporting ``ssl.MemoryBIO`` and
``ssl.SSLObject`` would avoid the need to embed pyOpenSSL, and would fix the
bootstrap issue (python -> ensurepip -> pip -> requests -> MemoryBIO).


Changes
===

Add ``MemoryBIO`` and ``SSLObject`` classes to the ``ssl`` module of
Python 2.7.

The code will be backported and adapted from the master branch
(Python 3).

The backport also significantly reduced the size of the Python 2/Python
3 difference of the ``_ssl`` module, which make maintenance easier.


Links
=

* :pep:`543`
* `[backport] ssl.MemoryBIO
  `_: Implementation of this PEP
  written by Alex Gaynor (first version written at October 2014)
* :pep:`466`


Discussions
===

* `[Python-Dev] Backport ssl.MemoryBIO on Python 2.7?
  `_
  (May 2017)


Copyright
=

This document has been placed in the public domain.
__

Re: [Python-Dev] PEP 544: Protocols - second round

[Ivan Levkivskyi]

On 31 May 2017 at 00:58, Guido van Rossum  wrote:
[...]

Thank you for very detailed answers! I have practically nothing to add.
It seems to me that most of the Kevin's questions stem from unnecessary
focus
on runtime type checking. Here are two ideas about how to fix this:

* Add the word "static" somewhere in the PEP title.
* Add a short note at the start mentioning this is an extension of the type
system proposed in PEP 484 and recommending to read PEP 484 first.

What do you think?

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Put token information in one place

[Serhiy Storchaka]

Currently when you add a new token you need to change a couple of files:

* Include/token.h
* _PyParser_TokenNames in Parser/tokenizer.c
* PyToken_OneChar(), PyToken_TwoChars() or PyToken_ThreeChars() in 
Parser/tokenizer.c

* Lib/token.py (generated from Include/token.h)
* EXACT_TOKEN_TYPES in Lib/tokenize.py
* Operator, Bracket or Special in Lib/tokenize.py
* Doc/library/token.rst

It is possible to generate all this information from a single source. 
Proposed in [1] patch uses Lib/token.py as an initial source. But maybe 
Lib/token.py also should be generated from some file in general format? 
Some information can be derived from Grammar/Grammar, but not all. 
Needed also a mapping between token strings ('(' or '>=') and names 
(LPAR, GREATEREQUAL). Can this be added in Grammar/Grammar or a new file?


There is a related problem, the tokenize module uses three additional 
tokens not used by the C tokenizer. It modifies the content of the token 
module after importing it, that is not good. [2] One of solutions is 
making a copy of tok_names in tokenize before modifying it, but this 
doesn't work, because third-party code search tokenize constants in 
token.tok_names. Other solution is adding tokenize specific constants to 
the token module. Is this good to expose in the token module tokens not 
used in the C tokenizer?


Non-terminal symbols are generated automatically, Lib/symbol.py from 
Include/graminit.h, and Include/graminit.h and Python/graminit.c from 
Grammar/Grammar by Parser/pgen. Is it worth to generate Lib/symbol.py by 
pgen too? Can pgen be implemented in Python?


See also similar issue for opcodes. [3]

[1] https://bugs.python.org/issue30455
[2] https://bugs.python.org/issue25324
[3] https://bugs.python.org/issue17861

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[INADA Naoki]

Hi,

I'm interested in startup time too, not only execution time.
Here is very rough test:


with open('with_abc.py', 'w') as f:
print("import abc", file=f)
for i in range(1, 1001):
print(f"class A{i}(metaclass=abc.ABCMeta): pass", file=f)

with open('without_abc.py', 'w') as f:
print("import abc", file=f)
for i in range(1, 1001):
print(f"class A{i}: pass", file=f)


$  time python3 -c 'import abc'

real 0m0.051s
user 0m0.035s
sys 0m0.013s

$  time python3 -c 'import with_abc'

real 0m0.083s
user 0m0.063s
sys 0m0.017s

$  time python3 -c 'import without_abc'

real 0m0.055s
user 0m0.042s
sys 0m0.011s


It seems 1000 ABC classes takes less than 30ms but 1000 normal
classes takes less than 10ms.

I don't know this penalty is acceptable or not.
But how about making ABC optional?


I don't want to use ABC so frequently when there is no real requirement of ABC.

ABC implementation is very complex and sometimes ABC cause unexpected
performance issue, like you fixed in https://github.com/python/typing/pull/383

If we start with "Protocol is always ABC" and we face unexpected
performance penalty later, it may be difficult to find and optimize it.

# If we can stop using ABC for io.IOBase, Python startup time will be
few ms faster.
# Maybe, I should implement weakset and abc in  C.

Regards,
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

[Ivan Levkivskyi]

On 30 May 2017 at 23:02, Guido van Rossum  wrote:

>
> All in all I'm still leaning towards Naomi's original proposal -- it looks
> simpler to implement as well.
>
>
OK, I think having a bit of verbosity is absolutely fine if we win
simplicity of implementation (for both static and runtime purposes).

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Cory Benfield]

> On 31 May 2017, at 08:42, Victor Stinner  wrote:
> 
> Hi,
> 
> I wrote a PEP based on the previous thread "Backport ssl.MemoryBIO on
> Python 2.7?". Thanks for Cory Benfield, Alex Gaynor and Nick Coghlan
> who helped me to write it!

It probably goes without saying, given that I helped with the drafting for the 
PEP, but I’m strongly in favour of this PEP. Just in case it helps to get that 
reaffirmation here. ;)

Cory

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 7 and braces { .... } on if

[Victor Stinner]

Hi,

I have a question on the CPython coding code for C code, the PEP 7:
https://www.python.org/dev/peps/pep-0007/

"""
Code structure: (...); braces are strongly preferred but may be
omitted where C permits, and they should be formatted as shown:

if (mro != NULL) {
...
}
else {
...
}
"""

This section was updated after the creation of CPython in 1991, so as
you may expect, a lot of existing C code doesn't respect this coding
style. I'm trying to slowly add "missing" braces and indent as the
example *when I have to modify existing code*.

The problem is the "strongly preferred" and "omitted where C permits"
part. I would like to make the PEP 7 more explicit on that part since
in each review, I disagree with Serhiy who wants to omit them.
Example:

if (func == NULL)
return NULL;

versus

if (func == NULL) {
return NULL;
}

I now prefer the version with braces. It's more verbose, but it
prevents bugs when the code is modified later for whatever reasons.
IMHO it also makes the code easily to read and to understand.

So I would suggest to modify the PEP 7 to *always* require braces for if.

I would also suggest to require braces on "for(...) { ... }" and
"while(...) { ... }". But only if the code has to be modified, not
only to update the coding style.

Changes which are pure coding style changes should be rejected to
avoid conflicts with other pending pull requests and "code churn".

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Victor Stinner]

Previous discussion which added "strongly preferred" to the PEP 7, January 2016:
https://mail.python.org/pipermail/python-dev/2016-January/142746.html

Victor

2017-05-31 16:11 GMT+02:00 Victor Stinner :
> Hi,
>
> I have a question on the CPython coding code for C code, the PEP 7:
> https://www.python.org/dev/peps/pep-0007/
>
> """
> Code structure: (...); braces are strongly preferred but may be
> omitted where C permits, and they should be formatted as shown:
>
> if (mro != NULL) {
> ...
> }
> else {
> ...
> }
> """
>
> This section was updated after the creation of CPython in 1991, so as
> you may expect, a lot of existing C code doesn't respect this coding
> style. I'm trying to slowly add "missing" braces and indent as the
> example *when I have to modify existing code*.
>
> The problem is the "strongly preferred" and "omitted where C permits"
> part. I would like to make the PEP 7 more explicit on that part since
> in each review, I disagree with Serhiy who wants to omit them.
> Example:
>
> if (func == NULL)
> return NULL;
>
> versus
>
> if (func == NULL) {
> return NULL;
> }
>
> I now prefer the version with braces. It's more verbose, but it
> prevents bugs when the code is modified later for whatever reasons.
> IMHO it also makes the code easily to read and to understand.
>
> So I would suggest to modify the PEP 7 to *always* require braces for if.
>
> I would also suggest to require braces on "for(...) { ... }" and
> "while(...) { ... }". But only if the code has to be modified, not
> only to update the coding style.
>
> Changes which are pure coding style changes should be rejected to
> avoid conflicts with other pending pull requests and "code churn".
>
> Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Barry Warsaw]

On May 31, 2017, at 04:13 PM, Victor Stinner wrote:

>Previous discussion which added "strongly preferred" to the PEP 7, January
>2016: https://mail.python.org/pipermail/python-dev/2016-January/142746.html

I had to go back to make sure I wouldn't contradict myself.  +1 then, +1
now for requiring braces. :)

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Paul Moore]

On 31 May 2017 at 15:11, Victor Stinner  wrote:
> So I would suggest to modify the PEP 7 to *always* require braces for if.
>
> I would also suggest to require braces on "for(...) { ... }" and
> "while(...) { ... }". But only if the code has to be modified, not
> only to update the coding style.
>
> Changes which are pure coding style changes should be rejected to
> avoid conflicts with other pending pull requests and "code churn".

As a practical compromise, I'd argue that if you're changing
*existing* code, you should retain the current style. For *new* code,
always add braces. So, changing

if (func == NULL)
return NULL;
/* further code */

to

if (func1 == NULL)
return NULL;
/* further code */

you would leave out the braces. But changing

/* Code using func */

to

if (func == NULL) {
return NULL;
}
/* Code using func */

you include the braces, because it's new code.

I'm not against making the PEP mandate braces, but I don't think it's
necessary to resolve the situation you describe as having triggered
the suggestion :-)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Oren Milman]

the 'goto fail' bug is a somewhat extreme reminder for why such braces are
a good idea (as Victor said) -
https://www.imperialviolet.org/2014/02/22/applebug.html

On Wed, May 31, 2017 at 6:25 PM Paul Moore  wrote:

> On 31 May 2017 at 15:11, Victor Stinner  wrote:
> > So I would suggest to modify the PEP 7 to *always* require braces for if.
> >
> > I would also suggest to require braces on "for(...) { ... }" and
> > "while(...) { ... }". But only if the code has to be modified, not
> > only to update the coding style.
> >
> > Changes which are pure coding style changes should be rejected to
> > avoid conflicts with other pending pull requests and "code churn".
>
> As a practical compromise, I'd argue that if you're changing
> *existing* code, you should retain the current style. For *new* code,
> always add braces. So, changing
>
> if (func == NULL)
> return NULL;
> /* further code */
>
> to
>
> if (func1 == NULL)
> return NULL;
> /* further code */
>
> you would leave out the braces. But changing
>
> /* Code using func */
>
> to
>
> if (func == NULL) {
> return NULL;
> }
> /* Code using func */
>
> you include the braces, because it's new code.
>
> I'm not against making the PEP mandate braces, but I don't think it's
> necessary to resolve the situation you describe as having triggered
> the suggestion :-)
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/orenmn%40gmail.com
>
-- 

-Oren
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Jim Baker]

Jython 2.7.1 is about to be released, with full support of upstream pip
(9.0.1), and corresponding vendored libraries, including requests.

However, this proposed new feature for CPython 2.7, and its usage, will
likely break pip on Jython 2.7.x going forward, given that future versions
of pip will depend on requests requiring MemoryBIO. Or am I wrong in this
analysis? This means we have to get back on the 2.7 development treadmill
just as we were about to focus on finally working on Jython 3 (
https://github.com/jython/jython3 previews this work).

Given that this proposed new feature is for 2.7 to support event loop usage
and not a security fix, I'm -1 on this change. In particular, it runs
counter to the justification policy stated in PEP 466.

- Jim


On Wed, May 31, 2017 at 7:25 AM, Cory Benfield  wrote:

>
> > On 31 May 2017, at 08:42, Victor Stinner 
> wrote:
> >
> > Hi,
> >
> > I wrote a PEP based on the previous thread "Backport ssl.MemoryBIO on
> > Python 2.7?". Thanks for Cory Benfield, Alex Gaynor and Nick Coghlan
> > who helped me to write it!
>
> It probably goes without saying, given that I helped with the drafting for
> the PEP, but I’m strongly in favour of this PEP. Just in case it helps to
> get that reaffirmation here. ;)
>
> Cory
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jbaker%40zyasoft.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Serhiy Storchaka]

31.05.17 17:11, Victor Stinner пише:

I have a question on the CPython coding code for C code, the PEP 7:
https://www.python.org/dev/peps/pep-0007/

"""
Code structure: (...); braces are strongly preferred but may be
omitted where C permits, and they should be formatted as shown:

if (mro != NULL) {
 ...
}
else {
 ...
}
"""

This section was updated after the creation of CPython in 1991, so as
you may expect, a lot of existing C code doesn't respect this coding
style. I'm trying to slowly add "missing" braces and indent as the
example *when I have to modify existing code*.

The problem is the "strongly preferred" and "omitted where C permits"
part. I would like to make the PEP 7 more explicit on that part since
in each review, I disagree with Serhiy who wants to omit them.
Example:

if (func == NULL)
 return NULL;

versus

if (func == NULL) {
 return NULL;
}

I now prefer the version with braces. It's more verbose, but it
prevents bugs when the code is modified later for whatever reasons.
IMHO it also makes the code easily to read and to understand.

So I would suggest to modify the PEP 7 to *always* require braces for if.

I would also suggest to require braces on "for(...) { ... }" and
"while(...) { ... }". But only if the code has to be modified, not
only to update the coding style.


Thank you for opening this discussion Victor. I was going to open it myself.

I strongly prefer using braces around "for" and "while" bodies. I 
dislike unconditional adding braces, but after changing PEP 7 I became 
always adding them in my code and asking this in reviewed code. I even 
made Argument Clinic generating braces around gotos. But when I asked 
Barry to add braces around simple statements to satisfy PEP 7 
requirements, he pointed out that braces are only "strongly preferred". 
Since that time I allowed myself to left braces out in cases when they 
look cluttering.


I think that PEP 7 should be more explicit about cases in what braces 
are required and in what they are optional. I prefer to make them 
optional (or even make the variant without braces preferable) in case of 
"if" (without "else") or "case" body containing only the single "goto", 
"break", "continue" or simple "return". Especially if it follows by an 
empty line or closing brace. Adding braces in these cases decreases 
readability IMHO. This is one peculiarity for which many people like (or 
hate) Python.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Victor Stinner]

2017-05-31 17:45 GMT+02:00 Jim Baker :
> Given that this proposed new feature is for 2.7 to support event loop usage
> and not a security fix, I'm -1 on this change. In particular, it runs
> counter to the justification policy stated in PEP 466.

Hum, it seems like the PEP 546 abstract is incomplete. The final goal
of the PEP is to make Python 3 more secure thanks to all goodness of
the PEP 543. The PEP 546 tries to explain why Python 2.7 is blocking
the adoption of the PEP 543 in practice.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Guido van Rossum]

I interpret the PEP as saying that you should use braces everywhere but not
to add them in code that you're not modifying otherwise. (I.e. don't go on
a brace-adding rampage.) If author and reviewer of a PR disagree I would go
with "add braces" since that's clearly the PEP's preference. This is C
code. We should play it safe.

On Wed, May 31, 2017 at 9:02 AM, Serhiy Storchaka 
wrote:

> 31.05.17 17:11, Victor Stinner пише:
>
> I have a question on the CPython coding code for C code, the PEP 7:
>> https://www.python.org/dev/peps/pep-0007/
>>
>> """
>> Code structure: (...); braces are strongly preferred but may be
>> omitted where C permits, and they should be formatted as shown:
>>
>> if (mro != NULL) {
>>  ...
>> }
>> else {
>>  ...
>> }
>> """
>>
>> This section was updated after the creation of CPython in 1991, so as
>> you may expect, a lot of existing C code doesn't respect this coding
>> style. I'm trying to slowly add "missing" braces and indent as the
>> example *when I have to modify existing code*.
>>
>> The problem is the "strongly preferred" and "omitted where C permits"
>> part. I would like to make the PEP 7 more explicit on that part since
>> in each review, I disagree with Serhiy who wants to omit them.
>> Example:
>>
>> if (func == NULL)
>>  return NULL;
>>
>> versus
>>
>> if (func == NULL) {
>>  return NULL;
>> }
>>
>> I now prefer the version with braces. It's more verbose, but it
>> prevents bugs when the code is modified later for whatever reasons.
>> IMHO it also makes the code easily to read and to understand.
>>
>> So I would suggest to modify the PEP 7 to *always* require braces for if.
>>
>> I would also suggest to require braces on "for(...) { ... }" and
>> "while(...) { ... }". But only if the code has to be modified, not
>> only to update the coding style.
>>
>
> Thank you for opening this discussion Victor. I was going to open it
> myself.
>
> I strongly prefer using braces around "for" and "while" bodies. I dislike
> unconditional adding braces, but after changing PEP 7 I became always
> adding them in my code and asking this in reviewed code. I even made
> Argument Clinic generating braces around gotos. But when I asked Barry to
> add braces around simple statements to satisfy PEP 7 requirements, he
> pointed out that braces are only "strongly preferred". Since that time I
> allowed myself to left braces out in cases when they look cluttering.
>
> I think that PEP 7 should be more explicit about cases in what braces are
> required and in what they are optional. I prefer to make them optional (or
> even make the variant without braces preferable) in case of "if" (without
> "else") or "case" body containing only the single "goto", "break",
> "continue" or simple "return". Especially if it follows by an empty line or
> closing brace. Adding braces in these cases decreases readability IMHO.
> This is one peculiarity for which many people like (or hate) Python.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Put token information in one place

[Brett Cannon]

On Wed, 31 May 2017 at 04:01 Serhiy Storchaka  wrote:

> Currently when you add a new token you need to change a couple of files:
>
> * Include/token.h
> * _PyParser_TokenNames in Parser/tokenizer.c
> * PyToken_OneChar(), PyToken_TwoChars() or PyToken_ThreeChars() in
> Parser/tokenizer.c
> * Lib/token.py (generated from Include/token.h)
> * EXACT_TOKEN_TYPES in Lib/tokenize.py
> * Operator, Bracket or Special in Lib/tokenize.py
> * Doc/library/token.rst
>
> It is possible to generate all this information from a single source.
> Proposed in [1] patch uses Lib/token.py as an initial source. But maybe
> Lib/token.py also should be generated from some file in general format?
>

I don't think it matters really. Whatever is most convenient.


> Some information can be derived from Grammar/Grammar, but not all.
> Needed also a mapping between token strings ('(' or '>=') and names
> (LPAR, GREATEREQUAL). Can this be added in Grammar/Grammar or a new file?
>

Maybe Grammar/Tokens?


>
> There is a related problem, the tokenize module uses three additional
> tokens not used by the C tokenizer. It modifies the content of the token
> module after importing it, that is not good. [2] One of solutions is
> making a copy of tok_names in tokenize before modifying it, but this
> doesn't work, because third-party code search tokenize constants in
> token.tok_names. Other solution is adding tokenize specific constants to
> the token module. Is this good to expose in the token module tokens not
> used in the C tokenizer?
>

No opinion from me.


>
> Non-terminal symbols are generated automatically, Lib/symbol.py from
> Include/graminit.h, and Include/graminit.h and Python/graminit.c from
> Grammar/Grammar by Parser/pgen. Is it worth to generate Lib/symbol.py by
> pgen too? Can pgen be implemented in Python?
>

I assume there's a build rule for Python/graminit.c and porting pgen to
Python would require Python be installed to do a build from scratch. Have
we made that a requirement yet? If so then rewriting pgen in Python would
make it easier to maintain (although when was the last time anyone touched
that file?).

-Brett


>
> See also similar issue for opcodes. [3]
>
> [1] https://bugs.python.org/issue30455
> [2] https://bugs.python.org/issue25324
> [3] https://bugs.python.org/issue17861
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Put token information in one place

[Serhiy Storchaka]

31.05.17 20:58, Brett Cannon пише:
I assume there's a build rule for Python/graminit.c and porting pgen to 
Python would require Python be installed to do a build from scratch. 
Have we made that a requirement yet? If so then rewriting pgen in Python 
would make it easier to maintain (although when was the last time anyone 
touched that file?).


That is why generated C files are added to the repository. There are 
precedences: Include/Python-ast.h, Python/Python-ast.c, 
Include/opcode.h, Python/opcode_targets.h, Objects/typeslots.inc, 
Objects/unicodetype_db.h, Modules/cjkcodecs/mappings_*.h, 
Modules/sre_constants.h, Modules/_ssl_data.h, and a number of 
**/clinic/*.h files are generated by Python scripts.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Jim Baker]

So we have two distinct changes that are proposed here:

1. Support alternative implementations of TLS instead of OpenSSL. In
particular this will enable the use of system trust stores for certificates.

2. Implement ABCs and concrete classes to support MemoryBIO, etc., from 3.7.

Supporting system trust stores is a valid security fix for 2.7, and I have
no such problem with such changes as long as they are narrowed to this
specific change.

But I object to a completely new feature being added to 2.7 to support the
implementation of event loop SSL usage. This feature cannot be construed as
a security fix, and therefore does not qualify as a feature that can be
added to CPython 2.7 at this point in its lifecycle.

The discussion that implementing such new features for 2.7 will improve
their adoption for Python 3 is a red herring. We could enumerate many such
features, but https://www.python.org/dev/peps/pep-0404/#upgrade-path is
rather clear here.

- Jim

On Wed, May 31, 2017 at 10:40 AM, Victor Stinner 
wrote:

> 2017-05-31 17:45 GMT+02:00 Jim Baker :
> > Given that this proposed new feature is for 2.7 to support event loop
> usage
> > and not a security fix, I'm -1 on this change. In particular, it runs
> > counter to the justification policy stated in PEP 466.
>
> Hum, it seems like the PEP 546 abstract is incomplete. The final goal
> of the PEP is to make Python 3 more secure thanks to all goodness of
> the PEP 543. The PEP 546 tries to explain why Python 2.7 is blocking
> the adoption of the PEP 543 in practice.
>
> Victor
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

[Barry Warsaw]

On May 31, 2017, at 02:09 PM, Jim Baker wrote:

>But I object to a completely new feature being added to 2.7 to support the
>implementation of event loop SSL usage. This feature cannot be construed as
>a security fix, and therefore does not qualify as a feature that can be
>added to CPython 2.7 at this point in its lifecycle.

The other problem with this is that there isn't just one CPython 2.7, there
are many.  It's likely that most people get their Python from whatever
distribution/package manager they use.  Looking at active Ubuntu/Debian
releases you can see Python 2.7's from 2.7.3 to 2.7.13.  Few if any of those
will get the new feature backported, so that makes it difficult to rely on
them being present.

I agree with Jim that it makes sense to backport security fixes.  Usually
those are more well contained, and thus easier to cherry pick into stable
releases.  New features are much tougher to justify the potential for
regressions.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

[Guido van Rossum]

On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi 
wrote:

> On 31 May 2017 at 00:58, Guido van Rossum  wrote:
> [...]
>
> Thank you for very detailed answers! I have practically nothing to add.
> It seems to me that most of the Kevin's questions stem from unnecessary
> focus
> on runtime type checking. Here are two ideas about how to fix this:
>
> * Add the word "static" somewhere in the PEP title.
>

So the title could become "Protocols: Static structural subtyping (duck
typing)" -- long, but not record-setting.

* Add a short note at the start mentioning this is an extension of the type
> system proposed in PEP 484 and recommending to read PEP 484 first.
>

Hm, the Abstract already spells that out. I suspect that many people react
to the discussion without first reading the PEP itself (I do this myself
:-). The only thing that could possibly be confusing about the abstract is
that it claims to specify "static and runtime semantics" -- but that's
reasonable, since the runtime semantics must somehow be specified even if
they're minimal.

-- 
--Guido van Rossum (python.org/~guido )
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

[Guido van Rossum]

On Wed, May 31, 2017 at 6:16 AM, Ivan Levkivskyi 
wrote:

> On 30 May 2017 at 23:02, Guido van Rossum  wrote:
>
>> All in all I'm still leaning towards Naomi's original proposal -- it
>> looks simpler to implement as well.
>>
>
> OK, I think having a bit of verbosity is absolutely fine if we win
> simplicity of implementation (for both static and runtime purposes).
>

Then I propose to do it this way. We can always add Jukka's way as an
alternative notation later. I'd like to hear from Jukka before I merge the
PR for PEP-484.

In the meantime, Naomi, are you interested in trying to implement this?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Greg Ewing]

Seems like a good idea to tighten it up.

If a style guide is going to say "you can either do X or
not do X", it might as well not mention X at all. :-)

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Put token information in one place

[Guido van Rossum]

On Wed, May 31, 2017 at 4:00 AM, Serhiy Storchaka 
wrote:

> [...] Can pgen be implemented in Python?
>

Yes, an implementation already exists in the stdlib under lib2to3/pgen2/...

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Benjamin Peterson]

Modern GCC can defend against these kinds of problems. If I introduce a
"goto fail" bug somewhere in Python, I get a nice warning:
../Objects/abstract.c: In function ‘PyObject_Type’:
../Objects/abstract.c:35:5: warning: this ‘if’ clause does not guard...
[-Wmisleading-indentation]
 if (o == NULL)
 ^~
../Objects/abstract.c:37:9: note: ...this statement, but the latter is
misleadingly indented as if it is guarded by the ‘if’
 2 + 3;
 ^

This is not to say that simply bracing everything isn't the right way to
go.

On Wed, May 31, 2017, at 15:59, Greg Ewing wrote:
> Seems like a good idea to tighten it up.
> 
> If a style guide is going to say "you can either do X or
> not do X", it might as well not mention X at all. :-)
> 
> -- 
> Greg
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/benjamin%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Serhiy Storchaka]

31.05.17 20:27, Guido van Rossum пише:
I interpret the PEP as saying that you should use braces everywhere but 
not to add them in code that you're not modifying otherwise. (I.e. don't 
go on a brace-adding rampage.) If author and reviewer of a PR disagree I 
would go with "add braces" since that's clearly the PEP's preference. 
This is C code. We should play it safe.


Thank you. I'll be interpreting it the same way.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 7 and braces { .... } on if

[Serhiy Storchaka]

01.06.17 09:36, Benjamin Peterson пише:

Modern GCC can defend against these kinds of problems. If I introduce a
"goto fail" bug somewhere in Python, I get a nice warning:
../Objects/abstract.c: In function ‘PyObject_Type’:
../Objects/abstract.c:35:5: warning: this ‘if’ clause does not guard...
[-Wmisleading-indentation]
  if (o == NULL)
  ^~
../Objects/abstract.c:37:9: note: ...this statement, but the latter is
misleadingly indented as if it is guarded by the ‘if’
  2 + 3;
  ^

This is not to say that simply bracing everything isn't the right way to
go.


Actually a bug with misleadingly indented statement in CPython sources 
was fixed just few months ago (thanks to modern GCC).


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com