Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Christian Heimes 
On 2016-08-29 04:38, Ned Deily wrote:
> On Aug 28, 2016, at 19:06, Benjamin Peterson  wrote:
>> On Sun, Aug 28, 2016, at 13:40, Christian Heimes wrote:
>>> Here is the deal for 2.7 to 3.5:
>>>
>>> 1) All versions older than 0.9.8 are completely out-of-scope and no
>>> longer supported.
>> +1
>>> 2) 0.9.8 is semi-support. Python will still compile and work with 0.9.8.
>>> However we do NOT promise that is secure to run 0.9.8. We also require a
>>> recent version. Patch level 0.9.8zc from October 2014 is reasonable
>>> because it comes with SCSV fallback (CVE-2014-3566).
>> I think we should support 0.9.8 for 2.7 and drop it for 3.6.
> 
> Sounds good to me, too.  I think we should also not change things for 3.5.x 
> at this point, e.g. continue to support 0.9.8 there.


In my proto-PEP I'm talking about different levels of support: full,
build-only and unsupported. Full support means that the combination of
Python and OpenSSL versions is reasonable secure and recommended.

On the other hand build-only support doesn't come with any security
promise. The ssl and hashlib module are source compatible with OpenSSL
0.9.8. You can still compile Python, do https connections but they might
not be secure. It's "Warranty void" mode.

>>> 3) 1.0.0 is irrelevant. Users are either stuck on 0.9.8 or are able to
>>> upgrade to 1.0.1+. Let's not support it.
>>>
>>> 4) 1.0.1 is discouraged but still supported until its EOL.
>>>
>>> 5) 1.0.2 is the recommend version.
>>>
>>> 6) 1.1 support will be added by #26470 soon.
> 
> [...]
> 
>>> For upcoming 3.6 I would like to limit support to 1.0.2+ and require
>>> 1.0.2 features for 3.7.
> 
> It's not clear to me what you are proposing as the differences between 3.6 
> ("limit support to 1.0.2+") and 3.7 ("require 1.0.2 features").  Could you 
> elaborate?

For 3.6 I don't require any 1.0.2 feature yet. The 1.1.0 patch keeps
code compatible with 0.9.8zc to 1.1.0. But as soon as I use new
features, the ssl module will no longer be source and build compatible
with < 1.0.2. There is also the point of OpenSSL 1.0.1. It reaches
end-of-lifetime by the end if this year. 1.0.2 will be supported until 2019.

I'm tempted to require 1.0.2 for Python 3.6 but it's technically not
necessary yet.

#if OPENSSL_VERSION_INFO < 0x01000200L
#  error "OpenSSL 1.0.2+ required"
#endif


> 
>>> What is the status of Python.org's OSX builds?
>>> Is it possible to drop 0.9.8?
> 
> I think we can safely drop 0.9.8 support in 3.6.  If anyone is aware of any 
> supported platform where this will would cause a problem, please speak up now.
> 
> With regard to OS X (or macOS, as the upcoming next major release is called), 
> the 3.6.0 python.org OS X installer will supply a private copy of OpenSSL 
> 1.0.2+.  Most other third-party distributors of Python on OS X already do not 
> use the Apple-suplied deprecated system OpenSSL libs.  As of the current OS X 
> 10.11 El Capitan, Apple no longer supplies the header files for OpenSSL in 
> either Xcode macosx SDK or in the optional Command Line Tools /usr/include 
> headers so, if you want to build Python on OS X, you now need to use a 
> non-system copy of OpenSSL anyway (the devguide explains how to build with 
> OpenSSL libs from either Homebrew or MacPorts).  The shared libs are still 
> supplied for the benefit of applications built on older releases and for the 
> Apple-supplied system Pythons (2.6.x and 2.7
>  .x, still no 3.x).

I'm glad to hear that. Thanks :)

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] File system path encoding on Windows

2016-08-28 Thread Stephen J. Turnbull 
tritium-l...@sdamon.com writes:

 > Once you get to var lengths like that, arcane single character flags start
 > looking preferable.  How about "PYTHONWINLEGACY" to just turn it all on or
 > off.  If the code breaks on one thing, it obviously isn't written to use the
 > other two, so might as well shut them all off.

Since Steve is thinking about three separate PEPs (among other things,
they might be implemented on different timelines), that's not really
possible (placing the features under control of one switch at
different times would be an unacceptable compatibility break).

Anyway, it's not *obvious* that your premise is true, because code
isn't written to do any of those things.  It's written to process
bytes agnostically.  The question is what does the environment look
like.  Steve obviously has a perspective on environment which suggests
that these aspects are often decoupled because in Windows the actual
filesystem is never bytes-oriented.  I don't know if it's possible to
construct a coherent environment where these aspects are decoupled,
but I can't say it's impossible, either.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Ned Deily 
On Aug 28, 2016, at 19:06, Benjamin Peterson  wrote:
> On Sun, Aug 28, 2016, at 13:40, Christian Heimes wrote:
>> Here is the deal for 2.7 to 3.5:
>> 
>> 1) All versions older than 0.9.8 are completely out-of-scope and no
>> longer supported.
> +1
>> 2) 0.9.8 is semi-support. Python will still compile and work with 0.9.8.
>> However we do NOT promise that is secure to run 0.9.8. We also require a
>> recent version. Patch level 0.9.8zc from October 2014 is reasonable
>> because it comes with SCSV fallback (CVE-2014-3566).
> I think we should support 0.9.8 for 2.7 and drop it for 3.6.

Sounds good to me, too.  I think we should also not change things for 3.5.x at 
this point, e.g. continue to support 0.9.8 there.

>> 3) 1.0.0 is irrelevant. Users are either stuck on 0.9.8 or are able to
>> upgrade to 1.0.1+. Let's not support it.
>> 
>> 4) 1.0.1 is discouraged but still supported until its EOL.
>> 
>> 5) 1.0.2 is the recommend version.
>> 
>> 6) 1.1 support will be added by #26470 soon.

[...]

>> For upcoming 3.6 I would like to limit support to 1.0.2+ and require
>> 1.0.2 features for 3.7.

It's not clear to me what you are proposing as the differences between 3.6 
("limit support to 1.0.2+") and 3.7 ("require 1.0.2 features").  Could you 
elaborate?

>> What is the status of Python.org's OSX builds?
>> Is it possible to drop 0.9.8?

I think we can safely drop 0.9.8 support in 3.6.  If anyone is aware of any 
supported platform where this will would cause a problem, please speak up now.

With regard to OS X (or macOS, as the upcoming next major release is called), 
the 3.6.0 python.org OS X installer will supply a private copy of OpenSSL 
1.0.2+.  Most other third-party distributors of Python on OS X already do not 
use the Apple-suplied deprecated system OpenSSL libs.  As of the current OS X 
10.11 El Capitan, Apple no longer supplies the header files for OpenSSL in 
either Xcode macosx SDK or in the optional Command Line Tools /usr/include 
headers so, if you want to build Python on OS X, you now need to use a 
non-system copy of OpenSSL anyway (the devguide explains how to build with 
OpenSSL libs from either Homebrew or MacPorts).  The shared libs are still 
supplied for the benefit of applications built on older releases and for the 
Apple-supplied system Pythons (2.6.x and 2.7.x, still no 3.x).

> Thanks for writing this patch!

Ditto!

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Nick Coghlan 
On 29 August 2016 at 06:40, Christian Heimes  wrote:
> Hi,
>
> we need to talk about OpenSSL and LibreSSL before the next release of
> Python. I'm working on a PEP. Most likely it won't be ready before the
> feature freeze.

If it's just drafting work that you need help with on that front, feel
free to send me what you have and I can work it up into PEP form so
folks can see a consolidated list of the proposed changes.

> I like to reduce the maintenance burden and list of supported OpenSSL
> versions ASAP. OpenSSL has deprecated 0.9.8 and 1.0.0 last year. 1.0.1
> will reach EOL by the end of this year,
> https://www.openssl.org/policies/releasestrat.html . However OpenSSL
> 0.9.8 is still required for some platforms (OSX).

Back when I wrote PEP 466, Ned indicated he was in favour of switching
to static linking for the Mac OS X installers:
https://mail.python.org/pipermail/python-dev/2014-March/133347.html

So for 3.6, I agree with Benjamin's suggestion that we drop 0.9.8
support as well.

For 2.7, I think we should defer the decision on what to do to a
follow-up to PEP 466 that resyncs 2.7 with the Python 3.6 network
security stack (while 466 got 2.7 to parity with 3.4.3, even that's
starting to show its age now)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] File system path encoding on Windows

2016-08-28 Thread Nick Coghlan 
On 29 August 2016 at 06:39,   wrote:
>> -Original Message-
>> From: Python-Dev [mailto:python-dev-bounces+tritium-
>> list=sdamon@python.org] On Behalf Of Steve Dower
>> Sent: Wednesday, August 24, 2016 11:44 AM
>> To: Stephen J. Turnbull 
>> Cc: Nick Coghlan ; Python Dev > d...@python.org>
>> Subject: Re: [Python-Dev] File system path encoding on Windows
>>
>> On 23Aug2016 2150, Stephen J. Turnbull wrote:
>> > Steve Dower writes:
>> >
>> >  > * Stephen sees "no reason not to change
> locale.getpreferredencoding()"
>> >  > (default encoding for open()) at the same time with the same
> switches,
>> >  > while I'm not quite as confident. Do users generally specify an
> encoding
>> >  > these days? I know I always put utf-8 there.
>> >
>> > I was insufficiently specific.  "No reason not to" depends on separate
>> > switches for file system encoding and preferred encoding.  That makes
>> > things somewhat more complicated for implementation, and significantly
>> > so for users.
>>
>> Yes, it does, but it's about the only possible migration path.
>>
>> I know Nick and Victor like the idea of a -X flag (or a direct -utf8
>> flag),

Command line flags and environment variables aren't mutually exclusive
- sometimes you have easy access to add command line arguments,
sometimes you have easy access to set environment variables, sometimes
you have equally easy access to both.

The idea of a "-X" flag is to have an easy way to try out new default
settings that say "Ignore the nominal default encoding and just assume
UTF-8 everywhere", such that folks can get Python behaving that way,
even if they haven't quite figured out how to configure their
operating system itself to use those defaults (this is particularly
relevant for non-systemd based Linux systems, as other init systems
generally don't have the ability to universally override the default
POSIX locale, but even systemd based systems can still do the wrong
thing if "LANG=C" is explicitly specified instead of "LANG=C.UTF-8",
if the particular distro in use doesn't have the latter locale
available, or if a client's locale settings have been forwarded to a
server SSH session)

>> but I prefer more specific environment variables:
>>
>> - PYTHONWINDOWSLEGACYSTDIO (for the console changes)
>> - PYTHONWINDOWSLEGACYPATHENCODING (assuming
>> getfilesystemencoding() is utf8)
>> - PYTHONWINDOWSLEGACYLOCALEENCODING (assuming
>> getpreferredencoding() is
>> utf8)
>
> Once you get to var lengths like that, arcane single character flags start
> looking preferable.  How about "PYTHONWINLEGACY" to just turn it all on or
> off.  If the code breaks on one thing, it obviously isn't written to use the
> other two, so might as well shut them all off.

+1 for a single global on-off switch that means we (and everyone else)
only have two configurations to test (the old way and the new way),
rather than a combinatorial explosion of 8 or more possible settings.
If we get demand for more configurability, *then* we can increase the
complexity, but we shouldn't inflict that level of QA pain on
ourselves in the absence of vocal user demand.

If the more fine-grained settings are considered useful for debugging
purposes, then they can be added with underscore prefixes and
documented accordingly (i.e. as a way of figuring out precisely which
aspect of the new defaults is causing a problem, rather than as a
permanently supported variant configuration).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Cory Benfield 

> On 28 Aug 2016, at 16:40, Christian Heimes  wrote:
> 
> For upcoming 3.6 I would like to limit support to 1.0.2+ and require
> 1.0.2 features for 3.7. What is the status of Python.org's OSX builds?
> Is it possible to drop 0.9.8?

I strongly support this change. Python with old OpenSSL versions is an enormous 
source of difficult-to-debug problems for users attempting to work with the 
web, and puts our users at totally unnecessary risk. Let’s move on.

Cory

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Benjamin Peterson 


On Sun, Aug 28, 2016, at 13:40, Christian Heimes wrote:
> Here is the deal for 2.7 to 3.5:
> 
> 1) All versions older than 0.9.8 are completely out-of-scope and no
> longer supported.

+1

> 
> 2) 0.9.8 is semi-support. Python will still compile and work with 0.9.8.
> However we do NOT promise that is secure to run 0.9.8. We also require a
> recent version. Patch level 0.9.8zc from October 2014 is reasonable
> because it comes with SCSV fallback (CVE-2014-3566).

I think we should support 0.9.8 for 2.7 and drop it for 3.6.

> 
> 3) 1.0.0 is irrelevant. Users are either stuck on 0.9.8 or are able to
> upgrade to 1.0.1+. Let's not support it.
> 
> 4) 1.0.1 is discouraged but still supported until its EOL.
> 
> 5) 1.0.2 is the recommend version.
> 
> 6) 1.1 support will be added by #26470 soon.

Thanks for writing this patch!
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Christian Heimes 
Hi,

we need to talk about OpenSSL and LibreSSL before the next release of
Python. I'm working on a PEP. Most likely it won't be ready before the
feature freeze.

But first let me start with some good news. OpenSSL 1.1 was released a
couple of days ago. It changed a lot of aspects of its internal API,
e.g. all structs are opaque and must be allocated / freed with OpenSSL
API calls. Since I have been tracking changes in OpenSSL for the last
half year and have submitted a couple of patches to OpenSSL, we are in a
good shape. My patch https://bugs.python.org/issue26470 makes Python 2
and 3 compatible with OpenSSL 0.9.8 to 1.1.0 and with LibreSSL, too. It
needs to go through review, though. I have asked Alex to verify my patch.

Now to the bad news. The SSL module is a mess. It looks like a junk room
owned by collector of ancient OpenSSL versions. For example it contains
version checks for OpenSSL 0.9.5 -- which was decommissioned in 2000!
That pre-dates new style classes!

I like to reduce the maintenance burden and list of supported OpenSSL
versions ASAP. OpenSSL has deprecated 0.9.8 and 1.0.0 last year. 1.0.1
will reach EOL by the end of this year,
https://www.openssl.org/policies/releasestrat.html . However OpenSSL
0.9.8 is still required for some platforms (OSX).


Here is the deal for 2.7 to 3.5:

1) All versions older than 0.9.8 are completely out-of-scope and no
longer supported.

2) 0.9.8 is semi-support. Python will still compile and work with 0.9.8.
However we do NOT promise that is secure to run 0.9.8. We also require a
recent version. Patch level 0.9.8zc from October 2014 is reasonable
because it comes with SCSV fallback (CVE-2014-3566).

3) 1.0.0 is irrelevant. Users are either stuck on 0.9.8 or are able to
upgrade to 1.0.1+. Let's not support it.

4) 1.0.1 is discouraged but still supported until its EOL.

5) 1.0.2 is the recommend version.

6) 1.1 support will be added by #26470 soon.

7) LibreSSL 2.3 is supported but with a slightly limited feature set.
LibreSSL removed some features like SSL_CERT_FILE and OPENSSL_CONF env vars.


For upcoming 3.6 I would like to limit support to 1.0.2+ and require
1.0.2 features for 3.7. What is the status of Python.org's OSX builds?
Is it possible to drop 0.9.8?

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] File system path encoding on Windows

2016-08-28 Thread tritium-list 


> -Original Message-
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Steve Dower
> Sent: Wednesday, August 24, 2016 11:44 AM
> To: Stephen J. Turnbull 
> Cc: Nick Coghlan ; Python Dev  d...@python.org>
> Subject: Re: [Python-Dev] File system path encoding on Windows
> 
> On 23Aug2016 2150, Stephen J. Turnbull wrote:
> > Steve Dower writes:
> >
> >  > * Stephen sees "no reason not to change
locale.getpreferredencoding()"
> >  > (default encoding for open()) at the same time with the same
switches,
> >  > while I'm not quite as confident. Do users generally specify an
encoding
> >  > these days? I know I always put utf-8 there.
> >
> > I was insufficiently specific.  "No reason not to" depends on separate
> > switches for file system encoding and preferred encoding.  That makes
> > things somewhat more complicated for implementation, and significantly
> > so for users.
> 
> Yes, it does, but it's about the only possible migration path.
> 
> I know Nick and Victor like the idea of a -X flag (or a direct -utf8
> flag), but I prefer more specific environment variables:
> 
> - PYTHONWINDOWSLEGACYSTDIO (for the console changes)
> - PYTHONWINDOWSLEGACYPATHENCODING (assuming
> getfilesystemencoding() is utf8)
> - PYTHONWINDOWSLEGACYLOCALEENCODING (assuming
> getpreferredencoding() is
> utf8)

Once you get to var lengths like that, arcane single character flags start
looking preferable.  How about "PYTHONWINLEGACY" to just turn it all on or
off.  If the code breaks on one thing, it obviously isn't written to use the
other two, so might as well shut them all off.


> I'm open to dropping "WINDOWS" from these if others don't think it's
> necessary. Swap "LEGACY" for "UNICODE" if we just offer the option to
> upgrade now rather than changing the default.
> 
> (I could also see the possibility of PYTHONWINDOWSSTRICT* options to use
> the default encoding but raise if decoding bytes fails - mbcs:strict
> rather than mbcs:replace. For utf-8 mode I'd want to use surrogatepass
> throughout, so it will always raise on invalid encoding, but there will
> be far fewer occurrences than for mbcs.)
> 
> I'll transform my earlier post into a PEP (or probably three PEPs - one
> for each change), since it seems like the paper trail is going to be
> more valuable than discussing it now. Without an actual build to test
> it's pretty hard to evaluate the impact.
> 
> Cheers,
> Steve
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What do we do about bad slicing and possible crashes (issue 27867)

2016-08-28 Thread Nick Coghlan 
On 28 August 2016 at 08:25, Terry Reedy  wrote:
> Slicing can be made to malfunction and even crash with an 'evil' __index__
> method. https://bugs.python.org/issue27867
>
> The crux of the problem is this: PySlice_GetIndicesEx
> receives a slice object and a sequence length.  Calling __index__ on the
> start, stop, and step components can mutate the sequence and invalidate the
> length.  Adjusting the int values of start and stop according to an invalid
> length (in particular, one that is too long) will result in invalid results
> or a crash.
>
> Possible actions -- very briefly.  For more see end of
> https://bugs.python.org/issue27867?@ok_message=msg 273801
> 0. Do nothing.
> 1. Detect length change and raise.

I suggest taking this path - it's the lowest impact, and akin to the
"dictionary changed size during iteration" runtime error.

__index__ having side effects is pathological code behaviour, so we
really just need to prevent the interpreter crash, rather than trying
to make it sense of it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Review request: issue 27350, compact ordered dict

2016-08-28 Thread INADA Naoki 
On Sun, Aug 28, 2016 at 2:05 PM, Guido van Rossum  wrote:
> Hopefully some core dev(s) can work on this during the core sprint, which is
> from Sept 5-9.
>

OK. While I'm in Japan (UTC+9) and cannot join the sprint, I'll be
active as possible
while the sprint.

Thank you!


>
> --
> --Guido van Rossum (python.org/~guido)

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com