Re: [Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-09 Thread Guido van Rossum 
Well reasoned!

On Fri, Jun 9, 2017 at 4:56 PM, Benjamin Peterson 
wrote:

> The reason we're having this conversation at all is probably a matter of
> timing. If MemoryBIO was in Python 3 when PEP 466 was accepted, it surely
> would have come along for the ride to 2.7. I believe PEP 466 is generally
> considered to have produced positive results. PEP 546, carrying no breaking
> changes, is less risky than PEP 466.
>
> The reluctance to bend 2.7 rules is healthy. This PEP is part of the price
> we pay, though, for making a backwards-incompatible release. The security
> landscape has and will change over the 10+ python-dev-supported life span
> of 2.7. During that time, we have an obligation to keep Python 2 secure.
> Part of that is supporting modern security interfaces, which are features.
> This change is needed to make another stdlib feature, ensurepip (which is
> itself yet another 2.7.x backport) work well.
>
> Therefore, as 2.7 release manager, I'm accepting the PEP.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-09 Thread Benjamin Peterson 
The reason we're having this conversation at all is probably a matter of
timing. If MemoryBIO was in Python 3 when PEP 466 was accepted, it
surely would have come along for the ride to 2.7. I believe PEP 466 is
generally considered to have produced positive results. PEP 546,
carrying no breaking changes, is less risky than PEP 466.
The reluctance to bend 2.7 rules is healthy. This PEP is part of the
price we pay, though, for making a backwards-incompatible release. The
security landscape has and will change over the 10+ python-dev-supported
life span of 2.7. During that time, we have an obligation to keep Python
2 secure. Part of that is supporting modern security interfaces, which
are features. This change is needed to make another stdlib feature,
ensurepip (which is itself yet another 2.7.x backport) work well.
Therefore, as 2.7 release manager, I'm accepting the PEP.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove embedded expat library?

2017-06-09 Thread Terry Reedy 

On 6/9/2017 12:40 PM, Zachary Ware wrote:


PR 1783 (https://github.com/python/cpython/pull/1783); needs another
review from Steve to make sure I haven't made a complete mess of
things, then it should be ready to go.  Anyone else on Windows (Terry
Reedy?) who can test it and provide feedback, please do!


Something like the following condensed log?

f:\dev>git clone https://github.com/python/cpython tem
f:\dev>cd tem
f:\dev\tem>git remote add upstream https://github.com/python/cpython
f:\dev\tem>git pr 1783
f:\dev\tem>pcbuild\build.bat -e
Fetching external libraries...
Fetching bzip2-1.0.6...
Fetching openssl-1.0.2k...
Fetching sqlite-3.14.2.0...
Fetching tcl-core-8.6.6.0...
Fetching tk-8.6.6.0...
Fetching tix-8.4.3.6...
Fetching xz-5.2.2...
Fetching external binaries...
Fetching nasm-2.11.06...
...

# Appeared to finish with usual warnings, but no errors.
f:\dev\tem>python -m test -j14
...
367 tests OK.

38 tests skipped:
test_crypt test_curses test_dbm_gnu test_dbm_ndbm test_devpoll
test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl
test_kqueue test_nis test_openpty test_ossaudiodev test_pipes
test_poll test_posix test_pty test_pwd test_readline test_resource
test_smtpnet test_socketserver test_spwd test_syslog
test_threadsignals test_timeout test_tix test_tk test_ttk_guionly
test_urllib2net test_urllibnet test_wait3 test_wait4 test_winsound
test_xmlrpc_net test_zipfile64

Total duration: 2 min 15 sec
Tests result: SUCCESS
-

Duration aside, my working clone master, without the patch, gives the 
same result.  (The usual debug build takes twice as long.)  In this 
respect, the patch seems ready to go.  If you have any questions about 
the compile or test logs, I will try to keep them around at least a few 
hours.





--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Donald Stufft 

> On Jun 9, 2017, at 3:41 PM, Steve Dower  wrote:
> 
> On 09Jun2017 1118, Donald Stufft wrote:
>> A ensuressl style module that tries to install an OpenSSL module is actually 
>> fairly hard to do securely. The fundamental issue being that fetching a file 
>> securely from the network before you have the primary tool for fetching 
>> things securely from a network gets to be a bit of a chicken and egg 
>> problem. There are *possible* ways around that, but they essentially boil 
>> down to having to ship something akin to a CA Bundle inside of Python which 
>> we just shift the responsibility from needing to have an updated OpenSSL in 
>> all versions of Python.
> 
>> The good news here is that PEP 543 is the thing that will enable using 
>> SChannel on Windows, SecureTransport on macOS, and OpenSSL on Linux getting 
>> python-dev out of the business of shipping an OpenSSL at all [1]. 
>> Backporting ssl.MemoryBio represents the smallest reasonable change to 2.7 
>> we can make [2] that paves the path forward to being able to do that in some 
>> number of years, because we can’t do that until PEP 543 is a thing and 
>> people are using it.
> 
> One of the reasons I'm wanting to push this way is that there are other APIs 
> on Windows to do the core client use cases we seem to care about.
> 
> For example, 
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa384106(v=vs.85).aspx
>  could easily let us implement the requests API with a direct call to the 
> operating system, without having to get into the SChannel world of hurt. I 
> assume macOS has similarly high-level APIs.
> 
> These are totally fine for implementing a requests-like API that relies on 
> system configuration for HTTPS connections. They are not sufficient for 
> building a server, but they should be sufficient for downloading the packages 
> you need to build a server.
> 
> This is starting to feel to me like we're stuck on "building the thing right" 
> and have skipped over "building the right thing". But maybe it's just me…
> 


I’m personally not super interested in having platform specific HTTP handling. 
I honestly want as little platform specific code as I can get away with, and 
the primary reason why I’m okay with drawing the line at the TLS stack (instead 
of the HTTP stack) is because being able to get operating system updates for 
the TLS stack is of immense benefit in addition to the fact that it’s really 
the *only* reasonable way to get access to the platform specific trust store.

Relying on the OS to handle HTTP means that we cannot depend on anything that 
isn’t the lowest common denominator across all of the variety of platforms we 
support, which (as I see Cory has mentioned) isn’t really a thing outside of 
macOS/Windows. For those platforms we’d just be adding yet another C dependency 
to Python (or pip, or requests, or whatever). There’s a lot of HTTP handling 
code in pip that would need to be rewritten to support this, and I don’t think 
it would be of a particular benefit much less be doable in any reasonable 
timeframe.

This isn’t really being particularly innovative in this area, it’s essentially 
what the browsers are doing. It also matches what curl is doing.

All that being said, if someone *does* want pip to use WinHTTP, requests 
provides a mechanism where you can plug in your own network handling code, so 
someone could write a requests-winhttp adapter that did that, and we could 
possibly even expose the ability to ask pip to use that. However that’s 
entirely in the domain of pip/requests feature requests and not really related 
to python-dev/this PEP itself.


—
Donald Stufft



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Cory Benfield 

> On 9 Jun 2017, at 20:41, Steve Dower  wrote:
> 
> These are totally fine for implementing a requests-like API that relies on 
> system configuration for HTTPS connections. They are not sufficient for 
> building a server, but they should be sufficient for downloading the packages 
> you need to build a server.
> 
> This is starting to feel to me like we're stuck on "building the thing right" 
> and have skipped over "building the right thing". But maybe it's just me...

For the purposes of this PEP I think we should exclude this. The only way this 
works on a decent number of platforms (hi there BSDs and Linuxes) is if the 
option on those platforms is libcurl. Linux does not ship a high level HTTP 
client library: libcurl is basically the option. That would require pip binding 
libcurl, NSURLSession, and the Windows API. It’s not clear to me that the 
solution to “we don’t want to backport some SSL code” is “write a bunch of 
bindings to other third-party libraries”.

Cory 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Steve Dower 

On 09Jun2017 1118, Donald Stufft wrote:
A ensuressl style module that tries to install an OpenSSL module is 
actually fairly hard to do securely. The fundamental issue being that 
fetching a file securely from the network before you have the primary 
tool for fetching things securely from a network gets to be a bit of a 
chicken and egg problem. There are *possible* ways around that, but they 
essentially boil down to having to ship something akin to a CA Bundle 
inside of Python which we just shift the responsibility from needing to 
have an updated OpenSSL in all versions of Python.


The good news here is that PEP 543 is the thing that will enable using 
SChannel on Windows, SecureTransport on macOS, and OpenSSL on Linux 
getting python-dev out of the business of shipping an OpenSSL at all 
[1]. Backporting ssl.MemoryBio represents the smallest reasonable change 
to 2.7 we can make [2] that paves the path forward to being able to do 
that in some number of years, because we can’t do that until PEP 543 is 
a thing and people are using it.


One of the reasons I'm wanting to push this way is that there are other 
APIs on Windows to do the core client use cases we seem to care about.


For example, 
https://msdn.microsoft.com/en-us/library/windows/desktop/aa384106(v=vs.85).aspx 
could easily let us implement the requests API with a direct call to the 
operating system, without having to get into the SChannel world of hurt. 
I assume macOS has similarly high-level APIs.


These are totally fine for implementing a requests-like API that relies 
on system configuration for HTTPS connections. They are not sufficient 
for building a server, but they should be sufficient for downloading the 
packages you need to build a server.


This is starting to feel to me like we're stuck on "building the thing 
right" and have skipped over "building the right thing". But maybe it's 
just me...


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extracting python bytecode from a linux core dump?

2017-06-09 Thread Skip Montanaro 
> I have a core file (produced via the gcore command) of a linux python2.6 
> process.  I need to extract the byte code and de-compile it.

Following on Steve's comment, you might want to take a look at
Misc/gdbinit for some GDB command inspiration. You are correct, you
won't have a running process, but I think you should be able source
that file (maybe with tweaks, depending on the Python version you are
debugging), then move up and down the C call stack, poke around in the
C locals, then follow pointers to the currently active functions, then
for those which are Python functions, follow the func_code attribute
to get the code object. I can't remember what the actual bytecode
attribute is called in the code object. (It's been too many years.)

> However, these all seem to require either a running process and/or a binary 
> with debugging symbols.

Yeah, you're going to have *a lot* of fun with a stripped executable.
If you're debugging a core file from an interpreter compiled with much
in the way of compiler optimization, many of the local variables will
have been optimized out. You'll likely be stuck rummaging around until
you figure out the pattern of where the compiler put things
(register-wise).

> I'm thinking that the compiled bytecode is likely in an array or contiguous 
> set of memory within the python executable's image and that there's probably 
> a way to pull it out with gdb.  Unsurprisingly, the pyc 0xd1f20d0a magic 
> number isn't kept in memory. So, how do I find the memory holding the 
> compiled byte-code ?

Correct. The module level bytecode is executed once at import time,
then discarded, at least that used to be how it was done.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Barry Warsaw 
On Jun 09, 2017, at 08:43 PM, Nick Coghlan wrote:

>So honestly, I'd be +1 for either approach:
>
>- stdlib backport to make dual-stack maintenance easier for the
>current volunteers, and we'll see how things work out on the
>ease-of-adoption front

As I've said, I'm okay with this approach as long as we don't expose new
public APIs in 2.7.  That won't prevent other folks from using the private
APIs, but we have no responsibility to support them.

>- PyPI backport to make 2.7 adoption easier, and we'll continue
>pestering redistributors to actually fund maintenance of Python 2.7's
>SSL stack properly (and encourage customers of those redistributors to
>do the same)

Of course, lots of distributions are completely voluntary, so that kind of
pestering falls on underfunded ears. ;)  But a PyPI backport might still be
useful since those third party packages can be distro-fied into current and
backported channels.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Donald Stufft 

> On Jun 9, 2017, at 1:07 PM, Steve Dower  wrote:
> 
> On 09Jun2017 0343, Nick Coghlan wrote:
>> So honestly, I'd be +1 for either approach:
>> - stdlib backport to make dual-stack maintenance easier for the
>> current volunteers, and we'll see how things work out on the
>> ease-of-adoption front
>> - PyPI backport to make 2.7 adoption easier, and we'll continue
>> pestering redistributors to actually fund maintenance of Python 2.7's
>> SSL stack properly (and encourage customers of those redistributors to
>> do the same)
> 
> My draft reply to Donald sat overnight, so I abandoned it in favour of 
> agreeing with Nick.
> 
> I'm in principle in favour of anything that makes 2.7 less of a burden to 
> maintain (up to and including EOL :) ), so if backporting parts of ssl/_ssl 
> makes that easier then I'm +0.
> 
> However, I do prefer the PyPI backport with some tool bundled in order to 
> obtain it. In fact, given the nature of OpenSSL, I'd be in favour of that 
> approach for all versions of Python (at least on Windows it would likely work 
> well - probably less so on other platforms where we couldn't include a 
> prebuilt fallback easily, though those tend to include compilers...).
> 
> That hypothetical "_ensuressl" module in my mind really doesn't have to do 
> much other than determine which file to download and then download and 
> extract it, which can be done with OS level tools rather than needing our own 
> stack. It may also be the necessary mechanism to make ssl pip-updateable, 
> since we have the locking problem that prevents it being possible normally.
> 


A ensuressl style module that tries to install an OpenSSL module is actually 
fairly hard to do securely. The fundamental issue being that fetching a file 
securely from the network before you have the primary tool for fetching things 
securely from a network gets to be a bit of a chicken and egg problem. There 
are *possible* ways around that, but they essentially boil down to having to 
ship something akin to a CA Bundle inside of Python which we just shift the 
responsibility from needing to have an updated OpenSSL in all versions of 
Python.

The good news here is that PEP 543 is the thing that will enable using SChannel 
on Windows, SecureTransport on macOS, and OpenSSL on Linux getting python-dev 
out of the business of shipping an OpenSSL at all [1]. Backporting 
ssl.MemoryBio represents the smallest reasonable change to 2.7 we can make [2] 
that paves the path forward to being able to do that in some number of years, 
because we can’t do that until PEP 543 is a thing and people are using it.


[1] Ok, the ``ssl`` module is still going to be ssl dependent, and there is a 
lot of code out there using it so we would need to keep it for awhile to 
maintain compatibility…. but at some point we can stop bundling it in our 
installers under the guise of “we have a newer, better API for handling this, 
and if you still need 2.7 compatibility, here’s the pip installable libraries 
you need to install to get it”.

[2] Where reasonable means, it’s not sacrificing good things or making things 
harder to use for the sake of being smaller.

—
Donald Stufft



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Steve Dower 

On 09Jun2017 0343, Nick Coghlan wrote:

So honestly, I'd be +1 for either approach:

- stdlib backport to make dual-stack maintenance easier for the
current volunteers, and we'll see how things work out on the
ease-of-adoption front
- PyPI backport to make 2.7 adoption easier, and we'll continue
pestering redistributors to actually fund maintenance of Python 2.7's
SSL stack properly (and encourage customers of those redistributors to
do the same)


My draft reply to Donald sat overnight, so I abandoned it in favour of 
agreeing with Nick.


I'm in principle in favour of anything that makes 2.7 less of a burden 
to maintain (up to and including EOL :) ), so if backporting parts of 
ssl/_ssl makes that easier then I'm +0.


However, I do prefer the PyPI backport with some tool bundled in order 
to obtain it. In fact, given the nature of OpenSSL, I'd be in favour of 
that approach for all versions of Python (at least on Windows it would 
likely work well - probably less so on other platforms where we couldn't 
include a prebuilt fallback easily, though those tend to include 
compilers...).


That hypothetical "_ensuressl" module in my mind really doesn't have to 
do much other than determine which file to download and then download 
and extract it, which can be done with OS level tools rather than 
needing our own stack. It may also be the necessary mechanism to make 
ssl pip-updateable, since we have the locking problem that prevents it 
being possible normally.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove embedded expat library?

2017-06-09 Thread Zachary Ware 
On Fri, Jun 9, 2017 at 11:03 AM, Ned Deily  wrote:
> On Jun 9, 2017, at 08:43, Victor Stinner  wrote:
>> I expect that all Linux distributions build Python using
>> --with-system-expat. It may become the default? What about macOS and
>> other operating systems?
>
> The current default is --with-system-expat=no so, unless builders of Python 
> take explicit action, the bundled version of expat is used.  Using the 
> bundled version is also currently the case for the python.org macOS 
> installer, no idea what other distributors do.  Apple supplies a version of 
> expat with macOS so we presumably we could use the system version for the 
> installer.  Presumably (Zach?) we would need to continue to supply a version 
> of expat for Windows builds.  But do we need to for others?  If it were only 
> Windows, *then* perhaps it might make sense to make all the changes to move 
> expat out of cpython into the common repo for third-party Windows libs.

Yes, we would need to continue providing a version for Windows.  It
would be a relatively small change to move it to the externals
repository.

I would be fine with switching to `--with-system-expat=yes` by default
and building from externals on Windows in 3.7, and removing the
bundled expat in 3.8.

>> By the way, Zachary Ware is working on converting this repository to
>> Git. I don't know his progress:
>> - https://github.com/python/cpython-bin-deps
>> - https://github.com/python/cpython-source-deps

PR 1783 (https://github.com/python/cpython/pull/1783); needs another
review from Steve to make sure I haven't made a complete mess of
things, then it should be ready to go.  Anyone else on Windows (Terry
Reedy?) who can test it and provide feedback, please do!

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Donald Stufft 

> On Jun 9, 2017, at 6:43 AM, Nick Coghlan  wrote:
> 
> OTOH, if the aim is to make the change now, so it gets into Ubuntu
> 18.04, with a view to projects only starting to fully rely on it in
> mid-to-late 2018 or so? That timeline might actually work, and this
> approach has the benefit of actually making the Py2 SSL stack easier
> to maintain between now and 2020.


Cory can correct me if I’m wrong, but yea that’s the general idea here.

We need a tree, and the best time to plant a tree is 20 years ago, but the 
second best time to plant a tree is today ;)

I don’t think anyone is going to immediately start depending on a hypothetical 
2.7.14 with MemoryBio support, but getting it out now means that Cory can start 
working on PEP 543 as part of a longer term effort for requests as well as 
starting to work on async-ifying requests. Neither of those efforts are going 
to be quick or simple tasks, so the work to make them possible can happen while 
we wait for adoption to naturally grow on 2.7.14.

—
Donald Stufft



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2017-06-09 Thread Python tracker 

ACTIVITY SUMMARY (2017-06-02 - 2017-06-09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6035 (+10)
  closed 36335 (+51)
  total  42370 (+61)

Open issues with patches: 2369 


Issues opened (40)
==

#30555: _io._WindowsConsoleIO breaks in the face of fd redirection
http://bugs.python.org/issue30555  opened by Segev Finer

#30556: asyncio.wait very slow with FIRST_COMPLETED
http://bugs.python.org/issue30556  opened by samuelcolvin

#30560: Py_SetFatalErrorAbortFunc: Allow embedding program to handle f
http://bugs.python.org/issue30560  opened by thomas.perl

#30561: sync-up gammavariate and expovariate code
http://bugs.python.org/issue30561  opened by leodema

#30563: multiprocessing module with pool object issue
http://bugs.python.org/issue30563  opened by Julien Verger

#30565: PEP 538: default to skipping warning for implicit locale coerc
http://bugs.python.org/issue30565  opened by ncoghlan

#30566: punycode codec raises IndexError in decode_generalized_number(
http://bugs.python.org/issue30566  opened by Vikram Hegde

#30569: Tutorial section 2.1 has *nix example at 3.7, but Windows at 3
http://bugs.python.org/issue30569  opened by csabella

#30570: issubclass segfaults on objects with weird __getattr__
http://bugs.python.org/issue30570  opened by Daniel Lepage

#30571: Add integer formatting code for fixed-width signed arithmetic 
http://bugs.python.org/issue30571  opened by rhettinger

#30576: http.server should support HTTP compression (gzip)
http://bugs.python.org/issue30576  opened by quentel

#30578: Misleading example in sys.set_coroutine_wrapper docs
http://bugs.python.org/issue30578  opened by ncoghlan

#30579: Allow traceback objects to be instantiated/mutated/annotated
http://bugs.python.org/issue30579  opened by njs

#30581: os.cpu_count() returns wrong number of processors on system wi
http://bugs.python.org/issue30581  opened by robbuckley

#30585: [security][3.3] Backport smtplib fix for TLS stripping vulnera
http://bugs.python.org/issue30585  opened by haypo

#30586: Encode to EBCDIC doesn't take into account conversion table ir
http://bugs.python.org/issue30586  opened by Vladimir Filippov

#30587: Mock with spec object does not ensure method call signatures
http://bugs.python.org/issue30587  opened by cbelu

#30588: Missing documentation for codecs.escape_decode
http://bugs.python.org/issue30588  opened by mdartiailh

#30589: With forkserver, Process.exitcode does not get signal number
http://bugs.python.org/issue30589  opened by pitrou

#30593: sqlite3 executescript does not respect isolation_level?
http://bugs.python.org/issue30593  opened by Noah Levitt

#30594: Refcounting mistake in _ssl.c
http://bugs.python.org/issue30594  opened by njs

#30595: test_queue_feeder_donot_stop_onexc() of test_multiprocessing_s
http://bugs.python.org/issue30595  opened by haypo

#30596: Add close() to multiprocessing.Process
http://bugs.python.org/issue30596  opened by pitrou

#30597: Show expected input in custom "print" error message
http://bugs.python.org/issue30597  opened by ncoghlan

#30598: Py_NewInterpreter() leaks a reference on warnoptions in _PySys
http://bugs.python.org/issue30598  opened by haypo

#30599: os.register_at_fork(): allow to unregister callbacks -- test_t
http://bugs.python.org/issue30599  opened by haypo

#30600: Error message incorrect when index is called with keyword argu
http://bugs.python.org/issue30600  opened by SylvainDe

#30602: [Windows] os.spawn*() tests of test_os leak references on Wind
http://bugs.python.org/issue30602  opened by haypo

#30603: textwrap: declining indent level has no test case
http://bugs.python.org/issue30603  opened by jonathaneunice

#30604: co_extra_freefuncs is stored thread locally and can lead to cr
http://bugs.python.org/issue30604  opened by dino.viehland

#30605: re.compile fails when compiling bytes under `-bb` mode
http://bugs.python.org/issue30605  opened by Roy Williams

#30607: Extract documentation theme into a separate package
http://bugs.python.org/issue30607  opened by Jon Wayne Parrott

#30608: argparse calculates string widths incorrectly
http://bugs.python.org/issue30608  opened by Vanessa McHale

#30609: Python 3.6.1 fails to generate 256 colors on Cygwin based 64-b
http://bugs.python.org/issue30609  opened by rigordo

#30610: Python's libexpat vulnerable to CVE-2016-0718
http://bugs.python.org/issue30610  opened by Duy Phan Thanh

#30611: Windows HTML Help always opens maximized
http://bugs.python.org/issue30611  opened by Christian.Ullrich

#30612: Unusual Windows registry path syntax
http://bugs.python.org/issue30612  opened by Christian.Ullrich

#30613: Distutils register command creates non-standard multipart data
http://bugs.python.org/issue30613  opened by Kaeptm Blaubaer

#30614: [EASY][2.7] testInitNonExistentFile() of test_bz2 leaks refere

Re: [Python-Dev] Remove embedded expat library?

2017-06-09 Thread Ned Deily 
On Jun 9, 2017, at 08:43, Victor Stinner  wrote:
> Python embeds a copy of the expat library which already got two major
> security vulnerabilities:
> 
> "CVE-2016-0718: expat bug #537"
> http://python-security.readthedocs.io/vuln/cve-2016-0718_expat_bug_537.html
> 
> "Issue #26556: Expat 2.1.1"
> http://python-security.readthedocs.io/vuln/issue_26556_expat_2.1.1.html
> 
> Would it be possible to maintain this dependency on an external
> repository which would be easier to maintain? Like
> http://svn.python.org/projects/external/ used to build Python on
> Windows.

Considering how infrequently there have been releases of upstream expat over 
the past decade or so, even if we had caught every release, I don't see that 
the benefit to maintaining our embedded copy as a separate repo rather than 
have to apply the upstreams copies a few time (to each Python branch) is worth 
the disruptive effort in changing the current Python build process (assuming we 
don't change the default, see below).  Now, as to whether we should continue to 
provide our own copy of expat, that's a different question.  Are there 
differences between our bundled expat and upstream?

> I expect that all Linux distributions build Python using
> --with-system-expat. It may become the default? What about macOS and
> other operating systems?

The current default is --with-system-expat=no so, unless builders of Python 
take explicit action, the bundled version of expat is used.  Using the bundled 
version is also currently the case for the python.org macOS installer, no idea 
what other distributors do.  Apple supplies a version of expat with macOS so we 
presumably we could use the system version for the installer.  Presumably 
(Zach?) we would need to continue to supply a version of expat for Windows 
builds.  But do we need to for others?  If it were only Windows, *then* perhaps 
it might make sense to make all the changes to move expat out of cpython into 
the common repo for third-party Windows libs.

> By the way, Zachary Ware is working on converting this repository to
> Git. I don't know his progress:
> - https://github.com/python/cpython-bin-deps
> - https://github.com/python/cpython-source-deps

It appears that the project itself is moving from Sourceforge to Github:

https://github.com/libexpat/libexpat

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2017 Python Language Summit coverage finale

2017-06-09 Thread Guido van Rossum 
Also a good summary for those of us that *did* attend! Thanks Jake from all
participants.

(And Nick, I hope you won't make a habit of skipping the summit. :-)

On Fri, Jun 9, 2017 at 3:10 AM, Nick Coghlan  wrote:

> On 9 June 2017 at 01:52, Jake Edge  wrote:
> >
> > Hola python-dev,
> >
> > I (finally!) finished up the coverage from the language summit ...
> >
> > The starting point is the overview article, here:
> > https://lwn.net/Articles/723251/ which should now be free for anyone to
> > see (and the first seven articles too).
>
> Thanks once again for these - very helpful for those of us that
> weren't able to attend!
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Remove embedded expat library?

2017-06-09 Thread Victor Stinner 
Hi,

Python embeds a copy of the expat library which already got two major
security vulnerabilities:

"CVE-2016-0718: expat bug #537"
http://python-security.readthedocs.io/vuln/cve-2016-0718_expat_bug_537.html

"Issue #26556: Expat 2.1.1"
http://python-security.readthedocs.io/vuln/issue_26556_expat_2.1.1.html

Would it be possible to maintain this dependency on an external
repository which would be easier to maintain? Like
http://svn.python.org/projects/external/ used to build Python on
Windows.

I expect that all Linux distributions build Python using
--with-system-expat. It may become the default? What about macOS and
other operating systems?

By the way, Zachary Ware is working on converting this repository to
Git. I don't know his progress:
- https://github.com/python/cpython-bin-deps
- https://github.com/python/cpython-source-deps

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [python-committers] IMPORTANT: Python 3.6.2 Maintenance Release Release Candidate in 3+ days (Monday 2017-06-12 12:00 UTC)

2017-06-09 Thread Victor Stinner 
Oh, about very annoying 3.6 bug, there was a regression caused by
FASTCALL optimizations. It's now fixed in the 3.6 branch:
https://github.com/python/cpython/commit/f0ff849adc6b4a01f9d1f08d9ad0f1511ff84541

Victor



2017-06-09 5:34 GMT+02:00 Ned Deily :
> We are approaching the end of the second calendar quarter of 2017 and, 
> according to PEP 494, it's time to start producing the second maintenance 
> release for the 3.6 series.  The schedule calls for the release candidate to 
> be produced on Monday 2017-06-12 UTC.  As was the case with previous 3.6.x 
> releases, the plan is for the release candidate to be the same as the final 
> release, that is, no additional changes go in after the release candidate 
> except for any showstopper critical problems that might be discovered with 
> rc1.  So please plan to get any security fixes, bug fixes, and documentation 
> changes you think should be in 3.6.2 merged in ASAP.  The 3.6.2 final is 
> planned for two weeks following rc1, that is, on 2017-06-26.  The next 3.6 
> maintenance release (3.6.3) is planned to follow about 3 months later, so 
> most likely in 2017-09.
>
> A reminder that the 3.6 branch will remain open for checkins throughout the 
> 3.6.2rc and final cycle.  Once 3.6.2rc1 is tagged, new commits to the 3.6 
> branch will release with 3.6.3.  As always, if you find any problem in 
> 3.6.2rc1 that you may believe should be corrected in 3.6.2, please ensure the 
> problem is documented in a new or existing open issue on bugs.python.org, 
> ensure that the Priority field of the issue is set to "release blocker", and 
> that "Python 3.6" is included in the selected Versions.  Comments or tags on 
> github Pull Requests are NOT sufficient!
>
> Thanks again for all of your efforts in bringing 3.6 into the world and for 
> helping now to make it even better!
>
> https://www.python.org/dev/peps/pep-0494/
> http://cpython-devguide.readthedocs.io
>
> --
>   Ned Deily
>   n...@python.org -- []
>
> ___
> python-committers mailing list
> python-committ...@python.org
> https://mail.python.org/mailman/listinfo/python-committers
> Code of Conduct: https://www.python.org/psf/codeofconduct/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-09 Thread Nick Coghlan 
On 9 June 2017 at 05:51, Donald Stufft  wrote:
> And the downside here is pretty small honestly?

Unfortunately, one of the downsides is "Doesn't provide any clearly
compelling benefits to users of LTS distributions, so even those of us
in a position to advocate for such backports probably won't win the
related arguments".

So the question is whether or not we're OK with saying that users
affected by that will need to switch to a different set of Python
binaries to get the latest pip & requests (e.g. upgrading their base
distro, or else adopting Red Hat SCLs or one of the cross-platform,
and hence cross-distro, distributions).

> It’s not changing behavior of anything that currently exists since it’s
> adding a new thing inside the ssl module and Alex has already written the
> patch so there’s little extra work to do and it actually makes maintenance
> easier since patches can more readily be applied straight from `master`.
>
> The primary argument I can see against it, is one of purity, that 2.7
> shouldn’t get new features but as we know, practicality beats purity ;) (and
> we’ve already accepted that TLS is a special case, special enough to break
> the rules, so the main question is whether this specific thing is
> worthwhile— which given it’s benefits to the Python ecosystem and to
> maintaining Python, I think it is).

It isn't a purity argument so much as a "Will the change reach a
sufficiently large proportion of 2.7 users to actually solve the
compatibility problem it is aiming to solve?" argument (There's a
*bit* of a purity argument behind that, in that the PEP 466 backport
*did* break customer applications due to the incompatibilities with
some of the async frameworks that were using underscore prefixed APIs
that changed behaviour, so our credibility for "trust us, it won't
break anything" is low in this context, but it's not the primary
objection)

I'm 100% confident that we'd reach a large enough audience with a
compatibility shim that gets installed on disk as something other than
"ssl" (e.g. "backport.ssl" or "_ssl_bootstrap"), and that has the
virtue of enabling a multi-tiered distribution approach:

- backport.ssl on PyPI as an independently installed module
- _ssl_bootstrap as a runtime or redistributor provided module

This approach also allows the API to be updated as necessary to meet
the needs of PEP 543, without needing to go through the full PEP
process again. The downside is adding a 3rd stack to maintain (Py3,
Py2-compatible Py3 backport, Py2) without making maintenance on the
Py2 stack any easier.

I'm markedly less confident of reaching a sufficiently large audience
with a "So stop using the system Python, then" approach (Don't get me
wrong, I'd love for people to stop using the system Python for things
that aren't part of the operating system, but people also *really
like* the convenience of doing that).

OTOH, if the aim is to make the change now, so it gets into Ubuntu
18.04, with a view to projects only starting to fully rely on it in
mid-to-late 2018 or so? That timeline might actually work, and this
approach has the benefit of actually making the Py2 SSL stack easier
to maintain between now and 2020.

So honestly, I'd be +1 for either approach:

- stdlib backport to make dual-stack maintenance easier for the
current volunteers, and we'll see how things work out on the
ease-of-adoption front
- PyPI backport to make 2.7 adoption easier, and we'll continue
pestering redistributors to actually fund maintenance of Python 2.7's
SSL stack properly (and encourage customers of those redistributors to
do the same)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2017 Python Language Summit coverage finale

2017-06-09 Thread Nick Coghlan 
On 9 June 2017 at 01:52, Jake Edge  wrote:
>
> Hola python-dev,
>
> I (finally!) finished up the coverage from the language summit ...
>
> The starting point is the overview article, here:
> https://lwn.net/Articles/723251/ which should now be free for anyone to
> see (and the first seven articles too).

Thanks once again for these - very helpful for those of us that
weren't able to attend!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com