[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 99e69d44f499625786a2e6461a954adcd0037d69 by Victor Stinner in 
branch 'master':
bpo-36710: Fix compiler warning on PyThreadState_Delete() (GH-12962)
https://github.com/python/cpython/commit/99e69d44f499625786a2e6461a954adcd0037d69


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36724] Clear _PyRuntime at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 87d23a041d9efb743c5680ac23305f300e51 by Victor Stinner in 
branch 'master':
bpo-36724: Add _PyWarnings_Fini() (#12963)
https://github.com/python/cpython/commit/87d23a041d9efb743c5680ac23305f300e51


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36724] Clear _PyRuntime at exit

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12889
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36640] distutils.sysconfig.get_python_lib behaves incorrectly when pip cache is enabled

[Saba Kauser]

Saba Kauser  added the comment:

Experts, please help.
get_python_lib() is behaving strangely and I have detailed the problem in my 
last update.

--
title: python ibm_db setup.py post install script does not seem to work from 
Anaconda -> distutils.sysconfig.get_python_lib behaves incorrectly when pip 
cache is enabled

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12888

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36640] python ibm_db setup.py post install script does not seem to work from Anaconda

[Saba Kauser]

Change by Saba Kauser :


--
components: +Distutils -Build
nosy: +dstufft, eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36731] Add example to priority queue

[Windson Yang]

New submission from Windson Yang :

We don't have the base example for priority queue in 
https://docs.python.org/3.8/library/heapq.html#priority-queue-implementation-notes,
 We can add something like:

> q = Q.PriorityQueue()
> q.put(10)
> q.put(1)
> q.put(5)
> while not q.empty():
print q.get()

We may also need to add Notes about the PriorityQueue will block when we use 
max size 

> q = Q.PriorityQueue(1)
> q.put(10)
> q.put(1) # will block until the Queue is available again.

--
assignee: docs@python
components: Documentation
messages: 340878
nosy: Windson Yang, docs@python
priority: normal
severity: normal
status: open
title: Add example to priority queue
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7951] Should str.format allow negative indexes when used for __getitem__ access?

[Éric Araujo]

Éric Araujo  added the comment:

A side question: where is it defined that in `{thing[0]}`, 0 will be parsed as 
an integer?

The PEP shows `{thing[name]}` and mentions that this is not Python but a 
smaller mini-language, with `name` always a string, no quotes needed or 
permitted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36725] Reference leak regression with Python3.8a3

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12887

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36725] Reference leak regression with Python3.8a3

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4d29983185bc12ca685a1eb3873bacb8a7b67416 by Victor Stinner in 
branch 'master':
bpo-36725: regrtest: add TestResult type (GH-12960)
https://github.com/python/cpython/commit/4d29983185bc12ca685a1eb3873bacb8a7b67416


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36730] Change outdated references to macOS

[Sebastian Bassi]

New submission from Sebastian Bassi :

There are multiple occurences in the web page of "Mac OS X", like "Download the 
latest version for Mac OS X". This OS is called macOS since some years. It may 
be confusing for a new user.

--
assignee: docs@python
components: Documentation
messages: 340875
nosy: Sebastian Bassi, docs@python
priority: normal
severity: normal
status: open
title: Change outdated references to macOS
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36725] Reference leak regression with Python3.8a3

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12886
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36722] In debug build, load also C extensions compiled in release mode or compiled using the stable ABI

[STINNER Victor]

STINNER Victor  added the comment:

Oh, there is an issue: --with-trace-refs has a different ABI and so should not 
do that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36725] Reference leak regression with Python3.8a3

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36729] Delete unused text variable on tests

[Emmanuel Arias]

Change by Emmanuel Arias :


--
keywords: +patch
pull_requests: +12885
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9db0324712f6982d89620b420f507a6aa5da152f by Victor Stinner in 
branch 'master':
bpo-36710: Add runtime parameter in gcmodule.c (GH-12958)
https://github.com/python/cpython/commit/9db0324712f6982d89620b420f507a6aa5da152f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36729] Delete unused text variable on tests

[Emmanuel Arias]

New submission from Emmanuel Arias :

On ```test_custom_non_data_descriptor``` and
```test_custom_data_descriptor``` from Lib/test/test_pydoc.py there was a text 
variable not used.

--
components: Tests
messages: 340872
nosy: eamanu
priority: normal
severity: normal
status: open
title: Delete unused text variable on tests
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 10c8e6af910e3a26e59f913a3c1e4830ca71b1af by Victor Stinner in 
branch 'master':
bpo-36710: Add runtime variable in pystate.c (GH-12956)
https://github.com/python/cpython/commit/10c8e6af910e3a26e59f913a3c1e4830ca71b1af


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36424] Pickle fails on frozen dataclass that has slots

[Alessandro Cucci]

Change by Alessandro Cucci :


--
nosy: +acucci

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12884

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21536] extension built with a shared python cannot be loaded with a static python

[STINNER Victor]

STINNER Victor  added the comment:

Since this change is causing subtle issues like bpo-36721 "Add pkg-config 
python-3.8-embed", I don't think that it would be a good idea to backport this 
change to Python 2.7 or 3.7. I close the issue.

See also bpo-36722 "In debug build, load also C extensions compiled in release 
mode or compiled using the stable ABI".

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36722] In debug build, load also C extensions compiled in release mode or compiled using the stable ABI

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12883

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36722] In debug build, load also C extensions compiled in release mode or compiled using the stable ABI

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5422e3cfb7ffc50b147b4662d6f596cd61533754 by Victor Stinner in 
branch 'master':
bpo-36722: Debug build loads libraries built in release mode (GH-12952)
https://github.com/python/cpython/commit/5422e3cfb7ffc50b147b4662d6f596cd61533754


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12882

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36728] Remove PyEval_ReInitThreads() from the public C API

[STINNER Victor]

New submission from STINNER Victor :

PyEval_ReInitThreads() is used internally by PyOS_AfterFork_Child(). I don't 
see the point of calling it directly. If you care of threads after fork, just 
call PyOS_AfterFork_Child(), no?

I propose to remove PyEval_ReInitThreads() from the public C API.

Problem: it's documented as a public function in High-level API of the Python 
Initialization API:
https://docs.python.org/dev/c-api/init.html#c.PyEval_ReInitThreads

On the Internet, I found a single project calling directly this function, but 
it's only in a test used to the test Python threading API:
https://github.com/DataDog/go-python3/blob/master/thread_test.go


func TestThreadInitialization(t *testing.T) {
Py_Initialize()
PyEval_InitThreads()

assert.True(t, PyEval_ThreadsInitialized())

PyEval_ReInitThreads()
}

I don't think that the PyEval_ReInitThreads() call here makes any sense.

--
components: Interpreter Core
messages: 340868
nosy: vstinner
priority: normal
severity: normal
status: open
title: Remove PyEval_ReInitThreads() from the public C API
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36718] Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: relocation R_X86_64_PC32 against symbol ... can not be used ...

[STINNER Victor]

STINNER Victor  added the comment:

https://buildbot.python.org/all/#/builders/139/builds/252 is back to green. I 
close the issue. Thanks David!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36727] python 3.6+ docs use ul tags instead of ol tags

[Oliver Too, Eh?]

New submission from Oliver Too, Eh? :

Documentation contents are ordered, and readers familiar with the section 
numbers/ordering from the Python 2 or Python 3.5 documentation will have an 
easier time transitioning to Python 3.6 or later if the sections remain 
numbered.

--
assignee: docs@python
components: Documentation
messages: 340866
nosy: Oliver Too, Eh?, docs@python
priority: normal
severity: normal
status: open
title: python 3.6+ docs use ul tags instead of ol tags
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17349] wsgiref.simple_server.demo_app is not PEP-3333 compatible

[Berker Peksag]

Berker Peksag  added the comment:

Closing this as 'not a bug' since examples in both branches are correct. Thank 
you for the report!

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18099] wsgiref sets Content-Length: 0 on 304 Not Modified

[Berker Peksag]

Change by Berker Peksag :


--
pull_requests: +12881

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36726] Empty select() on windows gives error.

[Martin Panter]

Martin Panter  added the comment:

I think this is a duplicate of Issue 29256. Issue 25680 also discusses platform 
differences with no file descriptors.

--
nosy: +martin.panter
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Windows select() errors out when given no fds to select on, 
which breaks SelectSelector

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36718] Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: relocation R_X86_64_PC32 against symbol ... can not be used ...

[David Bolen]

David Bolen  added the comment:

Ok, I've resolved this, and the linker errors did actually point at the root 
issue.  They show trying to link extensions against 
/usr/local/lib/libpython2.7.a which was my test static build of 2.7.16.

Arguably this seems an issue in the buildbot build process in that it doesn't 
sandbox extension building to force linking to the python library being built 
rather than a random system library, but I can't see spending any time on that. 
 I've purged the 2.7.16 files from the /usr/local tree and things seem to be 
back to normal

I've started a new run for the failing build.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36726] Empty select() on windows gives error.

[Ralph Loader]

New submission from Ralph Loader :

The following one liner gives an error on windows but not on linux:

```
import selectors ; selectors.DefaultSelector().select(timeout=1)
```

It appears that the windows select() function with no FDs set gives an error 
but on Linux it is equivalent to a sleep.

The error on windows:
```
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Program Files\Python37\lib\selectors.py", line 323, in select
r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Program Files\Python37\lib\selectors.py", line 314, in _select
r, w, x = select.select(r, w, w, timeout)
OSError: [WinError 10022] An invalid argument was supplied
```

--
components: Windows
messages: 340862
nosy: Ralph Loader, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Empty select() on windows gives error.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36461] timeit: Additional changes for autorange

[Alessandro Cucci]

Change by Alessandro Cucci :


--
pull_requests: +12880

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36725] Reference leak regression with Python3.8a3

[Kay Hayen]

New submission from Kay Hayen :

Much like #9366 the same file can be used. This reference leaks according to 
Nuitka comparative testing:

simpleFunction59: FAILED 129511 129512 leaked 1

def simpleFunction59():
a = 3
b = 5

try:
a = a * 2

return a
finally:
return a / b


I would be guessing, that you are leaking the return value when returning again.

--
messages: 340861
nosy: kayhayen
priority: normal
severity: normal
status: open
title: Reference leak regression with Python3.8a3
type: resource usage
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-36724: Clear _PyRuntime at exit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36724] Clear _PyRuntime at exit

[STINNER Victor]

New submission from STINNER Victor :

_PyRuntime.warnings is not cleared at Python exit: 3 objects are kept alive 
even after Py_Finalize().

See bpo-36356 which cleared some other variables.

PR 12453 "bpo-36356: Destroy the GIL at exit" is still open.

--
components: Interpreter Core
messages: 340859
nosy: eric.snow, vstinner
priority: normal
severity: normal
status: open
title: Clear _PyRuntime at exit
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36718] Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: relocation R_X86_64_PC32 against symbol ... can not be used ...

[David Bolen]

David Bolen  added the comment:

Yes, it appears most likely to be the worker environment.  I did upgrade the 
kernel in between builds 250 (good) and 251 (bad), but reverting that still 
fails, even with the same commit as in build 250.

My current suspicion is that a test I did recently with a static 2.7.16 build 
is influencing the buildbot build process (which might explain why the error 
seems to involve an -fPIC mismatch in a library).  Not sure how yet though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36461] timeit: Additional changes for autorange

[Michele Angrisano]

Change by Michele Angrisano :


--
keywords: +patch
pull_requests: +12879
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36722] In debug build, load also C extensions compiled in release mode or compiled using the stable ABI

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12878
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36723] Unittest Discovery for namespace subpackages dot notation fails

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36723] Unittest Discovery for namespace subpackages dot notation fails

[Jonny Fuller]

New submission from Jonny Fuller :

Hi friends,

I noticed strange behavior involving unittest discovery with namespace 
packages. Using dot notation to discover test packages within a namespace 
package will fail, but will succeed when using path notation.

It feels awkward that dot path would fail but normal path would succeed. Is 
this the desired behavior?

I created a demo repo and fully documented this odd behavior here

https://github.com/JonnyWaffles/djangonamespacetestfail.

--
messages: 340857
nosy: mrwaffles
priority: normal
severity: normal
status: open
title: Unittest Discovery for namespace subpackages dot notation fails
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36722] In debug build, load also C extensions compiled in release mode or compiled using the stable ABI

[STINNER Victor]

New submission from STINNER Victor :

bpo-36465 modified the ABI of debug build so release and debug build now have 
the same ABI.

bpo-21536 modified how C extensions are built: they are no longer linked to 
libpython.

In a debug build, it becomes possible to load a C extension built in release 
mode:
https://bugs.python.org/issue21536#msg340821

But I had to modify SOABI for that.

I propose to modify how Python looks for C extensions: look also for dynamic 
libraries without the "d" SOABI flag and for C extensions built using the 
stable ABI.

Release build:

$ ./python -c 'import _imp; print(_imp.extension_suffixes())'
['.cpython-38-x86_64-linux-gnu.so', '.abi3.so', '.so']

Debug build, *WITHOUT* my change:

$ ./python -c 'import _imp; print(_imp.extension_suffixes())'
['.cpython-38d-x86_64-linux-gnu.so', '.so']

Debug build, *WITH* my change:

$ ./python -c 'import _imp; print(_imp.extension_suffixes())'
['.cpython-38d-x86_64-linux-gnu.so', '.cpython-38-x86_64-linux-gnu.so', 
'.abi3.so', '.so']

--
components: Build
messages: 340856
nosy: vstinner
priority: normal
severity: normal
status: open
title: In debug build, load also C extensions compiled in release mode or 
compiled using the stable ABI
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35920] Windows 10 ARM32 platform support

[Steve Dower]

Steve Dower  added the comment:


New changeset 62dfd7d6fe11bfa0cd1d7376382c8e7b1275e38c by Steve Dower (Paul 
Monson) in branch 'master':
bpo-35920: Windows 10 ARM32 platform support (GH-11774)
https://github.com/python/cpython/commit/62dfd7d6fe11bfa0cd1d7376382c8e7b1275e38c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36721] Add pkg-config python-3.8-embed

[STINNER Victor]

New submission from STINNER Victor :

The bpo-21536 modified how C extensions are built: they are no longer linked to 
libpython. The problem is that when an application embeds Python: the 
application wants to be linked to libpython.

commit 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b (HEAD -> master, 
upstream/master)
Author: Victor Stinner 
Date:   Thu Apr 25 20:13:10 2019 +0200

bpo-21536: C extensions are no longer linked to libpython (GH-12946)

On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.

I chose to modify distutils, python-config (shell) and python-config.py 
(Python), but *not* Misc/python.pc (pkg-config).

Previously, we had:

$ pkg-config python-3.7 --libs
-lpython3.7m 
$ python3.7-config --libs
-lpython3.7m -lcrypt -lpthread -ldl -lutil -lm 
$ python3.7-config.py --libs
-lpython3.7m -lcrypt -lpthread -ldl -lutil -lm

Now, we get:

$ pkg-config python-3.8 --libs
-lpython3.8
$ python3.8-config --libs
-lcrypt -lpthread -ldl -lutil -lm -lm 
$ python-config.py --libs
-lcrypt -lpthread -ldl -lutil -lm -lm

python-config and python-config.py can now be used to build C extensions, but 
not to build an application embedding Python.

pkg-config should not be used to build a C extenstion since it links to 
libpython, but we don't want to do that (see bpo-21536).

I'm not sure that different tools should return different results.

I propose:

* Add a new command "pkg-config python-3.8-embed"
* Add a new "--embed" option to python3.8-config and python3.8-config.py 
commands
* Remove "-lpython@VERSION@@ABIFLAGS@" from "Libs: -L${libdir} 
-lpython@VERSION@@ABIFLAGS@" of Misc/python.pc.in

On Windows, we already provide different binaries for embedded Python with 
"-embed" suffix:

* Download Windows x86-64 embeddable zip file: python-3.7.3-embed-amd64.zip
* Download Windows x86-64 executable installer: python-3.7.3-amd64.exe

https://www.python.org/downloads/windows/

--
components: Build
messages: 340853
nosy: doko, vstinner
priority: normal
severity: normal
status: open
title: Add pkg-config python-3.8-embed
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21536] extension built with a shared python cannot be loaded with a static python

[STINNER Victor]

STINNER Victor  added the comment:

> If an application is linked to libpython to embed Python, the application 
> build requires flags to link to libpython. (...)

I created bpo-36721 "Add pkg-config python-3.8-embed" to discuss this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21536] extension built with a shared python cannot be loaded with a static python

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b by Victor Stinner in 
branch 'master':
bpo-21536: C extensions are no longer linked to libpython (GH-12946)
https://github.com/python/cpython/commit/8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28533] Replace asyncore

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36720] Correct Should to Must in Definition of object.__len__

[Brett Cannon]

Brett Cannon  added the comment:

There is probably validation code in len(), but the method itself can do 
whatever it wants if you call it directly. So the documentation is accurate in 
saying the method _should_ return >= 0 but that there's nothing strictly 
enforcing that in Python for that specific method when defined and called 
directly.

--
nosy: +brett.cannon
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36718] Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: relocation R_X86_64_PC32 against symbol ... can not be used ...

[STINNER Victor]

STINNER Victor  added the comment:

@David Bolen: Would you mind to have a look at this issue on your buildbot?

I'm not sure that the regression is caused by my commit, since the command 
lines are the same in the previous build 250:

building 'time' extension
gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -g -O0 -Wall -Wstrict-prototypes 
-I. -IInclude -I./Include -I/usr/include/x86_64-linux-gnu -I/usr/local/include 
-I/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Include 
-I/srv/buildbot/buildarea/2.7.bolen-ubuntu/build -c 
/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.c -o 
build/temp.linux-x86_64-2.7-pydebug/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.o
gcc -pthread -shared 
build/temp.linux-x86_64-2.7-pydebug/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.o
 -L/usr/lib/x86_64-linux-gnu -L/usr/local/lib -L. -lm -lpython2.7 -o 
build/lib.linux-x86_64-2.7-pydebug/time.so

--
nosy: +db3l, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36424] Pickle fails on frozen dataclass that has slots

[Josh Rosenberg]

Change by Josh Rosenberg :


--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33150] Signature error for methods of class configparser.Interpolation

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33348] lib2to3 doesn't parse f(*[] or [])

[Tim Hatch]

Tim Hatch  added the comment:

Also see discussion about divergence on issue36541

--
nosy: +thatch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36541] Make lib2to3 grammar more closely match Python

[Tim Hatch]

Tim Hatch  added the comment:

My strong preference would be getting the lib2to3 grammar to be the python 
grammar + additions, to make future changes easier to merge.  The strongest 
argument against doing that is the backwards-incompatibility of patterns -- 
some won't compile, while others will compile but do something unexpected).

It's good to hear (or at least infer) that parsing modern code is also a goal 
of lib2to3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36716] Embedded Python fails to import module files with version_platform extensions

[Eric Cosatto]

Eric Cosatto  added the comment:

Hi,

I have a full install of Python from the installer in:
C:\Program Files\Python37
Then, I added libraries using pip.
Everything works fine from the command line Python.

Then, I build my C++ application with VisualStudio and link to the Python DLLs. 
I start Python inside my app using the usual code:

  Py_SetPath(...);
  Py_SetProgramName(...);
  Py_InitializeEx(0);

  PyRun_SimpleString("import sys");
  PyRun_SimpleString("print(sys.path)");
  PyRun_SimpleString("print(sys.prefix)");
  PyRun_SimpleString("print(sys.executable)");


>> ['C:\\Program Files\\Python37\\DLLs', 'C:\\Program Files\\Python37\\Lib', 
>> 'C:\\Program Files\\Python37', 'C:\\Program 
>> Files\\Python37\\Lib\\site-packages']
>> C:\Program Files\Python37
>> C:\Program Files\Python37\python.exe

So, everything works fine at this point. If I run the following:

  PyRun_SimpleString("import numpy");


>> ImportError("No module named 'numpy.core._multiarray_umath'")

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36469] Stuck during interpreter exit, attempting to take the GIL

[Remy Noel]

Remy Noel  added the comment:

closing the ticket for now.

--
resolution:  -> later
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36719] regrtest --findleaks should fail if an uncollectable object is found

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12877
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36680] duplicate method definition in Lib/test/test_importlib/test_util.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36681] duplicate method definition in Lib/test/test_logging.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[STINNER Victor]

STINNER Victor  added the comment:

> I don't really understand the rationale for these changes. What's wrong with 
> the global variable _PyRuntime?
> What's the long-term goal for _PyRuntime? If you can't get rid of all 
> occurrences of _PyRuntime, how does it help to get rid of *some* occurences?

The long term goal is to support multiple interpreter instances per process:

Eric Snow's PEP 554 "Multiple Interpreters in the Stdlib"
https://www.python.org/dev/peps/pep-0554/

Right now, there is a single instance of _PyRuntimeState: _PyRuntime. I would 
be interested to have one _PyRuntimeState per interpreter.

Maybe _PyRuntimeState should be reworked in the meanwhile. Right now, it's a 
strange beast: it contains things which are set before Python initialization 
and things which are set after. It contains C types and Python objects. Maybe 
some parts should be moved into PyInterpreterState or even PyThreadState. I 
don't know at this point. It takes time to look at each individual structure 
field...

Anyway, more generally, IMHO it's a bad practice to rely on a global variable. 
Python runtime should be "stateless".

The current implementation of CPython leaks dozens of *Python* objects at exit. 
For example, I started to work on this issue while working on 
https://bugzilla.redhat.com/show_bug.cgi?id=1696322 : Python doesn't clear 2 
warnings variables at exit. When I looked into this issue, I also noticed that 
_PyRuntime.gc.garbage remains *alive* after Py_Finalize().

That's plain wrong: *all* Python objects must be cleared by Py_Finalize(). Two 
interpreters must *not* share any Python object.

Well, the PEP should better explain the rationale than me :-)

--

When I wrote PR 12934, I noticed that even getting the current thread state 
rely on _PyRuntime:

#define _PyThreadState_GET() \

((PyThreadState*)_Py_atomic_load_relaxed(&_PyRuntime.gilstate.tstate_current))

That's wrong in the case of ceval.c: it should be possible to run 
_PyEval_EvalFrameDefault() twice at the same time in two threads using two 
"isolated" interpreters.

Well, PR 12934 doesn't fix all issues. It's just one small step towards 
"stateless" runtime and the even more long term of having one GIL per 
*interpeter*, rather than a single GIL per *process*.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16079] list duplicate test names with patchcheck

[Xavier de Gaye]

Change by Xavier de Gaye :


--
pull_requests: +12876

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19119] duplicate test name in Lib/test/test_heapq.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36683] duplicate method definition in Lib/test/test_utf8_mode.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36678] duplicate method definitions in Lib/test/test_dataclasses.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19113] duplicate test names in Lib/ctypes/test/test_functions.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36679] duplicate method definition in Lib/test/test_genericclass.py

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36720] Correct Should to Must in Definition of object.__len__

[Michal Kononenko]

New submission from Michal Kononenko :

The link below defines __len__

https://docs.python.org/3/reference/datamodel.html?highlight=__len__#object.__len__

However, I was reading in the StackOverflow thread below that CPython does some 
validation to check that the return value of __len__ should be >= 0. Does this 
mean that len must return a value >= 0, in the RFC 2119 sense of the word?

https://stackoverflow.com/questions/42521449/how-does-python-ensure-the-return-value-of-len-is-an-integer-when-len-is-cal

--
assignee: docs@python
components: Documentation
messages: 340844
nosy: Michal Kononenko, docs@python
priority: normal
severity: normal
status: open
title: Correct Should to Must in Definition of object.__len__
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36719] regrtest --findleaks should fail if an uncollectable object is found

[STINNER Victor]

New submission from STINNER Victor :

regrtest (python3 -m test) has a --findleaks option to log warning if the 
garbage collector finds uncollectable objects. Problem: regrtest only logs a 
warning, but exit with a success in that case.

Attached PR makes regrtest fail in that case and adds an unit test for 
--findleaks.

--
components: Tests
messages: 340843
nosy: vstinner
priority: normal
severity: normal
status: open
title: regrtest --findleaks should fail if an uncollectable object is found
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36710] Pass _PyRuntimeState as an argument rather than using the _PyRuntime global variable

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

I don't really understand the rationale for these changes. What's wrong with 
the global variable _PyRuntime?

What's the long-term goal for _PyRuntime? If you can't get rid of all 
occurrences of _PyRuntime, how does it help to get rid of *some* occurences?

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25545] email parsing docs: clarify that only ASCII strings are supported

[immerrr again]

immerrr again  added the comment:

Hi everyone,

It's the first time I'm using this bugtracker, so apologies in advance if I 
manage to break something from the first go.

Not sure if it's the right place to report this, but I have the following repro 
that involves email.message_from_bytes:

In [128]: import email 
 ...: msg_bytes = ( 
 ...: b'MIME-Version: 1.0\r\n' 
 ...: b'Content-Type: text/plain;\r\n' 
 ...: b' charset=utf-8\r\n' 
 ...: b'Content-Transfer-Encoding: 8bit\r\n' 
 ...: b'Content-Disposition: attachment;\r\n' 
 ...: b' filename="camper_store.csv"\r\n\r\n' 
 ...: ) + 'Beyoğlu-İst'.encode('utf8') 
 ...: email.message_from_bytes(msg_bytes).get_payload(decode=True)  

  
Out[128]: b'Beyo\xc4\x9flu-\xc4\xb0st'

I have read this and some previous bug reports where it was clearly explained 
that message_from_string has its limitations and message_from_bytes should be 
used for better results. And if I'm not mistaken my repro should have it all 
set up correctly: CTE=8bit, body encoded in utf8 which is explicitly indicated 
as the content charset, yet the result is still encoded with 
'raw-unicode-escape'.

Is there something wrong with the input or is it a bug?

Thanks!

--
nosy: +immerrr again

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36718] Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: relocation R_X86_64_PC32 against symbol ... can not be used ...

[STINNER Victor]

New submission from STINNER Victor :

https://buildbot.python.org/all/#/builders/139/builds/251

Example:

building 'time' extension
gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -g -O0 -Wall -Wstrict-prototypes 
-I. -IInclude -I./Include -I/usr/include/x86_64-linux-gnu -I/usr/local/include 
-I/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Include 
-I/srv/buildbot/buildarea/2.7.bolen-ubuntu/build -c 
/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.c -o 
build/temp.linux-x86_64-2.7-pydebug/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.o
gcc -pthread -shared 
build/temp.linux-x86_64-2.7-pydebug/srv/buildbot/buildarea/2.7.bolen-ubuntu/build/Modules/timemodule.o
 -L/usr/lib/x86_64-linux-gnu -L/usr/local/lib -L. -lm -lpython2.7 -o 
build/lib.linux-x86_64-2.7-pydebug/time.so
/usr/local/lib/libpython2.7.a(posixmodule.o): In function `posix_tmpnam':
/srv/buildbot/Python-2.7.16/./Modules/posixmodule.c:7648: warning: the use of 
`tmpnam_r' is dangerous, better use `mkstemp'
/usr/local/lib/libpython2.7.a(posixmodule.o): In function `posix_tempnam':
/srv/buildbot/Python-2.7.16/./Modules/posixmodule.c:7595: warning: the use of 
`tempnam' is dangerous, better use `mkstemp'
/usr/bin/ld: /usr/local/lib/libpython2.7.a(ceval.o): relocation R_X86_64_PC32 
against symbol `PyFunction_Type' can not be used when making a shared object; 
recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

It seems to be a regression caused by my change, commit 
f4edd39017a211d4544570a1e2ac2110ef8e51b4, PR 12875 for bpo-28552.

--
components: Build
messages: 340840
nosy: vstinner
priority: normal
severity: normal
status: open
title: Python 2.7 compilation fails on AMD64 Ubuntu Shared 2.7 buildbot with: 
relocation R_X86_64_PC32 against symbol ... can not be used ...
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36659] distutils UnixCCompiler: Remove standard library path from rpath

[STINNER Victor]

STINNER Victor  added the comment:

I discussed the "rpath question" with different people. At the end, I think 
that the Fedora patch is "temporary fix" which we kept for way too long :-) We 
should remove this patch and ensure that upstream C extensions respect our 
rpath policy (Debian, OpenSuse, Fedora are against the usage of rpath).

I discussed with Miro Hrončok who is part of FESco (Fedora Engineering Steering 
Committee) and he created an issue to propose to make the build of a package 
*fail* if it uses rpath:
https://pagure.io/packaging-committee/issue/886

That change would be affect the whole Fedora project, not just python packages. 
IMHO it's a better approach than trying to exclude rpath for LIBDIR which is 
equal to /usr/lib64.

For these reasons, I close the issue and we will change Fedora instead.

Thanks everybody who was involved in this discussion, I learnt a lot! :-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d7befad328ad1a6d1f812be2bf154c1cd1e01fbc by Victor Stinner in 
branch 'master':
bpo-35537: Document posix_spawn() change in subprocess (GH-11668)
https://github.com/python/cpython/commit/d7befad328ad1a6d1f812be2bf154c1cd1e01fbc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21536] extension built with a shared python cannot be loaded with a static python

[STINNER Victor]

STINNER Victor  added the comment:

If an application is linked to libpython to embed Python, the application build 
requires flags to link to libpython. Currently, there are:

$ python3-config --libs
-lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm 

$ pkg-config --libs python-3.7 
-lpython3.7m 

My PR doens't change Misc/python.pc.

Discussion on python-dev:

* https://mail.python.org/pipermail/python-dev/2019-April/157196.html
* https://mail.python.org/pipermail/python-dev/2019-April/157201.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7774] sys.executable: wrong location if zeroth command argument is modified.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f4edd39017a211d4544570a1e2ac2110ef8e51b4 by Victor Stinner in 
branch '2.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) 
(GH-12949)
https://github.com/python/cpython/commit/f4edd39017a211d4544570a1e2ac2110ef8e51b4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[STINNER Victor]

STINNER Victor  added the comment:

I fixed the bug in Python 2.7, 3.7 and master branch (future 3.8).

Thanks Alexander P for the bug report ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) 
(GH-12948)
https://github.com/python/cpython/commit/3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f4edd39017a211d4544570a1e2ac2110ef8e51b4 by Victor Stinner in 
branch '2.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) 
(GH-12949)
https://github.com/python/cpython/commit/f4edd39017a211d4544570a1e2ac2110ef8e51b4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7774] sys.executable: wrong location if zeroth command argument is modified.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) 
(GH-12948)
https://github.com/python/cpython/commit/3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12874

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35824] http.cookies._CookiePattern modifying regular expressions

[Martin Panter]

Martin Panter  added the comment:

Test_http_cookies line 19 has the following test case:

{'data': 'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"',
 'dict': {'keebler' : 'E=mc2; L="Loves"; fudge=\012;'},
 'repr': '',
 'output': 'Set-Cookie: keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"'}

This is similar to an example in the documentation:

>>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";')
>>> print(C)
Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;"

If you break parsing of this string in the “load” method, you break documented 
behaviour. The “http.cookie” module is documented to follow RFC 2109. I believe 
the strings are valid by RFC 2109, in which the value is allowed to use the 
HTTP “quoted-string” format.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33017] Special set-cookie setting will bypass Cookielib

[Martin Panter]

Martin Panter  added the comment:

I think LCatro is saying that Python should accept the cookies and discard only 
the offending attributes. This makes sense to me and tends to agree with the 
specifications, but the three cases seem all seem unimportant to me.

PoC 1, Max-age:

>>> from urllib2 import Request
>>> from test.test_cookielib import FakeResponse
>>> cookies = CookieJar(DefaultCookiePolicy())
>>> request = Request('http://127.0.0.1/requests_test.php')
>>> cookies.extract_cookies(FakeResponse(()), request)  # Issue 12144
>>> cookies.make_cookies(FakeResponse(('Set-Cookie: test=123; max-age=a',)), 
>>> request)  # No cookies returned
[]

RFC 6265 says Max-age should be ignored if not does not start with a digit or 
minus sign: . Netscape did 
not specify Max-age at all. So I agree that the cookie should be retained.

PoC 2, Domain: You have to omit the equals sign to satisfy “v is None” and 
discard the cookie record, otherwise “v” is just an empty string '':

>>> cookies.make_cookies(FakeResponse(('Set-Cookie: test=123; domain=;',)), 
>>> request)  # v == ''
[Cookie(version=0, name='test', value='123', port=None, port_specified=False, 
domain='.', domain_specified=True, domain_initial_dot=False, path='/', 
path_specified=False, secure=False, expires=None, discard=True, comment=None, 
comment_url=None, rest={}, rfc2109=False)]
>>> cookies.make_cookies(FakeResponse(('Set-Cookie: test=123; domain;',)), 
>>> request)  # v is None
[]

RFC 6265 says both these cases should be treated the same, and recommends 
ignoring Domain in these cases.

PoC 3, Version:

>>> cookies.make_cookies(FakeResponse(('Set-Cookie: test=123; version=a;',)), 
>>> request)  # No cookies returned
[]

The Version attribute is only specified by RFC 2109. Since the IETF has 
obsoleted it, I suggest to deprecate RFC 2109 support in the Python module. 
That way, if a real problem is demonstrated, we can remove the parts that are 
causing the problem.

--
nosy: +martin.panter
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7774] sys.executable: wrong location if zeroth command argument is modified.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12875

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[miss-islington]

Change by miss-islington :


--
pull_requests: +12872

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28552] Distutils fail if sys.executable is None

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0ef8c157e9195df0115c54ba875a5efb92ac22fb by Victor Stinner in 
branch 'master':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875)
https://github.com/python/cpython/commit/0ef8c157e9195df0115c54ba875a5efb92ac22fb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7774] sys.executable: wrong location if zeroth command argument is modified.

[miss-islington]

Change by miss-islington :


--
pull_requests: +12873

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7774] sys.executable: wrong location if zeroth command argument is modified.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0ef8c157e9195df0115c54ba875a5efb92ac22fb by Victor Stinner in 
branch 'master':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875)
https://github.com/python/cpython/commit/0ef8c157e9195df0115c54ba875a5efb92ac22fb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12144] cookielib.CookieJar.make_cookies fails for cookies with 'expires' set

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

> Karthikeyan, it looks like your test will pass even when the bug is not 
> fixed. A test calling code that writes error message does not necessarily 
> mean the test itself will fail, I don’t think.

You are right. Sorry, I got mislead by the Exception message and didn't notice 
the test was passing. The below patch to master ensures the test passes by 
asserting expires in the cookie. If @demian.brecht haven't had a chance to make 
a PR then I can try converting the to a PR adding them as co-author.

diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py
index db82382357..07105a7c20 100644
--- a/Lib/http/cookiejar.py
+++ b/Lib/http/cookiejar.py
@@ -1590,6 +1590,7 @@ class CookieJar:
 def make_cookies(self, response, request):
 """Return sequence of Cookie objects extracted from response object."""
 # get cookie-attributes for RFC 2965 and Netscape protocols
+self._policy._now = self._now = int(time.time())
 headers = response.info()
 rfc2965_hdrs = headers.get_all("Set-Cookie2", [])
 ns_hdrs = headers.get_all("Set-Cookie", [])
@@ -1672,8 +1673,6 @@ class CookieJar:
 _debug("extract_cookies: %s", response.info())
 self._cookies_lock.acquire()
 try:
-self._policy._now = self._now = int(time.time())
-
 for cookie in self.make_cookies(response, request):
 if self._policy.set_ok(cookie, request):
 _debug(" setting cookie: %s", cookie)
diff --git a/Lib/test/test_http_cookiejar.py b/Lib/test/test_http_cookiejar.py
index 22bf41cf1d..ad3364c950 100644
--- a/Lib/test/test_http_cookiejar.py
+++ b/Lib/test/test_http_cookiejar.py
@@ -585,6 +585,14 @@ class CookieTests(unittest.TestCase):
 # if expires is in future, keep cookie...
 c = CookieJar()
 future = time2netscape(time.time()+3600)
+
+headers = ["Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/; 
expires={0}".format(future)]
+req = urllib.request.Request("http://www.coyote.com/;)
+res = FakeResponse(headers, "http://www.coyote.com/;)
+cookies = c.make_cookies(res, req)
+self.assertEqual(len(cookies), 1)
+self.assertEqual(time2netscape(cookies[0].expires), future)
+
 interact_netscape(c, "http://www.acme.com/;, 'spam="bar"; expires=%s' %
   future)
 self.assertEqual(len(c), 1)

Failure without patch : 

./python.exe -m unittest -v test.test_http_cookiejar.CookieTests.test_expires
test_expires (test.test_http_cookiejar.CookieTests) ... 
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/http/cookiejar.py:1619: 
UserWarning: http.cookiejar bug!
Traceback (most recent call last):
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/http/cookiejar.py", 
line 1616, in make_cookies
ns_cookies = self._cookies_from_attrs_set(
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/http/cookiejar.py", 
line 1574, in _cookies_from_attrs_set
cookie = self._cookie_from_cookie_tuple(tup, request)
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/http/cookiejar.py", 
line 1546, in _cookie_from_cookie_tuple
elif expires <= self._now:
AttributeError: 'CookieJar' object has no attribute '_now'

  _warn_unhandled_exception()
FAIL

==
FAIL: test_expires (test.test_http_cookiejar.CookieTests)
--
Traceback (most recent call last):
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_http_cookiejar.py",
 line 593, in test_expires
self.assertEqual(len(cookies), 1)
AssertionError: 0 != 1

--
Ran 1 test in 0.017s

FAILED (failures=1)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36305] Inconsistent behavior of pathlib.WindowsPath with drive paths

[Maor Kleinberger]

Maor Kleinberger  added the comment:

Alright. And what is left to do with the current issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12144] cookielib.CookieJar.make_cookies fails for cookies with 'expires' set

[Martin Panter]

Martin Panter  added the comment:

Karthikeyan, it looks like your test will pass even when the bug is not fixed. 
A test calling code that writes error message does not necessarily mean the 
test itself will fail, I don’t think.

I suggest you look at raising an exception when the UserWarning is triggered, 
and/or check that the expected cookie is returned with the right “expires” 
value.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36624] cleanup the stdlib and tests with regard to sys.platform usage

[Michael Felt]

Michael Felt  added the comment:

On 23/04/2019 17:53, Steve Dower wrote:
> Steve Dower  added the comment:
>
>> Being 'runtime' rather than 'buildtime' seemed more precise - tests that
>> are not meant to be build-time dependent use run-time status while
>> leaving sys.platform for concerns that are directly related to builds
> This is a good point - perhaps we need both?
>
> Many of the current test skips are related to build-time switches, but 
> indeed, some relate to the runtime environment.
>
> One aspect I'm keeping in mind here is that with the Windows Subsystem for 
> Linux, many lines are becoming blurred with respect to which build of Python 
> people are using (I'm already getting bug reports from people who thought 
> they were getting the Windows build but actually the Linux one, or vice 
> versa). And mismatching builds causes some weird problems. Being able to 
> clearly identify "Windows build + WSL environment" or "Linux build + WSL 
> environment" will likely matter more over time.
>
> It would be nice to have all the checks centralized, perhaps a set of 
> decorators in a test.requires module?
>
> @test.requires.linux_platform
> @test.requires.darwin_platform
> @test.requires.macos_runtime(10, 9)
> def test_my_test(self): ...
>
> (It would require some clever decorator design, but we can make those work 
> like "ORs" rather than "ANDs".)
>
> Does it provide any benefit? I think it's clever,
Too clever for me to build. There is a lot about the Python language
(syntax) I do not understand well enough.
> but that doesn't mean it's worthwhile :) Using skipIf and skipUnless with 
> test.support.CONSTANTS is just as readable and just as obvious (once we're 
> using them consistently).

To that end, I modified another 60 lines, or so, of mainly sys.platform
== 'win32' to use MS_WINDOWS, not MS_WINDOWS, skipIf(MS_WINDOWS,...) or
SkipUnless(MS_WINDOWS,...) - or comparable for the other platforms.

I also replaced the use of (support.)is_android and (support.is_java)
with ANDROID and JYTHON.

Curious about comments from code owners. And suggestions re: the
sys.platform lines (roughly 50) that are left.

FYI: when I started there were 321 references to sys.platform with
roughly 315 involved in a string comparison of some kind. Now it is
closer to 50.

>
> 
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com