[issue36030] add internal API function to create tuple without items array initialization

[Sergey Fedoseev]

Sergey Fedoseev  added the comment:

> Can you please convert msg336142 into a perf script?
> And then run again these benchmarks on PR 12052.

++-+--+
| Benchmark  | ref | untracked|
++=+==+
| list_as_tuple(0)   | 50.7 ns | 45.5 ns: 1.12x faster (-10%) |
++-+--+
| list_as_tuple(1)   | 64.5 ns | 56.5 ns: 1.14x faster (-12%) |
++-+--+
| list_as_tuple(5)   | 72.0 ns | 62.6 ns: 1.15x faster (-13%) |
++-+--+
| list_as_tuple(10)  | 86.3 ns | 77.1 ns: 1.12x faster (-11%) |
++-+--+
| list_as_tuple(100) | 469 ns  | 450 ns: 1.04x faster (-4%)   |
++-+--+

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35892] Fix awkwardness of statistics.mode() for multimodal datasets

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Are you happy guaranteeing that it will always be the first
> mode encountered?

Yes.  

All of the other implementations I looked at make some guarantee about which 
mode is returned.  Maple, Matlab, and Excel all return the first encountered.¹  
That is convenient for us because it is what Counter(data).most_common(1) 
already does and does cheaply (single pass, no auxiliary memory).  It also 
matches what a number of our other tools do:

>>> max(3, 3.0)   # 3 is first encountered
3
>>> max(3.0, 3)   # 3.0 is first encountered
3.0
>>> list(dict.fromkeys('aabbaacc'))[0] # 'a' is first encountered
'a'
>>> sorted([3, 3.0])[0]  # 3 is first encountered (due to sort stability)
3
>>> sorted([3.0, 3])[0]  # 3.0 is first encountered (due to sort stability)
3.0

¹ Scipy returned the smallest value rather than first value but it algorithm 
was sorting based to accommodate running a parallel mode() computation on 
multiple columns of an array. For us, that approach would be much slow, would 
require more memory, and would require more bookkeeping. 

P.S. I'm no longer thinking this should be saved for Pycon sprints.  That is 
right at the beta 1 feature freeze.  We should aim to get this resolved well in 
advance of that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0

[Safihre]

Safihre  added the comment:

In the CherryPy project it is also observed on Windows with Python 3,7.2.
In CherryPy it's triggered by Checker plugin, which connects to the app 
listening to the socket port in TLS mode via plain HTTP during startup (from 
the same process).
It has been around for a while: 
https://github.com/cherrypy/cherrypy/issues/1618#issuecomment-454150794

--
nosy: +Safihre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36131] test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

These tests were skipped in Travis with issue35411

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36130] Pdb(skip=[...]) + module without __name__ => TypeError

[Anthony Sottile]

Change by Anthony Sottile :


--
keywords: +patch
pull_requests: +12088
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36131] test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
priority: normal -> high
versions: +Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36131] test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Several buildbots are failing due to this problem. Some examples:

https://buildbot.python.org/all/#builders/33/builds/520
https://buildbot.python.org/all/#builders/72/builds/503
https://buildbot.python.org/all/#builders/78/builds/419

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36131] test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable

[Karthikeyan Singaravelan]

New submission from Karthikeyan Singaravelan :

I am seeing this error on Windows and Mac CI builds where FTP related tests in 
test.test_urllib2net.TimeoutTest are failing. It's reproducible locally too 
where the tests are skipped on Mac and Ubuntu. Not sure if it's random since 
several PRs in the last few hours fail with this.

Appveyor : 

* https://ci.appveyor.com/project/python/cpython/builds/22675425#L2817

VSTS builds : 

* https://dev.azure.com/Python/cpython/_build/results?buildId=38631
* https://dev.azure.com/Python/cpython/_build/results?buildId=38625

$ ./python.exe -m unittest -vv test.test_urllib2net.TimeoutTest
test_ftp_basic (test.test_urllib2net.TimeoutTest) ... skipped "Resource 
'ftp://www.pythontest.net/' is not available"
test_ftp_default_timeout (test.test_urllib2net.TimeoutTest) ... skipped 
"Resource 'ftp://www.pythontest.net/' is not available"
test_ftp_no_timeout (test.test_urllib2net.TimeoutTest) ... 
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: 
ResourceWarning: unclosed 
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: 
ResourceWarning: unclosed 
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: 
ResourceWarning: unclosed 
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: 
ResourceWarning: unclosed 
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/encodings/idna.py:163: 
ResourceWarning: unclosed 
  for label in labels[:-1]:
ResourceWarning: Enable tracemalloc to get the object allocation traceback
skipped "Resource 'ftp://www.pythontest.net/' is not available"
test_ftp_timeout (test.test_urllib2net.TimeoutTest) ... skipped "Resource 
'ftp://www.pythontest.net/' is not available"
test_http_basic (test.test_urllib2net.TimeoutTest) ... ok
test_http_default_timeout (test.test_urllib2net.TimeoutTest) ... ok
test_http_no_timeout (test.test_urllib2net.TimeoutTest) ... ok
test_http_timeout (test.test_urllib2net.TimeoutTest) ... ok

--
Ran 8 tests in 33.617s

OK (skipped=4)

--
components: Tests
messages: 336729
nosy: benjamin.peterson, pablogsal, vstinner, xtreak
priority: normal
severity: normal
status: open
title: test.test_urllib2net.TimeoutTest ftp related tests fail due to 
ftp://www.pythontest.net/ being unavailable
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36130] Pdb(skip=[...]) + module without __name__ => TypeError

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xdegaye, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36130] Pdb(skip=[...]) + module without __name__ => TypeError

[Anthony Sottile]

New submission from Anthony Sottile :

Here's the simplest example I could come up with -- hit this while debugging 
pytest (which uses attrs which uses similar code to this to make classes)

import pdb; pdb.Pdb(skip=['django.*']).set_trace()
eval(compile("1", "", "exec"), {})
print('ok!')


When running this:

$ python3.8 t.py
> /home/asottile/workspace/setup-cfg-fmt/t.py(2)()
-> eval(compile("1", "", "exec"), {})
(Pdb) n
Traceback (most recent call last):
  File "t.py", line 2, in 
eval(compile("1", "", "exec"), {})
  File "", line 1, in 
  File "/usr/lib/python3.8/bdb.py", line 90, in trace_dispatch
return self.dispatch_call(frame, arg)
  File "/usr/lib/python3.8/bdb.py", line 128, in dispatch_call
if not (self.stop_here(frame) or self.break_anywhere(frame)):
  File "/usr/lib/python3.8/bdb.py", line 203, in stop_here
self.is_skipped_module(frame.f_globals.get('__name__')):
  File "/usr/lib/python3.8/bdb.py", line 194, in is_skipped_module
if fnmatch.fnmatch(module_name, pattern):
  File "/usr/lib/python3.8/fnmatch.py", line 34, in fnmatch
name = os.path.normcase(name)
  File "/usr/lib/python3.8/posixpath.py", line 54, in normcase
s = os.fspath(s)
TypeError: expected str, bytes or os.PathLike object, not NoneType


$ python3.8 --version --version
Python 3.8.0a2 (default, Feb 25 2019, 23:11:49) 
[GCC 7.3.0]

--
components: Library (Lib)
messages: 336728
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Pdb(skip=[...]) + module without __name__ => TypeError
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28441] Change sys.executable to include executable suffix

[miss-islington]

Change by miss-islington :


--
pull_requests: +12087

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36125] Cannot cross-compile to more featureful but same tune

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report. Unfortunately, as far as I know, we don't claim anywhere 
to support cross-compiling in general.  As you've probably noticed, the Python 
Makefile-based build system (for non-Windows platforms) is by no means a 
conventional GNU autotools one. Over the years there have been some efforts to 
add support for a handful of specific cross-compilation scenarios, for example 
compiling for Android.  But I'm afraid you are more or less on your own for the 
case here.  That said, if you or someone else is motivated to supply some fixes 
to make more cross-compiling cases work, feel free to reopen the issue and 
submit the fixes as PRs and they may be considered.  What would probably be the 
best would be to try to reimplement our build system from the ground up with 
modern Autotools or something similar but that's a *really* big deal.

--
nosy: +ned.deily
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] [subinterpreters] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +12086

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25872] multithreading traceback KeyError when modifying file

[Christopher Unkel]

Change by Christopher Unkel :


--
keywords: +patch
pull_requests: +12085
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ionel Cristian Mărieș]

Ionel Cristian Mărieș  added the comment:

On Wed, Feb 27, 2019 at 1:41 AM Barry A. Warsaw 
wrote:
> Basically yes, I’ve done this.  But think of the poor user who doesn’t
have that expertise or ability to hack on an installed Python’s site.py
file.  When their application breaks because some faulty pth was installed
behind their back, how do they debug their application when the breakage
has already occurred before Python even gets to their code?  How do they
answer questions like “where did that magical sys.path entry come from?” or
“how did that module get in sys.modules already?”

Aren't these sort of questions answered by using `strace python -v` or
similar? What information is missing more exactly?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ionel Cristian Mărieș]

Ionel Cristian Mărieș  added the comment:

On Wed, Feb 27, 2019 at 1:31 AM Barry A. Warsaw 
wrote:
> Your sudo may not be my sudo. :)  Let’s say I update my Ubuntu desktop
and a new version of package with a pth breaks.
> Maybe I didn’t even know I was doing that, via automated updates, or
management portal, etc.
> Now a poor user who depends on this has their code break.  How do *they*
debug the problem?

Well that's easy:

* update my Ubuntu desktop -> stuff breaks -> rollback/downgrade
* automated updates -> stuff breaks -> stop using them, and learn lesson ;)
* management portal -> stuff breaks -> complain to sysadmin

Desktop users don't need to debug problems, devs/sysadmins do. They have
sudo.

> FWIW, `sudo pip install` should just be banned IMHO :).

Lets also ban ctypes and threads right? :)

> For sure.  But here’s the thing: you need to know *which* pth file is
problematic.  Which means you have to debug the entire startup process
where pth files are loaded.

How many pth files could one have? 2-3 ... 5 at most. Just `locate .pth`
and rename the biggest one till the problem goes away.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36129] io documentation unclear about flush() and close() semantics for wrapped streams

[Josh Rosenberg]

Josh Rosenberg  added the comment:

The general rule of thumb is to have the API behave as if no wrapping is 
occurring. The outermost layer should still adhere to the documented API 
requirements, so both flush and close should cascade (flush says it flushes the 
"buffers" plural; all user mode buffers should be flushed, no matter the 
layer), closing the outer layer without closing the inner makes it 
non-intuitive as to *how* you close the inner layers at all.

So the answers are:

1. flush all layers when told to flush
2. close all layers when told to close (exception for when you accept a fd, you 
can take a closefd argument and avoid closing the fd itself, but all Python 
layers should be closed)
3. N/A (close cascades)
4. No.
5. Non-blocking I/O doesn't seem to have a completely consistent story, but 
from the existing classes, I'm guessing outer layers should try to write to the 
underlying stream when told to do so, and raise BlockingIOError if they'd would 
block. The close story should be roughly the same; try to flush, raise 
BlockingIOError if it would block, then close (which shouldn't do much since 
the buffers are flushed).
6. No.

#5 is weird, and we could probably use a more coherent story for how 
non-blocking I/O should be done (asyncio provides one story, selectors another, 
and the io module seems to sort of gloss over the design strategy for 
non-blocking I/O).

You can derive all but #5 from how the built-in open behaves; it returns 
different types (FileIO, Buffered*, TextIOWrapper), but whatever it returns, 
you don't need to pay attention to the layered aspect if you don't want to 
(most don't). flush works as expected (no flushing each layer independently), 
close works as expected (no leaked file handles from only closing the outer 
layer), using it with a with statement is identical to explicitly calling close 
with a try/finally block.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36128] ResourceReader for FileLoader inconsistently handles path separators

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 26, 2019, at 13:05, Gregory Szorc  wrote:
> 
> I would think the two would agree about whether a path with separators is a 
> resource or not. The documentation at 
> https://docs.python.org/3.7/library/importlib.html#importlib.abc.ResourceReader
>  implies that resources in subdirectories should not be allowed.

Historical context is in the standalone tracker:

https://gitlab.com/python-devs/importlib_resources/issues/58

Clearly, with the current definition of the API, open_resource() should not 
allow slashes.  Please follow up on the above link if you have opinions about 
changing the behavior.  We would have an opportunity to relax that constraint 
in Python 3.8, if it’s something we want to do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 26, 2019, at 13:23, Ivan Pozdeev  wrote:
> 
> Easy. Insert a chunk into site.py that would call pdb.set_trace() if an 
> envvar (e.g. `PYSITEDEBUG') or a command line switch is set.
> 
> Actually, why can't whoever has this problem add such a chunk themselves? Is 
> this really such a frequent and ubiquitous problem
> that this needs to be in the stock codebase? I suspect we're dealing with a 
> vocal minority here.

Basically yes, I’ve done this.  But think of the poor user who doesn’t have 
that expertise or ability to hack on an installed Python’s site.py file.  When 
their application breaks because some faulty pth was installed behind their 
back, how do they debug their application when the breakage has already 
occurred before Python even gets to their code?  How do they answer questions 
like “where did that magical sys.path entry come from?” or “how did that module 
get in sys.modules already?”

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 26, 2019, at 12:52, Ionel Cristian Mărieș  wrote:
> 
> Something bad was installed with sudo but suddenly sudo is not acceptable for 
> debugging? This seems crazy.

Your sudo may not be my sudo. :)  Let’s say I update my Ubuntu desktop and a 
new version of package with a pth breaks.  Maybe I didn’t even know I was doing 
that, via automated updates, or management portal, etc.  Now a poor user who 
depends on this has their code break.  How do *they* debug the problem?

FWIW, `sudo pip install` should just be banned IMHO :).

> How exactly are pth files hard to debug? Are those files hard to edit? They 
> sure are, but the problem ain't the point where they are run, it's the fact 
> that a big lump of code is stuffed on a single line. Lets fix that instead!

For sure.  But here’s the thing: you need to know *which* pth file is 
problematic.  Which means you have to debug the entire startup process where 
pth files are loaded.  That means you’re not really debugging pth files 
themselves (often), but site.py.  Debugging site.py for an installed Python is 
not trivial.  Hopefully you are at least not squeamish about editing a system 
file and breaking Python worse than the original bug. 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36046] support dropping privileges when running subprocesses

[Patrick McLean]

Patrick McLean  added the comment:

> Thanks for your explanation. In case of a privileged process, the behavior of 
> setreuid/setregid/setgroups does seem well-defined. But setuid/setgid change 
> all ids (real, effective, saved) too in this case. Do you prefer 
> setreuid/setregid because they provide stricter semantics in non-privileged 
> processes compared to setuid/setgid? (The latter ones change the effective id 
> only, potentially preserving the process ability to switch ids later).

Yes, exactly. The stricter semantics provide stronger security guarantees. The 
idea is to run code in an unprivileged context in a way that the code has no 
way to regain privileges.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[STINNER Victor]

STINNER Victor  added the comment:

> How much bigger does the core interpreter + built-in extension modules get 
> when you make this change? How much more memory is used by real world 
> programs?

Well, any optimization is a matter of trade-off between memory and CPU. Last 
years, CPU are not really getting way faster (especially when you consider that 
Python is usually only able to use a single CPU thread), whereas computers are 
getting more and more RAM.


> It should be tested on a number of systems too; any losses to cache 
> unfriendliness would be highly dependent on the size of the CPU cache.

I prefer to no pay too much attention to assumptions on the hardware. I prefer 
to only trust benchmarks :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

> if LTO is enabled, the same stack bloat issues are possible

Please test, I'm not interested to spend too much time on that topic.

To be clear, _Py_NO_INLINE was a hack and a micro-optimization. It doesn't 
solve a real bug. Python has very weak promises on the maximum stack depth.

My work on reducing the stack memory usage was mostly motivated by my work on 
FASTCALL, since some patches reduced the maximum stack depth. They increased 
the stack memory usage with micro-optimizations like "PyObject 
*small_stack[_PY_FASTCALL_SMALL_STACK];" which is allocated on ths stack: see 
Objects/call.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

> Victor/vstinner: Isn't PR 12032 reintroducing the issue fixed in #29234?

No according to manual tests:
https://github.com/python/cpython/pull/12032#issuecomment-467110233

Usage of the stack memory doesn't change with PR 12032.

FYI I also tested manually since I was very surprised, and I confirm that the 
PR doesn't change the usage of the stack memory :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Steve Dower]

Steve Dower  added the comment:

Barry is a steering council member now, so by definition he's 1/5th of the 
loudest possible minority ;)

I am totally okay with adding more diagnostics here. Frankly, if "-v" doesn't 
currently log info about .pth files (or other things that the site module does 
when it's active) then we should just do that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36129] io documentation unclear about flush() and close() semantics for wrapped streams

[Gregory Szorc]

New submission from Gregory Szorc :

As part of implementing io.RawIOBase/io.BufferedIOBase compatible stream types 
for python-zstandard, I became confused about the expected behavior of flush() 
and close() when a stream is wrapping another stream.

The documentation doesn't lay out explicitly when flush() and close() on the 
outer stream should be proxied to the inner stream, if ever.

Here are some specific questions (it would be great to have answers to these in 
the docs):

1. When flush() is called on the outer stream, should flush() be called on the 
inner stream as well? Or should we simply write() to the inner stream and not 
perform a flush() on that inner stream?

2. When close() is called on the outer stream, should close() be called on the 
inner stream as well?

3. If close() is not automatically called on the inner stream during an outer 
stream's close(), should the outer stream trigger a flush() or any other 
special behavior on the inner stream? Or should it just write() any lingering 
data and then go away?

4. Are any of the answers from 1-3 impacted by whether the stream is a reader 
or writer? (Obviously reader streams don't have a meaningful flush() 
implementation.)

5. Are any of the answers from 1-3 impacted by whether the stream is in 
blocking/non-blocking mode?

6. Do any of the answers from 1-3 vary depending on whether behavior is 
incurred by the outer stream's __exit__?

(This issue was inspired by 
https://github.com/indygreg/python-zstandard/issues/76.)

--
components: Interpreter Core
messages: 336715
nosy: indygreg
priority: normal
severity: normal
status: open
title: io documentation unclear about flush() and close() semantics for wrapped 
streams
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36046] support dropping privileges when running subprocesses

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

> 1) This is intentional, this is for dropping privileges before running some 
> (possibly untrusted) command, we do not want to leave a path for the 
> subprocess to gain root back. If there is a subprocess that needs root for 
> some operations, it would presumably have the ability to drop privileges 
> itself, and would not need the python script to do it before running it.

> 2) While POSIX leaves it unspecified what changes are permitted for an 
> unprivileged process, these are permitted for a privileged process, which is 
> the main use case for this functionality. In the case the OS does not support 
> it for an unpriviliged process, the syscall would fail with EPERM, which can 
> be handled from the calling python code.

Thanks for your explanation. In case of a privileged process, the behavior of 
setreuid/setregid/setgroups does seem well-defined. But setuid/setgid change 
all ids (real, effective, saved) too in this case. Do you prefer 
setreuid/setregid because they provide stricter semantics in non-privileged 
processes compared to setuid/setgid? (The latter ones change the effective id 
only, potentially preserving the process ability to switch ids later).

> I am fine removing the workaround, and allowing it to fail with EPERM. 
> Perhaps we could find another way around this for running the tests in an 
> unprivileged environment, or just leave the test only running the EPERM 
> case...

> I could change the API to have have group= and supp_groups=  if you prefer.

Personally, I do prefer separate arguments because subprocess is a relatively 
low-level module and tends to expose underlying operations, so it seems logical 
to have a knob that maps to setgid() and another one that maps to setgroups(). 
@gregory.p.smith: what do you think about it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

On 26.02.2019 23:37, Barry A. Warsaw wrote:

> My opposition would lessen if there were clear ways to debug, and preferably 
> also prevent, pth interpretation.

Easy. Insert a chunk into site.py that would call pdb.set_trace() if an envvar 
(e.g. `PYSITEDEBUG') or a command line switch is set.

Actually, why can't whoever has this problem add such a chunk themselves? Is 
this really such a frequent and ubiquitous problem
that this needs to be in the stock codebase? I suspect we're dealing with a 
vocal minority here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36128] ResourceReader for FileLoader inconsistently handles path separators

[Gregory Szorc]

New submission from Gregory Szorc :

The implementation of the ResourceReader API for the FileLoader class in 
importlib/_bootstrap_external.py is inconsistent with regards to handling of 
path separators.

Specifically, "is_resource()" returns False if "resource" has a path separator. 
But "open_resource()" will happily open resources containing a path separator.

I would think the two would agree about whether a path with separators is a 
resource or not. The documentation at 
https://docs.python.org/3.7/library/importlib.html#importlib.abc.ResourceReader 
implies that resources in subdirectories should not be allowed.

One can easily demonstrate this behavior oddity with Mercurial:

(Pdb) p 
sys.modules['mercurial'].__spec__.loader.get_resource_reader('mercurial').open_resource('help/config.txt')
<_io.FileIO name='/home/gps/src/hg/mercurial/help/config.txt' mode='rb' 
closefd=True>
(Pdb) p 
sys.modules['mercurial'].__spec__.loader.get_resource_reader('mercurial').is_resource('help/config.txt')
False

The behavior has been present since the functionality was added 
(https://github.com/python/cpython/pull/5168).

--
components: Library (Lib)
messages: 336712
nosy: barry, indygreg
priority: normal
severity: normal
status: open
title: ResourceReader for FileLoader inconsistently handles path separators
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords

[Michael Sullivan]

Change by Michael Sullivan :


--
nosy: +msullivan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ionel Cristian Mărieș]

Ionel Cristian Mărieș  added the comment:

> Because there’s no good place to stick a pdb/breakpoint to debug such issues 
> other than site.py, and that usually requires sudo.

Something bad was installed with sudo but suddenly sudo is not acceptable for 
debugging? This seems crazy.

How exactly are pth files hard to debug? Are those files hard to edit? They 
sure are, but the problem ain't the point where they are run, it's the fact 
that a big lump of code is stuffed on a single line. Lets fix that instead!

I've written pth files with lots of stuff in them, and my experience is quite 
the opposite - they help with debugging. A lot. It's an incredibly useful 
python feature.

> I don’t remember the details, but yes I have been caught in this trap. 

Maybe if you remember the details we can discuss what are the debugging 
options, and what can be improved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 26, 2019, at 12:32, Steve Dower  wrote:
> 
> There are 100 other ways to end up in this situation though. Why is *this* 
> one so much worse?

Because there’s no good place to stick a pdb/breakpoint to debug such issues 
other than site.py, and that usually requires sudo.

> Can you offer an issue you hit that was caused by a .pth file that *wasn't* 
> debuggable by listing sys.path?

I don’t remember the details, but yes I have been caught in this trap.  The 
thing is, by the time user code gets called, the damage is already done, so 
debugging is quite difficult.

This will be alleviated at least partially by deprecating the executing of 
random code.  Maybe just allowing sys.path hacking will be enough to make it 
not so terrible, especially if e.g. (and I haven’t check to see whether this is 
the case today), `python -v` shows you exactly which .pth file is extending 
sys.path.

The issue is discoverability.  Since pth files happen before you get an 
interpreter prompt, it’s too difficult to debug unexpected, wrong, or broken 
behavior.  My opposition would lessen if there were clear ways to debug, and 
preferably also prevent, pth interpretation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Steve Dower]

Steve Dower  added the comment:

> Extending sys.path is a useful use case, but doing so in pth files is 
> problematic.

There are 100 other ways to end up in this situation though. Why is *this* one 
so much worse?

Can you offer an issue you hit that was caused by a .pth file that *wasn't* 
debuggable by listing sys.path?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x

[Geoff Shannon]

Geoff Shannon  added the comment:

I'm aware of it. I actually wrote that patch as well. :D

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36118] Cannot correctly concatenate nested list that contains more than ~45 entries with other nested lists.

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Agreed, I cannot reproduce this (online link showing behavior): 
https://tio.run/##K6gsycjPM/7/v0LBViE6WilRSUdBKQlI6OnpKQCZTlFKsbFclWDJWB2FaEMdIx1jHRMdU5gKS0uQfLRBLFBJBZDiKijKzCvRAIlocv3/DwA

My guess is the code is subtly different, e.g. replacing:

>>> y[0] = x[0]
>>> print(y[0])

with:

>>> y[:1] = x[0]
>>> print(y)

would get roughly what the OP is seeing (and it would be the correct/expected 
result in that case). Either way, not a bug.

--
nosy: +josh.r
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Josh Rosenberg]

Josh Rosenberg  added the comment:

How much bigger does the core interpreter + built-in extension modules get when 
you make this change? How much more memory is used by real world programs?

I'm a little concerned when I see individual functions growing by 140 lines in 
the first file of the diff. Clearly the increase in memory usage wasn't enough 
to slow down the microbenchmarks (which can probably fit the entire hot code 
path in CPU on die cache), but I'd be worried about the aggregate effect on 
real world programs if we go from a handful of argument parsing code paths 
reused by every function (and therefore kept constantly hot) to hundreds (or 
thousands?) of unique argument parsing code paths, each one unique to a single 
function. It could easily look great when a single function is being called 
repeatedly, while looking much less great (possibly worse) when the many varied 
function calls are interleaved.

It should be tested on a number of systems too; any losses to cache 
unfriendliness would be highly dependent on the size of the CPU cache.

I'll grant, it doesn't seem like inlining positional argument parsing has 
caused problems, and it looks like we're still using _PyArg_UnpackKeywords, so 
argument parsing isn't completely devolved to each functions, but I think we 
need something more than microbenchmarks before we jump on this.

If my worries end up being unfounded, I'll be happy. Looks very cool if we can 
really get that sort of speed up for all function calls, not just positional 
args only functions. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36071] Add support for Windows ARM32 in ctypes/libffi

[Paul Monson]

Change by Paul Monson :


--
keywords: +patch
pull_requests: +12084
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Josh Rosenberg]

Change by Josh Rosenberg :


--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 26, 2019, at 05:19, Nick Coghlan  wrote:
> 
> I just don't want to lose the "add this location to sys.path" behaviour that 
> exists for lines in pth files that *don't* start with "import ", since that 
> has plenty of legitimate use cases, and the only downside of overusing it is 
> an excessively long default sys.path (which has far more consistent and 
> obvious symptoms than the arbitrary code execution case can lead to).

It’s also very difficult to debug because pth loading usually happens before 
the user has a chance to intervene with a debugger.  This means mysterious 
things can happen, like different versions of a package getting imported than 
you expect.

Extending sys.path is a useful use case, but doing so in pth files is 
problematic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

I am closing this as it has been fixed by this PR 


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

That's a lot faster and will be great if they make it to next alpha :) Adding 
Ammar Askar since they did review for positional arguments.

--
nosy: +ammar2, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36119] Can't add/append in set/list inside shared dict

[Josh Rosenberg]

Josh Rosenberg  added the comment:

As Karthikeyan, this is an inevitable, and documented, consequence of the 
proxies not being aware of in-place modification of their contents.

As your own example demonstrates, any approach that provides that information 
to the shared dict proxy will work; |= and += are almost the same as .update 
and .extend, but implemented such that the left hand side is always reassigned, 
even when the result of __ior__/__iadd__ is the same object it was called on. 
Thus, |=/+= work, while add/append/update/extend do not.

Note that as of 3.6, there is another option: Nested shared objects:

> Changed in version 3.6: Shared objects are capable of being nested. For 
> example, a shared container object such as a shared list can contain other 
> shared objects which will all be managed and synchronized by the SyncManager.

So the alternative solution in your case (assuming you're on 3.6 or later, 
which your bug version tags say you are) is to make the sub-lists 
manager.lists, or replace use of a set with manager.dict (as dicts with all 
values set to True, are largely compatible with set anyway, especially with the 
advent of dict views):

manager = Manager()
shared_dict = manager.dict()

shared_dict['test'] = manager.dict() # or shared_dict['test'] = manager.list()

shared_dict['test'][1234] = True # or shared_dict['test'].append(1234)

Downside: The repr of shared dicts/lists doesn't display the contents, so your 
example code won't make it obvious that the problem is fixed, but it does in 
fact work. I wrote a terrible JSON one-liner to check the contents, and it 
demonstrates that the shared dict/list work just fine:

import json
from multiprocessing import Manager
from multiprocessing.managers import DictProxy, ListProxy

manager = Manager()
shared_dict = manager.dict()

shared_dict['testset'] = set()
shared_dict['testlist'] = []
shared_dict['testshareddict'] = manager.dict()
shared_dict['testsharedlist'] = manager.list()

shared_dict['testset'].add(1234)
shared_dict['testlist'].append(1234)
shared_dict['testshareddict'][1234] = True
shared_dict['testsharedlist'].append(1234)

print(json.dumps(shared_dict, default=lambda x: dict(x) if isinstance(x, 
DictProxy) else
list(x) if isinstance(x, 
ListProxy) else
dict.fromkeys(x, True) if 
isinstance(x, (set, frozenset)) else
x))

The dump shows that the changes to the shared inner dict and list are reflected 
in the result directly, even with no assignment back to the keys of the outer 
dict (while, as you note, the plain set and list show no changes).

Closing as not a bug, since this is fully documented, with multiple workarounds 
available.

--
nosy: +josh.r
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Some examples:

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 -s "round_ = round" "round_(4.2)"
Mean +- std dev: [...] 110 ns +- 3 ns -> [...] 81.4 ns +- 2.2 ns: 1.35x faster 
(-26%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 -s "sum_ = sum" "sum_(())"
Mean +- std dev: [...] 88.0 ns +- 6.5 ns -> [...] 57.6 ns +- 1.1 ns: 1.53x 
faster (-35%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 -s "sum_ = sum; a = [1, 2]"  "sum_(a)"
Mean +- std dev: [...] 95.9 ns +- 2.1 ns -> [...] 70.6 ns +- 2.0 ns: 1.36x 
faster (-26%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "'abc'.split()"  
Mean +- std dev: [...] 102 ns +- 3 ns -> [...] 80.5 ns +- 2.1 ns: 1.26x faster 
(-21%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "b'abc'.split()"
Mean +- std dev: [...] 91.8 ns +- 2.3 ns -> [...] 75.1 ns +- 1.4 ns: 1.22x 
faster (-18%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "'abc'.split('-')"
Mean +- std dev: [...] 118 ns +- 2 ns -> [...] 89.2 ns +- 1.8 ns: 1.32x faster 
(-24%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "b'abc'.decode()"
Mean +- std dev: [...] 96.1 ns +- 3.6 ns -> [...] 78.9 ns +- 2.2 ns: 1.22x 
faster (-18%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "'abc'.encode()"
Mean +- std dev: [...] 72.4 ns +- 1.9 ns -> [...] 55.1 ns +- 1.8 ns: 1.31x 
faster (-24%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 -s "int_= int"  "int(4.2)"
Mean +- std dev: [...] 105 ns +- 4 ns -> [...] 78.8 ns +- 1.9 ns: 1.33x faster 
(-25%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 -s "int_= int"  "int('5')"
Mean +- std dev: [...] 154 ns +- 5 ns -> [...] 122 ns +- 4 ns: 1.26x faster 
(-21%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000  "42 .to_bytes(2, 'little')
Mean +- std dev: [...] 109 ns +- 3 ns -> [...] 72.4 ns +- 1.9 ns: 1.51x faster 
(-34%)

$ ./python -m perf timeit --compare-to=../cpython-release-baseline/python 
--duplicate=1000 "from_bytes = int.from_bytes" "from_bytes(b'ab', 'little')"
Mean +- std dev: [...] 138 ns +- 3 ns -> [...] 96.3 ns +- 3.0 ns: 1.43x faster 
(-30%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +12083
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36127] Argument Clinic: inline parsing code for functions with keyword parameters

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

This is a follow up of issue23867 and issue35582. The proposed PR makes 
Argument Clinic inlining parsing code for functions with keyword parameters, 
i.e. functions that use _PyArg_ParseTupleAndKeywordsFast() and 
_PyArg_ParseStackAndKeywords() now. This saves time for parsing format strings 
and calling few levels of functions.

--
assignee: serhiy.storchaka
components: Argument Clinic
messages: 336700
nosy: larry, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Argument Clinic: inline parsing code for functions with keyword 
parameters
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36126] Reference count leakage in structseq_repr

[zasdfgbnm]

New submission from zasdfgbnm :

In Python 2.7 structseq is not a tuple, and in `structseq_repr` a tuple is 
created to help extracting items. However when the check at 
https://github.com/python/cpython/blob/2.7/Objects/structseq.c#L268 fails, the 
reference count of this tuple is not decreased, causing memory leakage.

To reproduce, download the attached file, install and run the `quicktest.py` 
and watch the memory usage.

This bug only exists on python 2.7, because on python >3.2, no helper tuple is 
created.

--
components: Interpreter Core
files: structseq.tar.xz
messages: 336699
nosy: zasdfgbnm
priority: normal
pull_requests: 12082
severity: normal
status: open
title: Reference count leakage in structseq_repr
type: behavior
versions: Python 2.7
Added file: https://bugs.python.org/file48172/structseq.tar.xz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36125] Cannot cross-compile to more featureful but same tune

[Ross Burton]

Ross Burton  added the comment:

>From what I can tell:

configure.ac sets PYTHON_FOR_BUILD like this if cross-compiling:

PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) 
_PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f 
pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib 
_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) 
'$interp

Note how PYTHONPATH is set to the source and build paths for Lib/.

The intention appears to be that the sysconfig.py in the build is used.  In my 
case that directory is also full of shared libraries that Python happily loads, 
and then fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD

[Enji Cooper]

Change by Enji Cooper :


--
versions: +Python 2.7, Python 3.4, Python 3.5, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13501] Make libedit support more generic; port readline / libedit to FreeBSD

[Enji Cooper]

Enji Cooper  added the comment:

I'll try to rebase Martin's changes, as they don't apply to the master branch 
on GitHub today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24643] VS 2015 pyconfig.h #define timezone _timezone conflicts with timeb.h

[Steve Dower]

Steve Dower  added the comment:

Thanks, Zackery!

--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36124] Provide convenient C API for storing per-interpreter state

[Eric Snow]

Eric Snow  added the comment:

+1 from me

@Armin, thanks to Nick I understand your request better now.  I'll put up a PR 
by the end of the week if no one beats me to it.

--
nosy: +arigo, eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[miss-islington]

miss-islington  added the comment:


New changeset 2632474957fa9c6311af21be6906d1234853f288 by Miss Islington (bot) 
in branch '3.7':
bpo-36123: Fix test_socket.testWithTimeoutTriggeredSend() race condition 
(GH-12053)
https://github.com/python/cpython/commit/2632474957fa9c6311af21be6906d1234853f288


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Victor/vstinner: Isn't PR 12032 reintroducing the issue fixed in #29234? 
_PyStack_AsTuple was explicitly marked as _Py_NO_INLINE because inlining was 
creating excessive stack consumption in the callers (which were the bytecode 
interpreter loop), but the new _PyTuple_FromArray isn't marked as 
_Py_NO_INLINE, so the swap reintroduces the problem.

Seems like either:

1. _PyTuple_FromArray should also be marked _Py_NO_INLINE

or

2. _PyStack_AsTuple should continue to exist as the non-inlined version of 
_PyTuple_FromArray

It's possible this isn't as much of an issue because _PyTuple_FromArray is in 
tupleobject.c (where it's only called in one place), while _PyStack_AsTuple was 
in call.c and was called from within call.c in four places, but only if 
link-time optimization isn't involved (and in practice, most public 
distributions of CPython are built with link-time optimization now, correct?); 
if LTO is enabled, the same stack bloat issues are possible.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36106] resolve sinpi() name clash with libm

[Mark Dickinson]

Mark Dickinson  added the comment:

This one was my fault. Thanks for the fix!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35417] Double parenthesis in print function running 2to3 in already correct call

[STINNER Victor]

STINNER Victor  added the comment:

I suggest to close this issue as WONTFIX.

2to3 is designed as a tool to convert a Python 2 code base to Python 3 at once. 
I understand that once the code base is converted, you may want to revert some 
unwanted "useless" changes.

Python 2 and Python 3 languages have some intersections where it's hard to 
guess if the code is valid or not in Python 3. 2to3 uses heuristic which 
implements practical solutions. And you noticed, there are some corner cases 
where 2to3 generates useless changes.

IMHO 2to3 is fine. It may be nice to enhance it, but well, it's a trade-off, 
it's good as it is.

You may want to test other tools like modernize, 2to6, sixer, etc. which made 
different trade-offs.

Note: I'm the author of sixer and I'm not a big of 2to3 since it drops Python 2 
support and generate many changes which are not needed since Python 3.2.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[miss-islington]

Change by miss-islington :


--
pull_requests: +12081

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 53b9e1a1c1d86187ad6fbee492b697ef8be74205 by Victor Stinner 
(Joannah Nanjekye) in branch 'master':
bpo-36123: Fix test_socket.testWithTimeoutTriggeredSend() race condition 
(GH-12053)
https://github.com/python/cpython/commit/53b9e1a1c1d86187ad6fbee492b697ef8be74205


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26228] pty.spawn hangs on FreeBSD 9.3, 10.x

[STINNER Victor]

STINNER Victor  added the comment:

Please have a look at this pty.spawn() documentation change:
https://github.com/python/cpython/pull/11980

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36125] Cannot cross-compile to more featureful but same tune

[Ross Burton]

New submission from Ross Burton :

My build machine is a Haswell Intel x86-64. I'm cross-compiling to x86-64, with 
-mtune=Skylake -avx2.  During make install PYTHON_FOR_BUILD loads modules from 
the *build* Lib/ which contain instructions my Haswell can't execute:

| 
_PYTHON_PROJECT_BASE=/data/poky-tmp/master/work/corei7-64-poky-linux/python3/3.7.2-r0/build
 _PYTHON_HOST_PLATFORM=linux-x86_64 PYTHONPATH=../Python-3.7.2/Lib 
_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_m_linux_x86_64-linux-gnu python3.7 -v 
-S -m sysconfig --generate-posix-vars ;\
Illegal instruction

--
components: Build
messages: 336688
nosy: rossburton
priority: normal
severity: normal
status: open
title: Cannot cross-compile to more featureful but same tune
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] [subinterpreters] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[Eric Snow]

Eric Snow  added the comment:

FYI, I have a couple of small follow-up changes to land before I close this 
issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35459] Use PyDict_GetItemWithError() instead of PyDict_GetItem()

[Josh Rosenberg]

Josh Rosenberg  added the comment:

#36110 was closed as a duplicate; the superseder is #36109 (which has been 
fixed). The change should still be documented, just in case anyone gets bitten 
by it.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36103] Increase shutil.COPY_BUFSIZE

[Inada Naoki]

Inada Naoki  added the comment:

Read this file too.
http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/ioblksize.h

coreutils choose 128KiB for *minimal* buffer size to reduce syscall overhead.
In case of shutil, we have Python interpreter overhead adding to syscall 
overhead.
Who has deeper insights than coreutils author?

I think 128KiB is the best, but I'm OK to 64KiB for conservative decision.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

Can you please convert msg336142 into a perf script? See the doc:
https://perf.readthedocs.io/en/latest/developer_guide.html

And then run again these benchmarks on PR 12052.

If you have a script, you can do:

./python-ref script.py -o ref.json
./python-untracked script.py -o untracked.json
./python-untracked -m perf compare_to ref.json untracked.json

https://perf.readthedocs.io/en/latest/cli.html#compare-to-cmd

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

> This optimization also can be used for BUILD_TUPLE opcode and in pickle 
> module, if it's OK to add _PyTuple_StealFromArray() function :-)

I would like to see a micro-benchmark showing that it's faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Armin Rigo]

Armin Rigo  added the comment:

Done.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36117] Allow rich comparisons for real-valued complex objects.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

NaN and complex numbers are not orderable by definition. This is a feature, not 
a bug.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36117] Allow rich comparisons for real-valued complex objects.

[Brandt Bucher]

Change by Brandt Bucher :


--
type: enhancement -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31904] Python should support VxWorks RTOS

[STINNER Victor]

STINNER Victor  added the comment:

Kuhl, Brian started a new discussion: [Python-Dev] VxWorks and cpython?
https://mail.python.org/pipermail/python-dev/2019-January/156024.html

PR 11968 and PR 12051 are small and reasonable.

IMHO we can take decisions on a case by case basic. But WindRiver plans to 
provide a buildbot and is already showing their will to propose PRs, so it 
seems like things are moving on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

@arigo 

Yep, I am interested because I would like to execute the tests of the major 
projects/libraries (django, flasks, pandas, requests, ...) and create issues 
for the maintainer.

the sooner we get feedback, the sooner we can fix the bugs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] [subinterpreters] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[STINNER Victor]

STINNER Victor  added the comment:

> ...so it doesn't appear that my PR introduces a performance regression.

IMHO there is no performance regression at all. Just noice in the result which 
doesn't come with std dev.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36117] Allow rich comparisons for real-valued complex objects.

[Brandt Bucher]

Brandt Bucher  added the comment:

> The rules for when things are comparable or not should be kept simple.

I think that the sort of user who uses complex numbers for their numerical 
calculations would still find this behavior "simple", but that may just be me.

> We don't want to have objects that are orderable depending on their values. I 
> can't think of anywhere else we do this.

Well... tuples and lists behave this way. ;)

> An application relying on this would be a nightmare to write comprehensive 
> tests for.

I'm not arguing that applications should rely on this behavior as core 
functionality, just that Python's more advanced math functionality deserves to 
be correct.

With that said, there are already so many weird limitations on complex numbers 
and floating-point values in general. I wouldn't expect users to treat 
comparing complex numbers any differently than they would treat comparing 
floats. Check check the .imag value in the former, check math.isnan in the 
latter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Armin Rigo]

Armin Rigo  added the comment:

Cool.  Also, no bugfix release of cffi was planned, but I can make one if you 
think it might help for testing the current pre-release of CPython 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36122] Second run of 2to3 continues to modify output

[bers]

bers  added the comment:

Yes, understood! Thanks for the explanation.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35892] Fix awkwardness of statistics.mode() for multimodal datasets

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> Proposed spec:
> '''
> Modify the API statistics.mode to handle multimodal cases so that the 
> first mode encountered is the one returned.  If the input is empty, 
> raise a StatisticsError.

Are you happy guaranteeing that it will always be the first mode 
encountered? I'm not happy about it, but I'll follow you lead on this 
one.

> TestCases:
> mode([])   --> StatisticsError
> mode('aabbbcc') --> 'c'

That should be 'b'.

> mode(iter('aabbbcc')) --> 'c'

And again 'b'.

> mode('eeffgg') --> 'a'

If it is first encountered, that should be 'd'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36122] Second run of 2to3 continues to modify output

[Paul Ganssle]

Paul Ganssle  added the comment:

Because what's being printed is a tuple, I think it's not exactly the same as 
issue35417, because in fact this is the correct behavior for 2to3, note that in 
Python 2:

Python 2.7.15 (default, Jul 21 2018, 11:13:03) 
>>> print 1, 2 
1 2
>>> print(1, 2)
(1, 2)

And in Python 3:

Python 3.7.2 (default, Feb  9 2019, 13:18:43) 
>>> print(1, 2)
1 2
>>> print((1, 2))
(1, 2)

I think this bug report is based on an understandable misunderstanding of what 
2to3 does - 2to3 is not intended to be idempotent or to generate code the works 
for both Python 2 and Python 3, it's intended to translate Python 2 code into 
Python 3, so passing it something that is *already Python 3 code* you are not 
guaranteed to get a meaningful output from it.

In this case, it first translates `print 1, 2` (Python 2) into `print(1, 2)` 
(Python 3), then when you run it a second time, it translates `print(1, 2)` 
(Python 2) into `print((1, 2))` (Python 3) - in both cases it's doing the right 
thing.

@bers I hope that this has helped to clarify the situation. Thank you for 
taking the time to report this.

--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21879] str.format() gives poor diagnostic on placeholder mismatch

[Carlos Damázio]

Carlos Damázio  added the comment:

Ops, someone already patched it! Sorry guys.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36124] Provide convenient C API for storing per-interpreter state

[Nick Coghlan]

Change by Nick Coghlan :


--
stage:  -> needs patch
type:  -> enhancement
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Nick Coghlan]

Nick Coghlan  added the comment:

Oh, cool (both the fact the issue here is only with building cffi itself, and 
that cffi creates extension modules that build with PY_LIMITED_API).

I've filed https://bugs.python.org/issue36124 to follow up on the 
PyInterpreter_GetDict API idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36124] Provide convenient C API for storing per-interpreter state

[Nick Coghlan]

New submission from Nick Coghlan :

(New issue derived from https://bugs.python.org/issue35886#msg336501 )

cffi needs a generally available way to get access to a caching dict for the 
currently active subinterpreter. Currently, they do that by storing it as an 
attribute in the builtins namespace: 
https://bitbucket.org/cffi/cffi/src/07d1803cb17b230571e3155e52082a356b31d44c/c/call_python.c?fileviewer=file-view-default

As a result, they had to amend their code to include the CPython internal 
headers in 3.8.x, in order to regain access to the "builtins" reference.

Armin suggested that a nicer way for them to achieve the same end result is if 
there was a PyInterpreter_GetDict() API, akin to 
https://docs.python.org/3/c-api/init.html#c.PyThreadState_GetDict

That way they could store their cache dict in there in 3.8+, and only use the 
builtin dict on older Python versions.

--
messages: 336670
nosy: ncoghlan
priority: normal
severity: normal
status: open
title: Provide convenient C API for storing per-interpreter state

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21879] str.format() gives poor diagnostic on placeholder mismatch

[Carlos Damázio]

Carlos Damázio  added the comment:

I've noticed this issue is quite not active, but I'm up on participating in it 
if anyone is doing it already. Since then, this issue still persists:

>>> '{1}'.format()
Traceback (most recent call last):
  File "", line 1, in 
IndexError: tuple index out of range

Reading further, I agree that improving the message is suitable. We should 
leave the Index Exception while changing the text to "Replacement index %d out 
of range for positional args tuple". Even though I find it quite extent for an 
exception message, I guess it couldn't be any other way, unless someone gives a 
better suggestion.

--
nosy: +dmzz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[Aaryn Tonita]

Aaryn Tonita  added the comment:

Although I am not personally interested in backporting a fix for this issue, 
anyone that experiences this issue in python 3.5 can execute the following 
monkey patch to solve the issue:

def _fix_issue_36041_3_5():
from email._header_value_parser import QuotedString, ValueTerminal, 
quote_string
import email._header_value_parser

class BareQuotedString(QuotedString):

token_type = 'bare-quoted-string'

def __str__(self):
return quote_string(''.join(str(x) for x in self))

@property
def value(self):
return ''.join(str(x) for x in self)

@property
def parts(self):
parts = list(self)
escaped_parts = []
for part in parts:
if isinstance(part, ValueTerminal):
escaped = quote_string(str(part))[1:-1]
escaped_parts.append(ValueTerminal(escaped, 'ptext'))
else:
escaped_parts.append(part)
# Add quotes to the first and last parts.
escaped_parts[0] = ValueTerminal('"' + str(escaped_parts[0]), 
'ptext')
escaped_parts[-1] = ValueTerminal(str(escaped_parts[-1] + '"'), 
'ptext')
return escaped_parts

email._header_value_parser.BareQuotedString = BareQuotedString

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[Aaryn Tonita]

Aaryn Tonita  added the comment:

Sorry about the delay. I opened pull request 
https://github.com/python/cpython/pull/12054 for this. Let me know if you need 
anything else.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36085] Enable better DLL resolution

[Nick Coghlan]

Nick Coghlan  added the comment:

As a note in favour of the "Allow package nesting to be encoded in names, not 
just directories" approach, we actually have a similar problem affecting 
builtin modules: they're currently limited to top-level modules, with no way 
for the module to indicate that it's actually part of the parent package. 
Details at https://bugs.python.org/issue1644818 (yes, that's a SourceForge era 
issue number).

The solutions may not overlap in practice, but they're conceptually related 
(i.e. outside frozen modules, the package topology is pretty tightly coupled to 
the underlying filesystem layout)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36120] Regression - Concurrent Futures

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Thank you.  I just tried running the example and it worked OK for me in the 3.8 
master build.  I believe your report is a duplicate of issue 35797.

--
resolution:  -> duplicate
stage: test needed -> resolved
status: open -> closed
superseder:  -> concurrent.futures.ProcessPoolExecutor does not work in venv on 
Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +12080
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Armin Rigo]

Armin Rigo  added the comment:

@nick the C sources produced by cffi don't change.  When they are compiled, 
they use Py_LIMITED_API so you can continue using a single compiled module 
version for any recent-enough Python 3.x.  The required fix is only inside the 
cffi module itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31916] ensurepip not honoring value of $(DESTDIR) - pip not installed

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Closing as third-party as @yan12125's report pinpoints the cause of the issue 
as part of pip.

--
nosy: +cheryl.sabella
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Nick Coghlan]

Nick Coghlan  added the comment:

Yep, I completely understand (and agree with) the desire to eliminate the code 
injection exploit that was introduced decades ago by using exec() to run lines 
starting with "import " (i.e. "import sys; ").

I just don't want to lose the "add this location to sys.path" behaviour that 
exists for lines in pth files that *don't* start with "import ", since that has 
plenty of legitimate use cases, and the only downside of overusing it is an 
excessively long default sys.path (which has far more consistent and obvious 
symptoms than the arbitrary code execution case can lead to).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

@nick which indirectly broke httpbin and this one is used by python-requests 
and we can't execute the tests of requests.

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Nick Coghlan]

Nick Coghlan  added the comment:

(On closer inspection, that's actually be the same breakage as already 
mentioned above)

However, what I'm not clear on is how this would affect projects that had 
*already* generated their cffi code, and include that in their sdist. Are all 
those sdists going to fail to build on Python 3.8 now?

It's OK to require that extension modules be rebuilt for a new release, but 
breaking compatibility with a *code generator* that means a broad selection of 
projects are all going to fail to build is a much bigger problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

[Nick Coghlan]

Nick Coghlan  added the comment:

Next incompatibility: https://github.com/python-hyper/brotlipy/issues/147 
(which indirectly broke httpbin)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
keywords: +patch
pull_requests: +12079
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36123] Race condition in test_socket

[Joannah Nanjekye]

New submission from Joannah Nanjekye :

Looking at the buildbot failures, there is a race condition in a test_socket 
test:

def _testWithTimeoutTriggeredSend(self):
address = self.serv.getsockname()
with open(support.TESTFN, 'rb') as file:
with socket.create_connection(address, timeout=0.01) as sock:
meth = self.meth_from_sock(sock)
self.assertRaises(socket.timeout, meth, file)

def testWithTimeoutTriggeredSend(self):
conn = self.accept_conn()
conn.recv(88192)

on slow buildbot, create_connection() fails with a timeout exception sometimes 
because the server fails to start listing in less than 10 ms. 

https://buildbot.python.org/all/#/builders/167/builds/597

==
ERROR: testWithTimeoutTriggeredSend (test.test_socket.SendfileUsingSendTest)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_socket.py",
line 5796, in testWithTimeoutTriggeredSend
conn = self.accept_conn()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_socket.py",
line 5607, in accept_conn
conn, addr = self.serv.accept()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/socket.py",
line 212, in accept
fd, addr = self._accept()
socket.timeout: timed out

==
ERROR: testWithTimeoutTriggeredSend (test.test_socket.SendfileUsingSendTest)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_socket.py",
line 335, in _tearDown
raise exc
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_socket.py",
line 353, in clientRun
test_func()
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_socket.py",
line 5791, in _testWithTimeoutTriggeredSend
with socket.create_connection(address, timeout=0.01) as sock:
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/socket.py",
line 727, in create_connection
raise err
  File 
"/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/socket.py",
line 716, in create_connection
sock.connect(sa)
socket.timeout: timed out

Note: Reported my Victor. I created the bug to track.

--
components: Tests
messages: 336658
nosy: nanjekyejoannah
priority: normal
severity: normal
status: open
title: Race condition in test_socket
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36121] csv: Non global alternative to csv.field_size_limit

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi Carlos, I think this can be added as a new `field_size_limit` keyword 
argument to csv.reader().

I will work on it.

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36116] test_multiprocessing_spawn fails on AMD64 Windows8 3.x

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36098] asyncio: ssl client-server with "slow" read

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for the report.

We definitely have to fix `AttributeError`.
Unfortunately, SSL implementation in asyncio is very tricky.

Yuri has an experimental asyncio ssl replacement in his uvloop project 
rewritten from scratch.
There is a plan to port it into asyncio itself.

P.S.
`await writer.drain()` doesn't send all data to buffer but ensures that the 
write buffer size is below high watermark limit 
(https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.set_write_buffer_limits)

Actually, it is good. Even after pushing a data into socket buffer you have no 
knowledge when the data is delivered to peer (or maybe not delivered at all at 
the moment of socket closing).
Moreover, when I experimented with write buffer disabling 
(transport.set_write_buffer_limits(1)) in aiohttp server benchmarks was 
executed about 50% slower for simple 'ping' request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36122] Second run of 2to3 continues to modify output

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This seems to be same as issue35417 and some resolution on detecting these type 
of cases at issue10375.

--
nosy: +benjamin.peterson, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36120] Regression - Concurrent Futures

[Jonathan]

Jonathan  added the comment:

The "ProcessPoolExecutor Example" on this page breaks for me:

https://docs.python.org/3/library/concurrent.futures.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36122] Second run of 2to3 continues to modify output

[bers]

New submission from bers :

I did this on Windows 10:

P:\>python --version
Python 3.7.2

P:\>echo print 1, 2 > Test.py

P:\>python Test.py
  File "Test.py", line 1
print 1, 2
  ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1, 2)?

P:\>2to3 -w Test.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored Test.py
--- Test.py (original)
+++ Test.py (refactored)
@@ -1 +1 @@
-print 1, 2
+print(1, 2)
RefactoringTool: Files that were modified:
RefactoringTool: Test.py

P:\>python Test.py
1 2

P:\>2to3 -w Test.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored Test.py
--- Test.py (original)
+++ Test.py (refactored)
@@ -1 +1 @@
-print(1, 2)
+print((1, 2))
RefactoringTool: Files that were modified:
RefactoringTool: Test.py

P:\>python Test.py
(1, 2)

Note how "print 1, 2" first becomes "print(1, 2)" (expected), then becomes 
"print((1, 2))" in the following run. This changes the output of Test.py

--
components: 2to3 (2.x to 3.x conversion tool)
messages: 336653
nosy: bers
priority: normal
severity: normal
status: open
title: Second run of 2to3 continues to modify output
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36120] Regression - Concurrent Futures

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Thank you for the report.  Please add a script that contains the least amount 
of code that would replicate the error.  Thanks!

>From the devguide:
> last but not least, you have to describe the problem in detail, including 
> what you expected to happen, what did happen, and how to replicate the 
> problem in the Comment field. Be sure to include whether any extension 
> modules were involved, and what hardware and software platform you were using 
> (including version information as appropriate).

--
nosy: +cheryl.sabella
stage:  -> test needed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com