[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I'm unassigning this.  It has been blown profoundly out of proportion.  The 
recommendation of the senior most developer is being ignored, and now two 
developers are speaking in terms of strong belief systems and labeling 
long-stable code as "evil".  This doesn't bode well and it makes it difficult 
to conduct reasoned discourse.  

(Meanwhile, we have another have another long-term active developer who 
routinely adds new macros to existing code because he thinks that is an 
improvement. That portends a pointless tug-of-war predicated almost solely in 
not liking how other people code).

--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

[Kyle Stanley]

Kyle Stanley  added the comment:

Is there a currently reliable way of accessing the GIL functions within the 
sub-interpreters, without causing deadlock issues? I was trying to follow the 
advice in the documentation 
(https://docs.python.org/3/c-api/init.html?highlight=global%20interpreter%20lock#bugs-and-caveats).

"It is highly recommended that you don’t switch sub-interpreters between a pair 
of matching PyGILState_Ensure() and PyGILState_Release() calls."

But it seemed that any attempt to use any of the PyGIL* calls within 
``interp_destroy()`` in a meaningful way resulted in a deadlock, even if it was 
done away from the sub-interpreter switching.

My next idea would be to add a conditional check to see if the current thread 
has ownership of the GIL, and using ``PyEval_RestoreThread()`` to acquire it if 
it doesn't. This would be followed by releasing the GIL with 
``PyThreadState_Get()`` at the end of the function. I'll try experimenting with 
that idea next.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

[Greg Price]

Greg Price  added the comment:

I hesitate to come back to this thread, because as Raymond says it's consumed a 
lot of time already.  But I think this point is of broader interest than the 
specific few lines we've been discussing:

> When small integer are disabled at compilation time (NSMALLPOSINTS=0 and  a 
> NSMALLNEGINTS=0), I suggest to remove code related to small integers. IMHO 
> CHECK_SMALL_INT() macro is a good practical solution for that.

Victor, can you be more specific about the problem this is solving? I think the 
existing code in master really doesn't leave any problems in place that 
CHECK_SMALL_INT solves.

In master (as of 2702638ea), here's the code that CHECK_SMALL_INT would replace:

if (IS_SMALL_INT(ival)) {
return get_small_int((sdigit)ival);
}

When NSMALLPOSINTS=0 and NSMALLNEGINTS=0 , this expands in the preprocessor to 
the equivalent of:

if (0) {
return (Py_UNREACHABLE(), NULL);
}

(Specifically, it expands to whatever that expands to, since Py_UNREACHABLE and 
possibly NULL are also macros.)

A compiler that's attempting to do any significant optimization at all has to 
be able to discard code that's inside `if (0)`; dead-code elimination is a 
necessary primitive for lots of other optimizations.

(And at a quick empirical sanity-check: gcc, clang, and msvc all do so at any 
optimization setting other than "disabled".  In fact gcc and clang do so even 
with optimizations disabled.)


You made the case very nicely above that macros are evil.  The IS_SMALL_INT 
macro is fairly mild, and it has a performance benefit on some platforms to 
justify it.  But CHECK_SMALL_INT causes a return from the enclosing function, 
which seems quite high on the "evil macro" scale.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

[Kyle Stanley]

Kyle Stanley  added the comment:

> I'm one of the first to advocate to replace ugly macros with clean static 
> inline functions. Macros are evil and can be too easily misused.

As someone who has only more recently started learning the C-API (and C in 
general), I'm certainly in favor of replacing macros with functions when 
possible. I might be a bit biased, but it definitely makes the code a lot 
easier to understand (especially the macros with implicit returns). As long as 
there's not a significant performance loss and it doesn't introduce new 
complications, I don't see an issue with it.

>From my understanding, readability isn't as high of a priority in the C code, 
>but certainly there's some benefit to improving areas that are more difficult 
>to understand. The easier the code is to understand, the lower the overall 
>maintenance cost becomes.

Of course, this should certainly be done in moderation to reduce the 
introduction of unnecessary new bugs and the cost in reviewer time for more 
pressing concerns.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38216] Fix for issue30458 prevents crafting invalid requests

[Ammar Askar]

Ammar Askar  added the comment:

> What bothers me here is that we apparently changed de facto behavior between 
> maintenance releases, in the middle of 3.7's lifecycle, without warning, no 
> doubt because we didn't realize it would break third-party packages.

Arguably, I think the programs that are affected by this vulnerability far 
outnumber the amount of third-party packages that will be broken. The trade-off 
here seems to be between the promise of compatibility and the promise of 
security, choosing compatibility strikes me as odd.

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

[Stefan Behnel]

Stefan Behnel  added the comment:

My usual first reaction is: "if you need to find out whether the return value 
of a callable will be an Awaitable or not, without calling it, then you're 
probably doing something wrong in your design".

However,
a) there is code that tries this already (and it falls short in various ways 
while trying)
b) asyncio has a function `iscoroutinefunction` which *seems* to fulfil this 
need but does not achieve it (because not everything that returns an Awaitable 
is a "coroutine function")
c) asyncio has an internal protocol for marking things as "is a coroutine", 
which comes close to but isn't "returns an Awaitable when called"

So – should there be an official protocol for marking callables as returning an 
Awaitable? Should we look at annotations for that? Anything else? Or do we 
consider this intention inherently flawed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38216] Fix for issue30458 prevents crafting invalid requests

[Ned Deily]

Ned Deily  added the comment:

Thanks for your comments, Greg.  Here's my take as release manager for 3.7 (and 
for 3.6).  What bothers me here is that we apparently changed de facto behavior 
between maintenance releases, in the middle of 3.7's lifecycle, without 
warning, no doubt because we didn't realize it would break third-party 
packages.  But it has and that's a big no-no.  A very important part of our 
maintenance strategy is that we implicitly promise our users that they can 
easily upgrade from any older release in a release family to the most current 
release without fear of incompatibilities (e.g. 3.7.0 to 3.7.4).  In return for 
that, we will only provide fixes for the most recent maintenance release in a 
family (e.g. once 3.7.5 is released, 3.7.4 is dead and 2.7.3 through 3.7.0 were 
already dead).  So it seems here we have violated that compatibility promise 
for 3.7 and 3.6 and are about to do so for 3.5 and 2.7.  Since at least one 
project is known to have been impacted, it's not unreasonable to expect that 
more will be.  So I think we should avoid such breakage and undo the change in 
behavior for 3.7 (and the older releases as well).

Now, as for 3.8, as it hasn't released yet, we have more latitude and, although 
we're close to producing an RC, it may still be OK to document the changed 
behavior there.  That should be Łukasz's call.

Does that make sense?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38093] Update MagicMock __aenter__ and __aexit__ to return AsyncMock's

[Lisa Roach]

Lisa Roach  added the comment:


New changeset 8b03f943c37e07fb2394acdcfacd066647f9b1fd by Lisa Roach in branch 
'master':
bpo-38093: Correctly returns AsyncMock for async subclasses. (GH-15947)
https://github.com/python/cpython/commit/8b03f943c37e07fb2394acdcfacd066647f9b1fd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38161] AsyncMock add `.awaited` like `.called`

[Lisa Roach]

Lisa Roach  added the comment:

Yeah I think the current `awaited` ought to at least be a private variable, 
since people will assume it works like a boolean to match `.called`. Then we 
could add a new, proper, `awaited` that would function like:

>>> mock = AsyncMock()
>>> cr_mock = mock()
>>> mock.called
True
>>> mock.awaited
False
>>> await cr_mock
>>> mock.awaited
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38216] Fix for issue30458 prevents crafting invalid requests

[Gregory P. Smith]

Gregory P. Smith  added the comment:

What's needed here is a Decision.  (release managers and steering councils make 
those)

IMNSHO, this regression is intentional and does not feel like a bug.

The Python HTTP APIs were never designed with an explicit intent to allow 
violations of the protocol.  That it was "allowed" was an accident because the 
very old original API design didn't go out of its way to check much.  That 
doesn't mean it was intended.  Hyrum's Law applies as the (lack of) behavior 
has been around forever.  So of course someone has written code that depended 
on that.  But it doesn't make said code right.

We either "be strict in what we produce" or we ignore abuse of the APIs and 
effectively close all CVEs filed against them as "not a bug, this library is 
intended to allow abuse when given untrusted input."

If we take the latter route and intended to allow such things, we'd need to 
explicitly document it as such and guarantee their behaviors.  As noted, 
disallowing it has already shipped in two stable releases.

When writing tests of particular out of spec queries such that servers can 
implement good "be lenient in what you accept" behavior, it is better not to 
depend on specific behaviors of a given http client library to allow you to do 
so for such tests.

If the decision requires code changes, do not expect me to spend time creating 
them.  Please feel free to loop me in on PR reviews but I'd rather not review 
anything without release manager approval of the direction they take things in 
the first place.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34002] minor efficiency and clarity improvements in email package

[Abhilash Raj]

Abhilash Raj  added the comment:

PR is now merged.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34002] minor efficiency and clarity improvements in email package

[Abhilash Raj]

Abhilash Raj  added the comment:


New changeset 2702638eabe5f7b25f36d295f0ad78cb8d4eda05 by Abhilash Raj (Michael 
Selik) in branch 'master':
bpo-34002: Minor efficiency and clarity improvements in email package. (GH-7999)
https://github.com/python/cpython/commit/2702638eabe5f7b25f36d295f0ad78cb8d4eda05


--
nosy: +maxking

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38228] Missing documentation on strftime modifier O

[Eryk Sun]

Eryk Sun  added the comment:

> Have you tried this on Windows or macOS?

3.5+ in Windows uses ucrt, which quietly ignores 'E' and 'O' strftime 
modifiers. From ucrt\time\wcsftime.cpp:

// Skip ISO E and O alternative representation format modifiers.  We
// do not support alternative formats in any locale.
if (*format_it == L'E' || *format_it == L'O')
{
++format_it;
}

Example:

>>> locale.setlocale(locale.LC_ALL, 'ca_ES.utf8')
'ca_ES.utf8'
>>> time.strftime('%b|%Ob')
'set.|set.'

2.7 uses the old MSVC runtime, in which the 'E' and 'O' modifiers are invalid.

Example:

>>> locale.setlocale(locale.LC_ALL, 'cat_esp')
'Catalan_Spain.1252'
>>> time.strftime('%b')
'set.'
>>> time.strftime('%Ob')
Traceback (most recent call last):
  File "", line 1, in 
ValueError: Invalid format string

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38229] ClientConnectionError exception doesn't serialize property

[David Parks]

David Parks  added the comment:

Oh yes, this belongs to aiohttp. I thought that was handled here. I'll move 
this to a git issue on that page, and I'm closing this.

https://github.com/aio-libs/aiohttp/issues/4077

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed
title: ClientConnectionError exception doesn't serialize propertly -> 
ClientConnectionError exception doesn't serialize property

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[Kyle Stanley]

Kyle Stanley  added the comment:

> Thanks, Kyle!

No problem, and thanks for all of the help from Andrew, Yury, and Victor!

> IMHO it will make asyncio more reliable, especially for tests on the CI.

Awesome, that was my primary intention. (:

> If it becomes an issue in Python 3.9 (executor hangs forever),

Feel free to add me to the nosy list if you have to open an issue for it, I'd 
be glad to help out with this if it becomes an issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38229] ClientConnectionError exception doesn't serialize propertly

[ppperry]

ppperry  added the comment:

This seems more likely to be a bug in aiohttp than in python.

--
nosy: +ppperry
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38228] Missing documentation on strftime modifier O

[Eric V. Smith]

Eric V. Smith  added the comment:

Have you tried this on Windows or macOS?

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38229] ClientConnectionError exception doesn't serialize propertly

[David Parks]

David Parks  added the comment:

Minor correction to the minimal reproducible test case:

```
import multiprocessing
import aiohttp


connection_key = aiohttp.client_reqrep.ConnectionKey
ose = OSError('test')

e = aiohttp.client_exceptions.ClientConnectorError(connection_key, ose)
q = multiprocessing.Queue()

q.put(e)
q.get()
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38229] ClientConnectionError exception doesn't serialize propertly

[David Parks]

New submission from David Parks :

Original question posted here:

https://stackoverflow.com/questions/58019939/attributeerror-str-object-has-no-attribute-errno?noredirect=1#comment102443264_58019939

The following exception is encountered when placing an 
`aiohttp.client_exceptions.ClientConnectionError` on a `multiprocessing.Queue`. 
The problem appears to occur during deserialization.

```
Traceback (most recent call last):
  File "model_neural_simplified.py", line 318, in 
main(**arg_parser())
  File "model_neural_simplified.py", line 314, in main
globals()[command](**kwargs)
  File "model_neural_simplified.py", line 304, in predict
next_neural_data, next_sample = reader.get_next_result()
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 174, in 
get_next_result
result = future.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in 
__get_result
raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
  File "model_neural_simplified.py", line 245, in read_sample
f_bytes = s3f.read(read_size)
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 374, in read
size, b = self._issue_request(S3Reader.READ, (self.url, size, 
self.position))
  File "/project_neural_mouse/src/asyncs3/s3reader.py", line 389, in 
_issue_request
response = self.communication_channels[uuid].get()
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/client_exceptions.py", 
line 133, in __init__
super().__init__(os_error.errno, os_error.strerror)
AttributeError: 'str' object has no attribute 'errno'
```

A similar issue is reproducible using the following code snippet:

```
import multiprocessing
import aiohttp


connection_key = aiohttp.client_reqrep.ConnectionKey
ose = OSError('test')

e = aiohttp.client_exceptions.ClientConnectorError(connection_key, ose)
q = multiprocessing.Queue()

q.put(e)
q.get(e)
```

--
components: asyncio
messages: 352818
nosy: asvetlov, davidparks21, yselivanov
priority: normal
severity: normal
status: open
title: ClientConnectionError exception doesn't serialize propertly
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I think we need better name than just `_is_coroutine`.
All async function properties as dunder named, the new *official* name should 
follow this convention as well

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38228] Missing documentation on strftime modifier O

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +belopolsky, p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38228] Missing documentation on strftime modifier O

[Joan]

New submission from Joan :

In the documentation at 
https://docs.python.org/3.9/library/datetime.html#strftime-strptime-behavior 
there's an omission of one of the modifiers that can be add to the strftime 
parameters (O and E)


Quoting from  http://man7.org/linux/man-pages/man3/strftime.3.html


Some conversion specifications can be modified by preceding the
conversion specifier character by the E or O modifier to indicate
that an alternative format should be used.  If the alternative format
or specification does not exist for the current locale, the behavior
will be as if the unmodified conversion specification were used. (SU)
The Single UNIX Specification mentions %Ec, %EC, %Ex, %EX, %Ey, %EY,
%Od, %Oe, %OH, %OI, %Om, %OM, %OS, %Ou, %OU, %OV, %Ow, %OW, %Oy,
where the effect of the O modifier is to use alternative numeric
symbols (say, roman numerals), and that of the E modifier is to use a
locale-dependent alternative representation.


The modifier works as expected for the O modifier returning the values defined 
in ab_alt_mon and alt_mon from the locale (instead of the ones defined in abmon 
and mon.
I haven't been able to get any results with the E modifier (might not be yet 
defined in any locale)

A small snippet of code to see the difference:
import locale
from datetime import datetime
locale.setlocale(locale.LC_ALL, 'ca_AD.utf8')
locale.setlocale(locale.LC_ALL, 'ca_ES.utf8')
now = datetime.now() # current date and time
date_time = now.strftime("|%Ob|%b|||%OB|%B|")
print("date and time:",date_time)

--
assignee: docs@python
components: Documentation
messages: 352816
nosy: aseques, docs@python
priority: normal
severity: normal
status: open
title: Missing documentation on strftime modifier O
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38227] Setting a signal handler gets multiprocessing.Pool stuck

[Ionel Cristian Mărieș]

New submission from Ionel Cristian Mărieș :

Running `python3.8 mp-bug-python2.8.py` usually gets stuck after a dozen 
iterations or so.

It appears that if I stop setting that signal handler it doesn't get stuck. 
Unfortunately I need it to perform critical cleanup.

This is what I got from gdb:

(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 95, in 
__enter__
return self._semlock.__enter__()
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 355, in get
with self._rlock:
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 370, in worker
`func` and (a, b) becomes func(a, b).
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/process.py", line 569, in _bootstrap
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
  File "/usr/lib/python3.8/multiprocessing/context.py", line 276, in _Popen
return Popen(process_obj)
  File "/usr/lib/python3.8/multiprocessing/process.py", line 633, in start
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 838, in 
_repopulate_pool_static
self._length = None
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 559, in 
_repopulate_pool
outqueue.put(None)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 212, in __init__
self._repopulate_pool()
  File "/usr/lib/python3.8/multiprocessing/context.py", line 375, in Pool
  File "mp-bug-python2.8.py", line 21, in 



And without the macros:



#0  0x7f51c79fb6d6 in futex_abstimed_wait_cancelable (private=128, 
abstime=0x0, expected=0, futex_word=0x7f51c7e29000) at 
../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x7f51c7e29000, abstime=0x0) at 
sem_waitcommon.c:111
#2  0x7f51c79fb7c8 in __new_sem_wait_slow (sem=0x7f51c7e29000, abstime=0x0) 
at sem_waitcommon.c:181
#3  0x7f51c79fb839 in __new_sem_wait (sem=) at sem_wait.c:42
#4  0x7f51c58d51f7 in semlock_acquire (self=0x7f51c58a60b0, args=, kwds=) at ./Modules/_multiprocessing/semaphore.c:319
#5  0x0067d929 in method_vectorcall_VARARGS_KEYWORDS 
(func=, args=0x7f51c58a4eb8, 
nargsf=, kwnames=)
at ../Objects/descrobject.c:332
#6  0x0042c5dc in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=) at 
../Include/cpython/abstract.h:127
#7  call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x22bd160) at ../Python/ceval.c:4987
#8  _PyEval_EvalFrameDefault (f=, throwflag=) at 
../Python/ceval.c:3486
#9  0x00425ed7 in function_code_fastcall (co=, 
args=, nargs=1, globals=) at ../Objects/call.c:283
#10 0x006761e4 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, 
args=0x7ffe20bcc908, callable=) at 
../Include/cpython/abstract.h:127
#11 method_vectorcall (method=, args=, 
nargsf=, kwnames=0x0) at ../Objects/classobject.c:67
#12 0x0042857e in _PyObject_Vectorcall (kwnames=0x0, nargsf=0, 
args=0x0, callable=) at 
../Include/cpython/abstract.h:127
#13 _PyObject_CallNoArg (func=) at 
../Include/cpython/abstract.h:153
#14 _PyEval_EvalFrameDefault (f=, throwflag=) at 
../Python/ceval.c:3287
#15 0x00425ed7 in function_code_fastcall (co=, 
args=, nargs=1, globals=) at ../Objects/call.c:283
#16 0x00676245 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1, 
args=0x23cea18, callable=) at 
../Include/cpython/abstract.h:127
#17 method_vectorcall (method=, args=0x23cea20, 
nargsf=, kwnames=0x0) at ../Objects/classobject.c:60
#18 0x0042cf2f in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=) at 
../Include/cpython/abstract.h:127
#19 call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x22bd160) at ../Python/ceval.c:4987
#20 _PyEval_EvalFrameDefault (f=, throwflag=) at 
../Python/ceval.c:3500
#21 0x004f9f44 in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x23ce820, for file /usr/lib/python3.8/multiprocessing/pool.py, 
line 370, in worker (inqueue=, 
_writer=, _rlock=, acquire=, release=) at remote 
0x7f51c589d0d0>, _poll=, 
_wlock=, 
acquire=, release=) at remote 0x7f51c589deb0>) at remote 
0x7f51c589d3a0>, outqueue=, globals=, 
locals=locals@entry=0x0, args=, argcount=6, kwnames=0x0, 
kwargs=0x7f51c636e628, kwcount=0, kwstep=1,
defs=0x7f51c65a4828, defcount=4, kwdefs=0x0, closure=0x0, name='worker', 
qualname='worker') at ../Python/ceval.c:4298
#23 0x0043e7a2 in _PyFunction_Vectorcall (func=func@entry=, stack=, nargsf=nargsf@entry=6, 
kwnames=) at ../Objects/call.c:435
#24 0x0044143c in PyVectorcall_Call (callable=, tuple=, kwargs=) at 
../Objects/call.c:199
#25 0x00428aec in do_call_core (kwdict={},
callargs=(, _writer=, 
_rlock=, 
acquire=, release=) at remote 0x7f51c589d0d0>, _poll=, _wlock=, acquire=, 

[issue38222] pathlib Path objects should support __format__

[Brett Cannon]

Brett Cannon  added the comment:

I'm -1 as PEP 519 created __fspath__ specifically so that pathlib.Path wouldn't 
be confused as a string by accident. You can also use os.fspath() to get a 
string representation of the path itself.

I also don't know if Antoine wants to make this sort of call for pathlib 
anymore as he isn't listed in https://devguide.python.org/experts/ or 
https://github.com/python/cpython/blob/master/.github/CODEOWNERS for pathlib 
anymore (and that was an explicit choice based on code history for the experts 
index).

--
nosy: +brett.cannon
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38226] pickle.dump and load error message when file isn't opened in binary mode are not useful

[Борис Верховский]

New submission from Борис Верховский :

pickle.load() and pickle.dump() take a file object as an argument. That file 
object must be opened in binary mode ("rb" or "wb"). If it's not, pickle raises 
a confusing error message.

When pickle.dump()'ing to a file opened in "w" mode instead of "wb" mode it'll 
raise a TypeError:


TypeError: write() argument must be str, not bytes


I thought it might be getting this because I was pickling bytes objects. 

For pickle.load()'ing from a file opened in "r" mode instead of "rb" mode, 
it'll raise a UnicodeDecodeError


UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid 
start byte


There are a couple pages on the internet (just search "pickle" followed by 
either error message) with people being confused about this. 

pickle should catch these errors and report something more useful.

--
components: Library (Lib)
messages: 352813
nosy: boris
priority: normal
severity: normal
status: open
title: pickle.dump and load error message when file isn't opened in binary mode 
are not useful
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

[Zachary Ware]

Change by Zachary Ware :


--
nosy: +scoder
versions:  -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15363] Idle/tkinter ~x.py 'save as' fails. closes idle

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21937] IDLE interactive window doesn't display unsaved-indicator

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23544] IDLE hangs when selecting Stack View with debug active

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -kbk, roger.serwy
versions: +Python 3.9 -Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21674] Idle: Add 'find all' in current file

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24812] All standard keystrokes not recognized in IDLE dialogs on Mac

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -kbk, roger.serwy
versions: +Python 3.9 -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29926] IDLE: in shell, time.sleep ignores _thread.interrupt_main()

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

[David Hilton]

Change by David Hilton :


--
keywords: +patch
pull_requests: +15877
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16292

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38225] iscoroutinefunction broken with cython - allow tagging of functions as async?

[David Hilton]

New submission from David Hilton :

If a python piece of code imports cython code with async defs, 
`asyncio.iscoroutinefunction` cannot determine that the code is async.

https://github.com/cython/cython/issues/2273#issuecomment-531537624

scoder is open to marking async defs so that they can be identified, just like 
`asyncio.coroutine`:

https://github.com/python/cpython/blob/ae239f6b0626e926613a4a1dbafa323bd41fec32/Lib/asyncio/coroutines.py#L156

However, that is an internal interface and `@coroutine` is deprecated.

--

Can we have some official way of marking functions as async that will not be 
deprecated?

The easiest would be for `asyncio.iscoroutinefunction` to look for 
`_is_coroutine = True`, and promise to keep looking for that value.

This would also allow for functools.partial to easily mark that it's returning 
an async function, which some people seem to care about.

--
components: asyncio
messages: 352812
nosy: asvetlov, dhiltonp, yselivanov
priority: normal
severity: normal
status: open
title: iscoroutinefunction broken with cython - allow tagging of functions as 
async?
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22083] IDLE: Refactor breakpoint related methods

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: Refactor PyShell's breakpoint related methods -> IDLE: Refactor 
breakpoint related methods
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31930] IDLE: Pressing "Home" on Windows places cursor before ">>>"

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.9 -Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21359] IDLE macOS: Some Command shortcuts do not work correctly

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Still an issue.  Several shortcuts do not work right, so I am generalizing the 
issue. 

1. Redo command accelerator acts as Undo.
2. Cmd-M minimizes instead of open module (#35104 has long discussion). 

I at least one other issue mentions a similar problem.  The default mac 
keybinding should all be tested to determine the scope of the problem.  Then a 
new keyset designed (#32761)

3. #18444 and #32761 both discuss ^A and beginning of line.

--
title: IDLE Redo command accelerator acts as Undo with current OS X Cocoa Tk 
8.5.15 -> IDLE macOS: Some Command shortcuts do not work correctly
versions: +Python 3.9 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35104] IDLE: On macOS, Command-M minimizes & opens "Open Module..."

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I am closing this as a duplicate of #21359 as I believe there is at least one 
more non-working shortcut and because I believe the solution is to test all and 
patch the default mac keyset all at once.

--
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> IDLE Redo command accelerator acts as Undo with current OS X 
Cocoa Tk 8.5.15

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38140] Py_tp_dictoffset / Py_tp_finalize are unsettable in stable API

[Dino Viehland]

Change by Dino Viehland :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38045] enum.Flag instance creation is slow

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +15876
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16291

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15348] IDLE - shell becomes unresponsive if debugger windows is closed while active.

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The input glitch mentioned above no longer exists, so closing.

--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15347] IDLE - remove debugger 'interacting'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

fix-nested-mainloop.patch was superceded by fix-nested2.patch for #24455.
I verified that the initial test now passes.  When I click 'yes' in that box 
popped up by step 4, both Shell and debugger windows disappear and IDLE exists.

The only question left is whether to apply remove-interacting-debugger.patch, 
which was aimed at the deprecated but not removed -n mode.

--
nosy:  -Saimadhav.Heblikar, roger.serwy
title: IDLE - does not close if the debugger was active -> IDLE - remove 
debugger 'interacting'
versions: +Python 3.9 -Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38224] Windows embeddable distribution doesn't ship headers

[Steve Dower]

Steve Dower  added the comment:

The embeddable distro is meant to be a "redistributable" for end-user machines, 
rather than a development kit. "Build" and "distribution" are two clearly 
separated tasks (at least on Windows), and so they have different dependencies. 
We won't be adding headers to the embeddable distro.

To get the headers, you'll either want the regular installer or the package 
from https://www.nuget.org/packages/python (or pythonx86). Or you can grab the 
Include folder from git, provided you use PC/pyconfig.h and the correct tag, as 
none of the files are "cooked" on Windows.

To directly address your two examples:
- the linked SCons script is a build script, which should be run on a build 
machine against a development kit. But when you copy the built result to 
another machine, they'll only require the embeddable distro at runtime
- the "frozen" distribution will need the new .exe binary to be built against a 
development kit. But when you copy the built result to another machine, they'll 
only require the embeddable distro at runtime.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38208] Simplify string.Template by using __init_subclass__()

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5e1400a6bcbb3350a6665176980a2b8343075c63 by Victor Stinner in 
branch '3.7':
bpo-37531: sync regrtest with master branch (GH-16285) (GH-16289)
https://github.com/python/cpython/commit/5e1400a6bcbb3350a6665176980a2b8343075c63


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38140] Py_tp_dictoffset / Py_tp_finalize are unsettable in stable API

[Dino Viehland]

Dino Viehland  added the comment:


New changeset 3368f3c6ae4140a0883e19350e672fd09c9db616 by Dino Viehland (Eddie 
Elizondo) in branch 'master':
bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)
https://github.com/python/cpython/commit/3368f3c6ae4140a0883e19350e672fd09c9db616


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15875
pull_request: https://github.com/python/cpython/pull/16290

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15874
pull_request: https://github.com/python/cpython/pull/16289

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33694] test_asyncio: test_start_tls_server_1() fails on Python on x86 Windows7 3.7 and 3.x

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15873
pull_request: https://github.com/python/cpython/pull/16288

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset fb7746d5d10ec4a34198da672018ba15f5667079 by Victor Stinner in 
branch '3.8':
bpo-37531: sync regrtest with master branch (GH-16285)
https://github.com/python/cpython/commit/fb7746d5d10ec4a34198da672018ba15f5667079


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +15872
pull_request: https://github.com/python/cpython/pull/16287

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38147] add Py_ASSUME() macro for __builtin_unreachable()

[STINNER Victor]

Change by STINNER Victor :


--
title: add macro for __builtin_unreachable -> add Py_ASSUME() macro for 
__builtin_unreachable()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38147] add macro for __builtin_unreachable

[STINNER Victor]

STINNER Victor  added the comment:

> Here's example: 
> https://github.com/sir-sigurd/cpython/commit/c8699d0c614a18d558216ae7d432107147c95c28.

"_Py_ASSUME((size_t)size <= MAX_LONG_DIGITS);"

Typically, such code use assert() and is removed for release build.

assert() is more for contract base programming: when the error "cannot" happen 
at runtime (it would be a programming error).

For other cases, I prefer to always emit code to handle the error (the error 
can happen, for example, the function must check inputs), even in release build.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38216] Fix for issue30458 prevents crafting invalid requests

[Larry Hastings]

Larry Hastings  added the comment:

FWIW I planned to tag and release 3.5.8 final early next week.  I don't have 
the domain knowledge to assess the severity of this bug--much less pitch in and 
help fix it--so I suspect this will simply hold up 3.5.8 final.

Depending on the complexity of the fix for this issue, I may also insert a 
second rc into the 3.5.8 schedule.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +15871
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16286

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Miki Tebeka]

Miki Tebeka  added the comment:

I don't think it violates " Explicit is better than implicit."
There's a lot of work done to make pathlib.Path objects work in places where 
str or bytes is expected (e.g PEP 519), IMO this is another case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38147] add macro for __builtin_unreachable

[Sergey Fedoseev]

Sergey Fedoseev  added the comment:

> If you care of _PyLong_Copy() performance, you should somehow manually inline 
> _PyLong_New() inside _PyLong_Copy().

It doesn't solve this:

> We could add a function that bypass that check, but in LTO build 
> PyObject_MALLOC() is inlined into _PyLong_New() and it also checks the size. 
> Adding Py_ASSUME((size_t)size <= MAX_LONG_DIGITS) allows to bypass both 
> checks.

Here's example: 
https://github.com/sir-sigurd/cpython/commit/c8699d0c614a18d558216ae7d432107147c95c28.

I attach some disassembly from this example compiled with LTO, to demonstrate 
how the proposed macro affects generated code.

--
Added file: https://bugs.python.org/file48614/disasm.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25130] Make tests more PyPy compatible

[Ronan Lamy]

Change by Ronan Lamy :


--
nosy: +Ronan.Lamy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

_Py_NO_RETURN is a promise that the code past the function is unreachable, not 
that the function call is unreachable.

> I'm not sure how __builtin_unreachable could be used with Py_UNREACHABLE() 
> macro.

In the release mode Py_UNREACHABLE() can be replaced with 
__builtin_unreachable().

> For example, if a function accepts an enum, but is called with a value which 
> is not part of the enum: what should happen? Should Python crash?

If the accessibility of the code depends on the call from Python code, we 
should raise normal exception. If it is not possible to pass it out of the 
function, we should use PyErr_WriteUnraisable().

If the accessibility of the code depends on the call from C code, 
we can use PyErr_BadInternalCall() or raise a SystemError directly.

If we are in the situation where recovering from errors is not possible (for 
example if the memory manager is broken) we use Py_FatalError().

If we are absolutely sure that the code never can be executed (unless memory 
was corrupted or something like), we use Py_UNREACHABLE(). It will silence 
compiler complains, fail loudly in debug mode if our assumption is wrong (this 
is our programming bug), and allow the compiler to optimize out the code if 
substituted by __builtin_unreachable() in the release mode.

> I don't see how the compiler can guess that the code is never executed with 
> the current macro.

But we can make the macro to expand to __builtin_unreachable in the release 
mode. Or just to no-op if there is other way to silence the compiler warning.

> Using a function allows to put a breakpoint on it.

You can put a brackpoint on Py_FatalError().

> In fact, I can easily modify PR 16280 to keep the macro, since I only call 
> Py_FatalError() with a string.

It would be nice. We can consider using __builtin_unreachable() in different 
issue. I am also going to use Py_UNREACHABLE() more widely to write more 
uniform, but still optimal code. It could replace

if (cond1) { ...
} else if (cond2) { ...
} else if (cond3) { ...
} else {
assert (!cond4);
...
}

with

if (cond1) { ...
} else if (cond2) { ...
} else if (cond3) { ...
} else if (cond4) { ...
} else {
Py_UNREACHABLE();
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[STINNER Victor]

STINNER Victor  added the comment:

asyncio.run() now shutdowns the default executor: nice, thanks! That was my 
request. IMHO it will make asyncio more reliable, especially for tests on the 
CI.

If it becomes an issue in Python 3.9 (executor hangs forever), we can add new 
parameters (to run()?) to put a timeout for example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 079931d12223ec98cbf53185b90db48efa61f93f by Victor Stinner in 
branch 'master':
bpo-34037: test_asyncio uses shutdown_default_executor() (GH-16284)
https://github.com/python/cpython/commit/079931d12223ec98cbf53185b90db48efa61f93f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15870
pull_request: https://github.com/python/cpython/pull/16285

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b2dd2dd6e9212ba6b03885b998736a180ef283d6 by Victor Stinner in 
branch 'master':
bpo-37531: Skip test_regrtest.test_multiprocessing_timeout() on all platforms 
(GH-16282)
https://github.com/python/cpython/commit/b2dd2dd6e9212ba6b03885b998736a180ef283d6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Eric V. Smith]

Eric V. Smith  added the comment:

I'm also -0 on it. It's up to Antoine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am -0 on this. It is a can of worms. Once we add a trivial __format__ in 
Path, we will need to handle numerous requests for adding a trivial __format__ 
in many other classes. It is better to teach users to use !s (and this works 
also on older Python versions!).

As for Path, there many ways to convert it to string: str(), repr(), 
os.fspath(), os.fsdecode(), Path.as_posix(), Path.as_uri(), with possible 
normalization, converting to absolute or relative path. I would not bet that we 
will never have reasons to have a different format specifier language for Path.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15869
pull_request: https://github.com/python/cpython/pull/16284

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38147] add macro for __builtin_unreachable

[STINNER Victor]

STINNER Victor  added the comment:

> This check is redundant when _PyLong_New() is called from _PyLong_Copy().

If you care of _PyLong_Copy() performance, you should somehow manually inline 
_PyLong_New() inside _PyLong_Copy().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38147] add macro for __builtin_unreachable

[STINNER Victor]

STINNER Victor  added the comment:

> Real world example. _PyLong_Copy() [1] calls _PyLong_New() [2]. _PyLong_New() 
> checks the size, so that overflow does not occur. This check is redundant 
> when _PyLong_New() is called from _PyLong_Copy(). We could add a function 
> that bypass that check, but in LTO build PyObject_MALLOC() is inlined into 
> _PyLong_New() and it also checks the size. Adding Py_ASSUME((size_t)size <= 
> MAX_LONG_DIGITS) allows to bypass both checks. 

This sounds like a bad usage of __builtin_unreachable().

_PyLong_New() must always check that size <= MAX_LONG_DIGITS, the check must 
not be optimized by the compiler.

__builtin_unreachable() must only be used if the code really be reached by 
design.

For example:

if (...) { Py_FatalError("oops)"; __builtin_unreachable() }

But it's a bad example, since Py_FatalError is decorated with the "noreturn" 
attribute, so the compiler should already know that Py_FatalError() never 
returns.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38155] Add __all__ to the datetime module

[Paul Ganssle]

Paul Ganssle  added the comment:

Closing this as resolved. I don't think we should backport this, as it's more 
of an enhancement than a bug fix (and since no one has ever complained about it 
to my knowledge, I don't think there's any big rush to see this released).

Thanks Tahia and all the reviewers!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38155] Add __all__ to the datetime module

[Paul Ganssle]

Paul Ganssle  added the comment:


New changeset 96b1c59c71534db3f0f3799cd84e2006923a5098 by Paul Ganssle (t k) in 
branch 'master':
bpo-38155: Add __all__ to datetime module (GH-16203)
https://github.com/python/cpython/commit/96b1c59c71534db3f0f3799cd84e2006923a5098


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 9fdc64cf1266b6d5bf0503847b5c38e5edc53a14 by Andrew Svetlov (Kyle 
Stanley) in branch 'master':
bpo-34037: Fix test_asyncio failure and add loop.shutdown_default_executor() 
(GH-15735)
https://github.com/python/cpython/commit/9fdc64cf1266b6d5bf0503847b5c38e5edc53a14


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thnks, Kyle!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Eric V. Smith]

Eric V. Smith  added the comment:

I don't think anyone is suggesting reverting the decision on object.__format__. 
That decision should stay.

I think the suggestion is to add Path.__format__, which just converts to a 
string and formats with the given spec.

Unless someone can think of a reason to have a different format specifier 
language for Path, I think this is a good usability improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13743] xml.dom.minidom.Document class is not documented

[Henry Harutyunyan]

Change by Henry Harutyunyan :


--
nosy: +hharutyunyan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[Sergey Fedoseev]

Sergey Fedoseev  added the comment:

FWIW I proposed to add Py_ASSUME() macro that uses __builtin_unreachable() in 
bpo-38147.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Or use !s or !r.

In some cases it may be even better to use both convertions: 
f'{str(path)!r:>50}'.

This is all application specific, so I do not think we should rethink our old 
decision. Explicit is better than implicit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6559] add pass_fds paramter to subprocess.Popen()

[Orivej Desh]

Change by Orivej Desh :


--
pull_requests: +15868
pull_request: https://github.com/python/cpython/pull/16283

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15867
pull_request: https://github.com/python/cpython/pull/16282

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Eric V. Smith]

Eric V. Smith  added the comment:

Correct: this change was made specifically so that objects could add their own 
format specifiers at a later time. If I recall correctly, This change was 
precipitated by wanting to add datetime.__format__: this change broke existing 
uses for format() that formatted datetimes as strings.

But the result is that objects that really just want to be formatted as strings 
need to have their own __format__ which returns format(str(self), format_spec).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[STINNER Victor]

STINNER Victor  added the comment:

> While it is a macro, it can be made a no-op, or even with compiler-specific 
> instructions like __builtin_unreachable.

My PR 16280 uses _Py_NO_RETURN which uses __attribute__((__noreturn__)) with 
GCC and clang.

I'm not sure how __builtin_unreachable could be used with Py_UNREACHABLE() 
macro.


> While it is a macro, it can be made a no-op

I understood that Py_UNREACHABLE() is used on purpose to prevent undefined 
behavior. For example, if a function accepts an enum, but is called with a 
value which is not part of the enum: what should happen? Should Python crash? 
Usually, we try to be nice and return an error. But sometimes, you cannot 
report an error and so Py_UNREACHABLE() is a good solution.


> I prefer to keep it a macro. The compiler does not know that it is never 
> executed, so it can generate a suboptimal code.

I don't see how PR 16280 could have an effect on that. I don't see how the 
compiler can guess that the code is never executed with the current macro.

--

Using a function allows to put a breakpoint on it.

In fact, I can easily modify PR 16280 to keep the macro, since I only call 
Py_FatalError() with a string. The function body is simple.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I prefer to keep it a macro. The compiler does not know that it is never 
executed, so it can generate a suboptimal code.

While it is a macro, it can be made a no-op, or even with compiler-specific 
instructions like __builtin_unreachable. This can help the compiler to generate 
more optimal code. For example, the popular idiom:

switch (kind) {
case PyUnicode_1BYTE_KIND: {
...
break;
}
case PyUnicode_2BYTE_KIND: {
...
break;
}
case PyUnicode_4BYTE_KIND: {
...
break;
}
default: Py_UNREACHABLE();
}

could be compiled to the code equivalent to:

if (kind == PyUnicode_1BYTE_KIND) {
...
break;
}
else if (kind == PyUnicode_2BYTE_KIND) {
...
break;
}
else { // assuming (kind == PyUnicode_4BYTE_KIND)
...
break;
}

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

[STINNER Victor]

STINNER Victor  added the comment:

When small integer are disabled at compilation time (NSMALLPOSINTS=0 and  a 
NSMALLNEGINTS=0), I suggest to remove code related to small integers. IMHO 
CHECK_SMALL_INT() macro is a good practical solution for that. So I suggest to 
restore this macro.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[STINNER Victor]

STINNER Victor  added the comment:

I propose to restrict this issue to Py_UNREACHABLE() macro/function.

Please use bpo-37812 to discuss longobject.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Py_UNREACHABLE() no longer behaves as a function call

[STINNER Victor]

Change by STINNER Victor :


--
title: Python no longer compiles without small integer singletons -> 
Py_UNREACHABLE() no longer behaves as a function call

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38019] asyncio subprocess AttributeError: 'NoneType' object has no attribute '_add_reader' / '_remove_reader'

[Marat Sharafutdinov]

Marat Sharafutdinov  added the comment:

Andrew, it would be nice to have it fixed within 3.8. Hope you'll find time for 
this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38224] Windows embeddable distribution doesn't ship headers

[Leblond Emmanuel]

New submission from Leblond Emmanuel :

I don't understand why the headers are not provided as part of the distribution.

It would be really easy to rm them once the distribution extracted for usecases 
that don't need them.

On the other hand, usecases that need them cannot just download the Include 
folder from the CPython git repository given they get cooked during the build 
phase.

It's also really inconvenient to extract the headers from the executable 
distribution.

>From my personal experience, this prevented me from using the distribution 
>each time it would have been a good fit for my need:
- Embedding CPython into the Godot game engine (see 
https://github.com/touilleMan/godot-python/blob/284f092d9f7893403f7af32cac5978532c0e7d21/platforms/windows-64/SCsub#L55-L62)
- Providing a frozen distribution of python for a script with simple .exe 
binary that would initialize the CPython insterpeter and run a simple "import 
mymodule; mymodule.main()" command

--
components: Build, Windows
messages: 352778
nosy: paul.moore, steve.dower, tim.golden, touilleMan, zach.ware
priority: normal
severity: normal
status: open
title: Windows embeddable distribution doesn't ship headers
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38223] Reorganize test_shutil

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +15866
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16281

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38223] Reorganize test_shutil

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Currently most shutil tests are in a single class. Tests for the same function 
are partially grouped together and partially intermixed with other tests. This 
makes hard to find an appropriate place for adding new tests and increase risk 
of hiding test by other test with the same name.

The proposed PR reorganized shutil tests. Added separate classes for particular 
functions and groups of related functions. It also cleans up some code: 
creating temporary dirs and monkey-patching shutil.open. test.support.rmtree() 
is used instead of shutil.rmtree() for cleaning up temporary dirs (it is more 
reliable for testing purposes).

I am going to add new tests, so I need to clean up the existing code first.

--
components: Tests
messages: 352777
nosy: giampaolo.rodola, serhiy.storchaka, tarek
priority: normal
severity: normal
status: open
title: Reorganize test_shutil
type: enhancement
versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Python no longer compiles without small integer singletons

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 16280: "Convert Py_UNREACHABLE() macro to a function". This change 
fix this issue but also enhance Py_UNREACHABLE() which now dumps the Python 
traceback where the bug occurs. See my example:
https://github.com/python/cpython/pull/16280#issuecomment-533049748

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Python no longer compiles without small integer singletons

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +15865
pull_request: https://github.com/python/cpython/pull/16280

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Initially, the default __format__ converted the object to str and passed format 
specifier to str.__format__. This is defined in PEP 3101:

class object:
def __format__(self, format_spec):
return format(str(self), format_spec)

But later we changed the implementation, because this leaded to difficult to 
catch errors. Non-empty format specifier is now error in the default __format__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38205] Python no longer compiles without small integer singletons

[STINNER Victor]

STINNER Victor  added the comment:

I like the idea of implementing Py_UNREACHABLE() as a function. Would it make 
sense to even declare it as a regular function rather than a static inline 
function? What is the benefit of inlining here? Inlining can make the code 
larger, rather than a function call at the machine code level is shorter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Eric V. Smith]

Eric V. Smith  added the comment:

I agree with Serhiy that !s solves the problem. But as a convenience it might 
be nice to add __format__. This issue pops up from time to time on Path and 
other types, and I don't think !s is very discoverable. Especially because 
formatting works with a specifier, but fails with one:

>>> format(path)
'/path/to/enlightenment'
>>> format(path, '>50')
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported format string passed to PosixPath.__format__

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38219] Optimize dict.__init__ and dict.update for dict argument

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Convert it to string first:

print(f'path is: {path!s:>50}')

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38222] pathlib Path objects should support __format__

[Miki Tebeka]

New submission from Miki Tebeka :

Currently pathlib.Path cannot be used with string formatting directives. IMO it 
should.

>>> from pathlib import Path
>>> path = Path('/path/to/enlightenment')
>>> print(f'path is: {path:>50}')
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported format string passed to PosixPath.__format__

--
components: Library (Lib)
messages: 352771
nosy: tebeka
priority: normal
severity: normal
status: open
title: pathlib Path objects should support __format__
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com