[issue35286] wrong result for difflib.SequenceMatcher

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35299] LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found

[Jorge Ramos]

New submission from Jorge Ramos :

Build of Python 3.6.7 fails with error 
"LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found" 
I am using Windows 10 64bit.

Steps to reproduce this error:
1) Install Visual Studio 2017 (With the python development and native dev 
tools) (I have also installed NVIDIA CUDA Toolkit but I don't think this 
matters)
2) Download source code from Python to folder E:\RepoGit\3.6 (this is the 
download folder)
3) Open a command prompt, navigate to "E:\RepoGit" and run "PCBuild\build.bat 
-p x64"
4) Close the command prompt
5) Copy file pyconfig.h file from E:\RepoGit\3.6\PC and paste it to 
E:\RepoGit\3.6\include. Also, create an environment variable called "LIB" that 
contains the folder "E:\RepoGiT\3.6\PCbuild\amd64" (this folder was created at 
step 3) 
6) Open a command prompt and run "Tools\msi\buildrelease.bat -x64"

NOTE1: if step 5 is skipped, the overall build does not fail, but the 
PGInstrument x64 Interpreter fails in distutils with error 
"E:\RepoGiT\3.6\include\Python.h(8): fatal error C1083: Cannot open include 
file: 'pyconfig.h': No such file or directory", as shown in message 
https://bugs.python.org/msg329247 of issue https://bugs.python.org/issue35157

NOTE2: if pyconfig.h in step 5 is copied but the environment variable is not 
created, the PGInstrument x64 Interpreter fails in distutils with error "LINK : 
fatal error LNK1104: cannot open file 'python36.lib'", and the overall build 
fails with the same error shown in the title of this Issue.

NOTE3: If all the steps are run, the PGInstrument x64 Interpreter succeeds in 
its tests but the overall build fails with the error shown in the title of this 
issue. All the verbose of this compilation is included in the file attached.

--
components: Build, Windows
files: LIBenRepogiT.txt
messages: 330299
nosy: neyuru, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: LGHT0091: Duplicate symbol 'File:include_pyconfig.h' found
type: compile error
versions: Python 3.6
Added file: https://bugs.python.org/file47941/LIBenRepogiT.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31823] Opaque default value for close_fds argument in Popen.__init__

[Tal Einat]

Tal Einat  added the comment:

Danish, see Martin's previous comment (msg314413):

"... the signature at [the 3.6 docs] should not show close_fds=True, because 
that is not necessarily the default on Windows."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31823] Opaque default value for close_fds argument in Popen.__init__

[Danish Prakash]

Danish Prakash  added the comment:

I'd like to work on this.

I get that the 3.7+ doc still has the irrelevant change from 3.6 which needs to 
be removed but I'm not sure what's wrong with the 3.6 doc.

> The signature is indeed awkward for 3.6,

--
nosy: +danishprakash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[Jeremy Kloth]

Jeremy Kloth  added the comment:

As a heavy user of the non-limited Python C API, I would like to offer my 
suggestions for consideration.  (I'm not allowed to post in discourse)

First off, to me, 'unstable' comes off quite negative, i.e. risky or erratic.  
Brett's suggestion of 'broad' is, well, seemingly too "broad" :)

In no particular order, some ideas:

- 'extended' or 'extra'; as opposed to limited (from the macro)

- 'volatile'; synonym for 'unstable' but with the benefit of being a C concept. 
 I like this one as the APIs covered by this include would have access to the 
"volatile" object details.  Things like borrowed references, direct array 
access or even the structure fields of the objects themselves.

- 'crunchy'; Monty Python reference, "If we took the bones out, it wouldn't be 
crunchy, would it?"
  https://en.wikipedia.org/wiki/Crunchy_Frog

--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34812] [EASY] support.args_from_interpreter_flags() doesn't inherit -I (isolated) flag

[Danish Prakash]

Danish Prakash  added the comment:

Sorry for bumping this thread but Victor, could you please share your inputs on 
this if you have the time for it, thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20582] socket.getnameinfo() does not document flags

[Danish Prakash]

Danish Prakash  added the comment:

I would like to work on this if nobody else has.

--
nosy: +danishprakash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Dong-hee Na]

Dong-hee Na  added the comment:

@asvetlov

I agree with asvetlov :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35292] Make SimpleHTTPRequestHandler load mimetypes lazily

[Danish Prakash]

Danish Prakash  added the comment:

I would like to work on this if you have not already started, Steve. 

> We should check whether people read from 
> SimpleHTTPRequestHandler.extensions_map directly before calling guess_type(), 
> and decide how quickly we can make the change.

Are you implying we make the change based on whether someone use 
SimpleHTTPRequestHandler.extensions_map before calling guess_type()? Could you 
please elaborate that a bit, thanks.

--
nosy: +danishprakash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33211] lineno and col_offset are wrong on function definitions with decorators

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33211] lineno and col_offset are wrong on function definitions with decorators

[Ethan Smith]

Ethan Smith  added the comment:

This has been fixed, so the issue can be closed I believe. 

FWIW I didn't see anything useful to salvage from my PR that wasn't already 
tested by your tests Serhiy.

--
pull_requests: +9923

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29877] compileall hangs when accessing urandom even if number of workers is 1

[INADA Naoki]

INADA Naoki  added the comment:

I am not sure PR 4856 is enhancement or bugfix.
But I think it's worth enough to backport to 3.6 and 3.7.
It caused a real UX problem.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35059] Convert Py_INCREF() and PyObject_INIT() to inlined functions

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9922

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35298] Segfault in _PyObject_GenericGetAttrWithDict

[gilado]

New submission from gilado :

Running pulseeffects, it crashes occasionally.  I have yet to find a cause, it 
just happens. 
I decided to run it under gdb to catch the crash.  It worked  for a few days 
before crashing.

$ GDK_DPI_SCALE=0.6 gdb -args /usr/bin/python3 /usr/bin/pulseeffects 

gdb: Symbol `acs_map' has different size in shared object, consider re-linking
GNU gdb (GDB) 7.8-1pclos2014 (PCLinuxOS release 2014)
...
...
13:57:26.125 - PulseEffects - CRITICAL - PM - context operation  failed!
13:57:29.779 - PulseEffects - CRITICAL - PM - context operation  failed!
13:57:30.391 - PulseEffects - CRITICAL - PM - context operation  failed!
[New LWP 27696]

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 27696]
0x773bcae6 in ?? () from /lib64/libc.so.6
(gdb) bt
#0  0x773bcae6 in  () at /lib64/libc.so.6
#1  0x729f8e08 in  ()
at /usr/lib64/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
#2  0x779baf1e in _PyObject_GenericGetAttrWithDict ()
at /usr/lib64/libpython3.6m.so.1.0
#3  0x77a2b6b0 in _PyEval_EvalFrameDefault ()
at /usr/lib64/libpython3.6m.so.1.0
#4  0x77a271df in  () at /usr/lib64/libpython3.6m.so.1.0
#5  0x77a2fce0 in _PyFunction_FastCallDict ()
at /usr/lib64/libpython3.6m.so.1.0
#6  0x779719fe in _PyObject_FastCallDict ()
at /usr/lib64/libpython3.6m.so.1.0
#7  0x77971c0d in _PyObject_Call_Prepend ()
at /usr/lib64/libpython3.6m.so.1.0
#8  0x779717c8 in PyObject_Call () at /usr/lib64/libpython3.6m.so.1.0
#9  0x729f433d in  ()
at /usr/lib64/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
#10 0x73f73dcf in ffi_closure_unix64_inner () at /usr/lib64/libffi.so.6
#11 0x73f74148 in ffi_closure_unix64 () at /usr/lib64/libffi.so.6
#12 0x7fffde2abaf5 in  () at /usr/lib64/libpulse.so.0
#13 0x7fffddf27ce1 in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#14 0x7fffddf28033 in pa_pdispatch_run ()
at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#15 0x7fffde2aaa9e in  () at /usr/lib64/libpulse.so.0
#16 0x7fffddf2ab4f in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#17 0x7fffddf2d5db in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#18 0x7fffddf2d979 in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#19 0x7fffddf2e1df in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#20 0x7fffde2bea58 in pa_mainloop_dispatch () at /usr/lib64/libpulse.so.0
#21 0x7fffde2bee1e in pa_mainloop_iterate () at /usr/lib64/libpulse.so.0
#22 0x7fffde2beea0 in pa_mainloop_run () at /usr/lib64/libpulse.so.0
#23 0x7fffde2cc786 in  () at /usr/lib64/libpulse.so.0
#24 0x7fffddf3db48 in  () at /usr/lib64/pulseaudio/libpulsecommon-12.2.so
#25 0x776ed62e in  () at /lib64/libpthread.so.0
#26 0x77421def in clone () at /lib64/libc.so.6
(gdb) 


Seems to be a python bug not validating a data pointer.

A similar bug was fixed here  https://bugs.python.org/issue12149

reviewing the code 
https://github.com/python/cpython/blob/master/Objects/object.c

I think line 981 mentioned in the bug description is now line 1235

f = descr->ob_type->tp_descr_get;

Which now is protected inside an if (descr != NULL). So they fixed it here. 
Further just below that both 'f' and 'descr' are used like this


if (f != NULL && PyDescr_IsData(descr)) {
res = f(descr, obj, (PyObject *)obj->ob_type);

The problem now seem to be further below (line 1279)  where the code is

if (f != NULL) {
res = f(descr, obj, (PyObject *)Py_TYPE(obj));

The problem is that they forget to check that descr is valid.

--
components: Interpreter Core
messages: 330289
nosy: gilado
priority: normal
severity: normal
status: open
title: Segfault in _PyObject_GenericGetAttrWithDict
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32305] Namespace packages have inconsistent __file__ and __spec__.origin

[Géry]

Géry  added the comment:

@barry You gave 2 reasons for changing __spec__.origin and __file__ for 
namespace packages.

Your 1st reason:

> I don't particularly like that its origin is "namespace".  That's an odd 
> special case that's unhelpful to test against (what if you import a 
> non-namespace package from the directory "namespace"?)

As far as I know, a non-namespace package always has an __init__.py file, so if 
it is imported from a directory named "namespace" it has a __spec__.origin and 
__file__ attributes equal to "path/to/package/namespace/__init__.py". So I 
don’t see the problem here with having a "namespace" origin for namespace 
package specs.

In addition, PEP 420 that introduced implicit namespace packages in Python 3.3 
clearly stated that having no __file__ attribute was intended for namespace 
packages, and more generally was left a the discretion of the module’s loader 
and no more limited to built-in modules 
(https://www.python.org/dev/peps/pep-0420/#module-reprs):

> Previously, module reprs were hard coded based on assumptions about a 
> module's __file__ attribute. If this attribute existed and was a string, it 
> was assumed to be a file system path, and the module object's repr would 
> include this in its value. The only exception was that PEP 302 reserved 
> missing __file__ attributes to built-in modules, and in CPython, this 
> assumption was baked into the module object's implementation. Because of this 
> restriction, some modules contained contrived __file__ values that did not 
> reflect file system paths, and which could cause unexpected problems later 
> (e.g. os.path.join() on a non-path __file__ would return gibberish).
> This PEP relaxes this constraint, and leaves the setting of __file__ to the 
> purview of the loader producing the module. Loaders may opt to leave __file__ 
> unset if no file system path is appropriate. Loaders may also set additional 
> reserved attributes on the module if useful. This means that the definitive 
> way to determine the origin of a module is to check its __loader__ attribute.
> For example, namespace packages as described in this PEP will have no 
> __file__ attribute because no corresponding file exists.

Your 2nd reason:

> This is especially bad because the documentation for __spec__.origin implies 
> a correlation to __file__, and says:
> "Name of the place from which the module is loaded, e.g. “builtin” for 
> built-in modules and the filename for modules loaded from source. Normally 
> “origin” should be set, but it may be None (the default) which indicates it 
> is unspecified."

I agree here, so why not updating the documentation instead of changing the 
implementation which followed PEP 420?

--
nosy: +eric.snow, maggyero

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[Géry]

Change by Géry :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30951] Documentation error in inspect module

[Utkarsh Gupta]

Change by Utkarsh Gupta :


--
nosy: +utkarsh2102

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33416] Add endline and endcolumn to every AST node

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

issue22616 is a bit different (proposing line/column ranges instead of just 
endline/endcolumn). I am happy to close this one in favor of issue22616 if we 
agree that we will go with endline/endcolumn.

I can't guarantee, but likely I will work on this during winter holidays, 
unless there are some objections, or unless someone will do this before :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35035] Documentation for email.utils is named email.util.rst

[Julien Palard]

Change by Julien Palard :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35083] Fix documentation for __instancecheck__

[Julien Palard]

Julien Palard  added the comment:

Currently [1] there's no concensus about:

- Is this a documentation issue and we should document the optimisation?
- Is the optimisation over-zealeous?

[1]: https://mail.python.org/pipermail/python-ideas/2018-October/054342.html

--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[STINNER Victor]

STINNER Victor  added the comment:

I created a poll on discuss.python.org for the name of the new subdirectory :-)
https://discuss.python.org/t/poll-what-is-your-favorite-name-for-the-new-include-subdirectory/477

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35035] Documentation for email.utils is named email.util.rst

[miss-islington]

miss-islington  added the comment:


New changeset daa34b84e9f6680a4012d5138068236ccdd2d1bd by Miss Islington (bot) 
(Julien Palard) in branch '2.7':
[2.7] bpo-35035: Rename email.utils documentation to email.utils.rst (GH-10023) 
(GH-10667)
https://github.com/python/cpython/commit/daa34b84e9f6680a4012d5138068236ccdd2d1bd


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30951] Documentation error in inspect module

[Utkarsh Gupta]

Change by Utkarsh Gupta :


--
keywords: +patch
pull_requests: +9921
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[STINNER Victor]

STINNER Victor  added the comment:

Another proposal: Include\impl\ as in "implementation details".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[STINNER Victor]

STINNER Victor  added the comment:

Brett:
> The "unstable" name bugs me as it suggests we might change it without notice 
> which isn't true at all. It's more a limited versus broad API. So maybe 
> rename the directory "broad"?

Brett: Nick proposed "Include\cpython", do you prefer this name?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35297] untokenize documentation is not correct

[Zsolt Cserna]

Change by Zsolt Cserna :


--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35297] untokenize documentation is not correct

[Zsolt Cserna]

New submission from Zsolt Cserna :

untokenize documentation 
(https://docs.python.org/3/library/tokenize.html#tokenize.untokenize) states 
the following:

"""
Converts tokens back into Python source code. The iterable must return 
sequences with at least two elements, the token type and the token string. Any 
additional sequence elements are ignored.
"""

This last sentence is clearly not true because here:
https://github.com/python/cpython/blob/master/Lib/tokenize.py#L242

The code checks for the length of the input token there, and the code behaves 
differently, in terms of whitespace, when an iterator of 2-tuples are given and 
when there are more elements in the tuple. When there are more elements in the 
tuple, the function renders whitespaces as the same as they were present in the 
original source.

So this code:
tokenize.untokenize(tokenize.tokenize(source.readline))

And this:
tokenize.untokenize([x[:2] for x in tokenize.tokenize(source.readline)])

Have different results.

I don't know that it is a documentation issue  or a bug in the module itself, 
so I created this bugreport to seek for assistance in this regard.

--
assignee: docs@python
components: Documentation
messages: 330281
nosy: csernazs, docs@python
priority: normal
severity: normal
status: open
title: untokenize documentation is not correct
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35035] Documentation for email.utils is named email.util.rst

[Julien Palard]

Change by Julien Palard :


--
pull_requests: +9920

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Well, to satisfy everybody we need to:
1. Implement `isAlive` for dummy thread
2. Add a deprecation warning for both Thread.isAlive and _DummyThread.isAlive
3. Remove isAlive in future Python release (3.8 for the deprecation, 3.8+2 for 
removal).
4. Backporting the property down to 3.7 and 3.6 doesn't hurt, let's do it

Did I miss something?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[pmpp]

pmpp  added the comment:

about micropython, only unix port have thread basic implementation and garbage 
collector messes with EINTR actually so it is not very useable.

unix port is only one on many, and is the less interesting port apart from 
running quick simulations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[pmpp]

pmpp  added the comment:

> I guess the question is whether any other Python implementation is threadless?

emscripten python ( cpython on asm.js or webassembly ) is threadless

--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Brett Cannon]

Brett Cannon  added the comment:

I guess the question is whether any other Python implementation is threadless? 
E.g. is MicroPython? If it even has threads then I agree about deprecating the 
module.

But if MicroPython does support threads we should keep the module. That would 
mean updating all references of isAlive() to is_alive() and adding the name 
alias since that name aliasing still exists in 'master' right now (probably for 
Python 2 porting support): 
https://github.com/python/cpython/blob/3bb183d7fb83ad6a84ec13dea90f95d67be35c69/Lib/threading.py#L1094

@dmaurer totally understand about time restraints, but do realize even 
"trivial" fixes like this would still take at least an hour to do a proper job 
so there's no guarantee someone will get around to fixing this. But if you do 
change your mind and want to give it a try, then https://devguide.python.org/ 
is there to help.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[Brett Cannon]

Brett Cannon  added the comment:

The "unstable" name bugs me as it suggests we might change it without notice 
which isn't true at all. It's more a limited versus broad API. So maybe rename 
the directory "broad"?

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35296] Install Include/internal/ header files

[STINNER Victor]

STINNER Victor  added the comment:

TODO: Update the Windows installer:
https://github.com/python/cpython/pull/10624#issuecomment-441090519

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35296] Install Include/internal/ header files

[STINNER Victor]

STINNER Victor  added the comment:

Example of debuggers: vmprof or faulthandler (now part of the stdlib).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35081] Move internal headers to Include/internal/

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9919

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35296] Install Include/internal/ header files

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +9918
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35177] Add missing dependencies between AST/parser header files

[STINNER Victor]

STINNER Victor  added the comment:

Ok, I fixed the "Yield warning" again.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35177] Add missing dependencies between AST/parser header files

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 3bb183d7fb83ad6a84ec13dea90f95d67be35c69 by Victor Stinner in 
branch 'master':
bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)
https://github.com/python/cpython/commit/3bb183d7fb83ad6a84ec13dea90f95d67be35c69


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35296] Install Include/internal/ header files

[STINNER Victor]

New submission from STINNER Victor :

For some very specific use cases (ex: debuggers), it can be interesting to 
access the internal API of CPython. I propose to install the internal API: 
Include/internal/*.h header files.

Attached PR implements this idea.

--
components: Interpreter Core
messages: 330271
nosy: eric.snow, nascheme, ncoghlan, vstinner
priority: normal
severity: normal
status: open
title: Install Include/internal/ header files
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35177] Add missing dependencies between AST/parser header files

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9917
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c9b3fc6b59b625c36c31ad437253e7140938af1a by Victor Stinner in 
branch '3.6':
bpo-9263: _PyObject_Dump() detects freed memory (GH-10061) (GH-10662) (GH-10663)
https://github.com/python/cpython/commit/c9b3fc6b59b625c36c31ad437253e7140938af1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35265] Internal C API: pass the memory allocator in a context

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35265] Internal C API: pass the memory allocator in a context

[STINNER Victor]

STINNER Victor  added the comment:

Honestly, I'm not convinced that the change is worth it. It looks like a giant 
complication for a very specific use case, whereas the current 
_PyMem_SetDefaultAllocator() solution "works". I close the issue.

Moreover, bpo-35266 conflicts with issue, and bpo-35266 looks more important 
than this issue, since it impacts the future public API and remove redundancy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35206] [WIP] Add a new experimental _Py_CAPI2 API

[STINNER Victor]

STINNER Victor  added the comment:

According to discussions on python-dev, a PEP will be needed first. Moreover, I 
prefer to make some progress on https://bugs.python.org/issue35134 before. I 
close the issue.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9916

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 95036ea25d47f0081bda2ba96ea327f3375cb6a4 by Victor Stinner in 
branch '3.7':
[3.7] bpo-9263: _PyObject_Dump() detects freed memory (GH-10061) (GH-10662)
https://github.com/python/cpython/commit/95036ea25d47f0081bda2ba96ea327f3375cb6a4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9915

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 2cf5d32fd9e61488e8b0be55a2e92a752ba8b06b by Victor Stinner in 
branch 'master':
bpo-9263: Fix _PyObject_Dump() for freed object (#10661)
https://github.com/python/cpython/commit/2cf5d32fd9e61488e8b0be55a2e92a752ba8b06b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35239] _PySys_EndInit() doesn't copy main interpreter configuration

[STINNER Victor]

STINNER Victor  added the comment:

I has been decided to not backport the change to Python 3.7: see PR 10532 
discusssion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35239] _PySys_EndInit() doesn't copy main interpreter configuration

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a5194115733f6ca8fc1ddbee43eabbde536900e6 by Victor Stinner in 
branch '3.7':
Revert "bpo-35239: _PySys_EndInit() copies module_search_path (GH-10532)" 
(GH-10660)
https://github.com/python/cpython/commit/a5194115733f6ca8fc1ddbee43eabbde536900e6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9263] Try to print repr() when an C-level assert fails (in the garbage collector, beyond?)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9914

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35239] _PySys_EndInit() doesn't copy main interpreter configuration

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9913

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35266] Add _PyPreConfig and rework _PyCoreConfig and _PyMainInterpreterConfig

[STINNER Victor]

STINNER Victor  added the comment:

Hum, my split is incomplete. From a high level point of view, the 
initialization should be done in these steps:

1) select memory allocator, config made of C char* (bytes) and int types
2) select encodings, add wchar_t* (Unicode) strings to the config
3) compute the "path configuration" (used to initialize importlib and sys.path)
4) apply the config to Python

Step (3) should be optional. Currently, the path configuration can be set in 
_PyCoreConfig to avoid the need "calculate" it (operation which access the 
filesystem).

Technically, we may have to use wchar_t* in (1), since Windows uses wmain() 
which gets argv as wchar_t** (and environment variables as wchar_t* as well).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[miss-islington]

miss-islington  added the comment:


New changeset 0c15e508baec7e542933db2b31ea950a646cd968 by Miss Islington (bot) 
in branch '3.6':
bpo-24658: os.read() reuses _PY_READ_MAX (GH-10657)
https://github.com/python/cpython/commit/0c15e508baec7e542933db2b31ea950a646cd968


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[STINNER Victor]

STINNER Victor  added the comment:

> I think the rules for C includes are that `"path/header.h"` looks next to the 
> current file first, whereas `` looks only in include 
> directories.

Oh ok, thanks.


> However, given your technique of mostly hiding the new directory name from 
> API consumers, what do you think of calling the new directory "cpython" 
> rather than "unstable"?

I'm not comfortable with "CPython" name. For me, everything the "CPython C API" 
is the concatenation of all files in Include/ but also in subdirectories. Right 
now, it's unclear what is the "Python" API ("portable" API, without 
implemenetation details) vs the "CPython API" (implementation details).

"unstable" comes from the PEP 384: "Defining a Stable ABI". IMHO what is not in 
the "Stable ABI" is the "Unstable ABI". By extension, APIs excluded by 
Py_LIMITED_API make the "unstable API".

>From my point of view, "CPython API" would be more internal/ + unstable/ APIs.


> The idea there would be that the "unstable ABI" eventually become known as 
> "the CPython C API" (since it exposes a lot of CPython implementation 
> details", while the limited API could become known as "the portable 
> cross-implementation Python C API".

Everybody seems to be confused by what is the "Python C API"... I see even more 
confusion if we have a "CPython C API". Do you see? "CPython" vs "Python", 
"Python C" vs "CPython"...

IMHO "unstable" is more explicit :-) It means: "don't touch this" :-D

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[miss-islington]

miss-islington  added the comment:


New changeset 18f3327d9a99163a658697465eb00c31f86535eb by Miss Islington (bot) 
in branch '3.7':
bpo-24658: os.read() reuses _PY_READ_MAX (GH-10657)
https://github.com/python/cpython/commit/18f3327d9a99163a658697465eb00c31f86535eb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[miss-islington]

Change by miss-islington :


--
pull_requests: +9912

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[miss-islington]

Change by miss-islington :


--
pull_requests: +9911

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9a0d7a7648547ffb77144bf2480155f6d7940dea by Victor Stinner in 
branch 'master':
bpo-24658: os.read() reuses _PY_READ_MAX (GH-10657)
https://github.com/python/cpython/commit/9a0d7a7648547ffb77144bf2480155f6d7940dea


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

> I'm wondering if we may need to make test_c_locale_coercion less opinionated, 
> where for most platforms it's just a "don't segfault" test, and it's only for 
> ones where we're certain about how we expect it to behave (i.e. Linux, Mac OS 
> X, AIX, maybe FreeBSD) that we try to check the specifics.

For test_embed, I tried to be smart to guess the encodings (ex: hardcode some 
encodings depending on  the platform) and... I gave up. It's very hard when you 
consider Linux, Windows, macOS, FreeBSD, but also HP-UX and AIX. Each platform 
has weird cases about encodings... Now the test spawns a Python subprocess to 
get the encoding. See get_stdio_encoding() and get_filesystem_encoding() in 
Lib/test/test_embed.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[STINNER Victor]

STINNER Victor  added the comment:

Almost all compiler warnings on 64-bit Windows have been fixed: bpo-9566 
tracked most of them.

I extract the CryptGenRandom() fix from issue18407-2.diff. I rewrote the ast.c 
fix differently (use larger type, don't downcast).

The patch may still contains relevant changes on Python/dtoa.c and 
Python/getargs.c, but again, the most critical bugs and warnings have already 
been fixed. So I suggest to abandon issue18407-2.diff which has been written 5 
years ago.

Thanks Jeremy Kloth for the report and for the patch! I think that it's now 
time to close the issue. See bpo-9566 for the few remaining issues.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4d73ae776140a583fdfe8f016d88cc767791e481 by Victor Stinner in 
branch 'master':
bpo-18407: ast.c uses Py_ssize_t for asdl_seq_LEN() iterator (GH-10655)
https://github.com/python/cpython/commit/4d73ae776140a583fdfe8f016d88cc767791e481


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c48ff73dd60bec5dcbe64bedeff91e6db26d98bc by Victor Stinner in 
branch 'master':
bpo-18407: win32_urandom() uses PY_DWORD_MAX (GH-10656)
https://github.com/python/cpython/commit/c48ff73dd60bec5dcbe64bedeff91e6db26d98bc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT

[Kubilay Kocak]

Change by Kubilay Kocak :


--
nosy: +koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT

[Nick Coghlan]

Nick Coghlan  added the comment:

Ah, interesting. https://bugs.python.org/issue30672 covered the fact that this 
test was already weird on FreeBSD, but the current status is that it was 
expecting BSD variants to act somewhat like Mac OS X, not like Linux.

I'm wondering if we may need to make test_c_locale_coercion less opinionated, 
where for most platforms it's just a "don't segfault" test, and it's only for 
ones where we're certain about how we expect it to behave (i.e. Linux, Mac OS 
X, AIX, maybe FreeBSD) that we try to check the specifics.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24925] Allow doctest to find line number of __test__ strings if formatted as a triple quoted string.

[Sanyam Khurana]

Sanyam Khurana  added the comment:

Hey  R. David Murray,

Where do you think the unit test for `doctest` should be placed? 

Should it be in `test_doctest.py`?

--
assignee:  -> CuriousLearner
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17490] Improve ast.literal_eval test suite coverage

[Nick Coghlan]

Change by Nick Coghlan :


--
assignee:  -> ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24658] open().write() fails on 2 GB+ data (OS X)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9910

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9909

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35266] Add _PyPreConfig and rework _PyCoreConfig and _PyMainInterpreterConfig

[Nick Coghlan]

Nick Coghlan  added the comment:

I didn't know what was possible when I wrote PEP 432 either - instead, I wrote 
down an initial concept for what I *wanted*, and then started exploring the 
code to find out the barriers to achieving that.

We know enough now to know that original design concept isn't technically 
feasible, but that's OK - the general idea was just to get to a point where the 
startup code is better tested, easier to maintain, and easier to control in an 
embedding application, and everything outside that is negotiable.

The problem with the purely bottom-up approach is that we may end up with 
something that's better tested and easier to maintain, but find out that it 
hasn't actually helped us get to a point where we can make the interpreter 
easier for embedding applications to manage.

As far as Unicode goes, it isn't Unicode as a concept that's problematic, it's 
specifically the CPython Unicode type: that needs hash randomisation 
configured, and that means we need to have already processed the input settings 
that can affect the hash seed. And unlike UTF-8 mode, where there's a 
comparatively limited set of strings to recreate with a different decoding 
step, there's no escape hatch to let you cleanly recreate all previously 
created string objects with a different basis for their hash.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18407] Fix compiler warnings in pythoncore for Win64

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9908
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24746] doctest 'fancy diff' formats incorrectly strip trailing whitespace

[Sanyam Khurana]

Change by Sanyam Khurana :


--
assignee:  -> CuriousLearner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9566] Compilation warnings under x64 Windows

[STINNER Victor]

STINNER Victor  added the comment:

There is also:

commit cdbcb773f5db24e23fa90e644ec620d54bd08127
Author: Victor Stinner 
Date:   Thu Nov 22 10:25:46 2018 +0100

cjkcodecs: Fix compiler warning (GH-10651)

Fixed the following compiler warning in multibytecodec.c:

warning C4244: '=': conversion from 'Py_ssize_t'
to 'unsigned char', possible loss of data

Cast Py_ssize_t to unsigned char: the maximum value is checked
on the previous line.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9566] Compilation warnings under x64 Windows

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 28f468cb19e3097079b7ce7850e6048de99022fa by Victor Stinner in 
branch 'master':
bpo-9566: Fix compiler warnings in pyexpat.c (GH-10654)
https://github.com/python/cpython/commit/28f468cb19e3097079b7ce7850e6048de99022fa


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35295] Please clarify whether PyUnicode_AsUTF8AndSize() or PyUnicode_AsUTF8String() is preferred

[Marcin Kowalczyk]

New submission from Marcin Kowalczyk :

The documentation is silent whether PyUnicode_AsUTF8AndSize() or 
PyUnicode_AsUTF8String() is preferred.

We are under the assumption that both are acceptable for the given caller, i.e. 
the caller wants to access just the sequence of UTF-8 code units (e.g. for 
calling a C++ function which takes std::string_view or std::string as a 
parameter), and the caller either will copy the UTF-8 code units immediately or 
is willing to own a temporary object to ensure a lifetime of the UTF-8 code 
units.

File comments in unicodeobject.h about PyUnicode_AsUTF8AndSize() have a warning:

   *** This API is for interpreter INTERNAL USE ONLY and will likely
   *** be removed or changed in the future.
   *** If you need to access the Unicode object as UTF-8 bytes string,
   *** please use PyUnicode_AsUTF8String() instead.

The discrepancy between these comments and the documentation should be fixed. 
Either the documentation is correct and the comment is outdated, or the comment 
is correct and the documentation is lacking guidance.

It is not even clear which function is better technically:

- PyUnicode_AsUTF8String() always allocates the string. 
PyUnicode_AsUTF8AndSize() does not allocate the string if the unicode object is 
ASCII-only (this is common) or if PyUnicode_AsUTF8AndSize() was already called 
before.

- If conversion must be performed, then PyUnicode_AsUTF8String() makes a single 
allocation, while PyUnicode_AsUTF8AndSize() first calls 
PyUnicode_AsUTF8String() and then copies the string.

- If the string is converted multiple times, then PyUnicode_AsUTF8AndSize() 
caches the result - faster. If the string is converted once, then the result 
persists as long as the string persists - wastes memory.

I see the following possible resolutions:

1a. Declare both functions equally acceptable. Remove comments claiming that 
PyUnicode_AsUTF8AndSize() should be avoided.

1b. 1a, and change the implementation of PyUnicode_AsUTF8AndSize() to avoid 
allocating the string twice if it needs to be materialized, so that 
PyUnicode_AsUTF8AndSize() is never significantly slower than 
PyUnicode_AsUTF8String().

2a. Declare PyUnicode_AsUTF8String() preferred. Indicate this in the 
documentation.

2b. 2a, and provide a public interface to check and access UTF-8 code units 
without allocating a new string in case this is possible (I think 
PyUnicode_READY() + PyUnicode_IS_ASCII() + PyUnicode_DATA() + 
PyUnicode_GET_LENGTH() would work, but they are not documented; or possibly 
also check if the string has a cached UTF-8 representation without populating 
that cached representation), so that a combination of the check with 
PyUnicode_AsUTF8String() is rarely or never significantly slower than 
PyUnicode_AsUTF8AndSize().

--
assignee: docs@python
components: Documentation, Interpreter Core, Unicode
messages: 330249
nosy: Marcin Kowalczyk, docs@python, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Please clarify whether PyUnicode_AsUTF8AndSize() or 
PyUnicode_AsUTF8String() is preferred
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The only _DummyThread usage is `threading.current_thread()`:

> If the caller's thread of control was not created through the threading
> module, a dummy thread object with limited functionality is returned.

--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Dong-hee Na]

Dong-hee Na  added the comment:

@serhiy.storchaka @pitrou
Then should we add the deprecated message 
for isAlive of class "Thread" first?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests:  -9906

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35134] Add a new Include/unstable/ subdirectory for the "unstable" API

[Nick Coghlan]

Nick Coghlan  added the comment:

I think the rules for C includes are that `"path/header.h"` looks next to the 
current file first, whereas `` looks only in include directories.

However, given your technique of mostly hiding the new directory name from API 
consumers, what do you think of calling the new directory "cpython" rather than 
"unstable"?

The idea there would be that the "unstable ABI" eventually become known as "the 
CPython C API" (since it exposes a lot of CPython implementation details", 
while the limited API could become known as "the portable cross-implementation 
Python C API".

(I know, I know, you were aiming to avoid further bikeshedding on the name, but 
"cpython" would namespace things nicely even if a compiler does something weird 
with header file lookups, and helps make it clearer to CPython contributors 
that we still need to care about public API stability in that directory, we 
just don't need to care about cross-implementation portability)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9566] Compilation warnings under x64 Windows

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9907

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35284] Incomplete error handling in Python/compile.c:compiler_call()

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Antoine Pitrou]

Antoine Pitrou  added the comment:

If it's not already deprecated, I'd say deprecate it first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

And since threads support no longer optional, are there reasons to keep 
_DummyThread?

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

More FreeBSD commits, "Teach man(1) about C.UTF-8":
https://svnweb.freebsd.org/base?view=revision=340128

---


EXPECTED_C_LOCALE_STREAM_ENCODING = 'ascii'
EXPECTED_C_LOCALE_FS_ENCODING = 'ascii'

I guess that the problem comes from that. But I'm not sure of what is the FS 
encoding for LC_ALL=C or LC_ALL=C.UTF-8.

--
title: [FreeBSD] test_c_locale_coercion: fsencoding is ASCII instead of UTF-8 
on FreeBSD CURRENT buildbot -> [FreeBSD] test_c_locale_coercion doesn't support 
new C.UTF-8 locale of FreeBSD CURRENT

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

isAlive() is the part of the old API. It is not even documented in Python 3, 
and can be removed in future. It is better to remove it from docstrings.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion: fsencoding is ASCII instead of UTF-8 on FreeBSD CURRENT buildbot

[STINNER Victor]

STINNER Victor  added the comment:

> AVAILABLE_TARGETS = ['C.UTF-8']

Aha, that's something new in FreeBSD. In FreeBSD 11.2, this locale is invalid. 
The locale has been added to FreeBSD CURRENT (future FreeBSD 12) at November 4:
https://reviews.freebsd.org/D17833

See discussion on freebsd-current:

http://freebsd.1045724.x6.nabble.com/C-UTF-8-as-default-locale-td6292359.html

So it's unrelated to my change, but it's a bug in test_c_locale_coercion on 
FreeBSD 12.

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Dong-hee Na]

Change by Dong-hee Na :


--
keywords: +patch
pull_requests: +9906
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24746] doctest 'fancy diff' formats incorrectly strip trailing whitespace

[Sanyam Khurana]

Sanyam Khurana  added the comment:

Thanks for your suggestion, Tim.

I've fixed the patch and it is ready for review at 
https://github.com/python/cpython/pull/10639

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9566] Compilation warnings under x64 Windows

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9905

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35059] Convert Py_INCREF() and PyObject_INIT() to inlined functions

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a42de742e7c20eeb64699b5785543fea65b2e8d3 by Victor Stinner in 
branch 'master':
bpo-35059: Cast void* to PyObject* (GH-10650)
https://github.com/python/cpython/commit/a42de742e7c20eeb64699b5785543fea65b2e8d3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35290] [FreeBSD] test_c_locale_coercion: fsencoding is ASCII instead of UTF-8 on FreeBSD CURRENT buildbot

[STINNER Victor]

STINNER Victor  added the comment:

Result of the additional debug info, on the FreeBSD CURRENT buildbot:

AVAILABLE_TARGETS = ['C.UTF-8']
EXPECTED_C_LOCALE_EQUIVALENTS = ['C', 'invalid.ascii']
EXPECTED_C_LOCALE_STREAM_ENCODING = 'ascii'
EXPECTED_C_LOCALE_FS_ENCODING = 'ascii'
EXPECT_COERCION_IN_DEFAULT_LOCALE = True
_C_UTF8_LOCALES = ('C.UTF-8', 'C.utf8', 'UTF-8')
_check_nl_langinfo_CODESET = True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35283] "threading._DummyThread" redefines "is_alive" but forgets "isAlive"

[Dong-hee Na]

Dong-hee Na  added the comment:

Hi, Can I work with this issue if no one works on it?

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35059] Convert Py_INCREF() and PyObject_INIT() to inlined functions

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9904

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35293] make doctest (Sphinx) emits a lot of warnings

[Dong-hee Na]

Dong-hee Na  added the comment:

Looks like they already fixed it.

- jinja2: 
https://github.com/pallets/jinja/commit/31bf9b7e71c3fee3b7866ffdc0f70f4525a490d9
- babel: 
https://github.com/python-babel/babel/commit/65d6bf39f2a2b4c02cd1e00c44ce9d7c1c5b

We should wait their release :)

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35293] make doctest (Sphinx) emits a lot of warnings

[STINNER Victor]

STINNER Victor  added the comment:

> They are warnings related to third party packages used for doctest and some 
> of them might become incompatible with 
> https://github.com/python/cpython/pull/10596. Related discussion : issue35109

We should help these packages to be fixed :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com