date:20190211



Guido van Rossum  added the comment:

Thanks for the report!

I think I see a path through the code that doesn't free the memory.

Does this patch fix it?

diff --git a/Parser/parsetok.c b/Parser/parsetok.c
index 1fa4a1286b..6a96f6bc5a 100644
--- a/Parser/parsetok.c
+++ b/Parser/parsetok.c
@@ -370,7 +370,6 @@ parsetok(struct tok_state *tok, grammar *g, int start, 
perrdetail *err_ret,
 type_ignores.items[i], 0);
 }
 }
-growable_int_array_deallocate(_ignores);
 
 #ifndef PGEN
 /* Check that the source for a single input statement really
@@ -405,6 +404,8 @@ parsetok(struct tok_state *tok, grammar *g, int start, 
perrdetail *err_ret,
 else
 n = NULL;
 
+growable_int_array_deallocate(_ignores);
+
 #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD
 *flags = ps->p_flags;
 #endif

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35973] `growable_int_array type_ignores` in parsetok.c is not always freed.



Change by Karthikeyan Singaravelan :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35934] Add socket.bind_socket() utility function



Giampaolo Rodola'  added the comment:

Closing this out as duplicate. Will continue in issue35934.

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17561] Add socket.bind_socket() convenience function



Giampaolo Rodola'  added the comment:

After iterating over this over the last few days I realized it makes more sense 
to implement and discuss the whole thing in here and as a single PR, so sorry 
about previously splitting this in a separate ticket/PR. Relevant PR is now 
this one and is ready for review:
https://github.com/python/cpython/pull/11784
Relevant changes which probably require attention/discussion are: 

1) since it was sort of ambiguous I renamed "has_dual_stack()" to 
"supports_hybrid_ipv46()". There could be space for some bikeshed as we already 
have "socket.has_ipv6" so this may be "has_hybrid_ipv46()" called instead

2) if family is unspecified and determined from *host* (e.g. "localhost" or "") 
the function sorts getaddrinfo() results preferring AF_INET over AF_INET6

3) doc includes link to my http://code.activestate.com/recipes/578504 recipe 
for platforms not supporting hybrid_ipv46 natively

4) it may be worthwhile (or maybe not?) to have another complementary 
bind_sockets() (plural) function returning all items from getaddrinfo(). That 
would be useful for non-blocking apps/frameworks and could be reused by asyncio.

Also, I'm CC-ing people from issue20215 as it contains relevant comments.

--
keywords:  -easy
nosy: +Carlos.Ralli, Paul Marks, andreasr, berker.peksag, dazhaoyu, 
gregory.p.smith, jleedev, jpokorny, martin.panter, nirs, r.david.murray
pull_requests: +11856
title: Add socket.create_server_sock() convenience function -> Add 
socket.bind_socket() convenience function

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35976] PCBuild file changes for arm32 should be separated from code changes for review

2019-02-11 Thread Paul Monson



Change by Paul Monson :


--
keywords: +patch
pull_requests: +11855
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35976] PCBuild file changes for arm32 should be separated from code changes for review

2019-02-11 Thread Paul Monson



Change by Paul Monson :


--
components: Build, Windows
nosy: Paul Monson, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: PCBuild file changes for arm32 should be separated from code changes for 
review
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-11 Thread Davin Potts



Davin Potts  added the comment:

@giampaolo.rodola: It definitely helps.


Conceptually, SyncManager provides "distributed shared memory" where lists, 
dicts, etc. are held in memory by one process but may be accessed remotely from 
another via a Proxy Object.  Mutating a dict from one process requires sending 
a message to some other process to request the change be made.

In contrast, SharedMemoryManager provides non-distributed shared memory where a 
special region of memory is held by the OS kernel (not a process) and made 
directly addressable to many processes simultaneously.  Modifying any data in 
this special region of memory requires zero process-to-process communication; 
any of the processes may modify the data directly.

In a speed contest, the SharedMemoryManager wins in every use case -- and it is 
not a close race.  There are other advantages and disadvantages to each, but 
speed is the key differentiator.


Thinking ahead to the future of SharedMemoryManager, there is the potential for 
a POSIX shared memory based semaphore.  The performance of this semaphore 
across processes should drastically outperform SyncManager's semaphore.  It 
might be something we will want to support in the future.  SharedMemoryManager 
needs a synchronization mechanism now (in support of common use cases) to 
coordinate across processes, which is why I initially thought 
SharedMemoryManager should expose the Lock, Semaphore, Event, Barrier, etc. 
powered by distributed shared memory.  I am no longer sure this is the right 
choice for three reasons:
(1) it unnecessarily complicates and confuses the separation of what is powered 
by fast SystemV-style shared memory and what is powered by slow distributed 
shared memory,
(2) it would be a very simple example in the docs to show how to add our 
existing Lock or Semaphore to SharedMemoryManager via register(),
(3) if we one day implement POSIX shared memory semaphores (and equivalent 
where POSIX is not supported), we will have the burden of an existing 
lock/semaphore creation methods and apis with behavioral differences.


I propose that it would be clearer but no less usable if we drop these 
registered object types (created via calls to register()) from 
SharedMemoryManager.  It is one line of code for a user to add "Lock" to 
SharedMemoryManager, which I think we can demonstrate well with a simple 
example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35971] Documentation should warn about code injection from current working directory



Karthikeyan Singaravelan  added the comment:

The change in behavior of Perl was discussed in 
https://mail.python.org/pipermail/python-ideas/2017-June/045842.html

--
nosy: +vstinner, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-11 Thread Raymond Hettinger



Raymond Hettinger  added the comment:

FWIW, I read the draft docs and found them to be at the right level throughout. 
I definitely wouldn't want anything more terse.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-11 Thread Eric Snow



Eric Snow  added the comment:

Is "long long" (AKA int64_t) 32 bits or 64 bits on a 32-bit platform?  It it's 
always 32 bits then there is no problem here.  Otherwise I agree we have a 
problem to fix.  My understanding is that it is the former (always 32 bits).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords

2019-02-11 Thread Ethan Smith



Change by Ethan Smith :


--
nosy: +Ethan Smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken



Giampaolo Rodola'  added the comment:


New changeset 58f05ce059cc5207320fef27a9fbc0ffdc2b1d1a by Giampaolo Rodola 
(Miss Islington (bot)) in branch '3.7':
bpo-35918: Remove broken has_key method and add test (GH-11819) (#11824)
https://github.com/python/cpython/commit/58f05ce059cc5207320fef27a9fbc0ffdc2b1d1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken

2019-02-11 Thread miss-islington



Change by miss-islington :


--
pull_requests: +11854

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Giampaolo Rodola'  added the comment:


New changeset a31f4cc881992e84d351957bd9ac1a92f882fa39 by Giampaolo Rodola 
(Rémi Lapeyre) in branch 'master':
bpo-35918: Remove broken has_key method and add test (#11819)
https://github.com/python/cpython/commit/a31f4cc881992e84d351957bd9ac1a92f882fa39


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords

2019-02-11 Thread Ivan Levkivskyi



Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35505] Test test_imaplib fail in test_imap4_host_default_value

2019-02-11 Thread Matej Cepl



Change by Matej Cepl :


--
keywords: +patch
pull_requests: +11853
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Change by Guido van Rossum :


--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35766] Merge typed_ast back into CPython



Guido van Rossum  added the comment:

See https://bugs.python.org/issue35975

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords


New submission from Guido van Rossum :

Now that the ast module can be used to parse type comments, there’s one more 
feature I’d like to lobby for – a flag that modifies the grammar slightly so it 
resembles an older version of Python (going back to 3.4).

This is used in mypy to decouple the Python version you’re running from the 
Python version for which you’re checking compatibility (useful when checking 
code that will be deployed on a system with a different Python version 
installed). I imagine this would be useful to other linters as well, and the 
implementation is mostly manipulating whether `async` and `await` are keywords. 
But if there’s pushback to this part I can live without it – the rest of the 
work is still useful.

The implementation in typed_ast takes the form of a feature_version flag which 
is set to the minor Python version to be parsed. Setting it to 4 or lower would 
make async/await never keywords, setting it to 5 or 6 would make them 
conditional per PEP 492, and setting it to 7 or higher would make these 
unconditional keywords. A few minor uses of the same flag also reject 
f-strings, annotated assignments (PEP 526), and matrix multiply for versions 
where those don't exist.

But I don't need all of those -- I really just need to be able to select the 
3.5/3.6 rules for conditional async/await keywords, since all the other 
features are purely backwards compatible, whereas with async/await there is 
legal (and plentiful!) code that uses these as variable or function names that 
should be supported in 3.5/3.6 mode.

Of course having it be a (minor) version number would still be more 
future-proof -- if say in 3.10 we add a match expression using some kind of 
conditional keyword hack, we might have to dust it off.

Note that much of what I'm asking for would effectively roll back 
https://bugs.python.org/issue30406 -- sorry Jelle! (Though there's more to it 
-- Serhiy's introduction of Grammar/Token followed, and I would still need to 
thread some kind of flag all the way from ast.parse() to tokenizer.c.

--
messages: 335277
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: Put back the ability to parse files where async/await aren't keywords

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35766] Merge typed_ast back into CPython



Guido van Rossum  added the comment:

I have some follow-up wishes but I'll create a new issue.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Change by Guido van Rossum :


--
stage: resolved -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35974] os.DirEntry.inode() returns invalid value within Docker container

2019-02-11 Thread Marat Sharafutdinov



New submission from Marat Sharafutdinov :

I'm trying to build Python 3.7.2 within official CentOS 7.6.1810 image 
(https://hub.docker.com/_/centos) and getting the following error during 
testing:

==
FAIL: test_attributes (test.test_os.TestScandir)
--
Traceback (most recent call last):
  File "/usr/src/python/Lib/test/test_os.py", line 3367, in test_attributes
self.check_entry(entry, 'dir', True, False, False)
  File "/usr/src/python/Lib/test/test_os.py", line 3319, in check_entry
os.stat(entry.path, follow_symlinks=False).st_ino)
AssertionError: 28093768 != 85098458

I guess this bug applies to Docker containers in general. For instance it's 
reproduced with the official Python 3.7.2-stretch image based on the Debian 
Stretch (https://hub.docker.com/_/python):

$ docker run --rm -it python:3.7.2-stretch
Python 3.7.2 (default, Feb  6 2019, 12:04:03) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir('/test_dir')
>>> for entry in os.scandir('/'):
...   if entry.name == 'test_dir':
... break
... 
>>> print(entry, entry.inode(), os.stat(entry.path, 
>>> follow_symlinks=False).st_ino)
 23898155 85118011
>>> assert entry.inode() == os.stat(entry.path, follow_symlinks=False).st_ino
Traceback (most recent call last):
  File "", line 1, in 
AssertionError
>>> 

In case of using host volume when running container it works ok, - the problem 
occurs when using default Docker volume:

$ docker run --rm -it -v /home/decaz/workspace:/host_dir python:3.7.2-stretch
Python 3.7.2 (default, Feb  6 2019, 12:04:03) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir('/host_dir/test_dir')
>>> for entry in os.scandir('/host_dir'):
...   if entry.name == 'test_dir':
... break
... 
>>> print(entry, entry.inode(), os.stat(entry.path, 
>>> follow_symlinks=False).st_ino)
 12873222 12873222
>>> assert entry.inode() == os.stat(entry.path, follow_symlinks=False).st_ino
>>> 

Similar issue - https://bugs.python.org/issue32811.

--
components: Build
messages: 335275
nosy: decaz
priority: normal
severity: normal
status: open
title: os.DirEntry.inode() returns invalid value within Docker container
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35973] `growable_int_array type_ignores` in parsetok.c is not always freed.

2019-02-11 Thread Brennan Vincent



New submission from Brennan Vincent :

To reproduce:

(1) build python: `../configure --prefix=$HOME/prefix --with-pydebug 
--without-pymalloc && make install`

(2) run with valgrind: `valgrind --leak-check=full ~/prefix/bin/python3`

(3) exit immediately from the interpreter by pressing ^D

(4) Note the following output from Valgrind:

```
==3810071== 40 bytes in 1 blocks are definitely lost in loss record 3 of 527
==3810071==at 0x4C28B5F: malloc (vg_replace_malloc.c:299)
==3810071==by 0x59ED58: growable_int_array_init (parsetok.c:27)
==3810071==by 0x59EE14: parsetok (parsetok.c:235)
==3810071==by 0x59F697: PyParser_ParseFileObject (parsetok.c:176)
==3810071==by 0x522E85: PyParser_ASTFromFileObject (pythonrun.c:1224)
==3810071==by 0x5231E9: PyRun_InteractiveOneObjectEx (pythonrun.c:238)
==3810071==by 0x5234D0: PyRun_InteractiveLoopFlags (pythonrun.c:120)
==3810071==by 0x523BF2: PyRun_AnyFileExFlags (pythonrun.c:78)
==3810071==by 0x4204FE: pymain_run_stdin (main.c:1185)
==3810071==by 0x42126B: pymain_run_python (main.c:1675)
==3810071==by 0x422EE0: pymain_main (main.c:1820)
==3810071==by 0x422F75: _Py_UnixMain (main.c:1857)
```

Reproduced on git commit hash 522346d792d9013140a3f4ad3534ac10f38d9085 .

--
components: Interpreter Core
messages: 335274
nosy: umanwizard
priority: normal
severity: normal
status: open
title: `growable_int_array type_ignores` in parsetok.c is not always freed.
type: resource usage
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes



Giampaolo Rodola'  added the comment:

I submitted an initial review / comments in the PR. I think this is good for a 
first iteration in order to understand what APIs to expose publicly (also, keep 
in mind I may not have a full picture of how this is intended to be used 
precisely).

As for the doc, I find it a bit too verbose. If the namespace change is 
accepted consider doing the following:

* document `SharedMemoryManager` right after `SyncManager`. That would let you 
reuse the description for `Semaphore`, `Barrier`, `Lock`, etc.
* within `SharedMemoryManager` doc link/point to a different section of the doc 
where you explain the whole thing in more details, and also document the 
remaining APIs 

Just an idea. Writing good doc is not easy and it requires actually putting 
hands on it. 

Hope this helps.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-11 Thread Alexey Izbyshev



Change by Alexey Izbyshev :


--
keywords: +patch
pull_requests: +11852
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-11 Thread Alexey Izbyshev



New submission from Alexey Izbyshev :

Victor Stinner pointed out that on x86 Gentoo Installed with X 3.x buildbot, 
there is a compiler warning:

Python/pystate.c:1483:18: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]

(https://buildbot.python.org/all/#/builders/103/builds/2067)

This warning reveals a bug:

static int
_long_shared(PyObject *obj, _PyCrossInterpreterData *data)
{
int64_t value = PyLong_AsLongLong(obj);
if (value == -1 && PyErr_Occurred()) {
if (PyErr_ExceptionMatches(PyExc_OverflowError)) {
PyErr_SetString(PyExc_OverflowError, "try sending as bytes");
}
return -1;
}
data->data = (void *)value;

A 64-bit value is converted to void *, which is 32-bit on 32-bit platforms.

I've implemented a PR that uses Py_ssize_t instead to match the pointer size 
and to preserve the ability to work with negative numbers.

--
assignee: izbyshev
components: Extension Modules
messages: 335272
nosy: eric.snow, izbyshev, vstinner
priority: normal
severity: normal
status: open
title: _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35971] Documentation should warn about code injection from current working directory

2019-02-11 Thread Gabriel Corona

New submission from Gabriel Corona :

The CLI tools shipped in Debian python-rdflib-tools package can load modules
from the current directory [1]:

$ echo 'print("Something")' > cgi.py
$ rdf2dot
INFO:rdflib:RDFLib Version: 4.2.2
Something
Reading from stdin as None...

This could be a security issue because an attacker could possibly exploit this
behavior to execute arbitrary code.

This happens because these CLI tools are implemented as:

#!/bin/sh

exec /usr/bin/python -m rdflib.tools.rdfpipe $*

"python -m $module", "python -c $code" and "$command | python" prepend the
current working directory in the Python path. The Python documentation [2]
should probably warn about this. In Python 3, "-I" could be suggested to
prevent the script/current directory to be added to the Python path. However,
this flag has other effects.

The Python documentation suggests "python -m" commands at some places [3-5]:
some form of warning at those places might be nice as well.

See the related behavior of Perl. Perl used to include "." in @INC but this was
removed for security reasons [6].

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921751.
[2] https://docs.python.org/3/using/cmdline.html
[3] https://docs.python.org/3.1/library/json.html
[4] https://docs.python.org/3/library/http.server.html
[5] https://docs.python.org/3/library/zipapp.html
[6]
https://metacpan.org/pod/release/XSAWYERX/perl-5.26.0/pod/perldelta.pod#Removal-of-the-current-directory-%28%22.%22%29-from-@INC

--
messages: 335271
nosy: Gabriel Corona
priority: normal
severity: normal
status: open
title: Documentation should warn about code injection from current working
directory
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

2019-02-11 Thread Stéphane Wirtel


Stéphane Wirtel  added the comment:

irc pong

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch

2019-02-11 Thread Tim Peters



Tim Peters  added the comment:

It's probably OK, but there's no "pure win" to be had here.  There's generally 
more than one way to convert one string to another, and what "looks right" to 
humans depends a whole lot on context.

For example, consider these strings:

"private Thread currentThread;"
"private volatile Thread currentThread;"

"It's obvious" someone inserted "volatile" into the first string, and that's 
what ndiff's default says:

- private Thread currentThread;
+ private volatile Thread currentThread;
? +

However, pass `charjunk=None` instead, and ndiff claims someone inserted "e 
volatil" after the "t" in "private":

- private Thread currentThread;
+ private volatile Thread currentThread;
?   +

Which is also a correct way, but - to human eyes - an insane way ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch



Jason R. Coombs  added the comment:

Nice insight Tim.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

2019-02-11 Thread Antoine Pitrou



Antoine Pitrou  added the comment:

notifying a lot of people

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue



Ernest W. Durbin III  added the comment:

test

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"

2019-02-11 Thread STINNER Victor



STINNER Victor  added the comment:

Thanks for the fix, Guido ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Guido van Rossum  added the comment:


New changeset 522346d792d9013140a3f4ad3534ac10f38d9085 by Guido van Rossum in 
branch 'master':
Complete switch cases in symtable.c; fixes bpo-35963 (GH-11821)
https://github.com/python/cpython/commit/522346d792d9013140a3f4ad3534ac10f38d9085


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue



Ernest W. Durbin III  added the comment:

should be fast send.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-11 Thread Davin Potts



Davin Potts  added the comment:

@terry.reedy and @ronaldoussoren: I have asked Van again to provide comments 
here clarifying the topics of (1) copyright notices and (2) requiring the 
BSD-licensed-work's author to sign a contributor agreement.

Specifically regarding the appearance of __copyright__, I added my agreement to 
your comments on GH-11816 on this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import



Karthikeyan Singaravelan  added the comment:

Sorry by previous reports I was talking about some of the issues where the 
installers were not correct though I couldn't find the issue at the moment. I 
just presented it as a data point where some necessary modules that cannot be 
imported could lead to crash. But if this can be dealt with in a better way I 
think it will be a good improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue



Ernest W. Durbin III  added the comment:

test mail sending

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch


Karthikeyan Singaravelan  added the comment:

Thanks for the explanation. This seems to give the desired diff with 
charjunk=None passed to multiline string comparison helper. I am not sure how 
useful it would be to pass it to sequence and dict comparison that also use 
ndiff. I can open a PR if it's okay with the set of strings in the report as a 
test case. There are no test case failures in existing unittest folder test 
suite so this seems like a safe change to me.


# With patch charjunk=None

./python.exe ../backups/bpo35955_1.py
F
==
FAIL: test_foo (__main__.FooTestCase)
--
Traceback (most recent call last):
  File "../backups/bpo35955_1.py", line 6, in test_foo
self.assertEqual("drwxrwxr-x 2 2000  2000\n", "drwxr-xr-x 2 2000  2000\n")
AssertionError: 'drwxrwxr-x 2 2000  2000\n' != 'drwxr-xr-x 2 2000  2000\n'
- drwxrwxr-x 2 2000  2000
?  ^
+ drwxr-xr-x 2 2000  2000
?  ^


--
Ran 1 test in 0.003s

FAILED (failures=1)

# Without patch

➜  cpython git:(master) ✗ python3.7 ../backups/bpo35955_1.py
F
==
FAIL: test_foo (__main__.FooTestCase)
--
Traceback (most recent call last):
  File "../backups/bpo35955_1.py", line 6, in test_foo
self.assertEqual("drwxrwxr-x 2 2000  2000\n", "drwxr-xr-x 2 2000  2000\n")
AssertionError: 'drwxrwxr-x 2 2000  2000\n' != 'drwxr-xr-x 2 2000  2000\n'
- drwxrwxr-x 2 2000  2000
?  ---
+ drwxr-xr-x 2 2000  2000
?+++


--
Ran 1 test in 0.002s

FAILED (failures=1)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-11 Thread Paul Ganssle



Paul Ganssle  added the comment:

@Karthikeyan I would certainly consider this a duplicate of the encodings 
bug/behavior that you demonstrate. I don't see an existing bug on the tracker 
for it, though.

I think it's pretty clear that the interpreter needs to fail, but it seems to 
*crash* rather than exit gracefully with a non-zero return code, which I think 
is the main problem. I'll note that it *does* print some stuff out before it 
crashes, which makes me think that even though io and/or encodings hasn't 
loaded, you still are able to do the "print an error message" part of "print an 
error message and exit with non-zero error code".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35970] no help flag in base64 util

2019-02-11 Thread Robert Kuska



New submission from Robert Kuska :

I failed today to print help message for base64 utility without an error:

 $ python3 -m base64 -help
option -h not recognized
usage: 
/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/base64.py
 [-d|-e|-u|-t] [file|-]
-d, -u: decode
-e: encode (default)
-t: encode and decode string 'Aladdin:open sesame

So I felt like this is a ripple in time space continuum worth adjusting.

I already opened a PR to address this (probably not so worthy) issue (link 
attached).

--
messages: 335255
nosy: rkuska
priority: normal
pull_requests: 11851
severity: normal
status: open
title: no help flag in base64 util

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue



Ernest W. Durbin III  added the comment:

test comment

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue



Ernest W. Durbin III  added the comment:

comment on test issue to test the request time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch

2019-02-11 Thread Tim Peters



Tim Peters  added the comment:

difflib generally synchs on the longest contiguous matching subsequence that 
doesn't contain a "junk" element.  By default, `ndiff()`'s optional `charjunk` 
argument considers blanks and tabs to be junk characters.

In the strings:

"drwxrwxr-x 2 2000  2000\n"
"drwxr-xr-x 2 2000  2000\n"

the longest matching substring not containing whitespace is "rwxr-x", of length 
6, starting at index 4 in the first string and at index 1 in the second.  So 
it's aligning the strings like so:

"drwxrwxr-x 2 2000  2000\n"
   "drwxr-xr-x 2 2000  2000\n"
 123456

That's why it wants to delete the 1:4 slice in the first string and insert 
"r-x" after the longest matching substring.

The default is aimed at improving results for human-readable text, like prose 
and Python code, where stuff between whitespace is often read "as a whole" 
(words, keywords, identifiers, ...).

For cases like this one, where character-by-character differences are 
important, it's often better to pass `charjunk=None`.  Then the longest 
matching substring is "xr-x 2 2000  2000" at the tail end of both strings, and 
you get the output you're expecting.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35967] Better platform.processor support

2019-02-11 Thread Ned Deily



Change by Ned Deily :


--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23460] Decimals do not obey ':g' exponential notation formatting rules

2019-02-11 Thread Brennan D Baraban



Brennan D Baraban <3...@holbertonschool.com> added the comment:

Well, Tuomas, I defer submission of a PR to you, as its your original patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35968] lib2to3 cannot parse rf''

2019-02-11 Thread Ned Deily



Ned Deily  added the comment:

@xtreak, I concur.  While it would have better to originally fix this in the 
release where f strings first appeared, the window to do that has closed.  
Sorry!

--
nosy: +ned.deily
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> 2to3 doesn't parse all valid string literals

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import


Karthikeyan Singaravelan  added the comment:

I hope there are certain modules that CPython interpreter expects to load 
properly. There were some cases in the past where encodings module caused the 
interpreter to crash. A similar scenario.

➜  cpython git:(master) echo "raise Exception('a')" > encodings.py
➜  cpython git:(master) ✗ PYTHONPATH=: ./python.exe
Fatal Python error: initfsencoding: failed to get the Python codec of the 
filesystem encoding
Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/encodings.py", line 
1, in 
Exception: a
[1]29005 abort  PYTHONPATH=: ./python.exe

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch



Karthikeyan Singaravelan  added the comment:

I am not sure this is a duplicate since the other issue was about newline at 
the end of strings. This is about the diff being little irrelevant even with 
newline in the end for strings. Sample program where change in 5th character 
gives the reported diff.

import difflib

for i in range(7):
print(f"Change character at {i}")
a = list("drwxrwxr-x 2 2000  2000\n")
b = "drwxrwxr-x 2 2000  2000\n"
a[i] = '-'
a = ''.join(a)
print(''.join(difflib.ndiff([a], [b])))

Change character at 0
- -rwxrwxr-x 2 2000  2000
? ^
+ drwxrwxr-x 2 2000  2000
? ^

Change character at 1
- d-wxrwxr-x 2 2000  2000
?  ^
+ drwxrwxr-x 2 2000  2000
?  ^

Change character at 2
- dr-xrwxr-x 2 2000  2000
?   ^
+ drwxrwxr-x 2 2000  2000
?   ^

Change character at 3
- drw-rwxr-x 2 2000  2000
?^
+ drwxrwxr-x 2 2000  2000
?^

Change character at 4
- drwx-wxr-x 2 2000  2000
? ^
+ drwxrwxr-x 2 2000  2000
? ^

Change character at 5
- drwxr-xr-x 2 2000  2000
?---
+ drwxrwxr-x 2 2000  2000
?  +++

Change character at 6
- drwxrw-r-x 2 2000  2000
?   ^
+ drwxrwxr-x 2 2000  2000
?   ^

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch



Jason R. Coombs  added the comment:

I don't think so, because the issue happens on a single line diff... although 
it's plausible there's a common-mode fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch

2019-02-11 Thread Chris Jerdonek



Chris Jerdonek  added the comment:

Is this a duplicate of issue24780?

--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-11 Thread Paul Ganssle



Paul Ganssle  added the comment:

Tested with 3.6 and 2.7 - core dump on 3.6, no core dump on 2.7. The crash on 
2.7 goes through a different path, it goes site.py > os.py >= UserDict.py > 
_abcoll.py > abc.py. That may account for why it's not crashing the interpreter 
itself.

--
versions: +Python 3.6 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-11 Thread Paul Ganssle



New submission from Paul Ganssle :

Just noticed this (tested on Python 3.7 and 3.8):

mkdir /tmp/demo
cd /tmp/demo
cat << EOF > abc.py
raise Exception("Hi")
EOF
PYTHONPATH=: python -c ""


This will crash the interpreter with:

Fatal Python error: init_sys_streams: can't initialize sys standard 
streams
Traceback (most recent call last):
  File ".../lib/python3.7/io.py", line 52, in 
  File "/tmp/demo/abc.py", line 1, in 
Exception: Hi
Aborted (core dumped)


It seems that the problem is that the io module imports the abc module, which 
raises an exception, so io fails to load. Evidently the io module is necessary 
to load the interpreter, so the interpreter crashes.

Here's the backtrace for 3.7.2 on Arch Linux:

(gdb) bt
#0  0x7f234b3f0d7f in raise () from /usr/lib/libc.so.6
#1  0x7f234b3db672 in abort () from /usr/lib/libc.so.6
#2  0x7f234b7db490 in fatal_error (prefix=prefix@entry=0x7f234b9d5fe0 
<__func__.16645> "init_sys_streams", 
msg=msg@entry=0x7f234ba01f60 "can't initialize sys standard streams", 
status=status@entry=-1)
at Python/pylifecycle.c:2179
#3  0x7f234b8460cb in _Py_FatalInitError (err=...) at 
Python/pylifecycle.c:2198
#4  0x7f234b8495a9 in pymain_init (pymain=0x7fff971cca70) at 
Modules/main.c:3019
#5  0x555dfa560050 in ?? ()
#6  0x7fff971ccbc0 in ?? ()
#7  0x in ?? ()


I'm not sure if anything can or should be done about this. It's very fair for 
the interpreter to fail to start, though I would guess that it should do that 
without dumping a core.

--
messages: 335244
nosy: p-ganssle
priority: normal
severity: normal
status: open
title: Interpreter crashes with "can't initialize init_sys_streams" when abc 
fails to import
type: crash
versions: Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager



Karthikeyan Singaravelan  added the comment:

No problem, after some debugging the statement finally executed is 
'exec(compile("set.add(0)", "foo.py", "exec"))' . In Python interpreter this 
will give the expected error. It also gives the same error before 
coverage.start() [0] and gives a different one as in the report once the 
statement is executed after coverage.start(). So this might be something to do 
with coverage and I have less knowledge about the internals.

I am closing this as third party since I couldn't see any issue with CPython 
and you might want to follow up coverage issue tracker. Thanks for the report 
though!

[0] 
https://github.com/nedbat/coveragepy/blob/f12db7fd4b3ab288f7f770f7138062951feaa6c9/coverage/cmdline.py#L639

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35378] multiprocessing.Pool.imaps iterators do not maintain alive the multiprocessing.Pool objects



Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35378] multiprocessing.Pool.imaps iterators do not maintain alive the multiprocessing.Pool objects



Pablo Galindo Salgado  added the comment:


New changeset 3766f18c524c57784eea7c0001602017d2122156 by Pablo Galindo in 
branch 'master':
bpo-35378: Fix multiprocessing.Pool references (GH-11627)
https://github.com/python/cpython/commit/3766f18c524c57784eea7c0001602017d2122156


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35968] lib2to3 cannot parse rf''


Karthikeyan Singaravelan  added the comment:

The reported example works fine on master and 3.7 . The related issue's fix was 
not backported to 3.6 . 3.6 is in security fixes only mode. So upgrading to 3.7 
will help and I propose closing this as a duplicate of issue33266 .

➜  cpython git:(master) ✗ cat foo.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-

regex_format = rf''
➜  cpython git:(master) ✗ ./python.exe ./Tools/scripts/2to3 foo.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: No changes to foo.py
RefactoringTool: Files that need to be modified:
RefactoringTool: foo.py
➜  cpython git:(master) ✗ python3.7 ./Tools/scripts/2to3 foo.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: No changes to foo.py
RefactoringTool: Files that need to be modified:
RefactoringTool: foo.py
➜  cpython git:(master) ✗ python3.6 ./Tools/scripts/2to3 foo.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Can't parse foo.py: ParseError: bad input: type=3, value="''", 
context=('', (4, 17))
RefactoringTool: No files need to be modified.
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse foo.py: ParseError: bad input: type=3, value="''", 
context=('', (4, 17))

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35968] lib2to3 cannot parse rf''



Karthikeyan Singaravelan  added the comment:

Seems related : issue33266

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

2019-02-11 Thread SylvainDe



SylvainDe  added the comment:

I have no explanation whatsoever but removing the call to coverage seems to 
remove the issue: 
https://travis-ci.org/SylvainDe/DidYouMean-Python/builds/491727724 .

I guess we can close this issue - sorry for the inconvenience.

@nedbat: do you want me to open an issue in coverage ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35968] lib2to3 cannot parse rf''

2019-02-11 Thread Dylan Lloyd



New submission from Dylan Lloyd :

```
#!/usr/bin/env python
# -*- coding: utf-8 -*-

regex_format = rf''
```

```
yapf poc.py
```

```
Traceback (most recent call last):
  File "lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 115, in 
ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
  File "lib/python3.6/lib2to3/pgen2/driver.py", line 107, in parse_string
return self.parse_tokens(tokens, debug)
  File "lib/python3.6/lib2to3/pgen2/driver.py", line 72, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
  File "lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=3, value="''", context=('', (4, 
17))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/yapf", line 10, in 
sys.exit(run_main())
  File "lib/python3.6/site-packages/yapf/__init__.py", line 326, in run_main
sys.exit(main(sys.argv))
  File "lib/python3.6/site-packages/yapf/__init__.py", line 213, in main
verbose=args.verbose)
  File "lib/python3.6/site-packages/yapf/__init__.py", line 263, in FormatFiles
in_place, print_diff, verify, verbose)
  File "lib/python3.6/site-packages/yapf/__init__.py", line 289, in _FormatFile
logger=logging.warning)
  File "lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 91, in 
FormatFile
verify=verify)
  File "lib/python3.6/site-packages/yapf/yapflib/yapf_api.py", line 129, in 
FormatCode
tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "lib/python3.6/site-packages/yapf/yapflib/pytree_utils.py", line 121, in 
ParseCodeToTree
tree = parser_driver.parse_string(code, debug=False)
  File "lib/python3.6/lib2to3/pgen2/driver.py", line 107, in parse_string
return self.parse_tokens(tokens, debug)
  File "lib/python3.6/lib2to3/pgen2/driver.py", line 72, in parse_tokens
if p.addtoken(type, value, (prefix, start)):
  File "lib/python3.6/lib2to3/pgen2/parse.py", line 159, in addtoken
raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=3, value="''", context=('', (4, 
17))
```

--
components: 2to3 (2.x to 3.x conversion tool)
messages: 335238
nosy: majuscule
priority: normal
severity: normal
status: open
title: lib2to3 cannot parse rf''
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

2019-02-11 Thread SylvainDe



SylvainDe  added the comment:

This is a brilliant idea! I'll give it a try and keep you posted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Change by Guido van Rossum :


--
keywords: +patch, patch, patch
pull_requests: +11848, 11849, 11850
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Change by Guido van Rossum :


--
keywords: +patch, patch
pull_requests: +11848, 11849
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Change by Guido van Rossum :


--
keywords: +patch
pull_requests: +11848
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35963] Python/symtable.c: warning: enumeration value ‘FunctionType_kind’ not handled in switch [-Wswitch]"



Guido van Rossum  added the comment:

Sure, I'll fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch



Karthikeyan Singaravelan  added the comment:

I have tried with different places where only '-' and 'w' differ. They seemed 
to produce correct diff except for this once case where the diff was confusing.

--
nosy: +tim.peters
type:  -> behavior
versions: +Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35766] Merge typed_ast back into CPython



Guido van Rossum  added the comment:


New changeset 4b250fc1da9c893803cf724a4974450b5e10bd8a by Guido van Rossum in 
branch 'master':
bpo-35766 follow-up: Add an error check to new_type_comment() (#11766)
https://github.com/python/cpython/commit/4b250fc1da9c893803cf724a4974450b5e10bd8a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager



Karthikeyan Singaravelan  added the comment:

Is this something related to running under coverage? With the failure commit on 
Travis I built the binary and there are no failures with unittest but the tests 
fail with coverage. Running 'set.add(0)' under coverage and as normal file 
generates different error messages. Can you try running with unittest to see if 
there are any discrepancies? I am adding @nedbat who might have a better 
explanation.

# Python version

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python
Python 3.8.0a1+ (tags/v3.8.0a1-23-g8a03ff2ff4:8a03ff2ff4, Feb 11 2019, 15:51:54)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

# set.add(0) under coverage and as a file

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ echo "set.add(0)" > 
foo.py
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ coverage run foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in 
set.add(0)
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in 
set.add(0)
TypeError: descriptor 'add' requires a 'set' object but received a 'int'

# Running the reported case will fail while running under coverage

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python -m unittest 
discover --start-directory=/tmp/suggest --pattern=*.py
..
--
Ran 2 tests in 0.002s

OK
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ coverage run -m 
unittest discover --start-directory=/tmp/suggest --pattern=*.py
FF
==
FAIL: test_assertRaisesRegex (foo.FooTest)
--
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/suggest/foo.py", line 8, in test_assertRaisesRegex
self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE, set.add, 0)
AssertionError: "descriptor '\w+' requires a 'set' object but received a 'int'" 
does not match "descriptor 'add' for 'set' objects doesn't apply to 'int' 
object"

==
FAIL: test_assertRaisesRegex_contextman (foo.FooTest)
--
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/suggest/foo.py", line 12, in test_assertRaisesRegex_contextman
set.add(0)
AssertionError: "descriptor '\w+' requires a 'set' object but received a 'int'" 
does not match "descriptor 'add' for 'set' objects doesn't apply to 'int' 
object"

--
Ran 2 tests in 0.003s

FAILED (failures=2)


# Executed test

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ cat /tmp/suggest/foo.py
import unittest

DESCRIPT_REQUIRES_TYPE_RE = r"descriptor '\w+' requires a 'set' object but 
received a 'int'"

class FooTest(unittest.TestCase):

def test_assertRaisesRegex(self):
self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE, set.add, 0)

def test_assertRaisesRegex_contextman(self):
with self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE):
set.add(0)

if __name__ == "__main__":
unittest.main()

--
nosy: +nedbat, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken

2019-02-11 Thread Antoine Pitrou



Antoine Pitrou  added the comment:

That sounds fine to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35955] difflib reports incorrect location of mismatch



Jason R. Coombs  added the comment:

I'm re-opening this issue as it does seem to apply stdlib (difflib.ndiff), 
which is why I encountered it both in unittest and pytest. Thanks xtreak for 
the distilled example.

--
resolution: third party -> 
stage: resolved -> 
status: closed -> open
title: unittest assertEqual reports incorrect location of mismatch -> difflib 
reports incorrect location of mismatch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35889] sqlite3.Row doesn't have useful repr



Change by Roundup Robot :


--
keywords: +patch, patch, patch, patch
pull_requests: +11844, 11845, 11846, 11847
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35889] sqlite3.Row doesn't have useful repr



Change by Roundup Robot :


--
keywords: +patch, patch
pull_requests: +11844, 11845
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35889] sqlite3.Row doesn't have useful repr



Change by Roundup Robot :


--
keywords: +patch
pull_requests: +11844
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35889] sqlite3.Row doesn't have useful repr



Change by Roundup Robot :


--
keywords: +patch, patch, patch
pull_requests: +11844, 11845, 11847
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Rémi Lapeyre  added the comment:

has_key method has been removed in Python3 so I think we can remove it from the 
DictProxy as well as nobody should be relying on it anymore.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Change by Rémi Lapeyre :


--
keywords: +patch, patch, patch
pull_requests: +11841, 11842, 11843
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +11841
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Change by Rémi Lapeyre :


--
keywords: +patch, patch
pull_requests: +11841, 11842
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault



Pablo Galindo Salgado  added the comment:

@Mark

   long x = i_result * b;

is still on the PR but the check for overflow does not use x. I will update the 
PR to move that line inside the overflow-safe block to prevent UB from 
happening and affecting the check itself (or the rest of the code).

Thanks for pointing that out!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7850] platform.system() should be "macosx" instead of "Darwin" on OSX

2019-02-11 Thread Ronald Oussoren



Ronald Oussoren  added the comment:

This issue can be closed. Status quo wins due to backward compatibility 
concerns.

BTW. The primary reason for proposing "macosx" as the return value for 
platform.system() is that the platform name as used in egg files (at the time, 
now wheels) is "macosx". That, and having two clearly different platforms with 
the same name (macOS and iOS).

And one final note: I definitely wouldn't mind having an API in platform that 
returns the marketing name and version of the current system (like mac_ver and 
linux_distribution).  But that API cannot be system_alias because its API 
contract does not give it access to enough information. Such an API could be 
designed in a new issue.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26024] Non-ascii Windows locale names

2019-02-11 Thread Steve Dower



Steve Dower  added the comment:

We should switch to _wsetlocale, or else come up with a more sensible mapping 
that makes sense between platforms (like we have for encodings already).

I suspect the latter requires proper design and discussion, so it's worth doing 
the first part immediately.

--
versions: +Python 3.8 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault

2019-02-11 Thread Mark Dickinson



Mark Dickinson  added the comment:

@Pablo: Thanks; I saw the PR.

It looks as though the troublesome line

   long x = i_result * b;

is still there in that PR, though. Or am I misreading? The difficulty here is 
that the check for overflow has to happen as a means of preventing invoking 
undefined behaviour; it doesn't really work to invoke the undefined behaviour 
first and then check to see what went wrong.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault



Pablo Galindo Salgado  added the comment:

@Mark Dickinson Thanks for the links. I had opened yesterday PR11809 addressing 
the UB and adding more tests. Could you review the PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18283] shutil.which() should support bytes

2019-02-11 Thread Cheryl Sabella



Change by Cheryl Sabella :


--
pull_requests: +11840
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault



Karthikeyan Singaravelan  added the comment:

@pablogsal I am reopening this since this has an open PR though original issue 
was fixed. Feel free to close this if this can be discussed in a separate issue.

--
resolution: fixed -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault

2019-02-11 Thread Mark Dickinson



Mark Dickinson  added the comment:

See below for a link to the Python 2.7 code for int * int multiplication (which 
needs to detect overflow so that it knows when to promote to long):

https://github.com/python/cpython/blob/2f1a317d5fdc45b9d714b067906f612f636ba08e/Objects/intobject.c#L496-L518

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken


Rémi Lapeyre  added the comment:

Hi @giampaolo.rodola, I will post a PR for this issue.

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35959] math.prod(range(10)) caues segfault

2019-02-11 Thread Mark Dickinson



Mark Dickinson  added the comment:

This approach to overflow checking needs reworking; the current code, in lines 
like:

long x = i_result * b;

induces undefined behaviour on overflow. That's something we need to avoid.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35967] Better platform.processor support

New submission from Jason R. Coombs :

or: Unable to implement 'uname' on Python due to recursive call
or: platform.uname() should avoid calling `uname` in a subprocess

In [this issue](https://github.com/jaraco/cmdix/issues/1), I stumbled across a
strange and somewhat unintuitive behavior. This project attempts to supply a
`uname` executable implemented in Python, so that such functionality could be
exposed on any platform including Windows.

What I found, however, was that because the stdlib `platform` module actually
invokes `sh -c uname -p` during most any call of the module
(https://github.com/python/cpython/blob/9db56fb8faaa3cd66e7fe82740a4ae4d786bb27f/Lib/platform.py#L836),
attempting to use that functionality on a system where `uname` is implemented
by Python (and on the path), will probably fail after a long delay due to
infinite recursion.

Moreover, the _only_ call that's currently invoking `uname` in a subprocess is
the `processor` resolution, which I suspect is rarely used, in part because the
results from it are inconsistent and not particularly useful.

For example, on Windows, you get a detailed description from the hardware:
'Intel64 Family 6 Model 142 Stepping 9, GenuineIntel'

On macOS, you get just 'i386'.

And on Linux, I see 'x86_64' or sometimes just '' (in docker).

To make matters even worse, this `uname -p` call happens unconditionally on
non-Windows systems for nearly any call in platform. As a result, it's
impossible to suppress the invocation of `uname`, especially when functionality
like `pkg_resources` and its environment markers is invoked early.

I suggest instead the platform module should (a) resolve processor information
in a more uniform manner and (b) not ever call uname, maybe [with something
like
this](https://github.com/jaraco/cmdix/blob/d61e6d3b40032a25feff0a9fb2a79afaa7dcd4e0/cmdix/command/uname.py#L53-L77).

At the very least, the `uname` call should be late-bound so that it's not
invoked unconditionally for rarely-used information.

After some period for comment, I'll draft an implementation.

--
messages: 335220
nosy: jaraco
priority: normal
severity: normal
status: open
title: Better platform.processor support

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7850] platform.system() should be "macosx" instead of "Darwin" on OSX

2019-02-11 Thread STINNER Victor



STINNER Victor  added the comment:

I suggest to close this issue.

@Ronald: Or do you think that something should still be done on macOS in the 
platform module?

This issue looks like a duplicate of bpo-35516. It has been decided that 
platform.system_alias() doesn't replace Darwin with macOS (it's now explained 
in a comment: see my commit 60875db2f67815d7d181c552bfac59e8c97619e3).

I modified platform.platform() to return "macOS" rather than "darwin" on macOS: 
"bpo-35344: platform.platform() uses mac_ver() on macOS (GH-10780)" (commit 
ea0ca218b0c28b2af2b1f6a5d3383569de7fc2c1).

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7850] platform.system() should be "macosx" instead of "Darwin" on OSX