[issue8706] accept keyword arguments on most base type methods and builtins

[INADA Naoki]

INADA Naoki added the comment:

TL;DR

Changing positional argument name doesn't break backward compatibility.
After start accepting keyword argument, it's name is part of API and should be 
stable.

For example, document says `str.startswith(prefix[, start[, end]])`.
But this patch seems using `sub` instead of `prefix`.
https://docs.python.org/3.5/library/stdtypes.html#str.startswith

Keyword name should be chosen carefully, like choosing method name.
So I'm -1 to adding keyword argument support to many method in one issue.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29337] BytesWarning at compile time

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

BytesWarning can be raised during compilation.

$ ./python -Wa -b -c "lambda: 'a'; lambda: b'a'"
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string

_PyCode_ConstantKey() should produce keys that don't allow comparing bytes with 
strings nested in tuples or frozensets. Currently it returns (tuple, ('a',), 
((str, 'a'),)) for ('a',) and (tuple, (b'a',), ((bytes, b'a'),)) for (b'a',). 
Key tuples have the same size and the same first element. Comparing second 
elements emits a BytesWarning.

--
components: Interpreter Core
messages: 285940
nosy: haypo, inada.naoki, serhiy.storchaka
priority: normal
severity: normal
status: open
title: BytesWarning at compile time
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29314] asyncio.async deprecation warning is missing stacklevel=2

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Attached patch adds stacklevel=2 to the deprecation warning.

--
keywords: +patch
nosy: +Mariatta
Added file: http://bugs.python.org/file46369/issue29314.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29336] merge tuples in module

[INADA Naoki]

Changes by INADA Naoki :


Added file: http://bugs.python.org/file46368/mergetuples2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29336] merge tuples in module

[INADA Naoki]

INADA Naoki added the comment:

Oh, thanks.  I hadn't checked the warning.
Since bytes are not important in this time, I'll remove bytes support.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29336] merge tuples in module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

$ ./python -Wa -b -c "lambda: 'a'; lambda: b'a'"
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string
sys:1: BytesWarning: Comparison between bytes and string

Look at _PyCode_ConstantKey(), it may help.

--
components: +Interpreter Core
nosy: +serhiy.storchaka
type:  -> enhancement
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29332] Uniform SelectSelector._select behavior

[Wen Adam]

Wen Adam added the comment:

I know it works on python3.4, but black magic still exists in selectors lib，and 
3rd-party module have to prepare two function to make compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29332] Uniform SelectSelector._select behavior

[Berker Peksag]

Berker Peksag added the comment:

Great! Since this is no longer an issue on Python 3.4+, I'm going to close this 
as a duplicate of issue 27634 and merge 
https://github.com/berkerpeksag/selectors34/pull/4.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> selectors.SelectSelectors fails if select.select was patched

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29332] Uniform SelectSelector._select behavior

[Wen Adam]

Wen Adam added the comment:

Thx for your replying, Peksag.

I encountered this problem is really because of Gevent and Selector34. Gevent 
fixed this issue but only works for 
python3.4+(https://github.com/gevent/gevent/blob/master/src/gevent/monkey.py#L497)
 by a tricking way. python 2.x still has this problem

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29336] merge tuples in module

[INADA Naoki]

INADA Naoki added the comment:

I tried this patch with attached script.

```
$ venv/bin/pip install django flask sqlalchemy
$ PYTHONTRACEMALLOC=5 venv/bin/python3 tuplemem.py > tuples.txt
$ sort tuples.txt | uniq -c | sort -nr > tuplecount
```

## default
memory: (32254693, 32292635)
tuples: 64968

head -n10 tuplecount-default
   5479 (None,)
   3069 ('self',)
727 (,)
688 ('__class__',)
321 ('NotImplementedError',)
287 ('self', 'other')
264 (None, None)
207 (False,)
193 (None, 0)
176 (None, False)

## patched
memory: (31224697, 31261892)
tuples: 51298

head -n10 tuplecount-patched
   1437 (None,)
727 (,)
328 ('self',)
264 (None, None)
207 (False,)
193 (None, 0)
138 ('__class__',)
114 (True,)
112 (None, False)
110 ('target', 'fn')


I'll try again with my company's private codebase in next week.

--
Added file: http://bugs.python.org/file46367/tuplemem.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29336] merge tuples in module

[INADA Naoki]

New submission from INADA Naoki:

Tuples consists of None, True, False, str and bytes can be merged safely.
To avoid memory leak, this patch shares such tuples in compile unit (module, in 
most case.)

--
files: merge-tuples.patch
keywords: patch
messages: 285933
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: merge tuples in module
Added file: http://bugs.python.org/file46366/merge-tuples.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29328] struct module should support variable-length strings

[Cameron Simpson]

Changes by Cameron Simpson :


--
nosy: +cameron

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28595] shlex.shlex should not augment wordchars

[Evan Andrews]

Evan Andrews added the comment:

Unfortunately shlex.shlex's defaults are probably going to remain that way for 
a long time in order to avoid breaking backwards compatibility. Presumably 
shlex.split was added so you didn't have to remember to set posix and 
whitespace_split yourself.

The particular problem I'm addressing in this issue is that the new 
punctuation_chars argument doesn't currently work with whitespace_split.

>>> def split(text, ws=False, pc=False):
... s = shlex.shlex(text, posix=True, punctuation_chars=pc)
... s.whitespace_split = ws
... return list(s)
...
>>> split('foo,bar>baz')
['foo', ',', 'bar', '>', 'baz']
>>> split('foo,bar>baz', ws=True)
['foo,bar>baz']
>>> split('foo,bar>baz', pc=True)
['foo', ',', 'bar', '>', 'baz']
>>> split('foo,bar>baz', ws=True, pc=True)
['foo,bar>baz']

With my patch, the last example outputs ['foo,bar', '>', 'baz'].

Before the release of 3.6 I was arguing that punctuation_chars should not 
attempt to augment wordchars at all, since the idea of wordchars is inherently 
incorrect as you point out. Now I think it's too late to change, hence my patch 
treats this as a new feature in 3.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29326] Blank lines in ._pth file are not ignored

[Ammar Askar]

Ammar Askar added the comment:

I've attached a patch that makes the site package ignore empty lines in .pth 
files. Along with a test for this behavior. I'm not really familiar with the 
site machinery so I hope the test and code are fine.

--
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46365/empty_path.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28595] shlex.shlex should not augment wordchars

[Gustavo Goretkin]

Gustavo Goretkin added the comment:

Sorry, I typo'd that last example pretty badly. Should be

>>> list(shlex.shlex(b"mkdir Bad\xffButLegalPath".decode("utf-8", 
>>> "surrogateescape")))
['mkdir', 'Bad', '\udcff', 'ButLegalPath']

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29331] Simplify argument parsing in sorted() and list.sort()

[Josh Rosenberg]

Changes by Josh Rosenberg :


--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28595] shlex.shlex should not augment wordchars

[Gustavo Goretkin]

Gustavo Goretkin added the comment:

>Instead of trying to enumerate all possible wordchars, I think a more robust 
>solution is to use whitespace_split to include *all* characters not otherwise 
>considered special.

I agree with that approach.

Also note that dash/hyphen gets incorrectly tokenized.

>>> import shlex
>>> list(shlex.shlex("mkdir -p somepath"))
['mkdir', '-', 'p', 'somepath']

White listing all valid word characters is not good, because the 
surrogateescape mechanism can include all sorts of "characters".

In bash:

$ echo mkdir $(echo -ne "Bad\xffButLegalPath")
mkdir Bad?ButLegalPath

the path is one token.

However currently in shlex, it gets broken into multiple tokens:

>>> list(shlex.shlex(b"mkdir Bad\ffButLegalPath".decode("utf-8", 
>>> "surrogoateescape")))
['mkdir', 'Bad', '\x0c', 'fButLegalPath']

--
nosy: +Gustavo Goretkin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29332] Uniform SelectSelector._select behavior

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the report! This is a duplicate of issue 27634.

Since we got multiple reports [1][2] about the same problem on selectors34 
(backport of selector module) and use of Gevent is very popular, I think we 
should reconsider the idea of wrapping select.select with staticmethod().

Also, if I remember correctly, this was fixed on Gevent's side before (although 
I know that you didn't mention Gevent in your report)

Victor and Charles-François, what do you think?

[1] https://github.com/berkerpeksag/selectors34/pull/4
[2] https://github.com/berkerpeksag/selectors34/issues/5

--
nosy: +berker.peksag, haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29328] struct module should support variable-length strings

[Elizabeth Myers]

Elizabeth Myers added the comment:

Also, to add to the discussion:

* Rejecting this because "it doesn't cover every use case" is a red herring at 
best. If this can't cover your use case, odds are the struct module can *never* 
cover it. That is no reason to reject it alone; you would need something more 
heavyweight than the struct module anyway.

* If the module can't cover your use case with this feature, it can't cover it 
right now, so why obstruct it? It covers my use cases for it just fine.

* Not everyone needs something more heavyweight, or wants to import some bigger 
module just because they need variable-length strings.

* If the real goal is to discourage use of the struct module, too bad. People 
are actually using it in production and it serves its (rather small) purpose 
very well. Other people would like to use the module for their use cases, but 
presently cannot, and this proposal would help cover their particular cases.

* The fact that the netstruct module exists with this feature is proof enough 
there's demand; not to mention the discussion on the python-ideas ML shows that 
many people already would find this very useful. It's not like I'm proposing 
adding braces or some horrible huge proposal, I'm adding two format specifiers. 
*Two.*

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29328] struct module should support variable-length strings

[Elizabeth Myers]

Elizabeth Myers added the comment:

Hi,

After discussing this on the python-ideas ML a bit more, this is actually a 
feature people want a great deal. It can't cover every use case, but to expand 
it further than this proposal and make it do so is way beyond the scope of this 
proposal.

It may not be completely useful for every protocol, but it is sufficiently 
useful for many people who have simpler use cases. There is no reason to 
prevent the addition of this feature other than what boils down to "well code 
will have to be written..."

I don't buy the argument that it's "outside the scope of the module" more than 
I think it's more "I don't like the idea of struct being used for non-fixed 
data." C structures support zero-terminated char arrays, and there is already a 
Pascal string type.

I didn't realise there'd be this much opposition to just adding two format 
specifiers... :/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28735] Mock is equal to ANY but MagicMock is not

[Berker Peksag]

Berker Peksag added the comment:

Serhiy's patch looks pretty good to me, thanks! Also, thanks for the report, 
Rafael. I definitely forgot to fix this case after fixing issue 25195.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8706] accept keyword arguments on most base type methods and builtins

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Most overhead of supporting keyword arguments when pass positional arguments 
was removed in issue29029. But still there is an overhead of passing argument 
by keywords. It can be decreased when convert functions to Argument Clinic or 
use new private argument parsing API. This is not very easy issue.

--
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26729] Incorrect __text_signature__ for sorted

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue29327 and issue29331.

sorted_3.patch LGTM.

--
stage: patch review -> commit review
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Ned Deily]

Changes by Ned Deily :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29334] ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange

[Greg Stark]

Greg Stark added the comment:

adding script the illustrates the bug.

--
Added file: http://bugs.python.org/file46364/sslbugs.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29335] Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD

[Zach Riggle]

New submission from Zach Riggle:

The attached script hits some "This should never happen" code in the subprocess 
module.

These lines here:
https://github.com/python/cpython/blob/2.7/Lib/subprocess.py#L1036-L1038

The root cause is a lack of checking WIFSTOPPED and WSTOPSIG in the handler.

When a process elects into being ptraced via PTRACE_TRACEME, it is stopped on 
the SIGSEGV instead of terminating, allowing the user to attach a debugger 
before the kernel destroys the process.

This bug makes it impossible to wait on any process which crashes, which is set 
up to wait for a debugger.

--
components: Library (Lib)
files: bug.py
messages: 285921
nosy: Zach Riggle
priority: normal
severity: normal
status: open
title: Python 2.7 subprocess module does not check WIFSTOPPED on SIGCHLD
versions: Python 2.7
Added file: http://bugs.python.org/file46363/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file46362/enumobject-docstrings.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch that restores ol docstrings for enumerate() and reversed(). But 
it may be better to change pydoc so that it would output a text signature of 
class constructor if available.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Microbenchmarks:

$ ./python -m perf timeit --duplicate 100 "enumerate('abc')"
Unpatched:  Median +- std dev: 1.76 us +- 0.10 us
Patched:Median +- std dev: 1.61 us +- 0.07 us

$ ./python -m perf timeit --duplicate 100 "enumerate('abc', 1)"
Unpatched:  Median +- std dev: 2.14 us +- 0.09 us
Patched:Median +- std dev: 1.76 us +- 0.07 us

$ ./python -m perf timeit --duplicate 100 "reversed('abc')"
Unpatched:  Median +- std dev: 1.20 us +- 0.06 us
Patched:Median +- std dev: 1.20 us +- 0.07 us

enumerate() is 9-21% faster (due to avoiding of tuple creating), reversed() is 
not changed (Argument Clinic generates the same parsing code for it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28735] Mock is equal to ANY but MagicMock is not

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch fixes comparing MagicMock with ANY.

--
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file46361/magicmock_eq.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Raymond Hettinger]

Raymond Hettinger added the comment:

When reviewing AC patches, we should always compare the help() before and 
after.  

Also, if the code already had fast parsing like:
   if (!PyArg_UnpackTuple(args, "reversed", 1, 1, ) )
there needs to be a before and after timing to make sure there wasn't a 
performance regression.

When PyArg_NoKeywords was present, we need to verify that the AC version also 
precludes keyword arguments (to prevent the creation of unhelpful keyword args 
and to keep compatibility with other versions of Python).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Raymond Hettinger]

Raymond Hettinger added the comment:

The application of AC to enumerate() lost information about the start argument 
and the signature of the call.   We're going backwards.

 New help -
class enumerate(object)
 |  Return an enumerate object.
 |
 |iterable
 |  an object supporting iteration
 |
 |  The enumerate object yields pairs containing a count (from start, which
 |  defaults to zero) and a value yielded by the iterable argument.
 |
 |  enumerate is useful for obtaining an indexed list:
 |  (0, seq[0]), (1, seq[1]), (2, seq[2]), ...

 Old help -

class enumerate(object)
 |  enumerate(iterable[, start]) -> iterator for index, value of iterable
 |  
 |  Return an enumerate object.  iterable must be another object that supports
 |  iteration.  The enumerate object yields pairs containing a count (from
 |  start, which defaults to zero) and a value yielded by the iterable argument.
 |  enumerate is useful for obtaining an indexed list:
 |  (0, seq[0]), (1, seq[1]), (2, seq[2]), ...


Also, reversed() lost the indication of its signature:
  reversed(sequence) -> reverse iterator over values of the sequence

And the help doesn't have the usual:

iterable
|  an object supporting iteration

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16285] Update urllib quoting to RFC 3986

[Piotr Dobrogost]

Changes by Piotr Dobrogost :


--
nosy: +piotr.dobrogost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20186] Derby #18: Convert 31 sites to Argument Clinic across 23 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are patches based on modules_issue20186.patch that convert to Argument 
Clinic 4 modules: _csv, _lsprof, _tracemalloc and symtable. They are 
synchronized with current sources and updated to current Argument Clinic. 
Other changes:

* Addressed Larry's comments.

* Used converter names instead of format units (object instead of 'O' etc).

* Removed unneeded self declarations in _lsprof.

* Removed changes for _csv.Dialect.__new__. Generated signature has None as 
default values, but this is not true. Actually parameters don't have default 
values. This function can't be converted.

* Parameters of _lsprof.Profiler.enable should have the int converter, not 
bool. Default values are -1 (meaning "don't change current value"), not True.

* Made the parameter of _tracemalloc._get_object_traceback() positional-only.

* Simplified code for creating a result in _traceback functions.

--
Added file: http://bugs.python.org/file46357/csv_clinic.patch
Added file: http://bugs.python.org/file46358/lsprof_clinic.patch
Added file: http://bugs.python.org/file46359/tracemalloc_clinic.patch
Added file: http://bugs.python.org/file46360/symtable_clinic.patch

___
Python tracker 

___diff -r f44f44b14dfc Modules/_csv.c
--- a/Modules/_csv.cFri Jan 20 08:35:18 2017 +0200
+++ b/Modules/_csv.cFri Jan 20 18:17:14 2017 +0200
@@ -13,6 +13,12 @@ module instead.
 #include "Python.h"
 #include "structmember.h"
 
+#include "clinic/_csv.c.h"
+/*[clinic input]
+module _csv
+class _csv.Dialect "PyObject *" "_Type"
+[clinic start generated code]*/
+/*[clinic end generated code: output=da39a3ee5e6b4b0d input=6f3170c88d1c5a7a]*/
 
 typedef struct {
 PyObject *error_obj;   /* CSV exception */
@@ -1421,12 +1427,21 @@ csv_writer(PyObject *module, PyObject *a
 /*
  * DIALECT REGISTRY
  */
+
+/*[clinic input]
+_csv.list_dialects
+
+Return a list of all know dialect names.
+[clinic start generated code]*/
+
 static PyObject *
-csv_list_dialects(PyObject *module, PyObject *args)
+_csv_list_dialects_impl(PyObject *module)
+/*[clinic end generated code: output=a5b92b215b006a6d input=a31ccb1071637d7c]*/
 {
 return PyDict_Keys(_csvstate_global->dialects);
 }
 
+
 static PyObject *
 csv_register_dialect(PyObject *module, PyObject *args, PyObject *kwargs)
 {
@@ -1454,36 +1469,69 @@ csv_register_dialect(PyObject *module, P
 return Py_None;
 }
 
+/*[clinic input]
+_csv.unregister_dialect
+
+name: object
+/
+
+Delete the name/dialect mapping associated with a string name.
+[clinic start generated code]*/
+
 static PyObject *
-csv_unregister_dialect(PyObject *module, PyObject *name_obj)
+_csv_unregister_dialect(PyObject *module, PyObject *name)
+/*[clinic end generated code: output=cf44401586cd32eb input=24474725768445c0]*/
 {
-if (PyDict_DelItem(_csvstate_global->dialects, name_obj) < 0)
+if (PyDict_DelItem(_csvstate_global->dialects, name) < 0)
 return PyErr_Format(_csvstate_global->error_obj, "unknown dialect");
-Py_INCREF(Py_None);
-return Py_None;
+Py_RETURN_NONE;
 }
 
+
+/*[clinic input]
+_csv.get_dialect
+
+name: object
+/
+
+Return the dialect instance associated with name.
+[clinic start generated code]*/
+
 static PyObject *
-csv_get_dialect(PyObject *module, PyObject *name_obj)
+_csv_get_dialect(PyObject *module, PyObject *name)
+/*[clinic end generated code: output=8f66fc0a27b68e24 input=c97af3b62e402afa]*/
 {
-return get_dialect_from_registry(name_obj);
+return get_dialect_from_registry(name);
 }
 
+
+/*[clinic input]
+_csv.field_size_limit
+
+[
+limit: object
+]
+/
+
+Sets an upper limit on parsed fields.
+
+Returns old limit. If limit is not given, no new limit is set and
+the old limit is returned.
+[clinic start generated code]*/
+
 static PyObject *
-csv_field_size_limit(PyObject *module, PyObject *args)
+_csv_field_size_limit(PyObject *module, int group_right_1, PyObject *limit)
+/*[clinic end generated code: output=49e0056f0b1e96f7 input=76b66600a7195ac4]*/
 {
-PyObject *new_limit = NULL;
 long old_limit = _csvstate_global->field_limit;
 
-if (!PyArg_UnpackTuple(args, "field_size_limit", 0, 1, _limit))
-return NULL;
-if (new_limit != NULL) {
-if (!PyLong_CheckExact(new_limit)) {
+if (group_right_1) {
+if (!PyLong_CheckExact(limit)) {
 PyErr_Format(PyExc_TypeError,
  "limit must be an integer");
 return NULL;
 }
-_csvstate_global->field_limit = PyLong_AsLong(new_limit);
+_csvstate_global->field_limit = PyLong_AsLong(limit);
 if (_csvstate_global->field_limit == -1 && PyErr_Occurred()) {
 _csvstate_global->field_limit = old_limit;
 return NULL;
@@ -1583,44 +1631,22 @@ PyDoc_STRVAR(csv_writer_doc,
 "\n"
 "The \"fileobj\" argument 

[issue29281] json.loads documentation missing "versionchanged" statement

[Ammar Askar]

Ammar Askar added the comment:

Which part is misleading, do you think the use of "encoding" could be confused 
with the argument encoding? 

There is a note right above the versionchanged block that says:

`The other arguments have the same meaning as in load(), except encoding which 
is ignored and deprecated.`


If that is the concern then I think changing the wording to be, "the encoding 
of the input string should be ..."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29316] Keep typing.py provisional for the duration of the Python 3.6 release cycle

[Ned Deily]

Ned Deily added the comment:

Patch LGTM, thanks Ivan!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29316] Keep typing.py provisional for the duration of the Python 3.6 release cycle

[Roundup Robot]

Roundup Robot added the comment:

New changeset 72c64b6e5486 by Ned Deily in branch '3.6':
Issue #29316: Restore the provisional status of typing module and add
https://hg.python.org/cpython/rev/72c64b6e5486

New changeset ef416878495f by Ned Deily in branch 'default':
Issue #29316: merge 3.6
https://hg.python.org/cpython/rev/ef416878495f

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29316] Keep typing.py provisional for the duration of the Python 3.6 release cycle

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Yes, I attached the patch.

--
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46356/provisional-typing-patch.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28735] Mock is equal to ANY but MagicMock is not

[Kushal Das]

Kushal Das added the comment:

Now I understood, thanks. I think I should have more coffee before commenting 
on bugs :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28735] Mock is equal to ANY but MagicMock is not

[Xiang Zhang]

Xiang Zhang added the comment:

Kushal I think the OP is reporting `mock.MagicMock() == mock.ANY` is False 
while `mock.ANY == mock.MagicMock()` is True which is still the case in 3.7. A 
related issue is #25195. In that issue Serhiy has mentioned this 
behaviour(http://bugs.python.org/issue25195#msg259822).

--
nosy: +berker.peksag, serhiy.storchaka, xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28735] Mock is equal to ANY but MagicMock is not

[Kushal Das]

Kushal Das added the comment:

>From a checkout in last week.

Python 3.7.0a0 (default:c163a76163f7, Jan 15 2017, 22:20:24) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from unittest import mock
>>> mock.ANY == mock.Mock() 
True
>>> mock.ANY == mock.MagicMock() 
True

--
nosy: +kushal.das

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29325] pysqlite: Evaluate removal of sqlite3_stmt_readonly

[R. David Murray]

R. David Murray added the comment:

Python ships with RH6, and includes a working sqlite3 module.  So if you are 
restricting yourself to software directly supported by RH6, you are fine.  If 
you are installing python3.6 on your rh6 box, then you can also install a more 
up to date sqlite in order to get it to compile the sqlite3 module.  So I don't 
see any actionable issue because of RH6 here, either for us or for pysqlite.  
The other issue is a different story, of course.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29334] ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange

[Greg Stark]

Greg Stark added the comment:

Christian,

I will gladly do so a little later today. Thanks for your quick response.

--greg

On Fri, Jan 20, 2017 at 7:29 AM, Christian Heimes 
wrote:

>
> Christian Heimes added the comment:
>
> Hi Grek,
>
> can you provide a script to reproduce the problem, please?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29334] ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange

[Christian Heimes]

Christian Heimes added the comment:

Hi Grek,

can you provide a script to reproduce the problem, please?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29319] Embedded 3.6.0 distribution cannot run pyz files

[Nick Coghlan]

Nick Coghlan added the comment:

I think this is just a straight-up interaction bug where we never accounted for 
the combination of -I with directory and zipfile execution back when -I was 
added.

>From a bug fix perspective, I think the right thing to do in all affected 
>versions would be to add a check to RunMainFromImporter such that it does an 
>insert rather than a set when -I is used.

For 3.7+, it may make sense to change the signature of 
runpy._run_module_as_main to accept the path entry as Steve suggests, as that 
could also resolve the quirk where we'll run a __main__ module from *anywhere* 
on sys.path, rather than only from the given path entry.

The core of the current approach was implemented back in the 2.6 time frame, 
and then tweaked a bit in 2.7, and hence predates all the niceties offered by 
the importlib API.

However, the latter change would be a separate RFE rather than a bug fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29334] ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange

[Greg Stark]

New submission from Greg Stark:

In my experiments with the relatively new class SSLObject from the ssl module 
I've noticed the following behavior(s) which I think can be described as bugs.

The getpeercert() method raises a ValueError exception "handshake not done" 
even after the handshake has successfully completed. If, however, I call the 
do_handshake() method *after* the handshake completes, then getpeercert() 
correctly runs and returns the peer's certificate. So now let's focus on 
do_handshake(). This method is basically undocumented, which I thought was ok 
because what it does should be obvious. It does seem to initiate a handshake if 
it's the first method call after the SSLObject is created. If called afterward, 
it doesn't outwardly appear to do anything, but as mentioned previously it does 
magically make the getpeercert() method start working.

--
assignee: christian.heimes
components: SSL
messages: 285903
nosy: Greg Stark, christian.heimes
priority: normal
severity: normal
status: open
title: ssl.SSLObject method getpeercert() is buggy, do_handshake() is strange
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29333] ConfigParser calls Interpolation.before_read after reading

[Gereon Kaiping]

New submission from Gereon Kaiping:

In its current implementation, `ConfigParser` calls its Interpolation's 
`before_read` method in the very last step of `_read`, when all properties have 
already been overwritten by their new uninterpolated values.

I am developing a program with modular config files: It is possible to supply 
several configuration files on the command line, and they are all fed through 
the `.read` method.

Now it would be amazing to use `read` time interpolation instead of `get` time 
interpolation to construct things like

outputfilename = %(outputfilename)s_extension_from_this_ini_module

By looking at the `Interpolation` class, it seems that behaviour like this 
should be supported by supplying a `before_read` as follows.

def before_read(self, parser, section, option, value):
L = []
interpolations = parser[section]
self._interpolate_some(
parser, option, L, value, section, interpolations, 1)
return ''.join(L)

However, this is not possible, because `before_read` is only called *after* all 
values in the config file have been read and all old values in the ConfigParser 
object have been overridden.

The attached file contains a subclass of `BasicInterpolation` and a subclass of 
`ConfigParser` which in concert allow me to write recursive property 
definitions as given above.

The downside of this change is that (a) interpolation values can't be defined 
*after* they are used any more, and (b) it is not possible to hack this parser 
to accept multi-line option names as interpolations. (To me personally, both of 
these don't sound like useful features to begin with.)

--
files: earlyinterpolation.py
messages: 285902
nosy: Anaphory
priority: normal
severity: normal
status: open
title: ConfigParser calls Interpolation.before_read after reading
Added file: http://bugs.python.org/file46355/earlyinterpolation.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29331] Simplify argument parsing in sorted() and list.sort()

[STINNER Victor]

STINNER Victor added the comment:

sort-arg-parse-2.patch now LGTM with the comment, thanks :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29327] SystemError or crash in sorted(iterable=[])

[STINNER Victor]

Changes by STINNER Victor :


--
title: SystemError or crash in sorted(iterable= -> SystemError or crash in 
sorted(iterable=[])

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29327] SystemError or crash in sorted(iterable=

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you for your reviews Victor and Raymond.

As for more general Raymond comments, I agreed with many of them in principle, 
but this isn't directly related to this issue. Victor opened a topic on the 
python-committers mailing list:

https://mail.python.org/pipermail/python-committers/2017-January/004129.html

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29327] SystemError or crash in sorted(iterable=

[STINNER Victor]

STINNER Victor added the comment:

Raymond Hettinger: "A few random thoughts that may or may not be helpful: (...)"

I replied on the python-committers mailing list:
https://mail.python.org/pipermail/python-committers/2017-January/004129.html

I'm not sure that this specific issue is the best place to discuss, I propose 
to continue the discussion there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29051] Improve error reporting involving f-strings (PEP 498)

[Mark Shannon]

Mark Shannon added the comment:

It is also worth mentioning that incorrect line numbers means that tools like 
pyflakes, pylint, mypy, lgtm, etc, need to reimplement parsing of f-strings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Christian Heimes]

Christian Heimes added the comment:

PS: I'm also against a hashlib.fips module in stdlib. FIPS mode is irrelevant 
for majority of users and countries. I neither want to confuse people nor 
introduce more maintenance and documentation burden than necessary. Antoine 
gave another good reason against a fips module, too.

I'm fine with a used_for_security flag and functions to get/set FIPS state. 
Something like hashlib.get_fips_mode() is useful for testing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29051] Improve error reporting involving f-strings (PEP 498)

[Mark Shannon]

Mark Shannon added the comment:

This problem is the parsing of f-strings. 

The expressions in an f-string are not "eval"ed in the sense of the eval() 
function. They are evaluated exactly the same as any other Python expression. 
However the parsing of f-strings does not provide correct line numbers.

This problem also manifests itself in the ast and tokenize modules.

>>> m = ast.parse("""f'''
... {
... FOO
... }
... '''
... """)

>>> m.body[0].value.values[1].value.id
'FOO'
>>> m.body[0].value.values[1].value.lineno
2

That 2 should be a 3, and yet
eval(compile(m, "test2", "exec"))
  File "", line 1, in 
  File "test2", line 5, in 
NameError: name 'FOO' is not defined

gives line 5 for the error, so not only are the line numbers wrong they are 
inconsistent.

The problem is that the internals of the f-string are not tokenized and parsed 
using the normal mechanism, but in an ad-hoc fashion in Python-ast.c as 
demonstrated when tokenizing the source

$ python3.6 -m tokenize test2
0,0-0,0:ENCODING   'utf-8'
1,0-5,3:STRING "f'''\n{\nFOO\n}\n'''"
5,3-5,4:NEWLINE'\n'   
6,0-6,0:ENDMARKER  ''

The f-string could should be tokenized as something like:
FSTRING_START f'''
STRING_PART \n 
LEFT_BRACE {
NEWLINE
IDENTIFIER FOO
NEWLINE
RIGHT_BRACE }
STRING_PART \n
FSTRING_END '''

Although this would complicate the tokenizer, it would mean that the internals 
of f-strings could be made explicit in the grammar, and that the compiler could 
generate correct offsets.

--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Christian Heimes]

Christian Heimes added the comment:

Objection from hashlib maintainer: I will reject a used_for_security flag with 
default of False. I'm slowly moving Python to a secure-by-default policy. 
Therefore used_for_security must be an explicit opt-out.

I'm aware that the policy will require modifications to all software that uses 
MD5. To be honest that's my goal. If you care about FIPS, then any use of MD5 
must be a concious and careful decision. I want developers to move away from 
MD5 and replace it with SipHash24, Blake2 or SHA-2. MD5 should *only* remain 
when backwards incompatibility prevent migration.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29328] struct module should support variable-length strings

[Antoine Pitrou]

Antoine Pitrou added the comment:

To add a bit to what Yury said, even framing isn't always compatible with this 
proposal.  For example, in dask/distributed, we first have a word for the 
number of frames, then one word per frame to indicate each frame's length, then 
the frame bodies.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29160] pow with three int arguments works like it had two arguments

[Mark Dickinson]

Changes by Mark Dickinson :


--
resolution: not a bug -> third party

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9924] sqlite3 SELECT does not BEGIN a transaction, but should according to spec

[Aviv Palivoda]

Changes by Aviv Palivoda :


--
nosy: +palaviv

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28518] execute("begin immediate") throwing OperationalError

[Aviv Palivoda]

Aviv Palivoda added the comment:

Uploading a new patch with fixes from Ma Lin comments.
Two points:
1. Should we add the VACUUM with a explicit commit? Maybe there should be an 
implicit commit before VACUUM?
2. Should a SELECT start a transaction? I think it should according to PEP 249. 
There is a open issue on the case (#9924). Should we just change this on this 
patch?

--
Added file: http://bugs.python.org/file46354/sqlite-ddl-dml-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29331] Simplify argument parsing in sorted() and list.sort()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file46353/sort-arg-parse-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29281] json.loads documentation missing "versionchanged" statement

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The wording looks little misleading. There is the encoding parameter in 
json.loads(). It is deprecated and ignored.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29332] Uniform SelectSelector._select behavior

[Wen Adam]

New submission from Wen Adam:

SelectSelector._select is differently on different platforms.

On win32, SelectSelector._select is a unbound/bound method and pass instance as 
the first argument, but on *nix, SelectSelector._select is a 
builtin_function_or_method(Although self.select still works)

If someone want to replace(or patch) select, he must implement two select 
function(one is a function, the other is a instance method), 
Otherwise you will get「TypeError: select() takes at most 4 arguments (5 given)」.

And I think make built-in funtion(like len, select.select) as a class 
attr/method is a black magic(it's still a function) and hard to maintain

--
components: Library (Lib)
files: uniform_select_behaviour.patch
keywords: patch
messages: 285891
nosy: Wen Adam, neologix
priority: normal
severity: normal
status: open
title: Uniform SelectSelector._select behavior
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46352/uniform_select_behaviour.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com