[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
Added file: 
http://bugs.python.org/file45947/faster_keyword_args_parse_alt2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Just for the history, there are two alternative patches. They unpack keyword 
arguments to linear array. I expected this approach can add more optimization, 
but actually the benefit is too small or negative.

--
Added file: http://bugs.python.org/file45946/faster_keyword_args_parse_alt.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, I came to conclusion than needed to push existing issues for separate 
files. I'm sure there are ready patches waiting for review. Now there is 
additional reason for converting to Argument Clinic. But some files contain 
only one PyArg_ParseTupleAndKeywords(), I think we can convert them in one 
patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Brett Cannon]

Brett Cannon added the comment:

I think for converting uses to Argument Clinic it can be done in a more 
iterative process on a per-module basis. How many modules do we have left to 
convert? If it isn't ridiculously huge we could open individual issues to 
convert them each.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I left this issue open for three reasons.

1. I had ideas and almost finished patch for different optimization. 
Unfortunately my hope was not justified, new implementation is slower. If I 
fail to fix it in few days, I'll close the issue.

2. For bikeshedding in case somebody want to suggest different names or 
interface.

3. I was going to convert most occurrences of PyArg_ParseTupleAndKeywords() to 
Argument Clinic for achieving larger effect of this optimization. But this 
patch was larger than I expected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[STINNER Victor]

STINNER Victor added the comment:

The issue can now be closed no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Roundup Robot]

Roundup Robot added the comment:

New changeset e527715bd0b3 by Serhiy Storchaka in branch 'default':
Issue #27574: Decreased an overhead of parsing keyword arguments in functions
https://hg.python.org/cpython/rev/e527715bd0b3

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Raymond Hettinger]

Raymond Hettinger added the comment:

Normally, LGTM is an almost useless comment, but the patch does in fact look 
good to me.  I like how compact and straight-forward the changes are to the 
individual parsing calls.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Indeed, in issue17170 this issue was discussed first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[STINNER Victor]

STINNER Victor added the comment:

See also the old issue #17170 "string method lookup is too slow".

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Updated patch addresses Antoine's comments. All checks of format string are 
moved into parser_init.

I experimented with Antoine's idea about making vgetargskeywords a simple 
wrapper around vgetargskeywordsfast with one-shot parser, but this slows down 
parsing positional arguments too much (due to creating Python strings for 
unused keyword names).

--
Added file: http://bugs.python.org/file43996/faster_keyword_args_parse_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ping.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Brett Cannon]

Brett Cannon added the comment:

I haven't reviewed the patch, but the idea is great as I know one of Larry's 
hopes of using Argument Clinic was to allow for this kind of speed-up.

--
nosy: +brett.cannon, larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27574] Faster parsing keyword arguments

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Parsing keyword arguments is much more slow than parsing positional arguments. 
Parsing time can be larger that useful execution time.

$ ./python -m timeit "b'a:b:c'.split(b':', 1)"
100 loops, best of 3: 0.638 usec per loop
$ ./python -m timeit "b'a:b:c'.split(b':', maxsplit=1)"
100 loops, best of 3: 1.64 usec per loop

The main culprit is that Python strings are created for every keyword name on 
every call.

Proposed patch adds alternative API that caches keyword names as Python strings 
in special object. Argument Clinic is changed to use this API in generated 
file. An effect of the optimization:

$ ./python -m timeit "b'a:b:c'.split(b':', maxsplit=1)"
100 loops, best of 3: 0.826 usec per loop

Invocations of PyArg_ParseTupleAndKeywords() in non-generated code are kept, 
since API is not stable yet. Later I'm going to cache parsed format strings and 
speed up parsing positional arguments too.

--
components: Interpreter Core
files: faster_keyword_args_parse.patch
keywords: patch
messages: 270832
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Faster parsing keyword arguments
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file43794/faster_keyword_args_parse.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com