[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: Indeed keeping this issue open wouldn't be very productive since it relates to the more general problem of Python's slow interpretation. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Mark Lawrence added the comment: I don't think there's anything to do here so can it be closed? If anything else needs discussing surely it can go to python-ideas, python-dev or a new issue as appropriate. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Josh Rosenberg shadowran...@gmail.com: -- nosy: +ShadowRanger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Mark Lawrence added the comment: What's the status of this issue? Code was committed to the default branch over a year ago, see msg182250 -- nosy: +BreamoreBoy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Yury Selivanov yselivanov...@gmail.com: -- versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Yury Selivanov yselivanov...@gmail.com: -- nosy: +yselivanov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Martin Morrison m...@ensoft.co.uk: -- nosy: +isoschiz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
STINNER Victor added the comment: More generally though, this would be improved by precompiling some of the information (like Argument Clinic does, perhaps). The same idea was already proposed to optimize str%args and str.format(args). struct.unpack() does also compile the format into an optimize structure (and have a cache). We may do something like Martin von Loewis's _Py_IDENTIFIER API: compile at runtime at the first call, and cache the result in a static variable. It's not a tiny project, and I don't know exactly how to build a JIT compiler for getargs.c, nor how complex it would be. But it would speed up *all* Python calls, so any Python application. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Mark Shannon m...@hotpy.org: -- nosy: +Mark.Shannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Stefan Behnel added the comment: Let me throw in a quick reminder that Cython has substantially faster argument parsing than the C-API functions provide because it translates function signatures like def func(int a, b=1, *, list c, d=2): ... into tightly specialised unpacking code, while keeping it as compatible as possible with the equivalent Python function (better than manually implemented C functions, BTW). Might be an alternative to the Argument Clinic, one that has been working for a couple of years now and has already proven its applicability to a large body of real world code. -- nosy: +scoder ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Terry J. Reedy added the comment: (Stefan) into tightly specialised unpacking code, Are you suggesting that func.__call__ should be specialized to func's signature, more than it is now (which is perhaps not at all), or something else? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Stefan Behnel added the comment: Cython does that in general, sure. However, this ticket is about a specific case where string methods (which are implemented in C) are slow when called from Python. Antoine found out that the main overhead is not so much from the method lookup itself but from argument parsing inside of the function. The unpacking code that Cython generates for the equivalent Python signature would speed this up, while keeping or improving the compatibility with Python call semantics. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Roundup Robot added the comment: New changeset 4e985a96a612 by Antoine Pitrou in branch 'default': Issue #17170: speed up PyArg_ParseTuple[AndKeywords] a bit. http://hg.python.org/cpython/rev/4e985a96a612 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Larry Hastings added the comment: Argument Clinic has languished for lack of time. I didn't get much feedback, though a couple people were shouting for a PEP, which I was resisting. I figured, if they have something to say, they can go ahead and reply on the tracker issue, and if they don't have something to say, why do we need a PEP? I need to reply to one bit of thorough feedback, and after that--I don't know. I'd like to get things moving before PyCon so we can point sprinters at it. -- nosy: +larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Larry Hastings added the comment: Oh, and, as to whether Argument Clinic would solve this problem, the answer is not yet. Right now Argument Clinic literally generates calls to PyArg_ParseTupleAndKeywords. (In special cases it switches to PyArg_ParseTuple.) I'm more interested in Argument Clinic from the API perspective; I wanted to make a better way of specifying arguments to functions so we got all the metadata we needed without having to endlessly repeat ourselves. Truthfully I was hoping someone else would pick up the gauntlet once it was checked in and make a new argument processing API / hack up the Argument Clinic output to make it faster. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: Truthfully I was hoping someone else would pick up the gauntlet once it was checked in and make a new argument processing API / hack up the Argument Clinic output to make it faster. Argument Clinic's preprocessing would be a very nice building block to generate faster parsing sequences. Like Nick I'd still like to see a PEP, though ;-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Barry A. Warsaw ba...@python.org: -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Amaury Forgeot d'Arc added the comment: I left some comments on Rietveld. I wonder if PyArg_ParseTupleAndKeywords can be replaced by something that would compute and cache the set of keywords; a bit like _Py_IDENTIFIER. -- nosy: +amaury.forgeotdarc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Guido van Rossum added the comment: What's the status of Argument Clinic? Won't that make this obsolete? --Guido van Rossum (sent from Android phone) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: I left some comments on Rietveld. I wonder if PyArg_ParseTupleAndKeywords can be replaced by something that would compute and cache the set of keywords; a bit like _Py_IDENTIFIER. It would make sense indeed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Nick Coghlan added the comment: To answer Guido's question about clinic, see http://bugs.python.org/issue16612 Mostly positive feedback, but several of us would like a PEP to make sure we're happy with the resolution of the limited negative feedback. -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Florent Xicluna florent.xicl...@gmail.com: -- nosy: +flox, haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Terry J. Reedy added the comment: A related issue: the speed of finding and hence replacing chars in strings is known to have regressed in 3.3 relative to 3.2, especially on Windows. For long strings, that will negate in 3.3 the speedup for the initial method call. See #16061, with patches. The holdup seems to be deciding which of two good patches to apply. -- nosy: +terry.reedy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Guido van Rossum added the comment:
Hm, you seem to be right. Changing the bug title.
So, can we speed up method lookup? It's a shame that I have to start promoting
this ugly idiom. There's a similar issue where s[:5]=='abcde' is faster than
s.startswith('abcde'):
./python.exe -m timeit -s a = 'hundred' a.startswith('foo')
100 loops, best of 3: 0.281 usec per loop
./python.exe -m timeit -s a = 'hundred' a[:3] == 'foo'
1000 loops, best of 3: 0.158 usec per loop
--
title: string replace is too slow - string method lookup is too slow
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17170
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti, serhiy.storchaka versions: +Python 3.4 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Serhiy Storchaka added the comment:
There are two overheads: an attribute lookup and a function call.
$ ./python -m timeit -s a = 'hundred' 'x' in a
1000 loops, best of 3: 0.0943 usec per loop
$ ./python -m timeit -s a = 'hundred' a.__contains__('x')
100 loops, best of 3: 0.271 usec per loop
$ ./python -m timeit -s a = 'hundred' a.__contains__
1000 loops, best of 3: 0.135 usec per loop
Time of a.__contains__('x') is greater than the sum of times of
a.__contains__ and 'x' in a.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17170
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment:
Indeed the function call cost actually dominates:
$ ./python -m timeit -s a = 'hundred' a.find('x')
100 loops, best of 3: 0.206 usec per loop
$ ./python -m timeit -s a = 'hundred'; f=a.find f('x')
1000 loops, best of 3: 0.176 usec per loop
$ ./python -m timeit -s a = 'hundred' 'x' in a
1000 loops, best of 3: 0.0431 usec per loop
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17170
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: Some crude C benchmarking on this computer: - calling PyUnicode_Replace is 35 ns (per call) - calling hundred.replace is 125 ns - calling PyArg_ParseTuple with the same signature as hundred.replace is 80 ns Therefore, most of the overhead (125 - 35 = 90 ns) is in calling PyArg_ParseTuple() to unpack the method arguments. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Serhiy Storchaka added the comment: And PyArg_ParseTupleAndKeywords() is even more slow. $ ./python -m timeit str(b'', 'utf-8', 'strict') 100 loops, best of 3: 0.554 usec per loop $ ./python -m timeit str(object=b'', encoding='utf-8', errors='strict') 100 loops, best of 3: 1.74 usec per loop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: Here is a patch yielding a decent speedup (~ 40%) on PyArg_ParseTuple itself. More generally though, this would be improved by precompiling some of the information (like Argument Clinic does, perhaps). (note: PyArg_ParseTupleAndKeywords is a completely separate implementation...) -- keywords: +patch Added file: http://bugs.python.org/file29021/getargs_freelist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: Updated patch to also handle PyArg_ParseTupleAndKeywords. -- Added file: http://bugs.python.org/file29023/getargs_freelist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Antoine Pitrou pit...@free.fr: Removed file: http://bugs.python.org/file29023/getargs_freelist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Antoine Pitrou pit...@free.fr: Removed file: http://bugs.python.org/file29021/getargs_freelist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Changes by Antoine Pitrou pit...@free.fr: Added file: http://bugs.python.org/file29024/getargs_freelist.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Guido van Rossum added the comment: Great to see some action. Would there be a problem in backporting this? It's not a new feature after all... -- stage: patch review - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17170] string method lookup is too slow
Antoine Pitrou added the comment: That would be left to the discretion of release managers. In all honesty the real-world benefit should be small (around 2% on the benchmark suite, apparently). Also, the principle of this patch doesn't apply to 2.7. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
