[issue2459] speedup loops with better bytecode
Changes by Jesús Cea Avión [EMAIL PROTECTED]: -- nosy: +jcea __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by P. Henrique Silva [EMAIL PROTECTED]: -- nosy: +phsilva __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Armin Rigo [EMAIL PROTECTED] added the comment: Can you see if this simpler patch also gives speed-ups? (predict_loop.diff) -- nosy: +arigo Added file: http://bugs.python.org/file9877/predict_loop.diff __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: Armin, your patch gives a speed-up for for loops and comprehensions, although a bit less. Also, it doesn't speed up while loops and if statements at all. For some reasons it also appears to make pystone a bit slower. Here are some micro-benchmarks: ./python -m timeit for x in xrange(1): pass Before: 1000 loops, best of 3: 758 usec per loop After: 1000 loops, best of 3: 483 usec per loop ./python -m timeit x=100 while x: x -= 1 Before: 1 loops, best of 3: 21.8 usec per loop After: 1 loops, best of 3: 21.6 usec per loop ./python -m timeit -s l = range(100) [x for x in l] Before: 10 loops, best of 3: 14.9 usec per loop After: 10 loops, best of 3: 13.3 usec per loop ./python -m timeit -s l = range(100) [x for x in l if x] Before: 1 loops, best of 3: 23.9 usec per loop After: 1 loops, best of 3: 22.3 usec per loop ./python -m timeit -s l = range(100) [x for x in l if not x] Before: 10 loops, best of 3: 15.8 usec per loop After: 10 loops, best of 3: 13.9 usec per loop ./python Tools/pybench/pybench.py -t IfThenElse Before: 164ms per round After: 166ms per round __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Lauro Moura [EMAIL PROTECTED]: -- nosy: +lauromoura __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Jeffrey Yasskin [EMAIL PROTECTED]: -- nosy: +jyasskin __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: This new patch completes the bytecode modifications. For/while loops as well as list comprehensions and generator expressions are a bit faster now. Also, as a side effect I've introduced a speed improvement for if statements and expressions... Some micro-benchmarks (completing the ones already given above): ./python Tools/pybench/pybench.py -t IfThenElse Before: 167ms per round After: 136ms per round ./python -m timeit -s y=range(100) sum(x for x in y) Before: 1 loops, best of 3: 20.4 usec per loop After: 10 loops, best of 3: 17.9 usec per loop ./python -m timeit -s y=range(100) sum(x for x in y if x) Before: 1 loops, best of 3: 28.5 usec per loop After: 1 loops, best of 3: 23.3 usec per loop ./python -m timeit -s y=range(100) sum(x for x in y if not x) Before: 10 loops, best of 3: 16.4 usec per loop After: 10 loops, best of 3: 12.1 usec per loop ./python -m timeit -s x,y,z=1,2,3 x if y else z Before: 100 loops, best of 3: 0.218 usec per loop After: 1000 loops, best of 3: 0.159 usec per loop A couple of tests seem to be failing in obscure ways in the test suite, I'll try to examine them. Most of the test suite runs fine though. Added file: http://bugs.python.org/file9870/loops6.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: Ok, the fix for the bizarre failures was really simple. Now the only failing tests are in test_trace (because it makes assumptions regarding the bytecode that aren't true anymore, I'll have to adapt the tests). Added file: http://bugs.python.org/file9871/loops7.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Neal Norwitz [EMAIL PROTECTED] added the comment: Antoine, I hope to look at this patch eventually. Unfortunately, there are too many build/test problems that need to be resolved before the next release. If you can help out with those, I will be able to review this patch sooner. -- nosy: +nnorwitz __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Gregory P. Smith [EMAIL PROTECTED]: -- nosy: +gregory.p.smith __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Fred L. Drake, Jr. [EMAIL PROTECTED]: __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Fred L. Drake, Jr. [EMAIL PROTECTED]: __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: By the way, the compiler package fix has been isolated and cleaned up as part of #2472. Maybe it would be nice to start with that one. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: This new patch also updates the code generation for list comprehensions. Here are some micro-benchmarks: ./python -m timeit -s l = range(100) [x for x in l] Before: 10 loops, best of 3: 14.9 usec per loop After: 10 loops, best of 3: 12.5 usec per loop ./python -m timeit -s l = range(100) [x for x in l if x] Before: 1 loops, best of 3: 24.1 usec per loop After: 10 loops, best of 3: 18.8 usec per loop ./python -m timeit -s l = range(100) [x for x in l if not x] Before: 10 loops, best of 3: 15.5 usec per loop After: 10 loops, best of 3: 11.9 usec per loop Please note that this patch is orthogonal with Neal's patch in #2183, so the two combined should be quite interesting for the list comprehensions bytecode. Added file: http://bugs.python.org/file9863/loops5.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: This new patch includes surgery to the compiler package (especially flowgraph trickery) in order to make it work with the new opcodes. I think my changes are sane but since the package seems basically untested, unmaintained and almost unused, there may be some glitches. However, test_compiler passes. (test_dis will need to be updated for the new opcodes, not a big deal) Added file: http://bugs.python.org/file9827/loops3.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Antoine Pitrou [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file9827/loops3.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: Removed latest patch, it was half-baked. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: This new patch should be ok. The block ordering algorithm in compiler.pyassem looks entirely clean now, to the extent that the previous fixup hacks have been disabled. Attaching loops3.py. Added file: http://bugs.python.org/file9829/loops3.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Antoine Pitrou [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file9822/loops2.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: loops4.patch adds a mechanism to avoid blocking signal catching in empty loops (such as for x in it: pass or while x: pass). Much of the speedup is still retained. ./python -m timeit for x in xrange(1): pass Before: 1000 loops, best of 3: 737 usec per loop After: 1000 loops, best of 3: 438 usec per loop ./python -m timeit x=100 while x: x -= 1 Before: 1 loops, best of 3: 21.7 usec per loop After: 10 loops, best of 3: 16.6 usec per loop ./python Tools/pybench/pybench.py -t ForLoops Before: 364ms per round After: 242ms per round Added file: http://bugs.python.org/file9832/loops4.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
New submission from Antoine Pitrou [EMAIL PROTECTED]: This is a preliminary patch to speedup for and while loops (it will also be able to speedup list comps and gen comps, if I get to do it). The patch works by putting the loop condition test at the end of loop, which allows removing one JUMP_ABSOLUTE byte code out of the critical path. For this two new opcodes are introduced: - FOR_ITER2 is the same as FOR_ITER except that it does an /absolute/ jump if the iterator is /not/ exhausted (when other uses of FOR_ITER are replaced with FOR_ITER2, we can of course restore the original naming) - JUMP_ABS_IF_TRUE /pops/ the top of the stack and does an absolute jump if its value is true Some micro-benchmarks: ./python -m timeit for x in xrange(1): pass Before: 1000 loops, best of 3: 782 usec per loop After: 1000 loops, best of 3: 412 usec per loop ./python -m timeit -s x=1 while x: x -= 1 Before: 100 loops, best of 3: 0.237 usec per loop After: 1000 loops, best of 3: 0.176 usec per loop ./python Tools/pybench/pybench.py -t ForLoops Before: 365ms per round After: 234ms per round Also, pystone gets 5% faster (from 43300 to 45800). Now for the less shiny things: 1. I'm having problems modifying the pure Python compiler module. For some reasons it seems to have stricter requirements than compile.c itself does (!); I get some assertion error in compiler.pyassem.FlowGraph.fixupOrderHonorNext as soon as a loop gets non-trivial. 2. Line numbers probably need to be fixed. The lnotab format may even have to be adapted in order to accomodate non-monotonically increasing line numbers. Is there some interest in this patch? If yes, I'd like to have your input on the two bullet points above :) -- components: Interpreter Core files: loops.patch keywords: patch messages: 64342 nosy: pitrou severity: normal status: open title: speedup loops with better bytecode type: performance versions: Python 2.6 Added file: http://bugs.python.org/file9820/loops.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: This is a preliminary patch to speedup for and while loops (it will also be able to speedup list comps and gen comps, if I get to do it). The patch works by putting the loop condition test at the end of loop, which allows removing one JUMP_ABSOLUTE byte code out of the critical path. For this two new opcodes are introduced: - FOR_ITER2 is the same as FOR_ITER except that it does an /absolute/ jump if the iterator is /not/ exhausted (when other uses of FOR_ITER are replaced with FOR_ITER2, we can of course restore the original naming) - JUMP_ABS_IF_TRUE /pops/ the top of the stack and does an absolute jump if its value is true Some micro-benchmarks: ./python -m timeit for x in xrange(1): pass Before: 1000 loops, best of 3: 782 usec per loop After: 1000 loops, best of 3: 412 usec per loop ./python -m timeit x=100 while x: x -= 1 Before: 1 loops, best of 3: 22.1 usec per loop After: 10 loops, best of 3: 16.6 usec per loop ./python Tools/pybench/pybench.py -t ForLoops Before: 365ms per round After: 234ms per round Also, pystone gets 5% faster (from 43300 to 45800). Now for the less shiny things: 1. I'm having problems modifying the pure Python compiler module. For some reasons it seems to have stricter requirements than compile.c itself does (!); I get some assertion error in compiler.pyassem.FlowGraph.fixupOrderHonorNext as soon as a loop gets non-trivial. 2. Line numbers probably need to be fixed. The lnotab format may even have to be adapted in order to accomodate non-monotonically increasing line numbers. Is there some interest in this patch? If yes, I'd like to have your input on the two bullet points above :) __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Antoine Pitrou [EMAIL PROTECTED] added the comment: Sorry for the double post, the second message contains fixed benchmark numbers. __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Antoine Pitrou [EMAIL PROTECTED]: Added file: http://bugs.python.org/file9822/loops2.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2459] speedup loops with better bytecode
Changes by Antoine Pitrou [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file9820/loops.patch __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2459 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
