[issue13279] Add memcmp into unicode_compare for optimizing comparisons
Richard Saunders richismyn...@mac.com added the comment: Here's a test demonstrating the memcmp optimization effect: --- more ~/PickTest5/Python/string_test3.py a = [] b = [] c = [] d = [] for x in range(0,1000) : a.append(the quick brown fox+str(x)) b.append(the wuick brown fox+str(x)) c.append(the quick brown fox+str(x)) d.append(the wuick brown fox+str(x)) count = 0 for x in range(0,20) : if a==c : count += 1 if a==c : count += 2 if a==d : count += 3 if b==c : count += 5 if b==d : count += 7 if c==d : count += 11 print(count) - # plain Python 3.3 no memcmp, no UCS specialization % time ./python ~/PickTest5/Python/string_test3.py 200 28.918u 0.014s 0:29.02 99.6%0+0k 0+0io 0pf+0w % setenv CFLAGS -fno-builtin-memcmp # (reconfigure and remake) % time ./python ~/PickTest5/Python/string_test3.py 200 29.783u 0.017s 0:29.88 99.6%0+0k 0+0io 0pf+0w - # Python 3.3 with memcmp optimizations and UCS2/4 specialization (no CFLAGS) % time ./python ~/PickTest5/Python/string_test3.py 200 37.126u 0.046s 0:37.35 99.4%0+0k 0+0io 0pf+0w % setenv CFLAGS -fno-builtin-memcmp # (reconfigure and remake) % time ./python ~/PickTest5/Python/string_test3.py 200 18.621u 0.013s 0:18.69 99.6%0+0k 0+0io 0pf+0w - Note we only really see the effect if we make sure that gcc isn't emitting its special memcmp: that's why the -fno-builtin-memcmp is SO important on gcc builds! See http://www.picklingtools.com/study.pdf I am currently working with Bob Arendt (a colleague who works frequently with Fedora) to try to put the -fno-builtin-memcmp in the .spec file for their Python -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13279] Add memcmp into unicode_compare for optimizing comparisons
Richard Saunders richismyn...@mac.com added the comment: Added branches for specializing for UCS2 and UCS4 types -- Added file: http://bugs.python.org/file23574/unicode_with_memcmp_and_ucs_specialization.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13279] Add memcmp into unicode_compare for optimizing comparisons
Richard Saunders richismyn...@mac.com added the comment: Some more information: Bob Arendt and I have been playing with the Fedora Core .spec file for python on Fedora Core 15: the compile options we found seem to automatically (as we did non invoke this option) invoke '-fno-builtin-memcmp' somehow? We disassembled the Python binary we built for the machine ourselves (via the spec file) code and saw, yes, it was calling memcmp on the system, even though we didn't bypass it explicitly. Our conjecture is that the -m32 or -mtune=atom automatically turns the builtin memcmp off, but we're not sure (we're still playing with it). However, perhaps Fedora builds on a more generic machine: that generic build keeps the 'rep cmbsb'? Frustrating. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13279] Add memcmp into unicode_compare for optimizing compares
New submission from Richard Saunders richismyn...@mac.com: In discussions of memcmp performance, (http://www.picklingtools.com/study.pdf) it was noted how well Python 2.7 can take advantage of faster memcmps (indeed, the rich comparisons are all memcmp calls). There have been some discussion on python-...@python.org as well. With unicode and Python 3.3 (and anyPython 3.x) there are a few places we could call memcmp to make string comparisons faster, but they are not completely trivial. Basically, if the unicode strings are 1 byte kind, then memcmp can be used almost as is. If the unicode strings are the same kind, they can at least use memcmp to compare for equality or inequality. There is also a minor optimization laying in unicode_compare: if you are comparing two strings for equality/inequality, there is no reason to look at the entire string if the lengths are different. These 3 minor optimizations can make unicode_compare faster. -- components: Interpreter Core messages: 146508 nosy: RichIsMyName, asmodai, loewis, pitrou, scoder priority: normal severity: normal status: open title: Add memcmp into unicode_compare for optimizing compares type: performance versions: Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13279] Add memcmp into unicode_compare for optimizing comparisons
Richard Saunders richismyn...@mac.com added the comment: This is a potential patch: I believe it follows the C-style of PEP 7 There is a test as well, testing 1 and 2 byte kinds. I have run it through the python tests and have added no new breakages (there were some tests that failed, but they failed with and without the patch) -- keywords: +patch title: Add memcmp into unicode_compare for optimizing compares - Add memcmp into unicode_compare for optimizing comparisons Added file: http://bugs.python.org/file23537/unicode_with_memcmp.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
