On Sun, 2010-02-21 at 23:05 +0100, Martin v. Loewis wrote: > > It's far from scientific, but I've seen behaviour that's close to a 100% > > performance penalty on a dual-core linux system: > > > > http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2 > > > > Short story: a particular test suite of mine used to run in around 25 > > seconds, but a bit of ctypes magic to set thread affinity dropped the > > running time to under 13 seconds. > > Indeed, it's not scientific - but with a few more details, you could > improve it quite a lot: what specific Linux distribution (the posting > doesn't even say it's Linux), what specific Python version had you been > using? (less important) what CPUs? If you can: what specific test suite?
I'm on Ubuntu Karmic, Python 2.6.4, an AMD Athlon 7750 dual core. Unfortunately the test suite is for a proprietary application. I've been able to reproduce similar behaviour with an open-source test suite, using the current trunk of the "pyfilesystem" project: http://code.google.com/p/pyfilesystem/ In this project "OSFS" is an object-oriented interface to the local filesystem. The test case "TestOSFS.test_cases_in_separate_dirs" runs three theads, each doing a bunch of IO in a different directory. Running the tests normally: r...@durian:/storage/software/fs$ nosetests fs/tests/test_fs.py:TestOSFS.test_cases_in_separate_dirs . ---------------------------------------------------------------------- Ran 1 test in 9.787s That's the best result from five runs - I saw it go as high as 12 seconds. Watching it in top, I see CPU usage at around 150%. Now using threading2 to set the process cpu affinity at the start of the test run: r...@durian:/storage/software/fs$ nosetests fs/tests/test_fs.py:TestOSFS.test_cases_in_separate_dirs . ---------------------------------------------------------------------- Ran 1 test in 3.792s Again, best of five. The variability in times here is much lower - I never saw it go above 4 seconds. CPU usage is consistently 100%. Cheers, Ryan -- Ryan Kelly http://www.rfk.id.au | This message is digitally signed. Please visit r...@rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details
signature.asc
Description: This is a digitally signed message part
-- http://mail.python.org/mailman/listinfo/python-list