Hi, Brian helped me fix my problems with ipython and it then was super easy to create parallel testing for sympy. Currently in my branch here:
http://github.com/certik/sympy/tree/test2 e.g. start the cluster: $ ipcluster local -n 8 Install sympy: $ git clone git://github.com/certik/sympy.git $ cd sympy $ git checkout -b test2 origin/test2 $ python setup.py install --home=~/lib (make sure ~/lib/lib/python is in your PYTHONPATH) test in parallel: $ python t.py number of tests: 1376 distributing jobs collecting results processor: 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions ________________________________________________________________________________ File "/home/ondrej/lib/lib/python/sympy/mpmath/tests/test_basic_ops.py", line 115, in test_complex_misc assert mpc(2+1e-15j).ae(2) File "/home/ondrej/lib/lib/python/sympy/mpmath/tests/test_basic_ops.py", line 115, in test_complex_misc assert mpc(2+1e-15j).ae(2) processor: 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions processor: 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions processor: 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F . . . . . exceptions ________________________________________________________________________________ File "/home/ondrej/lib/lib/python/sympy/utilities/tests/test_lambdify.py", line 97, in test_mpmath_lambda assert -prec < f(mpmath.mpf("0.2")) - sin02 < prec File "/home/ondrej/lib/lib/python/sympy/utilities/tests/test_lambdify.py", line 97, in test_mpmath_lambda assert -prec < f(mpmath.mpf("0.2")) - sin02 < prec processor: 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions processor: 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions processor: 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions processor: 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exceptions All tests pass, except 4 mpmath tests, but I suspect it could be some bug in mpmath. Compare the timing: Sequential: $ time bin/test sympy [...] real 1m4.295s user 1m3.872s sys 0m0.332s Parallel: $ time python t.py [..] real 0m10.206s user 0m0.308s sys 0m0.064s And this doesn't do any load balancing, so in the late phase (last 2s) it is basically just computing on one processor only. I am looking forward when I run this on our UNR cluster, I guess all sympy tests could be done in just couple seconds. Now some thoughts about load balancing: It starts here: http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L50 I create a tasks dictionary, which contains processors ids as keys and sympy tests numbers as values (those tests should be executed on that processor), where one test is just one testfunction in a file (e.g. there is almost 1400 tests). Then I distribute the tests on the processors here using nonblocking mec.execute: http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L64 And finally I collect the results: http://github.com/certik/sympy/blob/55de47b6f0a7bee01249fc24c03e5567695c4569/t.py#L69 and report the results to the user. This should of course be integrated into our testing framework, so that it looks exactly the same. The goal should be that one would just use bin/test or bin/test -j8 but otherwise the output doesn't change, only it will be 8x faster. Ondrej --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to sympy@googlegroups.com To unsubscribe from this group, send email to sympy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sympy?hl=en -~----------~----~----~----~------~----~------~--~---