On 12/07/11 01:20, Maciej Fijalkowski wrote: > Hi > > I'm a bit worried with our current benchmarks state. We have around 4 > benchmarks that had reasonable slowdowns recently [cut]
Ok, let's try to make a summary of what we discovered about benchmark regressions. > > Current list: > > http://speed.pypy.org/timeline/?exe=1&base=none&ben=spectral-norm&env=tannit&revs=50 this is weird. Maybe I did something wrong (like picking the wrong revisions to test), but I cannot reproduce the slowdown. This is on my machine, 32-bit chroot: $ ./pypy-c-jit-45360-867e8ffff7a8-linux/bin/pypy ~/pypy/benchmarks/own/spectral-norm.py --take_geo_mean 0.0256132620823 $ ./pypy-c-jit-45373-582113929b62-linux/bin/pypy ~/pypy/benchmarks/own/spectral-norm.py --take_geo_mean 0.0256120378632 $ ./pypy-c-jit-45412-3476b9be3cec-linux/bin/pypy ~/pypy/benchmarks/own/spectral-norm.py --take_geo_mean 0.025725552797 on tannit, I get similar results (a bit faster than on my machine, but all the same). > http://speed.pypy.org/timeline/?exe=1&base=none&ben=spitfire&env=tannit&revs=50 I think that armin investigated this, and the outcome was that it's because of the changes we did in the GC during the sprint. Armin, do you confirm? Do we have a solution? > This is a good example why we should not work the way we work now: > > http://speed.pypy.org/timeline/?exe=1&base=none&ben=slowspitfire&env=tannit&revs=200 > > There was an issue, then the issue was fixed, but apparently not quite > (7th of June is quite a bit slower than 25th of May) and then recently > we introduced something that make it faster alltogether. Can we even > fish the original issue? did anybody look at this? > http://speed.pypy.org/timeline/?exe=1&base=none&ben=bm_mako&env=tannit&revs=200 I investigated this. The culprit is 5b62f71347c8, in particular the change to policy.py which enables tracing inside lltypesystem.rbuilder. With this patch, mako is fast again: http://paste.pocoo.org/show/439257/ The weird thing is that in jit-summary there is no obvious difference between the runs, apart the total time: http://paste.pocoo.org/show/439263/ Alex, do you feel like investigating more? > http://speed.pypy.org/timeline/?exe=1&base=none&ben=nbody_modified&env=tannit&revs=50 > (is it relevant or just noise?) > > http://speed.pypy.org/timeline/?exe=1&base=none&ben=telco&env=tannit&revs=50 I did not look into these two. Anybody? ciao, Anto _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev