Re: Benchmark Stuff
--- Jerome Quelin <[EMAIL PROTECTED]> wrote: > - I'm deleting, recreating and repopulating the rrd > files each time, but this way I'm free regarding > Joshua's way of parsing I am not exactly happy with it myself. I wanted to get something out there so people could comment on it. The first comment was to try and use parrotbench.pl instead, so I am doing that. I have quite a few changes planned which should make the data more useable for general purposes. > Anyway, it works. Now, I'm a bit concerned about the > frequency: is once a week is enough? > Joshua, I don't know if that's possible to run > benchmarks once a day? When I first brought this up in IRC, Dan said if I came up with something to through the stuff into a database, he would volunteer to set up the automation of it. The web page was an afterthought. I can't promise everyday because I do this from a work machine that I don't have permission to "schedule" jobs for, but I can say every day I am at work I can do this. > If you can / think it's ok, then I'll update the > script to use new values for rrd creation. > Additionaly, maybe later it would be better to just > drop the sqlite database and ue rrd directly? I still have a few ideas for improving parrotbench.pl. When I am done with those, I will confer with you on getting the storage format worked out. > Regards, > Jerome > -- > [EMAIL PROTECTED] Cheers Joshua Gatcomb a.k.a. Limbic~Region ___ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush
Re: Benchmark Stuff
On Tuesday 24 August 2004 20:29, Joshua Gatcomb wrote: > I don't know what kind of layout/data would be useful > to people so let me know if you want something else. > I will update every Tuesday morning. And here's a script that use your sqlite database to generate rrd file and graph results. You will need RRDs.pm installed (Perl binding of rrdtool) - download at www.rrdtool.org or use your favorite packaging tool. The script is quite dirty: - it's not really portable for non-unix platforms (although it might not be difficult to change) - it's not really portable for perl < 5.8.0 (although it might not be difficult to change) - I'm deleting, recreating and repopulating the rrd files each time, but this way I'm free regarding Joshua's way of parsing - I'm generating fixed pngs with the results of the last 8 weeks - I'm just generating graphs, not the html that wrap them Anyway, it works. Now, I'm a bit concerned about the frequency: is once a week is enough? Joshua, I don't know if that's possible to run benchmarks once a day? If you can / think it's ok, then I'll update the script to use new values for rrd creation. Additionaly, maybe later it would be better to just drop the sqlite database and ue rrd directly? Regards, Jerome -- [EMAIL PROTECTED] rrdit.pl Description: Perl program
Re: Benchmark Stuff
Joshua Gatcomb <[EMAIL PROTECTED]> wrote: > parrotbench.pl didn't correctly handle hyphens in > names (it was using \w char class) - fixed Updated in CVS. Thanks. leo
RE: Benchmark Stuff
> > Here is the modified parrotbench script along with > > the > > new data collection script and the script to make it > > into a web page. > > > > I don't know what kind of layout/data would be > > useful > > to people so let me know if you want something else. > > > > I will update every Tuesday morning. > > I guess it helps if I share the web page to look at: > > http://perlmonk.org/~limbicregion/parrot_bench.html > good stuff. it seems that there are many more benchmark tests than runtimes. perhaps switching the axes would make it easier to read (or maybe it's just my preference for vertical scrolling over horizontal.) also, highlighting the lowest value for each bench test (perhaps by adding a little background color) might make the output easier to parse, at a glance. hrmm. looking some more, it seems 'total' is hard to interpret, since some runtimes don't run some bench tests. maybe also listing '# tests run', and 'total # tests' alongside the 'total' would help in the interpretation. ~jerry ** This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance.
Re: Benchmark Stuff
--- Joshua Gatcomb <[EMAIL PROTECTED]> wrote: > Here is the modified parrotbench script along with > the > new data collection script and the script to make it > into a web page. > > I don't know what kind of layout/data would be > useful > to people so let me know if you want something else. > > I will update every Tuesday morning. I guess it helps if I share the web page to look at: http://perlmonk.org/~limbicregion/parrot_bench.html Cheers Joshua Gatcomb a.k.a. Limbic~Region ___ Do you Yahoo!? Express yourself with Y! Messenger! Free. Download now. http://messenger.yahoo.com
Re: Benchmark Stuff
--- Sebastian Riedel <[EMAIL PROTECTED]> wrote: > > > Take a look at tools/dev/parrotbench.pl > It already does most of the things you want, you > just have to parse it's > output and feed it to your database. > > > Cheers, > Sebastian parrotbench.pl didn't correctly handle hyphens in names (it was using \w char class) - fixed parrotbench.pl doesn't properly handle foo.pasm and foo.imc, it quietly discards one parrotbench.pl only provided a single regex for you to specify which benchmarks to run. This made it difficult to say, run all but x. I added a skip benchmarks that match regex support. I also modified the regex to parse the configuration file (death to .*). Unfortunately, I also added some stuff that really doesn't need to be there to make my life easier. Here is the modified parrotbench script along with the new data collection script and the script to make it into a web page. I don't know what kind of layout/data would be useful to people so let me know if you want something else. I will update every Tuesday morning. Cheers Joshua Gatcomb a.k.a. Limbic~Region __ Do you Yahoo!? Yahoo! Mail is new and improved - Check it out! http://promotions.yahoo.com/new_mail parrotbench.gat Description: parrotbench.gat benchit.pl Description: benchit.pl webit.pl Description: webit.pl
Re: Benchmark Stuff
Well, after I had something that made comparing differences easier, I reverted back CVS. While I noticed a speed difference, it was very minor which means the significant speed bump is likely related to something non-parrot related. I am going to rebuild the latest/greatest gcc on Monday to see if I can't get back to what I was seeing before. Either way - I think it is a good thing to keep track of our performance impact when making changes - perhaps I will get more motivated and make the code a bit more robust and use parrotbench instead of my homegrown stuff. Cheers Joshua Gatcomb a.k.a. Limbic~Region --- Leopold Toetsch <[EMAIL PROTECTED]> wrote: > Joshua Gatcomb <[EMAIL PROTECTED]> wrote: > > > I recently noticed that the benchmarks in > > examples/benchmarks was running significantly > slower. > > That might very likely be related to current patches > WRT garbage > collection. I've turned off ARENA_DOD_FLAGS, which > was on since quite a > time. Turning that off can cause a lot slower DOD > sweeps for simple > stuff like integers. > > I'm not aware of other recent changes though, that > might have influenced > performance. But I didn't look at it for quite a > time. > > leo > ___ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush
Re: Benchmark Stuff
Joshua Gatcomb <[EMAIL PROTECTED]> wrote: > I recently noticed that the benchmarks in > examples/benchmarks was running significantly slower. That might very likely be related to current patches WRT garbage collection. I've turned off ARENA_DOD_FLAGS, which was on since quite a time. Turning that off can cause a lot slower DOD sweeps for simple stuff like integers. I'm not aware of other recent changes though, that might have influenced performance. But I didn't look at it for quite a time. leo
Re: Benchmark Stuff
Joshua Gatcomb wrote: I recently noticed that the benchmarks in examples/benchmarks was running significantly slower. I update Cygwin and Parrot daily - so there have been a lot of changes to account for. I idly asked on IRC if anyone was regularly tracking benchmark performance because I was feeling lazy. Dan said not that he was aware of but if I was willing to whip something up in Perl he would be more than happy to do so on a regular basis. Well, I can't get Pg working on Cygwin anymore (should have regression tests everytime I update) so after messing with it for an hour I decided to use SQLite. I also couldn't get File::Basename to work correctly so I gave up and rolled my own regex. Since at this point it was no longer portable I just threw caution to the wind and coded it *nix centric. It assumes the parrot executable is in your $PATH env It assumes it is being run from examples/benchmark It is also a very quick hack because I was on my lunch break. Take a look at tools/dev/parrotbench.pl It already does most of the things you want, you just have to parse it's output and feed it to your database. Enjoy Joshua Gatcomb a.k.a. Limbic~Region Cheers, Sebastian ___ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush
Benchmark Stuff
I recently noticed that the benchmarks in examples/benchmarks was running significantly slower. I update Cygwin and Parrot daily - so there have been a lot of changes to account for. I idly asked on IRC if anyone was regularly tracking benchmark performance because I was feeling lazy. Dan said not that he was aware of but if I was willing to whip something up in Perl he would be more than happy to do so on a regular basis. Well, I can't get Pg working on Cygwin anymore (should have regression tests everytime I update) so after messing with it for an hour I decided to use SQLite. I also couldn't get File::Basename to work correctly so I gave up and rolled my own regex. Since at this point it was no longer portable I just threw caution to the wind and coded it *nix centric. It assumes the parrot executable is in your $PATH env It assumes it is being run from examples/benchmark It is also a very quick hack because I was on my lunch break. Enjoy Joshua Gatcomb a.k.a. Limbic~Region ___ Do you Yahoo!? Win 1 of 4,000 free domain names from Yahoo! Enter now. http://promotions.yahoo.com/goldrush benchmark Description: benchmark
