Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-29 Thread Brad Anderson
On Wed, Mar 21, 2012 at 6:32 PM, Juan Manuel Cabo juanmanuel.c...@gmail.com wrote: This is a small util I wrote in D which is like the unix 'time' command but can repeat the command N times and show median, average, standard deviation, minimum and maximum. As you all know, it is not proper

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-27 Thread Don Clugston
On 23/03/12 16:25, Andrei Alexandrescu wrote: On 3/23/12 12:51 AM, Manfred Nowak wrote: Andrei Alexandrescu wrote: You may want to also print the mode of the distribution, nontrivial but informative In case of this implementation and according to the given link: trivial and noninformative,

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-26 Thread Ary Manzana
On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: Dude, this is awesome. I tend to just use time, but if I was doing anything more complicated, I'd use this. I would suggest changing the name while you still can. avgtime is not that

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-26 Thread Juan Manuel Cabo
On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: Dude, this is awesome. I tend to just use time, but if I was doing anything more complicated, I'd use this. I would suggest

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-26 Thread Ary Manzana
On Tuesday, 27 March 2012 at 01:19:22 UTC, Juan Manuel Cabo wrote: On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: Dude, this is awesome. I tend to just use time, but if I

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-26 Thread Juan Manuel Cabo
On Thursday, 22 March 2012 at 17:13:58 UTC, Manfred Nowak wrote: Juan Manuel Cabo wrote: like the unix 'time' command `version linux' is missing. -manfred Done!, it works in windows now too. (release 0.5 in github). --jm

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-26 Thread Juan Manuel Cabo
On Tuesday, 27 March 2012 at 03:39:56 UTC, Ary Manzana wrote: On Tuesday, 27 March 2012 at 01:19:22 UTC, Juan Manuel Cabo wrote: On Tuesday, 27 March 2012 at 00:58:26 UTC, Ary Manzana wrote: On 3/23/12 4:11 PM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 06:51:48 UTC, James Miller

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-25 Thread Manfred Nowak
Andrei Alexandrescu wrote: Wait, doesn't a benchmark always measure an algorithm with the same input? The fact that you formulate as a question indicates that you are unsure about the wright answer---me too, but 1) surely one can define a benchmark to have this property. But if one uses

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-24 Thread Marco Leise
Am Fri, 23 Mar 2012 09:02:01 +0100 schrieb Juan Manuel Cabo juanmanuel.c...@gmail.com: On Friday, 23 March 2012 at 05:16:20 UTC, Andrei Alexandrescu wrote: [.] (man, the gaussian curve is everywhere, it never ceases to perplex me). I'm actually surprised. I'm working on

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-24 Thread Andrei Alexandrescu
On 3/23/12 5:42 PM, Manfred Nowak wrote: Andrei Alexandrescu wrote: In the limit, taking the minimum over infinitely many measurements of X would yield T. True, if the thoretical variance of the distribution of T is close to zero. But horrible wrong, if T depends on an algorithm that is fast

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread James Miller
On 23 March 2012 17:53, Juan Manuel Cabo juanmanuel.c...@gmail.com wrote: But I think the most important change is that I'm now showing the 95% and 99% confidence intervals. (For the confidence intervals to mean anything, please everyone, remember to control your variables (don't defrag and

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 05:16:20 UTC, Andrei Alexandrescu wrote: [.] (man, the gaussian curve is everywhere, it never ceases to perplex me). I'm actually surprised. I'm working on benchmarking lately and the distributions I get are very concentrated around the minimum. Andrei

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Thursday, 22 March 2012 at 17:13:58 UTC, Manfred Nowak wrote: Juan Manuel Cabo wrote: like the unix 'time' command `version linux' is missing. -manfred Linux only for now. Will make it work in windows this weekend. I hope that's what you meant. --jm

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: Dude, this is awesome. I tend to just use time, but if I was doing anything more complicated, I'd use this. I would suggest changing the name while you still can. avgtime is not that informative a name given that it now does more

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 05:51:40 UTC, Manfred Nowak wrote: | For samples, if it is known that they are drawn from a symmetric | distribution, the sample mean can be used as an estimate of the | population mode. I'm not printing the population mode, I'm printing the 'sample mode'. It

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread James Miller
On 23 March 2012 21:37, Juan Manuel Cabo juanmanuel.c...@gmail.com wrote: PS: I should use the t student to make the confidence intervals, and for computing that I should use the sample standard deviation (/n-1), but that is a completely different story. The z normal with n30 aproximation is

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Don Clugston
On 23/03/12 09:37, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 05:51:40 UTC, Manfred Nowak wrote: | For samples, if it is known that they are drawn from a symmetric | distribution, the sample mean can be used as an estimate of the | population mode. I'm not printing the population

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Don Clugston
On 23/03/12 11:20, Don Clugston wrote: On 23/03/12 09:37, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 05:51:40 UTC, Manfred Nowak wrote: | For samples, if it is known that they are drawn from a symmetric | distribution, the sample mean can be used as an estimate of the | population

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Andrei Alexandrescu
On 3/23/12 12:51 AM, Manfred Nowak wrote: Andrei Alexandrescu wrote: You may want to also print the mode of the distribution, nontrivial but informative In case of this implementation and according to the given link: trivial and noninformative, because | For samples, if it is known that

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Andrei Alexandrescu
On 3/23/12 3:02 AM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 05:16:20 UTC, Andrei Alexandrescu wrote: [.] (man, the gaussian curve is everywhere, it never ceases to perplex me). I'm actually surprised. I'm working on benchmarking lately and the distributions I get are very

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Andrei Alexandrescu
On 3/23/12 5:51 AM, Don Clugston wrote: No, it's easy. Student t is in std.mathspecial. Aargh, I didn't get around to copying it in. But this should do it. [snip] Shouldn't put this stuff in std.numeric, or create a std.stat module? I think also some functions for t-test would be useful.

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 15:33:18 UTC, Andrei Alexandrescu wrote: On 3/23/12 3:02 AM, Juan Manuel Cabo wrote: On Friday, 23 March 2012 at 05:16:20 UTC, Andrei Alexandrescu wrote: [.] (man, the gaussian curve is everywhere, it never ceases to perplex me). I'm actually surprised. I'm

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 10:51:37 UTC, Don Clugston wrote: No, it's easy. Student t is in std.mathspecial. Aargh, I didn't get around to copying it in. But this should do it. /** Inverse of Student's t distribution * [.] Great!!! Thank you soo much Don!!! --jm

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Juan Manuel Cabo
On Friday, 23 March 2012 at 05:26:54 UTC, Nick Sabalausky wrote: Wow, that's just fantastic! Really, this should be a standard system tool. I think this guy would be proud: http://zedshaw.com/essays/programmer_stats.html Thanks for the good vibes! Hahahhah, that article is so ing

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Manfred Nowak
Andrei Alexandrescu wrote: In the limit, taking the minimum over infinitely many measurements of X would yield T. True, if the thoretical variance of the distribution of T is close to zero. But horrible wrong, if T depends on an algorithm that is fast only under amortized analysis, because

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-23 Thread Nick Sabalausky
Juan Manuel Cabo juanmanuel.c...@gmail.com wrote in message news:bqrlhcggehbrzyuhz...@forum.dlang.org... On Friday, 23 March 2012 at 06:51:48 UTC, James Miller wrote: Dude, this is awesome. I tend to just use time, but if I was doing anything more complicated, I'd use this. I would suggest

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-22 Thread Manfred Nowak
Juan Manuel Cabo wrote: like the unix 'time' command `version linux' is missing. -manfred

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-22 Thread Andrei Alexandrescu
On 3/21/12 7:32 PM, Juan Manuel Cabo wrote: avgtime -r 10 -q ls -lR /etc Total time (ms): 933.742 Repetitions : 10 Median time : 90.505 Avg time : 93.3742 Std dev. : 4.66808 Minimum : 88.732 Maximum : 101.225 Sweet! You may want to also print the mode of the

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-22 Thread Juan Manuel Cabo
On Thursday, 22 March 2012 at 22:22:31 UTC, Andrei Alexandrescu wrote: Sweet! You may want to also print the mode of the distribution, which is the time of the maximum sample density. http://en.wikipedia.org/wiki/Mode_(statistics) (Warning: nontrivial but informative.) Andrei Thanks for

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-22 Thread Andrei Alexandrescu
On 3/22/12 11:53 PM, Juan Manuel Cabo wrote: On Thursday, 22 March 2012 at 22:22:31 UTC, Andrei Alexandrescu wrote: Sweet! You may want to also print the mode of the distribution, which is the time of the maximum sample density. http://en.wikipedia.org/wiki/Mode_(statistics) (Warning:

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-22 Thread Nick Sabalausky
Juan Manuel Cabo juanmanuel.c...@gmail.com wrote in message news:mytcmgglyntqsoybj...@forum.dlang.org... On Thursday, 22 March 2012 at 22:22:31 UTC, Andrei Alexandrescu wrote: Sweet! You may want to also print the mode of the distribution, which is the time of the maximum sample density.

avgtime - Small D util for your everyday benchmarking needs

2012-03-21 Thread Juan Manuel Cabo
This is a small util I wrote in D which is like the unix 'time' command but can repeat the command N times and show median, average, standard deviation, minimum and maximum. As you all know, it is not proper to conclude that a program is faster than another program by running them just once.

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-21 Thread Tove
On Thursday, 22 March 2012 at 00:32:31 UTC, Juan Manuel Cabo wrote: This is a small util I wrote in D which is like the unix 'time' command but can repeat the command N times and show median, average, standard deviation, minimum and maximum. As you all know, it is not proper to conclude that a

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-21 Thread Juan Manuel Cabo
On Thursday, 22 March 2012 at 01:37:19 UTC, Tove wrote: Awesome, I do have a tiny feature request for the next version... a commandline switch to enable automatically discarding the first run as an outlier. /Tove Done, I just put it in github. (-d switch). But maybe you should be looking

Re: avgtime - Small D util for your everyday benchmarking needs

2012-03-21 Thread Nick Sabalausky
Juan Manuel Cabo juanmanuel.c...@gmail.com wrote in message news:zgjczrnyknqsiylhn...@forum.dlang.org... This is a small util I wrote in D which is like the unix 'time' command but can repeat the command N times and show median, average, standard deviation, minimum and maximum. As you all