Re: A microbenchmarking library

2018-10-02 Thread lemonboy
Thanks to @dm1try latest PR it now works fine on OSX!

Re: A microbenchmarking library

2018-10-02 Thread mratsim
I've left it running for longer, definitely stuck. Scale factor is 1.0

Re: A microbenchmarking library

2018-10-02 Thread lemonboy
2 minutes top on a very busy machine. I guess there's something wrong in the `getMonotonicTime` implementation for OSX then... What is the computed `scaleFactor` in `timer.nim` ?

Re: A microbenchmarking library

2018-10-02 Thread mratsim
How long is it supposed to run? It's also freezing for me on MacOS, killed it after a minute.

Re: A microbenchmarking library

2018-10-02 Thread lemonboy
Good news, criterion is now available on nimble! I also need your help, if you're on Windows or OSX can you please check if the `tfib.nim` example runs fine for you? I tried setting up Travis to run the test suite on OSX too but got a timeout after 10m so either the timing code is wrong for tha

Re: A microbenchmarking library

2018-08-26 Thread lemonboy
Oh I see what you mean. As usual one must be smart enough to set the CPU governor to 'performance' and pin the benchmarking thread to a single core to prevent measuring errors due to SMP migration. Benchmarking is twice as hard as writing code :)

Re: A microbenchmarking library

2018-08-26 Thread Stefan_Salewski
Indeed, gcc 8.2 generates perfectly optimized code using cmov instruction for "GOTO label", as [https://www.godbolt.org](https://www.godbolt.org)/ proves with compiler option -O3: int w1(int64_t a, int64_t b) { int r=0; if (a == b) r = 10; else

Re: A microbenchmarking library

2018-08-25 Thread mratsim
I don't see an issue with the gotos, those are direct jumps so there is no cost there. The real question is the if branching vs cmov. I do not now the impact of Meltdown, Spectre and L1TF mitigation on Haswell branch predictors though as one of the main goal was fixing speculative execution an

Re: A microbenchmarking library

2018-08-25 Thread Stefan_Salewski
Yes, as I already said, 100 cycles is fine for me. mratsim, what do you generally think about all the GOTOs? I had the feeling it makes it for C compiler a bit hard to 100 % optimize the code, for example it may be difficult to apply cmov instructions to get branchless code. I think it will mak

Re: A microbenchmarking library

2018-08-25 Thread mratsim
100 cycles is the cost of a cache miss so allocating then comparing 2 strings for 100 cycles seems pretty reasonable.

Re: A microbenchmarking library

2018-08-25 Thread Stefan_Salewski
For your example, my output is # $ nim c t.nim $ ./t Benchmark: fib5() Collected 241 samples Warning: Found 6 mild and 15 extreme outliers in the time measurements Warning: Found 8 mild and 10 extreme outliers in the cycles measurements Time Mean: 251.75

Re: A microbenchmarking library

2018-08-25 Thread lemonboy
> The example output on your github page is still wrong, the cycles count is > much too high. Is it? I've probably benchmarked it without the release switch. As usual beware of the optimizer, make sure the benchmark function doesn't elide the comparison completely. Introducing an argument using

Re: A microbenchmarking library

2018-08-25 Thread Stefan_Salewski
OK, here is the assembler listing... # nim c -d:release t.nim # gcc.options.speed = "-save-temps -march=native -O3 -fno-strict-aliasing" $ cat t.nim proc t(a, b: string) : bool = a == b proc main = var a = "Rust" var b = "Nim" echo t(a, b)

Re: A microbenchmarking library

2018-08-25 Thread Araq
> C code looks like And what does the produced assembler code look like?

Re: A microbenchmarking library

2018-08-25 Thread Stefan_Salewski
I guess you know well how it looks -- but wait a moment, I will generate it. I showed the C code only because someone wonders about 100 cycles -- I do not wonder, it is fine for me. But Python or Java people do not always know that string comparison is different from plain integer comparison. I

Re: A microbenchmarking library

2018-08-25 Thread Stefan_Salewski
Nice. The example output on your github page is still wrong, cycles count is much too high. Min cycle count seems to be 4 for most simple procs, but that is no problem. I have just tested a plain string comparison -- about 100 cycles, as expected from its C code. So string comparison is not a v

Re: A microbenchmarking library

2018-08-20 Thread lemonboy
That's now fixed! Thanks for the heads up!

Re: A microbenchmarking library

2018-08-20 Thread dataman
Cool! Where is dip module? :) (in statistics.nim)

A microbenchmarking library

2018-08-20 Thread lemonboy
Dear Nimmers, [Here's](https://github.com/LemonBoy/criterion.nim) a nice little library for all your microbenchmarking needs. Benchmarking is hard and the aim of this library is to abstract away most of the complexity and pitfalls and provide the user (hopefully) meaningful results. If you ever