On Monday 24 November 2008, Paul Biggar wrote: > Hi, > > I'm currently using cachegrind as a kind of a simulator to benchmark > my code. Using the 'I refs' field of cachegrind's output, I see that > revision A of my code runs in X cycles, and revision B runs in Y > cycles. Hence, the speedup between the two revisions in X/Y.
Hmm. Perhaps speedup in terms of executed instructions. However, the term "speedup" is usually used for runtime improvement... > I find > these results to be consistent on this machine, if I ignore that last > 4 digits or so. What do you mean by 'consistent' here? Real measurement of instructions executed via a performance counter? But if you talk about the relation of your instruction speedup to actual time speedup on your machine, then this is only true by chance, for your specific code. In another code, one instruction doing a memory access could need as much time as 100 instructions doing integer operations... > I'd like to know how reliable I can consider this number to be: > - Could it be considered an accurate reflection of how long my > program will take to run? No. > - Can I consider the number to be consistent between different > versions of cachegrind? As far as I know, the semantic behind Ir did not change in the past. However, note that 2 runs of the same code with the same cachegrind version can show slightly different numbers because of different OS behavior (doing reads/writes in different chunks, asking for EAGAIN and so on). > - Will it be forward-compatible with future versions? Probably, yes. > - Can I expect the results to be the same on different > OS/architecture combinations? No. Depending on the compiler & architecture, for the same problem there of course can be code generated with different numbers of instructions. > Finally, would you consider it to be 'safe' to include these numbers > in a submission to a peer-reviewed publication? The idea would be that > instead of using performance results that are fickle - depending on > the load on my machine, my configuration, CPU etc - I could quote a > number which would be reproducible on another machine. As said above, cachegrinds Ir numbers say nothing about performance on a real machine. They never will be a replacement for real time measurements. However, if you assume a simple machine model (doing 1 instruction per time step), the "Ir" numbers would be a good estimation of performance in this model. Josef PS: A better (but still rough) time estimation is to take the cache misses into account, which cachegrind can provide. > > > Thanks in advance, > Paul Biggar > ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Valgrind-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/valgrind-users
