On 22.2.2015 10:59, Andrew Gierth wrote: >>>>>> "Gavin" == Gavin Flower <gavinflo...@archidevsys.co.nz> writes: > > Gavin> Essentially, how consistent are the results, or how great is the > Gavin> noise? There may be better indicators than the ones I've > Gavin> suggested above. > > This is all rather missing the point. > > The relevant metric is not how much noise is introduced between runs > of the same code, but rather how much noise is introduced as a result > of non-consequential changes to the code. > > I can get variations of several percent - easily more than three > sigmas of the timing of repeated runs of unchanged code - in the time > taken to sort a float8 column simply from introducing varying amounts > of padding into the body of a function which is never called in the > test. Clearly, the only possible effect here is that the changed > memory addresses of functions must be resulting in different patterns > of cache misses / cache replacements, or TLB misses, or similar > low-level effects which have nothing to do with the code as such. > > (That this is a low-level alignment effect is supported by the fact > that the performance changes are not monotonic in the size of the > padding; adding more padding may cause either speedups or slowdowns.)
Interesting, but I think Gavin was asking about how much variation was there for each tested case (e.g. query executed on the same code / dataset). And in those cases the padding / alignment won't change, and thus the effects you describe won't influence the results, no? -- Tomas Vondra http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers