> I have just realized that the recent patches in pgbench have altered its > behavior in a way that destroys reproducibility of results --- I'm > seeing reported TPS numbers about twice what they were before that. > I'd love to say we did something in the past month that made the backend > 2X faster, but sadly, tain't so :-( > > I've only done limited investigation but what seems to be happening > is that the "-s" scale factor is no longer being applied to adjust the > range of the random variables, leading to only part of the database > being touched. For example, with -s = 10, the older code updates all 10 > "branches" rows but CVS HEAD only seems to touch bid = 1.
As far as I concern the bug is there, but in a different way what you said. It seems "-s" works. Problem is, pgbench does not get the default scaling factor (that is same as the number of rows in branches table), instead alway uses the default scaling factor 1. I have fixed the problem in CVS HEAD. > I see that the code still thinks it is scaling the numbers, but the > mechanism for doing that has changed drastically, and I suspect that's > the problem. I assume there's some garden-variety bug in there. > > At a more fundamental level, the recent patches seem to mean that > pg_dump is doing noticeably more work to issue its queries than it was > before, and I wonder whether this isn't going to destroy cross-version > reproducibility in its own way. Have you done any timing studies to > verify that the new and old code have comparable performance? I don't > mind enhancing pgbench's functionality for non-default tests, but I > think it's important that the default case remain comparable over time. > If there is a noticeable speed difference then I'd vote for going back > to the klugy old code for adjusting the default script's values. Let me do further performance testing. > And at the nitpicking level, is ":tps" a good name for the variable that > reflects the scaling factor? It seems awfully easy to confuse that with > the TPS numbers that pgbench reports. Perhaps ":scale" or some such > would be better. I replaced all occurenes of "tps" to scale. -- Tatsuo Ishii SRA OSS, Inc. Japan ---------------------------(end of broadcast)--------------------------- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match