On Mon, Mar 7, 2016 at 6:19 PM, Robert Haas <robertmh...@gmail.com> wrote:
> On Sat, Mar 5, 2016 at 7:22 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > > On Wed, Mar 2, 2016 at 11:05 AM, Dilip Kumar <dilipbal...@gmail.com> > wrote: > >> And this latest result (no regression) is on X86 but on my local > machine. > >> > >> I did not exactly saw what this new version of patch is doing different, > >> so I will test this version in other machines also and see the results. > > > > > > I tested this on PPC again, This time in various order (sometime patch > first > > and then base first). > > I tested with latest patch pinunpin-cas-2.patch on Power8. > > > > Shared Buffer = 8GB > > ./pgbench -j$ -c$ -T300 -M prepared -S postgres > > > > BASE > > ----- > > Clients run1 run2 run3 > > 1 21200 18754 20537 > > 2 40331 39520 38746 > > > > > > Patch > > ----- > > Clients run1 run2 run3 > > 1 20225 19806 19778 > > 2 39830 41898 36620 > > > > I think, here we can not see any regression, (If I take median then it > may > > looks low with patch so posting all 3 reading). > > If the median looks low, how is that not a regression? > I don't think we can rely on median that much if we have only 3 runs. For 3 runs we can only apply Kornfeld method which claims that confidence interval should be between lower and upper values. Since confidence intervals for master and patched versions are overlapping we can't conclude that expected TPS numbers are different. Dilip, could you do more runs? 10, for example. Using such statistics we would be able to conclude something. ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company