On Thu, Mar 10, 2016 at 8:26 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote:
> I don't think we can rely on median that much if we have only 3 runs. > For 3 runs we can only apply Kornfeld method which claims that confidence > interval should be between lower and upper values. > Since confidence intervals for master and patched versions are overlapping > we can't conclude that expected TPS numbers are different. > Dilip, could you do more runs? 10, for example. Using such statistics we > would be able to conclude something. > Here is the reading for 10 runs.... Median Result ----------------------- Client Base Patch ------------------------------------------- 1 19873 19739 2 38658 38276 4 68812 62075 Full Results of 10 runs... Base ------------- Runs 1 Client 2 Client 4 Client ----------------------------------------------------- 1 19442 34866 49023 2 19605 35139 51695 3 19726 37104 53899 4 19835 38439 55708 5 19866 38638 67473 6 19880 38679 70152 7 19973 38720 70920 8 20048 38737 71342 9 20057 38815 71403 10 20344 41423 77953 ----------------------------------------------------- Patch ----------- Runs 1 Client 2 Client 4 Client ------------------------------------------------------ 1 18881 30161 54928 2 19415 32031 59741 3 19564 35022 61060 4 19627 36712 61839 5 19670 37659 62011 6 19808 38894 62139 7 19857 39081 62983 8 19910 39923 75358 9 20169 39937 77481 10 20181 40003 78462 ------------------------------------------------------ -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com