On Sun, Mar 20, 2016 at 2:23 PM, David Rowley <david.row...@2ndquadrant.com> wrote: > > I've had a look over this. I had to first base it on the 0005 patch, > as it seemed like the pg_aggregate.h changes didn't include the > serialfn and deserialfn changes, and an OID was being consumed by > another function I added in patch 0003. > > On testing I also noticed some wrong results, which on investigation, > are due to the wrong array elements being added together. > > For example: > > postgres=# select stddev(num) from f; > stddev > ------------------ > 28867.5149028984 > (1 row) > > > postgres=# set max_parallel_degree=8; > SET > postgres=# select stddev(num) from f; > stddev > -------- > 0 > (1 row) > > + N += transvalues2[0]; > + sumX += transvalues2[1]; > + CHECKFLOATVAL(sumX, isinf(transvalues1[1]) || isinf(transvalues2[1]), true); > + sumX2 += transvalues1[2]; > > The last line should use transvalues2[2], not transvalues1[2].
Thanks. > There's also quite a few mistakes in float8_regr_combine() > > + sumX2 += transvalues2[2]; > + CHECKFLOATVAL(sumX2, isinf(transvalues1[2]) || isinf(transvalues2[1]), > true); > > Wrong array element on isinf() check > > + sumY2 += transvalues2[4]; > + CHECKFLOATVAL(sumY2, isinf(transvalues1[4]) || isinf(transvalues2[3]), > true); > > Wrong array element on isinf() check > > + sumXY += transvalues2[5]; > + CHECKFLOATVAL(sumXY, isinf(transvalues1[5]) || isinf(transvalues2[1]) || > + isinf(transvalues2[3]), true); > > Wrong array element on isinf() check, and the final > isinf(transvalues2[3]) check does not need to be there. Thanks for the changes, I just followed the float8_regr_accum function while writing float8_regr_combine function. Now I understood isinf proper usage. Regards, Hari Babu Fujitsu Australia -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers