pgsql: Adjust the integer overflow tests in the numeric code.

2021-08-06 Thread Dean Rasheed
Adjust the integer overflow tests in the numeric code. Formerly, the numeric code tested whether an integer value of a larger type would fit in a smaller type by casting it to the smaller type and then testing if the reverse conversion produced the original value. That's perfectly fine, except

pgsql: Adjust the integer overflow tests in the numeric code.

2021-08-06 Thread Dean Rasheed
Adjust the integer overflow tests in the numeric code. Formerly, the numeric code tested whether an integer value of a larger type would fit in a smaller type by casting it to the smaller type and then testing if the reverse conversion produced the original value. That's perfectly fine, except

pgsql: Adjust the integer overflow tests in the numeric code.

2021-08-06 Thread Dean Rasheed
Adjust the integer overflow tests in the numeric code. Formerly, the numeric code tested whether an integer value of a larger type would fit in a smaller type by casting it to the smaller type and then testing if the reverse conversion produced the original value. That's perfectly fine, except

Re: Numeric x^y for negative x

2021-08-06 Thread Dean Rasheed
On Fri, 6 Aug 2021 at 21:26, Tom Lane wrote: > > Dean Rasheed writes: > > On Fri, 6 Aug 2021 at 17:15, Tom Lane wrote: > >> Looks plausible by eyeball (I've not tested). > > > So, I have back-branch patches for this ready to go. The question is, > > is it bet

Re: Numeric x^y for negative x

2021-08-06 Thread Dean Rasheed
On Fri, 6 Aug 2021 at 17:15, Tom Lane wrote: > > > I guess the best thing to do is just test the value against > > PG_INT32_MIN/MAX, which is what int84() does. There are 2 other places > > in numeric.c that use similar code to check for int16/32 overflow, so > > it's possible that they're broken

Re: Numeric x^y for negative x

2021-08-06 Thread Dean Rasheed
On Fri, 6 Aug 2021 at 03:58, Tom Lane wrote: > > Dean Rasheed writes: > > On Thu, 5 Aug 2021 at 17:04, Tom Lane wrote: > >> It looks like castoroides is not happy with this patch: > >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=castoroides=2021-

pgsql: Improve numeric_power() tests for large integer powers.

2021-08-05 Thread Dean Rasheed
Improve numeric_power() tests for large integer powers. Two of the tests added by 4dd5ce2fd fail on buildfarm member castoroides, though it's not clear why. Improve the tests to report the actual values produced, if they're not what was expected. Apply to v11 only for now, until it's clearer

Re: Numeric x^y for negative x

2021-08-05 Thread Dean Rasheed
On Thu, 5 Aug 2021 at 17:04, Tom Lane wrote: > > It looks like castoroides is not happy with this patch: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=castoroides=2021-08-01%2008%3A52%3A43 > > Maybe there's some hidden C99 dependency in what you did? > Although pademelon, which is

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

pgsql: Fix division-by-zero error in to_char() with 'EEEE' format.

2021-08-05 Thread Dean Rasheed
Fix division-by-zero error in to_char() with '' format. This fixes a long-standing bug when using to_char() to format a numeric value in scientific notation -- if the value's exponent is less than -NUMERIC_MAX_DISPLAY_SCALE-1 (-1001), it produced a division-by-zero error. The reason for this

Re: Division by zero error in to_char(num, '9.9EEEE')

2021-08-04 Thread Dean Rasheed
On Fri, 30 Jul 2021 at 08:26, Dean Rasheed wrote: > > SELECT to_char(1.2e-1002, '9.9'); -- fails > ERROR: division by zero > > I think the simplest > solution is to just introduce a new local function, as in the attached > patch. This directly constructs 10^n, for integ

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

pgsql: Fix corner-case errors and loss of precision in numeric_power().

2021-07-31 Thread Dean Rasheed
nsistently return zero for cases like this where the result is indistinguishable from zero. Some paths through this code already returned zero in such cases, but others were throwing overflow errors. Dean Rasheed, reviewed by Yugo Nagata. Discussion: http://postgr.es/m/CAEZATCW6Dvq7+3wN3tt5jLj-FyOcUg

Division by zero error in to_char(num, '9.9EEEE')

2021-07-30 Thread Dean Rasheed
While testing the numeric_power() patch in [1], I found this problem trying to use to_char() to format very small numbers: SELECT to_char(1.2e-1001, '9.9'); -- OK to_char 1.2e-1001 SELECT to_char(1.2e-1002, '9.9'); -- fails ERROR: division by zero It turns out that the

Re: Numeric x^y for negative x

2021-07-29 Thread Dean Rasheed
On Thu, 22 Jul 2021 at 16:19, Dean Rasheed wrote: > > On Thu, 22 Jul 2021 at 06:13, Yugo NAGATA wrote: > > > > Thank you for updating the patch. I am fine with the additional comments. > > I don't think there is any other problem left, so I marked it > > Re

Re: Have I found an interval arithmetic bug?

2021-07-28 Thread Dean Rasheed
On Wed, 28 Jul 2021 at 00:08, John W Higgins wrote: > > It's nice to envision all forms of fancy calculations. But the fact is that > > '1.5 month'::interval * 2 != '3 month"::interval > That's not exactly true. Even without the patch: SELECT '1.5 month'::interval * 2 AS product, '3

Re: WIP: Relaxing the constraints on numeric scale

2021-07-26 Thread Dean Rasheed
On Fri, 23 Jul 2021 at 16:50, Tom Lane wrote: > > OK, I've now studied this more closely, and have some additional > nitpicks: > > * I felt the way you did the documentation was confusing. It seems > better to explain the normal case first, and then describe the two > extended cases. OK, that

pgsql: Allow numeric scale to be negative or greater than precision.

2021-07-26 Thread Dean Rasheed
of new inline functions. Bump the catversion because the allowed contents of atttypmod have changed for numeric columns. This isn't a change that requires a re-initdb, but negative scale values in the typmod would confuse old backends. Dean Rasheed, with additional improvements by Tom Lane. Reviewed

Re: WIP: Relaxing the constraints on numeric scale

2021-07-22 Thread Dean Rasheed
On Wed, 21 Jul 2021 at 22:33, Tom Lane wrote: > > I took a brief look at this and have a couple of quick suggestions: > Thanks for looking at this! > * As you mention, keeping some spare bits in the typmod might come > in handy some day, but as given this patch isn't really doing so. > I think

Re: Numeric x^y for negative x

2021-07-22 Thread Dean Rasheed
On Thu, 22 Jul 2021 at 06:13, Yugo NAGATA wrote: > > Thank you for updating the patch. I am fine with the additional comments. > I don't think there is any other problem left, so I marked it > Ready-for-Committers. > Thanks for looking. Barring any further comments, I'll push this in a few

Re: Use extended statistics to estimate (Var op Var) clauses

2021-07-21 Thread Dean Rasheed
On Tue, 20 Jul 2021 at 19:28, Tomas Vondra wrote: > > > The new code in statext_is_compatible_clause_internal() is a little > > hard to follow because some of the comments aren't right > > I ended up doing something slightly different - examine_opclause_args > now "returns" a list of expressions,

Re: Numeric x^y for negative x

2021-07-21 Thread Dean Rasheed
On Tue, 20 Jul 2021 at 10:17, Yugo NAGATA wrote: > > This patch fixes numeric_power() to handle negative bases correctly and not > to raise an error "cannot take logarithm of a negative number", as well as a > bug that a result whose values is almost zero is incorrectly returend as 1. > The

Re: Have I found an interval arithmetic bug?

2021-07-21 Thread Dean Rasheed
On Wed, 21 Jul 2021 at 03:48, Bruce Momjian wrote: > > this example now gives me concern: > > SELECT INTERVAL '1.06 months 1 hour'; >interval > --- > 1 mon 2 days 01:00:00 > > Notice that it rounds the '1.06 months' to '1 mon 2 days',

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-18 Thread Dean Rasheed
On Sat, 17 Jul 2021 at 05:24, Bharath Rupireddy wrote: > > I also think that if it is specified as CREATE FUNCTION ... STRICT > STRICT, CREATE FUNCTION ... CALLED ON NULL INPUT RETURNS NULL ON NULL > INPUT etc. isn't the syntaxer catching that error while parsing the > SQL text, similar to CREATE

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-18 Thread Dean Rasheed
On Fri, 16 Jul 2021 at 12:17, Dean Rasheed wrote: > > On Fri, 16 Jul 2021 at 10:26, Bharath Rupireddy > wrote: > > > > Thanks. The v5 patch LGTM. > > OK, I'll push that in a while. > Pushed, with some additional tidying up. In particular, I decided it was neater t

pgsql: Improve error checking of CREATE COLLATION options.

2021-07-18 Thread Dean Rasheed
Improve error checking of CREATE COLLATION options. Check for conflicting or redundant options, as we do for most other commands. Specifying any option more than once is at best redundant, and quite likely indicates a bug in the user's code. While at it, improve the error for conflicting locale

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Dean Rasheed
On Fri, 16 Jul 2021 at 10:26, Bharath Rupireddy wrote: > > Thanks. The v5 patch LGTM. OK, I'll push that in a while. > Comment on errorConflictingDefElem: > I think that the message in errorConflictingDefElem should say > <>. I'm not sure why it > wasn't done. Almost, all the cases where

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Dean Rasheed
On Fri, 16 Jul 2021 at 06:40, Bharath Rupireddy wrote: > > 1) Duplicate option check for "FROM" option is unnecessary and will be > a dead code. Because the syntaxer anyways would catch if FROM is > specified more than once, something like CREATE COLLATION mycoll1 FROM > FROM "C";. Hmm, it is

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-15 Thread Dean Rasheed
On Mon, 31 May 2021 at 15:10, vignesh C wrote: > > On Sat, May 29, 2021 at 9:20 PM Bharath Rupireddy > wrote: > > > > Thanks. PSA v3 patch. > > Thanks for the updated patch, the changes look good to me. > Hi, Having pushed [1], I started looking at this, and I think it's mostly in good shape.

Re: Enhanced error message to include hint messages for redundant options error

2021-07-15 Thread Dean Rasheed
On Tue, 13 Jul 2021 at 15:30, vignesh C wrote: > > On Tue, Jul 13, 2021 at 4:25 PM Dean Rasheed wrote: > > > > As it stands, the improvements from (3) seem quite worthwhile. Also, > > the patch saves a couple of hundred lines of code, and for me an > > optimi

pgsql: Improve reporting of "conflicting or redundant options" errors.

2021-07-15 Thread Dean Rasheed
Improve reporting of "conflicting or redundant options" errors. When reporting "conflicting or redundant options" errors, try to ensure that errposition() is used, to help the user identify the offending option. Formerly, errposition() was invoked in less than 60% of cases. This patch raises

Re: Enhanced error message to include hint messages for redundant options error

2021-07-13 Thread Dean Rasheed
On Mon, 12 Jul 2021 at 17:39, vignesh C wrote: > > Thanks for your comments, I have made the changes for the same in the > V10 patch attached. > Thoughts? > I'm still not happy about changing so many error messages. Some of the changes might be OK, but aren't strictly necessary. For example:

Re: Undocumented array_val[generate_series(...)] functionality?

2021-07-12 Thread Dean Rasheed
On Mon, 12 Jul 2021 at 02:39, David G. Johnston wrote: > > One, the select generate_series(1,3) function call causes multiple rows to be > generated where there would usually be only one. Yes. > Two, composition results in an inside-to-outside execution order: the SRF is > evaluated first,

Re: Enhanced error message to include hint messages for redundant options error

2021-07-11 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 18:09, vignesh C wrote: > > I'm planning to handle conflicting errors separately after this > current work is done, once the patch is changed to have just the valid > scenarios(removing the scenarios you have pointed out) existing > function can work as is without any

Re: Enhanced error message to include hint messages for redundant options error

2021-07-11 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 17:03, vignesh C wrote: > > > Also, I don't think this function should be marked inline -- using a > > normal function ought to help make the compiled code smaller. > > inline instructs the compiler to attempt to embed the function content > into the calling code instead of

Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 18:30, Tom Lane wrote: > > [ moving to pghackers for wider visibility ] > > Dean Rasheed writes: > > On Sat, 10 Jul 2021 at 16:01, Tom Lane wrote: > >> In general, I'm disturbed that we just threw away the previous > >> promise that

Re: pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 16:01, Tom Lane wrote: > > I think this needs a bit more thought. Before, a case like > select 1e-16000 * 1e-16000; > produced > ERROR: value overflows numeric format > Now you get an exact zero (with a lot of trailing zeroes, but still > it's just zero).

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

pgsql: Fix numeric_mul() overflow due to too many digits after decimal

2021-07-10 Thread Dean Rasheed
Fix numeric_mul() overflow due to too many digits after decimal point. This fixes an overflow error when using the numeric * operator if the result has more than 16383 digits after the decimal point by rounding the result. Overflow errors should only occur if the result has too many digits

Re: Enhanced error message to include hint messages for redundant options error

2021-07-10 Thread Dean Rasheed
On Sat, 10 Jul 2021 at 11:44, Dean Rasheed wrote: > > I'm inclined to think that it isn't worth the effort trying to > distinguish between conflicting options, options specified more than > once and faked-up options that weren't really specified. If we just > make errorConflictingD

Re: Enhanced error message to include hint messages for redundant options error

2021-07-10 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 14:40, vignesh C wrote: > > On Thu, Jul 8, 2021 at 1:52 AM Daniel Gustafsson wrote: > > > > I sort of like the visual cue of seeing ereport(ERROR .. since it makes it > > clear it will break execution then and there, this will require a lookup for > > anyone who don't know

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 14:38, David Rowley wrote: > > I gave it a bit of exercise by running pgbench and calling this procedure: > > It ran 8526956 times, so with the loop that's 8.5 billion random > numbers. No variations between the two functions. I got the same > after removing the 0 - to test

Re: rand48 replacement

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 09:26, Fabien COELHO wrote: > > > Finally, I think it would be better to treat the upper bound of the > > range as inclusive. > > This bothered me as well, but the usual approach seems to use range as the > number of values, so I was hesitant to depart from that. I'm still >

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-08 Thread Dean Rasheed
On Thu, 8 Jul 2021 at 05:30, David Rowley wrote: > > On Thu, 8 Jul 2021 at 13:31, David Rowley wrote: > > It feels like if we're going to fix this negative rounding thing then > > we should maybe do it and backpatch a fix then rebase this work on top > > of that. Yes, that was my thinking too.

Re: Numeric x^y for negative x

2021-07-07 Thread Dean Rasheed
On Wed, 7 Jul 2021 at 18:57, Zhihong Yu wrote: > > + (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE), > +errmsg("value overflows numeric format"))); > > Here is an example of existing error message which I think is more readable > than 'overflows numeric

Re: Numeric x^y for negative x

2021-07-07 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 14:17, Dean Rasheed wrote: > > On Tue, 29 Jun 2021 at 12:08, Dean Rasheed wrote: > > > > Numeric x^y is supported for x < 0 if y is an integer, but this > > currently fails if y is outside the range of an int32 > > I've been doing some mo

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-07 Thread Dean Rasheed
On Wed, 7 Jul 2021 at 03:47, David Rowley wrote: > > Updated patch attached. > Hmm, this looked easy, but... It occurred to me that there ought to be regression tests for the edge cases where it steps from one unit to the next. So, in the style of the existing regression tests, I tried the

Re: rand48 replacement

2021-07-07 Thread Dean Rasheed
On Wed, 7 Jul 2021 at 04:00, Yura Sokolov wrote: > > Anyway, excuse me for heating this discussion cause of such > non-essential issue. > I'll try to control myself and don't proceed it further. > Whilst it has been interesting learning and discussing all these different techniques, I think it's

Re: Using COPY FREEZE in pgbench

2021-07-06 Thread Dean Rasheed
On Sun, 4 Jul 2021 at 09:32, Tatsuo Ishii wrote: > > >> So overall gain by the patch is around 15%, whereas the last test > >> before the commit was 14%. It seems the patch is still beneficial > >> after the commit. > > > > Yes, that's good! > > Yeah! > I tested this with -s100 and got similar

Re: Planning time grows exponentially with levels of nested views

2021-07-06 Thread Dean Rasheed
On Sun, 18 Apr 2021 at 21:42, Tom Lane wrote: > > > If multiple references are actually possible then this'd break it. > > I think this patch doesn't make things any worse for such a case though. > If we re-introduced such a bug, the result would be an immediate null > pointer crash while trying

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread Dean Rasheed
On Tue, 6 Jul 2021 at 13:15, David Rowley wrote: > > Can you give an example where calling half_rounded too many times will > give the wrong value? Keeping in mind we call half_rounded the number > of times that the passed in value would need to be left-shifted by to > get the equivalent

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-07-06 Thread Dean Rasheed
On Tue, 6 Jul 2021 at 10:20, David Rowley wrote: > > I made another pass over this and ended up removing the doHalfRound > field in favour of just doing rounding based on the previous > bitshifts. > When I first read this: +/* half-round until we get down to unitBits */ +

Re: Use extended statistics to estimate (Var op Var) clauses

2021-07-05 Thread Dean Rasheed
On Sun, 13 Jun 2021 at 21:28, Tomas Vondra wrote: > > Here is a slightly updated version of the patch > Hi, I have looked at this in some more detail, and it all looks pretty good, other than some mostly cosmetic stuff. The new code in statext_is_compatible_clause_internal() is a little hard

Re: Numeric multiplication overflow errors

2021-07-05 Thread Dean Rasheed
On Sun, 4 Jul 2021 at 09:43, David Rowley wrote: > > On Sat, 3 Jul 2021 at 11:04, Dean Rasheed wrote: > > Thinking about this more, I think it's best not to risk back-patching. > > It *might* be safe, but it's difficult to really be sure of that. The > > bug itself is

Re: Numeric multiplication overflow errors

2021-07-05 Thread Dean Rasheed
On Fri, 2 Jul 2021 at 19:48, Ranier Vilela wrote: > > If you allow me a small suggestion. > Move the initializations of the variable tmp_var to after check if the > function can run. > Saves some cycles, when not running. > OK, thanks. I agree, on grounds of neatness and consistency with nearby

pgsql: Prevent numeric overflows in parallel numeric aggregates.

2021-07-05 Thread Dean Rasheed
Prevent numeric overflows in parallel numeric aggregates. Formerly various numeric aggregate functions supported parallel aggregation by having each worker convert partial aggregate values to Numeric and use numeric_send() as part of serializing their state. That's problematic, since the range of

Re: rand48 replacement

2021-07-04 Thread Dean Rasheed
On Sun, 4 Jul 2021 at 17:03, Fabien COELHO wrote: > > > As for determinism, the end result is still fully deterministic. > > The result is indeed deterministic of you call the function with the same > range. However, if you change the range value in one place then sometimes > the state can

Re: rand48 replacement

2021-07-04 Thread Dean Rasheed
On Sun, 4 Jul 2021 at 10:35, Fabien COELHO wrote: > > I did not understand why it is not correct. > Well, to make it easier to visualise, let's imagine our word size is just 3 bits instead of 64 bits, and that the basic prng() function generates numbers in the range [0,8). Similarly, imagine a

Re: rand48 replacement

2021-07-04 Thread Dean Rasheed
On Sat, 3 Jul 2021 at 08:06, Fabien COELHO wrote: > > Here is a v4, which: > > - moves the stuff to common and fully removes random/srandom (Tom) > - includes a range generation function based on the bitmask method (Dean) > but iterates with splitmix so that the state always advances once

Re: WIP: Relaxing the constraints on numeric scale

2021-07-03 Thread Dean Rasheed
Attached is a more complete patch, with updated docs and tests. I chose to allow the scale to be in the range -1000 to 1000, which, to some extent, is quite arbitrary. The upper limit of 1000 makes sense, because nearly all numeric computations (other than multiply, add and subtract) have that as

Re: Numeric multiplication overflow errors

2021-07-02 Thread Dean Rasheed
On Fri, 2 Jul 2021 at 12:56, David Rowley wrote: > > On Fri, 2 Jul 2021 at 22:55, Dean Rasheed wrote: > > Here's an update with the > > last set of changes discussed. > > Looks good to me. Thanks for the review and testing! > Just the question of if we have any probl

Re: Numeric multiplication overflow errors

2021-07-02 Thread Dean Rasheed
On Fri, 2 Jul 2021 at 10:24, David Rowley wrote: > > I ran this again with a few different worker counts after tuning a few > memory settings so there was no spilling to disk and so everything was > in RAM. Mostly so I could get consistent results. > > Here's the results. Average over 3 runs on

Re: rand48 replacement

2021-07-02 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 22:18, Fabien COELHO wrote: > > > However, this patch seems to be replacing that with a simple > > modulo operation, which is perhaps the worst possible way to do it. > > The modulo operation is biased for large ranges close to the limit, sure. > Also, the bias is somehow of

Re: rand48 replacement

2021-07-01 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 19:41, Tom Lane wrote: > > Anyway, after taking a very quick look at the patch itself, I've > got just one main objection: I don't approve of putting this in > port.h or src/port/. I haven't looked at the patch in detail, but one thing I object to is the code to choose a

Re: Numeric x^y for negative x

2021-07-01 Thread Dean Rasheed
On Tue, 29 Jun 2021 at 12:08, Dean Rasheed wrote: > > Numeric x^y is supported for x < 0 if y is an integer, but this > currently fails if y is outside the range of an int32 > I've been doing some more testing of this, and I spotted another problem with numeric_power(). This

Re: Numeric multiplication overflow errors

2021-07-01 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 06:43, David Rowley wrote: > > Master @ 3788c6678 > > Execution Time: 8306.319 ms > Execution Time: 8407.785 ms > Execution Time: 8491.056 ms > > Master + numeric-agg-sumX2-overflow-v2.patch > Execution Time: 6633.278 ms > Execution Time: 6657.350 ms > Execution Time:

Re: Numeric multiplication overflow errors

2021-07-01 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 10:27, Dean Rasheed wrote: > > I'll post an update in a while. Thanks for the review. > One other thing I'm wondering about is back-patching. I was originally thinking of these as back-patchable bug fixes, but changing the binary format of the aggregate serializati

Re: Numeric multiplication overflow errors

2021-07-01 Thread Dean Rasheed
On Thu, 1 Jul 2021 at 02:00, David Rowley wrote: > > I kinda disagree with the send/recv naming since all you're using them > for is to serialise/deserialise the NumericVar. Functions named > *send() and recv() I more expect to return a bytea so they can be used > for a type's send/recv function.

Re: WIP: Relaxing the constraints on numeric scale

2021-06-29 Thread Dean Rasheed
On Tue, 29 Jun 2021 at 21:34, Robert Haas wrote: > > I thought about this too, but > http://postgr.es/m/774767.1591985...@sss.pgh.pa.us made me think that > it would be an on-disk format break. Maybe it's not, though? > No, because the numeric dscale remains non-negative, so there's no change to

WIP: Relaxing the constraints on numeric scale

2021-06-29 Thread Dean Rasheed
When specifying NUMERIC(precision, scale) the scale is constrained to the range [0, precision], which is per SQL spec. However, at least one other major database vendor intentionally does not impose this restriction, since allowing scales outside this range can be useful. A negative scale implies

Numeric x^y for negative x

2021-06-29 Thread Dean Rasheed
Numeric x^y is supported for x < 0 if y is an integer, but this currently fails if y is outside the range of an int32: SELECT (-1.0) ^ 2147483647; ?column? - -1. (1 row) SELECT (-1.0) ^ 2147483648; ERROR: cannot take logarithm of a negative number

Re: Numeric multiplication overflow errors

2021-06-29 Thread Dean Rasheed
Thanks for looking! On Mon, 28 Jun 2021 at 12:27, David Rowley wrote: > > Instead of adding a send/recv function, unless I'm mistaken, it should > be possible to go the whole hog and optimizing this further by instead > of having numericvar_send(), add: > > static void

Numeric multiplication overflow errors

2021-06-28 Thread Dean Rasheed
I found a couple of places where numeric multiplication suffers from overflow errors for inputs that aren't necessarily very large in magnitude. The first is with the numeric * operator, which attempts to always produce the exact result, even though the numeric type has a maximum of 16383 digits

Re: Use extended statistics to estimate (Var op Var) clauses

2021-06-21 Thread Dean Rasheed
On Sun, 13 Jun 2021 at 21:28, Tomas Vondra wrote: > > Here is a slightly updated version of the patch > > The main issue I ran into > is the special case clauselist_selectivity, which does > > if (list_length(clauses) == 1) > return clause_selectivity_ext(...); > > which applies to

Re: Failures with gcd functions with GCC snapshots GCC and -O3 (?)

2021-06-03 Thread Dean Rasheed
On Thu, 3 Jun 2021 at 08:26, Michael Paquier wrote: > > Hi all, > > serinus has been complaining about the new gcd functions in 13~: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=serinus=2021-06-03%2003%3A44%3A14 > > The overflow detection is going wrong the way up and down, like

Re: pgbench test failing on 14beta1 on Debian/i386

2021-05-19 Thread Dean Rasheed
On Wed, 19 May 2021 at 12:07, Fabien COELHO wrote: > > Attached patch disactivates the test with comments to outline that there > is an issue to fix… so it is *not* removed. > I opted to just remove the test rather than comment it out, since the issue highlighted isn't specific to permute().

pgsql: Fix pgbench permute tests.

2021-05-19 Thread Dean Rasheed
Fix pgbench permute tests. One of the tests for the pgbench permute() function added by 6b258e3d68 fails on some 32-bit platforms, due to variations in the floating point computations in getrand(). The remaining tests give sufficient coverage, so just remove the failing test. Reported by

Re: pgbench test failing on 14beta1 on Debian/i386

2021-05-19 Thread Dean Rasheed
On Wed, 19 May 2021 at 11:32, Fabien COELHO wrote: > > >> Or, (3) remove this test? I am not quite sure what there is to gain > >> with this extra test considering all the other tests with permute() > >> already present in this script. > > > > Yes, I think removing the test is the best option.

Re: pgbench test failing on 14beta1 on Debian/i386

2021-05-19 Thread Dean Rasheed
On Wed, 19 May 2021 at 00:35, Thomas Munro wrote: > > FWIW this is reproducible on my local Debian/gcc box with -m32, Confirmed, thanks for looking. I can reproduce it on my machine with -m32. It's somewhat annoying that the buildfarm didn't pick it up sooner :-( On Wed, 19 May 2021 at 08:28,

Re: Does rewriteTargetListIU still need to add UPDATE tlist entries?

2021-04-26 Thread Dean Rasheed
On Mon, 26 Apr 2021 at 15:55, Tom Lane wrote: > > I checked into the commit history (how'd we ever survive without "git > blame"?) and found that my argument above is actually wrong in detail. > Before cab5dc5da of 2013-10-18, rewriteTargetListIU expanded non-updated > columns for all views not

Re: Does rewriteTargetListIU still need to add UPDATE tlist entries?

2021-04-26 Thread Dean Rasheed
On Mon, 26 Apr 2021 at 15:09, Tom Lane wrote: > > Thanks for looking at that. On reflection I think this must be so, > because those rewriter mechanisms were designed long before we had > trigger-updatable views, and rewriteTargetListIU has never added > tlist items like this for any other sort

Re: pgbench - add pseudo-random permutation function

2021-04-06 Thread Dean Rasheed
On Mon, 5 Apr 2021 at 13:07, Fabien COELHO wrote: > > Attached a v28 which I hope fixes the many above issues and stays with > ints. The randu64 is still some kind of a joke, I artificially reduced the > cost by calling jrand48 once and extending it to 64 bits, so it could give > an idea of the

pgsql: pgbench: Function to generate random permutations.

2021-04-06 Thread Dean Rasheed
pgbench: Function to generate random permutations. This adds a new function, permute(), that generates pseudorandom permutations of arbitrary sizes. This can be used to randomly shuffle a set of values to remove unwanted correlations. For example, permuting the output from a non-uniform random

Re: pgbench - add pseudo-random permutation function

2021-04-04 Thread Dean Rasheed
On Fri, 2 Apr 2021 at 06:38, Fabien COELHO wrote: > > >> r = (uint64) (pg_erand48(random_state.xseed) * size); > >> > >> I do not understand why the random values are multiplied by anything in > >> the first place… > > > > These are just random integers in the range [0,mask] and [0,size-1], > >

Re: pgbench - add pseudo-random permutation function

2021-04-01 Thread Dean Rasheed
On Wed, 31 Mar 2021 at 18:53, Fabien COELHO wrote: > > While looking at it, I have some doubts on this part: > > m = (uint64) (pg_erand48(random_state.xseed) * (mask + 1)) | 1; > r = (uint64) (pg_erand48(random_state.xseed) * (mask + 1)); > r = (uint64) (pg_erand48(random_state.xseed) *

<    1   2   3   4   5   6   7   8   9   10   >