Dean Rasheed writes:
> On 24 November 2015 at 16:20, Tom Lane wrote:
>> ... None of these effects are going to let the final div[qi+1] value
>> get to more than two or three times NBASE squared, which is still
>> an order of magnitude less than INT_MAX.
> Right. In fact I think div[qi+1] is even
On 24 November 2015 at 16:20, Tom Lane wrote:
> After further thought I've pretty well convinced myself that there is
> indeed no observable bug, at least as long as you assume that overflow
> within the multiplication will behave as stated above. The proof goes
> like this:
>
> We already know t
I wrote:
> Note that div[qi+1], and indeed all the remaining dividend digits, are
> large negative values. This is what you'd expect if the carry propagation
> step hasn't run for awhile, which is a precondition for div[qi] being
> large enough to cause an issue. When we compute 218943 * 1, w
Dean Rasheed writes:
> On 18 November 2015 at 22:19, Tom Lane wrote:
>> Still, this proves that we are onto a real problem.
> Agreed. I had a go at turning that example into something using
> log(base, num) so that the result would be visible in a pure SQL test,
> but I didn't have any luck.
I
On 18 November 2015 at 22:19, Tom Lane wrote:
> A bit of progress on this: I've found a test case, namely
>
> select sqrt(99...
>
Wow.
> Still, this proves that we are onto a real problem.
>
Agreed. I had a go at turning that example into something using
log(base, num) so that the result wo
I wrote:
> I'm kind of stuck on that too. I did some experimentation by tracking
> maximum values of outercarry in the regression tests (including
> numeric_big) and did not see it get larger than a couple hundred thousand,
> ie more or less INT_MAX/NBASE. But I don't have an argument as to why
>
Dean Rasheed writes:
> On 17 November 2015 at 23:57, Tom Lane wrote:
>> I'm not sure that it's provably okay though. The loose ends are to show
>> that div[qi] can't overflow an int during the divisor-subtraction step
>> and that "outercarry" remains bounded. Testing suggests that outercarry
>
On 17 November 2015 at 23:57, Tom Lane wrote:
> After thinking some more about what this is doing, it seems to me that
> we could avoid changing div[qi + 1] at all here, and instead deal with
> leftover dividend digits by shoving them into the floating-point part of
> the calculation. All that we
I wrote:
> I pushed this, but while looking at it, my attention was drawn to this
> bit down near the end of the loop:
> /*
> * The dividend digit we are about to replace might still be nonzero.
> * Fold it into the next digit position. We don't need to worry about
>
Dean Rasheed writes:
> On 17 November 2015 at 14:43, Tom Lane wrote:
>> Hm, good point. I don't feel a compulsion to have a test case that
>> proves it's broken before we fix it. Do you want to send a patch?
> OK, here it is. It's slightly different from mul_var, because maxdiv
> is tracking a
On 17 November 2015 at 14:43, Tom Lane wrote:
> Dean Rasheed writes:
>> I just noticed that div_var_fast() has almost identical code, and so
>> in principle it has the same vulnerability, although it obviously only
>> affects the transcendental functions.
>> I don't actually have a test case that
Dean Rasheed writes:
> I just noticed that div_var_fast() has almost identical code, and so
> in principle it has the same vulnerability, although it obviously only
> affects the transcendental functions.
> I don't actually have a test case that triggers it, but it's basically
> the same algorithm
On 21 September 2015 at 17:14, Tom Lane wrote:
> Dean Rasheed writes:
>> On 21 September 2015 at 16:09, Tom Lane wrote:
>>> After trying to rework the comment to explain what maxdig really meant
>>> after your changes, I came to the conclusion that it'd be better to do
>>> it as per attached. D
Dean Rasheed writes:
> On 21 September 2015 at 16:09, Tom Lane wrote:
>> After trying to rework the comment to explain what maxdig really meant
>> after your changes, I came to the conclusion that it'd be better to do
>> it as per attached. Does this look sane to you?
> Yes that looks better. I
On 21 September 2015 at 16:09, Tom Lane wrote:
> I wrote:
>> Dean Rasheed writes:
>>> The problem then arises in the final carry propagation pass. During
>>> this phase of the computation, the carry from one digit (which can be
>>> a shade under INT_MAX / NBASE) is added to the next digit, and th
I wrote:
> Dean Rasheed writes:
>> The problem then arises in the final carry propagation pass. During
>> this phase of the computation, the carry from one digit (which can be
>> a shade under INT_MAX / NBASE) is added to the next digit, and that's
>> where the overflow happens.
> Nice catch! I
Dean Rasheed writes:
> The problem then arises in the final carry propagation pass. During
> this phase of the computation, the carry from one digit (which can be
> a shade under INT_MAX / NBASE) is added to the next digit, and that's
> where the overflow happens.
Nice catch! I think the comment
Hi,
By chance, while testing the nearby numeric log/exp/pow patch, I came
across the following case which generates an initially puzzling
looking error on HEAD -- (5.6-1e-500) ^ (3.2-1e-200). This computation
actually works OK with that other patch, but only by blind luck. It
turns out that the un
18 matches
Mail list logo
