subject:"Always false float comparisons"


On 21.05.2016 20:14, Walter Bright wrote:

On 5/21/2016 10:03 AM, Timon Gehr wrote:

Check out section 5 for some convincing examples showing why the x87
is horrible.


The polio vaccine winds up giving a handful of people polio, too.
...


People don't get vaccinated without consent.


It's good to list traps for the unwary in FP usage. It's disingenuous to
list only problems with one design and pretend there are no traps in
another design.


Some designs are much better than others.

Re: Always false float comparisons

2016-05-21 Thread Walter Bright via Digitalmars-d


On 5/21/2016 11:36 AM, Tobias M wrote:

Sorry but this is a misrepresentation. I never claimed that the x87 doesn't
conform to the IEEE standard.


My point was directed to more than just you. Sorry I didn't make that clear.



The point is, that is IS possible to provide fairly reasonable and consistent
semantics within the existing standards (C, C++, IEEE, ...).


That implies what I propose, which is what many C/C++ compilers do, is 
unreasonable, inconsistent, not Standard compliant, and not IEEE. I.e. that the 
x87 is not conformant :-)


Read the documentation on the FP switches for VC++, g++, clang, etc. You'll see 
there are tradeoffs. There is no "obvious, sane" way to do it.


There just isn't.



They provide a
certain degree of freedom to accomodate for different hardware, but this doesn't
mean that software should use this freedom to do arbitrary things.


Nobody is suggesting doing arbitrary things, but to write portable fp, take into 
account what the Standard says rather than what your version of the compiler 
does with various default and semi-documented switches.




Regarding the decades of FP design, the initial edition of K C contained the
following clause:
"Notice that all floats in an expression are converted to double; all floating
point arithmethic in C is done in double precision".
That passus was removed quite quickly because users complained about it.


It was changed to allow floats to be computed as floats, not require it. And the 
reason at the time, as I recall, was to get faster floating point ops, not 
because anyone desired reduced precision.

Re: Always false float comparisons

2016-05-21 Thread Fool via Digitalmars-d


Reasons have been alleged. What's your final decision?

Re: Always false float comparisons


On 21.05.2016 19:58, Walter Bright wrote:

On 5/21/2016 2:26 AM, Tobias Müller wrote:

On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I
present Kahan
who concurrently designed the IEEE 754 spec and the x87.


Actually cited this *because* of you mentioning Kahan several times.
And because
you said "The people who design these things are not fools, and there
are good
reasons for the way things are."


I meant two things by this:

1. Do the homework before disagreeing with someone who literally wrote
the book and designed the hardware for it.
...


Sure.


2. Complaining that the x87 is not IEEE compliant, when the guy that
designed the x87 wrote the spec at the same time, suggests a
misunderstanding the spec.


Who claimed that the x87 is not IEEE compliant? Anyway, this is easy to 
resolve. IEEE 754-2008 requires FMA. x87 has no FMA.



Also, in practice, it is used to produce non-compliant results, as it 
has a default mode of being used that gives you results that differ from 
the specified results for single and double precision sources and 
destinations.


From IEEE 754-2008:

"1.2 Purpose
1.2.0
This standard provides a method for computation with floating-point 
numbers that will yield the same result whether the processing is done 
in hardware, software, or a combination of the two."


I.e. the only stated purpose of IEEE 754 is actually reproducibility.



"shall indicates mandatory requirements strictly to be followed in order 
to conform to the standard and from which no deviation is permitted 
(“shall” means “is required to”"


"3.1.2 Conformance
3.1.2 .0
A  conforming  implementation  of  any  supported  format  shall 
provide means to initialize that format and shall provide conversions 
between that format and all other supported formats.


A conforming implementation of a supported arithmetic format shall 
provide all the operations of this standard defined in Clause 5, for 
that format.


A conforming implementation of a supported interchange format shall 
provide means to read and write that

format using a specific encoding defined in this clause, for that format.

A programming environment conforms to this standard, in a particular 
radix, by implementing one or more of the basic formats of that radix as 
both a supported arithmetic format and a supported interchange format."



For the original IEEE 754-1985, the x87 seems to support all of those 
clauses for float, double and extended (if you use it in the right way, 
which is inefficient, as you need to spill the result to memory after 
each operation, and it is not the default way), but it also supports 
further operations that fulfill similar functions in a non-conforming 
manner, and compiler implementations use it that way.


Another way to read it is that the x86 conforms by supporting float and 
double as interchange formats and extended as arithmetic format.


What is the correct way to interpret the x87 as conforming?



I.e. again, gotta do the homework first.

Dismissing several decades of FP designs, and every programming
language, as being "obviously wrong" and "insane"


No need to unduly generalize the scope of my claims.


is an extraordinary claim,


Not really. There don't seem to be so many popular systems in computing 
that are not broken in one way or another. (And even when they are, they 
are a lot more useful than nothing. I'd rather have the x87 than no 
hardware floating point support at all.)



and so requires extraordinary evidence.
...


There are several decades of experience with the x87 to draw from. SSE 
does not suffer from those issues anymore. This is because flaws in the 
design were identified and fixed. Why is this evidence not extraordinary 
enough?



After all, what would your first thought be when a sophomore physics
student tells you that Feynman got it all wrong?


It's an oddly unspecific statement (what about his work is wrong?) and 
it's a completely different case. Feynman getting everything wrong is 
not a popular sentiment in the Physics community AFAICT.



It's good to revisit
existing dogma now and then, and challenge the underlying assumptions of
it, but again, you gotta understand the existing dogma all the way down
first.
...


What parts of "existing dogma" justify all the problems that x87 causes?
Also, if e.g.implicit 80 bit precision for CTFE floats truly is mandated 
by "floating point dogma" then "floating point dogma" clashes with 
"language design dogma".



If you don't, you're very likely to miss something fundamental and
produce a design that is less usable.


I haven't invented anything new either. SSE already fixes the issues.

(But yes, it is probably hard to support systems that only have the x87 
in a sane way. Tradeoffs might be necessary. Deliberately copying 
mistakes that x87 made to other contexts is not the right

Re: Always false float comparisons

2016-05-21 Thread Tobias M via Digitalmars-d


On Saturday, 21 May 2016 at 17:58:49 UTC, Walter Bright wrote:

On 5/21/2016 2:26 AM, Tobias Müller wrote:

On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may 
I present Kahan

who concurrently designed the IEEE 754 spec and the x87.


Actually cited this *because* of you mentioning Kahan several 
times. And because
you said "The people who design these things are not fools, 
and there are good

reasons for the way things are."


I meant two things by this:

1. Do the homework before disagreeing with someone who 
literally wrote the book and designed the hardware for it.


2. Complaining that the x87 is not IEEE compliant, when the guy 
that designed the x87 wrote the spec at the same time, suggests 
a misunderstanding the spec. I.e. again, gotta do the homework 
first.


Sorry but this is a misrepresentation. I never claimed that the 
x87 doesn't conform to the IEEE standard. That's completely 
missing the point. Again.


Dismissing several decades of FP designs, and every programming 
language, as being "obviously wrong" and "insane" is an 
extraordinary claim, and so requires extraordinary evidence.


After all, what would your first thought be when a sophomore 
physics student tells you that Feynman got it all wrong? It's 
good to revisit existing dogma now and then, and challenge the 
underlying assumptions of it, but again, you gotta understand 
the existing dogma all the way down first.


If you don't, you're very likely to miss something fundamental 
and produce a design that is less usable.


The point is, that is IS possible to provide fairly reasonable 
and consistent semantics within the existing standards (C, C++, 
IEEE, ...). They provide a certain degree of freedom to 
accomodate for different hardware, but this doesn't mean that 
software should use this freedom to do arbitrary things.


Regarding the decades of FP design, the initial edition of K C 
contained the following clause:
"Notice that all floats in an expression are converted to double; 
all floating point arithmethic in C is done in double precision".
That passus was removed quite quickly because users complained 
about it.

Re: Always false float comparisons

2016-05-21 Thread Walter Bright via Digitalmars-d


On 5/21/2016 10:03 AM, Timon Gehr wrote:

Check out section 5 for some convincing examples showing why the x87 is 
horrible.


The polio vaccine winds up giving a handful of people polio, too.

It's good to list traps for the unwary in FP usage. It's disingenuous to list 
only problems with one design and pretend there are no traps in another design.

Re: Always false float comparisons

2016-05-21 Thread Walter Bright via Digitalmars-d


On 5/21/2016 2:26 AM, Tobias Müller wrote:

On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I present Kahan
who concurrently designed the IEEE 754 spec and the x87.


Actually cited this *because* of you mentioning Kahan several times. And because
you said "The people who design these things are not fools, and there are good
reasons for the way things are."


I meant two things by this:

1. Do the homework before disagreeing with someone who literally wrote the book 
and designed the hardware for it.


2. Complaining that the x87 is not IEEE compliant, when the guy that designed 
the x87 wrote the spec at the same time, suggests a misunderstanding the spec. 
I.e. again, gotta do the homework first.


Dismissing several decades of FP designs, and every programming language, as 
being "obviously wrong" and "insane" is an extraordinary claim, and so requires 
extraordinary evidence.


After all, what would your first thought be when a sophomore physics student 
tells you that Feynman got it all wrong? It's good to revisit existing dogma now 
and then, and challenge the underlying assumptions of it, but again, you gotta 
understand the existing dogma all the way down first.


If you don't, you're very likely to miss something fundamental and produce a 
design that is less usable.

Re: Always false float comparisons


On 21.05.2016 15:45, Timon Gehr wrote:

On 21.05.2016 00:22, Walter Bright wrote:

...
may I present
Kahan who concurrently designed the IEEE 754 spec and the x87.



The x87 is by far not the slam-dunk design you seem to make it out to
be. ...


https://hal.archives-ouvertes.fr/hal-00128124v5/document
Check out section 5 for some convincing examples showing why the x87 is 
horrible.

Re: Always false float comparisons


On 17.05.2016 20:09, Max Samukha wrote:

On Monday, 16 May 2016 at 19:01:19 UTC, Timon Gehr wrote:


You are not even guaranteed to get the same result on two different x86
implementations.


Without reading the x86 specification, I think it is safe to claim
that you actually are guaranteed to get the same result.


Correct. Sorry for the noise.


Interestingly, I was actually wrong:
https://hal.archives-ouvertes.fr/hal-00128124v5/document

I think that document is worth a read in any case.

Page 18: "Note, also, that different processors within the same 
architecture can implement the same transcendental functions with 
different accuracies. We already noted the difference between the AMD-K5 
and the K6 and following processors with respect to angular reduction. 
Intel also notes that the algorithms’ precision was improved between the 
80387 / i486DX processors and the Pentium processors. [Int, 1997,§7.5.10]"


The language should ideally not leak such differences to the user.

Re: Always false float comparisons


On 20.05.2016 14:34, Johan Engelen wrote:

On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00

Better?


Ignore if you think it's not relevant.
I am a bit lost in this thread. I thought this was about floating point
CTFE,


Yes, this is how it started out. :)
At some point, I made the comment that the compiler should never be 
allowed to use a higher precision than the one specified, which 
broadened the scope of the discussion somewhat. But the kinds of 
problems caused by using higher precision than requested can be similar 
for CTFE and runtime.



but I think what you are seeing here (in the 2nd case) is the DMD
optimizer allowing reassociation of fp expressions?


I don't believe it does (Walter knows /very/ well that a+(b+c) is not 
the same as (a+b)+c), but I have not looked at the assembly output. The 
result becomes wrong when 80 bit precision is used behind the scenes for 
some (but not all) of the computation, so this could be the explanation.



LDC does not allow
that (per default) and produces the 1001.00 result also
with optimization on.

-Johan


Thanks!

Re: Always false float comparisons


On 20.05.2016 08:25, Walter Bright wrote:

On 5/19/2016 12:49 AM, Max Samukha wrote:

People are trying to get across that, if they wanted to maximize
accuracy, they
would request the most precise type explicitly. D has 'real' for that.
This
thread has shown unequivocally that the semantics you are insisting on
is bound
to cause endless confusion and complaints.


C++ programs do the same thing, and have for decades, and such behavior
is allowed by the C++ Standard.
...


And reported as a gcc bug regularly, and there seems to be a flag to 
turn x87-induced strangeness off completely. Also, the C99 standard 
rules out compiler-specific results. (And allows the program to query 
whether it is being compiled with the set of flags it requires: 
https://en.wikipedia.org/wiki/C99#IEEE.C2.A0754_floating_point_support .)



People have been confused by and complained about floating point
behavior at least since I started programming 40 years ago.
...


That's mostly an incidental problem. (Sharing language syntax with 
non-rounding operations, needing to map to mis-designed hardware, etc.)



I have not invented anything new here.



You may not have invented anything new, but the old invention is broken 
and in the (slow and painful) process of getting fixed. There is no 
reason to build new things that use the same principle, as there is 
already a better way available.

Re: Always false float comparisons


On 20.05.2016 13:32, Joakim wrote:

On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:

On 20.05.2016 11:14, Joakim wrote:

On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

On 19.05.2016 08:04, Joakim wrote:

On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

It's not just slightly worse, it can cut the number of useful bits in
half or more! It is not unusual, I have actually run into those
problems in the past, and it can break an algorithm that is in Phobos
today!


I wouldn't call that broken.  Looking at the hex output by
replacing %f
with %A in writefln, it appears the only differences in all those
results is the last byte in the significand.


Argh...

// ...

void main(){
//double[] data=[1e16,1,-9e15];
import std.range;
double[] data=1e16~repeat(1.0,1).array~(-9e15);
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00


Better?

Obviously there is more structure in the data that I invent manually
than in a real test case where it would go wrong. The problems carry
over though.


I looked over your code a bit.  If I define sum and c as reals in
"kahanBroken" at runtime, this problem goes away.


Yes. That's absolutely obvious, and I have noted it before, but
thanks. Maybe try to understand why this problem occurs in the first
place.


Yet you're the one arguing against increasing precision everywhere in CTFE.
...


High precision is usually good (use high precision, up to arbitrary 
precision or even symbolic arithmetic whenever it improves your results 
and you can afford it). *Implicitly* increasing precision during CTFE is 
bad. *Explicitly* using higher precision during CTFE than at running 
time /may or may not/ be good. In case it is good, there is often no 
reason to stop at 80 bits.



Since that's what the
CTFE rule is actually doing, ie extending all floating-point to reals at
compile-time, I don't see what you're complaining about.  Try it, run
even your original naive summation algorithm through CTFE and it will
produce the result you want:

enum double[] ctData=[1e16,1,-9e15];
enum ctSum = sum(ctData);
writefln("%f", ctSum);
...


This example wasn't specifically about CTFE, but just imagine that
only part of the computation is done at CTFE, all local variables are
transferred to runtime and the computation is completed there.


Why would I imagine that?


Because that's the most direct way to go from that example to one where 
implicit precision enhancement during CTFE only is bad.



And this whole discussion is about what
happens if you change the precision of all variables to real when doing
CTFE, so what's the point of giving an example that isn't "specifically"
about that?
...


Walter's request wasn't specifically about that, and it's the first 
thing I came up with:


> On 17.05.2016 23:07, Walter Bright wrote:

I'd like to see an example of double rounding "completely defeating" an
algorithm,


I don't have infinite amounts of time. The (significant) time spent 
writing up all of this competes with time spent doing things that are 
guaranteed to yield (often immediate) tangible benefits.




And if any part of it is done at runtime using the algorithms you gave,
which you yourself admit works fine if you use the right
higher-precision types,


What's "right" about them? That the compiler will not implicitly 
transform some of them to even higher precision in order to break the 
algorithm again? (I don't think that is even guaranteed.)



you don't seem to have a point at all.
...


Be assured that I have a point. If you spend some time reading, or ask 
some constructive questions, I might be able to get it across to you. 
Otherwise, we might as well stop arguing.



As Don's talk pointed out,
all floating-point calculations will see loss of precision starting
there.
...



This is implicitly assuming a development model where the programmer
first writes down the computation as it would be correct in the real
number system and then naively replaces every operation by the
rounding equivalent and hopes for the best.


No, it is intrinsic to any floating-point calculation.
...


How do you even define accuracy if you don't specify an infinitely
precise reference result?


There is no such thing as an infinitely precise result.  All one can do
is compute using even higher precision and compare it to lower precision.
...


If I may ask, how much mathematics have you been exposed to?



It is a useful rule if that is what you're doing. One might be doing
something else. Consider the following paper for an example where the
last bit in the significant actually carries useful information for
many of the values used in the program.

Re: Always false float comparisons

2016-05-19 Thread Ola Fosheim Grøstad via Digitalmars-d

On 21.05.2016 00:22, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.

If we're going to use the fallacy of appeal to authority,

Authorities are not always wrong, the fallacy is to argue that they are
right *because* they are authorities. However, in this case, I don't
think you have provided a quote from Kahan that is applicable to the
case we have here. The Gustafson quote is formulated generally enough to
apply to the current case directly.

may I present
Kahan who concurrently designed the IEEE 754 spec and the x87.

The x87 is by far not the slam-dunk design you seem to make it out to
be. (And you don't need to take my word for it. Otherwise, why would
hardware designers prefer to phase it out in favour of systems that
don't do the implicit extra scratch space thing?)

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

Bug report for this problem in gcc. The discussion raises an issue I
wasn't aware of so far (or have forgotten about): Apparently the x87
does not even round to double correctly, it only rounds the mantissa but
does not deal with out-of-range-exponents correctly. What a mess.

http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html

x87 experience report.

http://www.vedetta.com/x87-fpu-php-bug-causes-infinite-loop-affected-websites-vulnerable-to-dos-via-php-get-function-2.2250738585072011e-308

DOS attack possible due to usage of x87 instructions.

http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it

Many complaints about x87, and some concerns about 80 bit floating point
support in the future.

http://www.vinc17.org/research/extended.en.html

Good summary of some x87 problems. (Slightly broken English though.)

http://www.realworldtech.com/physx87/

x87 instructions used to make the GPU look faster compared to the CPU.
(Not /directly/ a x87 design problem.)

http://www.cims.nyu.edu/~dbindel/class/cs279/87stack.pdf
http://www.cims.nyu.edu/~dbindel/class/cs279/stack87.pdf

Kahan criticizing the design of the register stack on the x87.

I haven't found any statement from Kahan on gcc bug 323, but this would
be really interesting, if anyone can find anything.

I think there is ample evidence that x87 is ultimately a design failure,
even though the intentions were good and based on real expertise.

Re: Always false float comparisons

2016-05-21 Thread Fool via Digitalmars-d


On Tuesday, 17 May 2016 at 21:07:21 UTC, Walter Bright wrote:
[...] why an unusual case of producing a slightly worse answer 
trumps the usual case of producing better answers.


'Sometimes worse' is not 'better', but that's not the point, here.

Even if you managed to consistently produce not less accurate 
results: if the results computed at compile time differ from 
those computed at run time you cannot safely substitute run time 
calls with compile time calls without altering the result. Even 
slightest differences can lead to completely different outcomes.


For example, if we are interested in the number of real roots of 
a quadratic polynomial we compute its discriminant D. There are 
no solutions if D is negative, there is one solution if D is zero 
and there are two solutions if D is positive. In a floating point 
environment the crucial test is often implemented with a certain 
tolerance and good implementations compute the discriminant with 
increased precision. In any case, the critical point is that we 
have to make a decision based on a single computed value. A 
difference of one unit in the last place can change the result.


Note that this in not about correctness of the result but about 
reproducibility and substitutability across run time and compile 
time boundaries.


Please make a decision, document it, and move on.

Fool

Re: Always false float comparisons

2016-05-21 Thread Tobias M via Digitalmars-d


On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I 
present Kahan who concurrently designed the IEEE 754 spec and 
the x87.


Since I'm just in the mood of appealing to authorities, let me 
cite Wikipedia:


The argument from authority (Latin: argumentum ad verecundiam) 
also appeal to authority, is a common argument form which can be 
fallacious, *such as when an authority is cited on a topic 
outside their area of expertise, when the authority cited is not 
a true expert or if the cited authority is stating a contentious 
or controversial position.*


(Emphasis is mine)

Re: Always false float comparisons

2016-05-21 Thread Tobias Müller via Digitalmars-d


On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I 
present Kahan who concurrently designed the IEEE 754 spec and 
the x87.


Actually cited this *because* of you mentioning Kahan several 
times. And because you said "The people who design these things 
are not fools, and there are good reasons for the way things are."


Obviously, the designer of x87 and IEEE 754 (which both 
explicitly allow such "magic" behind the back of the programmer) 
thinks this is a good thing.


But after reading the book, I tend to agree with Gustafson. In 
the book this is elaborated more in detail, I just cited the 
conclusion.

Re: Always false float comparisons

2016-05-21 Thread Max Samukha via Digitalmars-d


On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:

If you say so. I would like to see an example that 
demonstrates that the first

roundToDouble is required.


That's beside the point. If there are spots in the program that 
require rounding, what is wrong with having to specify it?


Because the programmer has already specified it with a type that 
requires rounding!


Why compromise speed & accuracy in the rest of the code that 
does not require it? (And yes, rounding in certain spots can 
and does compromise both speed and accuracy.)




Accuracy and speed won't be compromised if the programmer chooses 
an appropriate type - 'real' for the highest precision supported 
by the implementation.


BTW, you need to remove 'hardware' from the definition of 'real' 
in the spec.

Re: Always false float comparisons

2016-05-20 Thread Walter Bright via Digitalmars-d


On 5/20/2016 5:36 AM, Tobias M wrote:

Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I present Kahan 
who concurrently designed the IEEE 754 spec and the x87.

Re: Always false float comparisons

2016-05-20 Thread HorseWrangler via Digitalmars-d


On Friday, 20 May 2016 at 11:22:48 UTC, Timon Gehr wrote:

On 20.05.2016 08:12, Walter Bright wrote:
I'm curious if you know of any language that meets your 
requirements.

(Java 1.0 did, but Sun was forced to abandon that.)


x86_64 assembly language.


Similar discussion for Rust: 
https://internals.rust-lang.org/t/pre-rfc-dealing-with-broken-floating-point/2673/1

Re: Always false float comparisons

2016-05-20 Thread Tobias M via Digitalmars-d


On Friday, 20 May 2016 at 12:32:40 UTC, Tobias Müller wrote:

Let me cite Prof. John L Gustafson


Not "Prof." but "Dr.", sorry about that. Still an authority, 
though.

Re: Always false float comparisons

2016-05-20 Thread Johan Engelen via Digitalmars-d


On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00

Better?


Ignore if you think it's not relevant.
I am a bit lost in this thread. I thought this was about floating 
point CTFE, but I think what you are seeing here (in the 2nd 
case) is the DMD optimizer allowing reassociation of fp 
expressions? LDC does not allow that (per default) and produces 
the 1001.00 result also with optimization on.


-Johan

Re: Always false float comparisons

2016-05-20 Thread Tobias Müller via Digitalmars-d


On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:

On 5/19/2016 1:26 PM, Timon Gehr wrote:
Those two lines producing different results is unexpected, 
because you are
explicitly saying that y is a double, and the first line also 
does implicit
rounding (probably -- on all compilers and targets that will 
be relevant in the

near future -- to double).

> [...]

It's obviously bad language design on multiple levels.


Floating point semantics simply are not that simple, on any 
compiler, language or hardware. I recommend reading what the 
C/C++ Standards say about it, and look at the plethora of 
compiler switches for VC++/g++/clang++.


The people who design these things are not fools, and there are 
good reasons for the way things are.


Sorry, I don't agree.


Let me cite Prof. John L Gustafson from his Book "The End of 
Error" (it's worth reading):

---
The lesson taught by all these attempts at clever hidden 
scratchpad work is that computer users want *consistent* answers, 
and want to be *permitted the trade-of between speed and 
accuracy* depending on the situation. There is a subtle arrogance 
in any computer system that takes the position "You asked for a 
quick, cheap, approximate calculation, but I know what is good 
for you and will instead do a better job even if it takes more 
time and storage and energy and gives you a different answer. 
You're welcome."

---

Re: Always false float comparisons

2016-05-20 Thread Joakim via Digitalmars-d


On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:

On 20.05.2016 11:14, Joakim wrote:

On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

On 19.05.2016 08:04, Joakim wrote:

On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
It's not just slightly worse, it can cut the number of 
useful bits in
half or more! It is not unusual, I have actually run into 
those
problems in the past, and it can break an algorithm that is 
in Phobos

today!


I wouldn't call that broken.  Looking at the hex output by 
replacing %f
with %A in writefln, it appears the only differences in all 
those

results is the last byte in the significand.


Argh...

// ...

void main(){
//double[] data=[1e16,1,-9e15];
import std.range;
double[] data=1e16~repeat(1.0,1).array~(-9e15);
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00


Better?

Obviously there is more structure in the data that I invent 
manually
than in a real test case where it would go wrong. The 
problems carry

over though.


I looked over your code a bit.  If I define sum and c as reals 
in

"kahanBroken" at runtime, this problem goes away.


Yes. That's absolutely obvious, and I have noted it before, but 
thanks. Maybe try to understand why this problem occurs in the 
first place.


Yet you're the one arguing against increasing precision 
everywhere in CTFE.



Since that's what the
CTFE rule is actually doing, ie extending all floating-point 
to reals at
compile-time, I don't see what you're complaining about.  Try 
it, run
even your original naive summation algorithm through CTFE and 
it will

produce the result you want:

enum double[] ctData=[1e16,1,-9e15];
enum ctSum = sum(ctData);
writefln("%f", ctSum);
...


This example wasn't specifically about CTFE, but just imagine 
that only part of the computation is done at CTFE, all local 
variables are transferred to runtime and the computation is 
completed there.


Why would I imagine that?  And this whole discussion is about 
what happens if you change the precision of all variables to real 
when doing CTFE, so what's the point of giving an example that 
isn't "specifically" about that?


And if any part of it is done at runtime using the algorithms you 
gave, which you yourself admit works fine if you use the right 
higher-precision types, you don't seem to have a point at all.



As Don's talk pointed out,
all floating-point calculations will see loss of precision 
starting

there.
...



This is implicitly assuming a development model where the 
programmer
first writes down the computation as it would be correct in 
the real

number system and then naively replaces every operation by the
rounding equivalent and hopes for the best.


No, it is intrinsic to any floating-point calculation.
...


How do you even define accuracy if you don't specify an 
infinitely precise reference result?


There is no such thing as an infinitely precise result.  All one 
can do is compute using even higher precision and compare it to 
lower precision.


It is a useful rule if that is what you're doing. One might 
be doing
something else. Consider the following paper for an example 
where the
last bit in the significant actually carries useful 
information for

many of the values used in the program.

http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


Did you link to the wrong paper? ;)


No. That paper uses multiple doubles per approximated real 
value to implement arithmetic that is more precise than using 
just plain doubles. If any bit in the first double is off, this 
is no better than using a single double.



I skimmed it and that paper
explicitly talks about error bounds all over the place.


It is enough to read the abstract to figure out what the 
problem is. This demonstrates a non-contrived case where CTFE 
using enhanced precision throughout can break your program. 
Compute something as a double-double at compile-time, and when 
it is transferred to runtime you lose all the nice extra 
precision, because bits in the middle of the (conceptual) 
mantissa are lost.


That is a very specific case where they're implementing 
higher-precision algorithms using lower-precision registers.  If 
you're going to all that trouble, you should know not to blindly 
run the same code at compile-time.



The only mention of "the last bit" is


This part is actually funny. Thanks for the laugh. :-)
I was going to say that your text search was too naive, but 
then I double-checked your claim and there are actually two 
mentions of "the last bit", and close by to the other mention, 
the paper says that "the first double a_0 is a double-precision 
approximation to the number a, accurate to

Re: Always false float comparisons

2016-05-20 Thread Timon Gehr via Digitalmars-d


On 20.05.2016 08:12, Walter Bright wrote:

On 5/19/2016 1:26 PM, Timon Gehr wrote:

Those two lines producing different results is unexpected, because you
are
explicitly saying that y is a double, and the first line also does
implicit
rounding (probably -- on all compilers and targets that will be
relevant in the
near future -- to double).

 > [...]

It's obviously bad language design on multiple levels.


Floating point semantics simply are not that simple,


This is about how the language defines floating point semantics. They 
can be that simple if the specification says so.



on any compiler,


gdc/ldc seem to get it right. David, Iain, is this the case?


language


Anybody can invent a language and IEEE 754 semantics are possible to 
support. Recent hardware does it.



or hardware. I recommend reading what the C/C++ Standards say
about it, and look at the plethora of compiler switches for
VC++/g++/clang++.
...


I don't really need to look at examples of how to do it wrong, but I 
will read whatever relevant link you can give me carefully. In D you 
don't need compiler switches to change the semantics of your code. We 
have more or less sane conditional compilation.



The people who design these things are not fools,


I don't know those people, but I would like to know their thoughts about 
the current discussion.



and there are good reasons for the way things are.
...


It is not unusual that there is some good reason for a design mistake.


Sorry, I don't agree.


If you say so. I would like to see an example that demonstrates that
the first
roundToDouble is required.


That's beside the point.


Absolutely not.


If there are spots in the program that require
rounding, what is wrong with having to specify it?


I explained what is wrong with it.

What is wrong with having to specify the precision that is /required for 
your code to run correctly/?



Why compromise speed & accuracy in the rest of the code


You don't compromise speed or accuracy if the rest of the code just 
correctly specifies the precision it should be run at.



that does not require it? (And yes,
rounding in certain spots can and does compromise both speed and accuracy.)
...


Because it is the only sane thing to do, as explained from many 
different angles in this thread. If it does compromise accuracy, that 
means the code is bad (it relies on precision enhancements that are not 
guaranteed to happen, hence it is hardly correct).





And if you are not? (I find the standard assumption that
counterexamples to
wrong statements are one-off special cases tiresome.


You might, but I've implemented a number of FP algorithms, and none of
them required rounding to a lower precision.


double x=..., z=...;
double y=roundToDouble(x+z);

Lower than what precision? It is not a 'lower' precision, it is the 
precision that was specified.



I've also seen many that
failed due to premature rounding.
...


Yes. Your results might be garbage more often if you run at too low 
precision. I and everyone else are very aware of this. I am not arguing 
against any of this. There seems to be a total communication failure here.





Creating useful programs in D shouldn't require knowing (or thinking
about) an
excess amount of D-specific implicit arcane details of the language
semantics.
Being easy to pick up and use is a major selling point for D.


As pointed out innumerable times here, the same issues are in C and C++
(with even more variability!).


As also pointed out here, it does not matter. I'm *not* trying to make 
the point that C and C++ do it better or anything like that. (But well, 
in practice, the actual commonly used C and C++ implementations probably 
do get it right by default.)




When designing an algorithm that you know
requires rounding in a certain place, you are already significantly
beyond implementing a naive algorithm, and it would be time to start
paying attention to how FP actually works.

I'm curious if you know of any language that meets your requirements.
(Java 1.0 did, but Sun was forced to abandon that.)


x86_64 assembly language.

Re: Always false float comparisons

2016-05-20 Thread Timon Gehr via Digitalmars-d


On 20.05.2016 11:14, Joakim wrote:

On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

On 19.05.2016 08:04, Joakim wrote:

On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

It's not just slightly worse, it can cut the number of useful bits in
half or more! It is not unusual, I have actually run into those
problems in the past, and it can break an algorithm that is in Phobos
today!


I wouldn't call that broken.  Looking at the hex output by replacing %f
with %A in writefln, it appears the only differences in all those
results is the last byte in the significand.


Argh...

// ...

void main(){
//double[] data=[1e16,1,-9e15];
import std.range;
double[] data=1e16~repeat(1.0,1).array~(-9e15);
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00


Better?

Obviously there is more structure in the data that I invent manually
than in a real test case where it would go wrong. The problems carry
over though.


I looked over your code a bit.  If I define sum and c as reals in
"kahanBroken" at runtime, this problem goes away.


Yes. That's absolutely obvious, and I have noted it before, but thanks. 
Maybe try to understand why this problem occurs in the first place.



Since that's what the
CTFE rule is actually doing, ie extending all floating-point to reals at
compile-time, I don't see what you're complaining about.  Try it, run
even your original naive summation algorithm through CTFE and it will
produce the result you want:

enum double[] ctData=[1e16,1,-9e15];
enum ctSum = sum(ctData);
writefln("%f", ctSum);
...


This example wasn't specifically about CTFE, but just imagine that only 
part of the computation is done at CTFE, all local variables are 
transferred to runtime and the computation is completed there.




As Don's talk pointed out,
all floating-point calculations will see loss of precision starting
there.
...



This is implicitly assuming a development model where the programmer
first writes down the computation as it would be correct in the real
number system and then naively replaces every operation by the
rounding equivalent and hopes for the best.


No, it is intrinsic to any floating-point calculation.
...


How do you even define accuracy if you don't specify an infinitely 
precise reference result?



It is a useful rule if that is what you're doing. One might be doing
something else. Consider the following paper for an example where the
last bit in the significant actually carries useful information for
many of the values used in the program.

http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


Did you link to the wrong paper? ;)


No. That paper uses multiple doubles per approximated real value to 
implement arithmetic that is more precise than using just plain doubles. 
If any bit in the first double is off, this is no better than using a 
single double.



I skimmed it and that paper
explicitly talks about error bounds all over the place.


It is enough to read the abstract to figure out what the problem is. 
This demonstrates a non-contrived case where CTFE using enhanced 
precision throughout can break your program. Compute something as a 
double-double at compile-time, and when it is transferred to runtime you 
lose all the nice extra precision, because bits in the middle of the 
(conceptual) mantissa are lost.




The only mention of "the last bit" is


This part is actually funny. Thanks for the laugh. :-)
I was going to say that your text search was too naive, but then I 
double-checked your claim and there are actually two mentions of "the 
last bit", and close by to the other mention, the paper says that "the 
first double a_0 is a double-precision approximation to the number a, 
accurate to almost half an ulp."



when they say they calculated their
constants in arbitrary precision before rounding them for runtime use,
which is ironically similar to what Walter suggested doing for D's CTFE
also.
...


Nothing "ironic" about that. It is sometimes a good idea and I can do 
this explicitly and make sure the rounding is done correctly, just like 
they did. Also, it is a lot more flexible if I can specify the exact way 
the computation is done and the result is rounded. 80 bits might not be 
enough anyway. There is no reason for the language to apply potentially 
incorrect "hand holding" here.


Again, please understand that my point is not that lower precision is 
better. My point is that doing the same thing in every context and 
allowing the programmer to specify what happens is better.



In this case, not increasing precision gets the more accurate result,
but other examples could be constructed that _heavily_ favor increasing
precision.

Re: Always false float comparisons

2016-05-20 Thread Joakim via Digitalmars-d


On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

On 19.05.2016 08:04, Joakim wrote:

On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
It's not just slightly worse, it can cut the number of useful 
bits in
half or more! It is not unusual, I have actually run into 
those
problems in the past, and it can break an algorithm that is 
in Phobos

today!


I wouldn't call that broken.  Looking at the hex output by 
replacing %f
with %A in writefln, it appears the only differences in all 
those

results is the last byte in the significand.


Argh...

// ...

void main(){
//double[] data=[1e16,1,-9e15];
import std.range;
double[] data=1e16~repeat(1.0,1).array~(-9e15);
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00


Better?

Obviously there is more structure in the data that I invent 
manually than in a real test case where it would go wrong. The 
problems carry over though.


I looked over your code a bit.  If I define sum and c as reals in 
"kahanBroken" at runtime, this problem goes away.  Since that's 
what the CTFE rule is actually doing, ie extending all 
floating-point to reals at compile-time, I don't see what you're 
complaining about.  Try it, run even your original naive 
summation algorithm through CTFE and it will produce the result 
you want:


enum double[] ctData=[1e16,1,-9e15];
enum ctSum = sum(ctData);
writefln("%f", ctSum);


As Don's talk pointed out,
all floating-point calculations will see loss of precision 
starting there.

...



This is implicitly assuming a development model where the 
programmer first writes down the computation as it would be 
correct in the real number system and then naively replaces 
every operation by the rounding equivalent and hopes for the 
best.


No, it is intrinsic to any floating-point calculation.

It is a useful rule if that is what you're doing. One might be 
doing something else. Consider the following paper for an 
example where the last bit in the significant actually carries 
useful information for many of the values used in the program.


http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


Did you link to the wrong paper? ;) I skimmed it and that paper 
explicitly talks about error bounds all over the place.  The only 
mention of "the last bit" is when they say they calculated their 
constants in arbitrary precision before rounding them for runtime 
use, which is ironically similar to what Walter suggested doing 
for D's CTFE also.


In this case, not increasing precision gets the more accurate 
result,
but other examples could be constructed that _heavily_ favor 
increasing

precision.


Sure. In such cases, you should use higher precision. What is 
the problem? This is already supported (the compiler is not 
allowed to use lower precision than requested).


I'm not the one with the problem, you're the one complaining.


In fact, almost any real-world, non-toy calculation would
favor it.

In any case, nobody should depend on the precision out that 
far being

accurate or "reliable."



IEEE floating point has well-defined behaviour and there is 
absolutely nothing wrong with code that delivers more accurate 
results just because it is actually aware of the actual 
semantics of the operations being carried out.


You just made the case for Walter doing what he did. :)

Re: Always false float comparisons

2016-05-20 Thread Walter Bright via Digitalmars-d


On 5/19/2016 12:49 AM, Max Samukha wrote:

People are trying to get across that, if they wanted to maximize accuracy, they
would request the most precise type explicitly. D has 'real' for that. This
thread has shown unequivocally that the semantics you are insisting on is bound
to cause endless confusion and complaints.


C++ programs do the same thing, and have for decades, and such behavior is 
allowed by the C++ Standard.


People have been confused by and complained about floating point behavior at 
least since I started programming 40 years ago.


I have not invented anything new here.

Re: Always false float comparisons

2016-05-20 Thread Walter Bright via Digitalmars-d


On 5/19/2016 1:26 PM, Timon Gehr wrote:

Those two lines producing different results is unexpected, because you are
explicitly saying that y is a double, and the first line also does implicit
rounding (probably -- on all compilers and targets that will be relevant in the
near future -- to double).

> [...]

It's obviously bad language design on multiple levels.


Floating point semantics simply are not that simple, on any compiler, language 
or hardware. I recommend reading what the C/C++ Standards say about it, and look 
at the plethora of compiler switches for VC++/g++/clang++.


The people who design these things are not fools, and there are good reasons for 
the way things are.


Sorry, I don't agree.


If you say so. I would like to see an example that demonstrates that the first
roundToDouble is required.


That's beside the point. If there are spots in the program that require 
rounding, what is wrong with having to specify it? Why compromise speed & 
accuracy in the rest of the code that does not require it? (And yes, rounding in 
certain spots can and does compromise both speed and accuracy.)




And if you are not? (I find the standard assumption that counterexamples to
wrong statements are one-off special cases tiresome.


You might, but I've implemented a number of FP algorithms, and none of them 
required rounding to a lower precision. I've also seen many that failed due to 
premature rounding.




Creating useful programs in D shouldn't require knowing (or thinking about) an
excess amount of D-specific implicit arcane details of the language semantics.
Being easy to pick up and use is a major selling point for D.


As pointed out innumerable times here, the same issues are in C and C++ (with 
even more variability!). When designing an algorithm that you know requires 
rounding in a certain place, you are already significantly beyond implementing a 
naive algorithm, and it would be time to start paying attention to how FP 
actually works.


I'm curious if you know of any language that meets your requirements. (Java 1.0 
did, but Sun was forced to abandon that.)

Re: Always false float comparisons

2016-05-19 Thread Walter Bright via Digitalmars-d


On 5/18/2016 4:30 AM, Ethan Watson wrote:

I appreciate that it sounds like I'm starting to stretch to hold to my point,
but I imagine we'd also be able to control such things with the compiler - or at
least know what flags it uses so that we can ensure consistent behaviour between
compilation and runtime.


All compilers I know of use "round to nearest" for constant folding, and that is 
not adjustable.

Re: Always false float comparisons

2016-05-19 Thread Timon Gehr via Digitalmars-d


On 19.05.2016 09:09, Walter Bright wrote:

On 5/18/2016 10:10 AM, Timon Gehr wrote:

double kahan(double[] arr){
double sum = 0.0;
double c = 0.0;
foreach(x;arr){
double y=x-c;

 double y = roundToDouble(x - c);


Those two lines producing different results is unexpected, because you 
are explicitly saying that y is a double, and the first line also does 
implicit rounding (probably -- on all compilers and targets that will be 
relevant in the near future -- to double). It's obviously bad language 
design on multiple levels.



double t=sum+y;

 double t = roundToDouble(sum + y);

c = (t-sum)-y;
sum=t;
}
return sum;
}


Those are the only two points in the program that depend on rounding.


If you say so. I would like to see an example that demonstrates that the 
first roundToDouble is required.


Also, in case that the compiler chooses to internally use higher 
precision for all local variables in that program, the 'roundToDouble's 
you have inserted will reduce precision in comparison to the case where 
they are not inserted, reducing magical precision enhancement that would 
otherwise happen. (I.e., assuming that some of the ideas are valid and 
it is in fact desirable to have variable precision enhancement depending 
on target, if I find a precision bug I can fix by adding roundToDouble, 
this introduces a potential regression on other targets. And as none of 
it was guaranteed to work in the first place, the resulting mess is all 
my fault.)



If you're implementing Kahan,


And if you are not? (I find the standard assumption that counterexamples 
to wrong statements are one-off special cases tiresome. This is not 
usually the case, even if you cannot construct other examples right away.)



you'd know that,


I would, but what often happens in practice is that people don't. For 
example because they wouldn't expect roundToDouble to be required 
anywhere in code that only uses doubles. They are right.



so there shouldn't be anything difficult about it.
...


https://issues.dlang.org/show_bug.cgi?id=13474

Creating useful programs in D shouldn't require knowing (or thinking 
about) an excess amount of D-specific implicit arcane details of the 
language semantics. Being easy to pick up and use is a major selling 
point for D.


I'm not convinced there is nothing difficult about it. It's certainly a 
generator of incidental complexity. Also, error prone is not the same as 
difficult. Almost everybody makes some mistakes occasionally.



There's no reason to make the rest of the program suffer inaccuracies.


Indeed there isn't. I'm not suggesting to allow using a precision lower 
than the one specified.


If you are talking about programs that "benefit" from automatic 
extension of precision: do you think their authors are aware that e.g. 
their 32 bit release builds with gdc/ldc are producing wrong results?

Re: Always false float comparisons

2016-05-19 Thread Timon Gehr via Digitalmars-d


On 18.05.2016 19:10, Timon Gehr wrote:

implementation-defined behaviour


Maybe that wasn't the right term (it's worse than that; I think the 
documentation of the implementation is not even required to tell you 
precisely what it does).

Re: Always false float comparisons

2016-05-19 Thread Timon Gehr via Digitalmars-d


On 19.05.2016 08:04, Joakim wrote:

On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

It's not just slightly worse, it can cut the number of useful bits in
half or more! It is not unusual, I have actually run into those
problems in the past, and it can break an algorithm that is in Phobos
today!


I wouldn't call that broken.  Looking at the hex output by replacing %f
with %A in writefln, it appears the only differences in all those
results is the last byte in the significand.


Argh...

// ...

void main(){
//double[] data=[1e16,1,-9e15];
import std.range;
double[] data=1e16~repeat(1.0,1).array~(-9e15);
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000.00
1001.00
1000.00

dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00


Better?

Obviously there is more structure in the data that I invent manually 
than in a real test case where it would go wrong. The problems carry 
over though.




As Don's talk pointed out,
all floating-point calculations will see loss of precision starting there.
...



This is implicitly assuming a development model where the programmer 
first writes down the computation as it would be correct in the real 
number system and then naively replaces every operation by the rounding 
equivalent and hopes for the best.


It is a useful rule if that is what you're doing. One might be doing 
something else. Consider the following paper for an example where the 
last bit in the significant actually carries useful information for many 
of the values used in the program.


http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


In this case, not increasing precision gets the more accurate result,
but other examples could be constructed that _heavily_ favor increasing
precision.


Sure. In such cases, you should use higher precision. What is the 
problem? This is already supported (the compiler is not allowed to use 
lower precision than requested).



In fact, almost any real-world, non-toy calculation would
favor it.

In any case, nobody should depend on the precision out that far being
accurate or "reliable."



IEEE floating point has well-defined behaviour and there is absolutely 
nothing wrong with code that delivers more accurate results just because 
it is actually aware of the actual semantics of the operations being 
carried out.

Re: Always false float comparisons


On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:

Computer scientists are no good if they don't know any science.


Even the computer scientists that does not know any science are 
infinitely better than those who refuse to read papers and debate 
on a rational level.


Blind D zealotry at its finest.

Re: Always false float comparisons

2016-05-19 Thread Joseph Rushton Wakeling via Digitalmars-d

On Thursday, 19 May 2016 at 12:00:33 UTC, Joseph Rushton Wakeling 
wrote:

On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:
The example he refers to is laughable because it also checks 
for equality.


With good reason, because it's intended to illustrate the point 
that two calculations that _look_ identical in code, that 
intuitively should produce identical results, actually don't, 
because of under-the-hood precision differences.


And that's my point too, that floating point will always subvert 
your expectations, particularly such simplistic notions of 
equality, which is why you should always use approxEqual or it's 
analogues.

Re: Always false float comparisons


On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:
The example he refers to is laughable because it also checks 
for equality.


With good reason, because it's intended to illustrate the point 
that two calculations that _look_ identical in code, that 
intuitively should produce identical results, actually don't, 
because of under-the-hood precision differences.

Re: Always false float comparisons

2016-05-19 Thread Ola Fosheim Grøstad via Digitalmars-d

On Thursday, 19 May 2016 at 11:00:31 UTC, Ola Fosheim Grøstad 
wrote:

On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:
On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
wrote:

On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
In this case, not increasing precision gets the more 
accurate result, but other examples could be constructed 
that _heavily_ favor increasing precision.  In fact, almost 
any real-world, non-toy calculation would favor it.


Please stop saying this. It is very wrong.


I will keep saying it because it is _not_ wrong.


Can you then please read this paper in it's entirety before 
continuing saying it. Because changing precision breaks 
properties of the semantics of IEEE floating point.


What Every Computer Scientist Should Know About Floating-Point 
Arithmetic


https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377

«Conventional wisdom maintains that extended-based systems must 
produce results that are at least as accurate, if not more 
accurate than those delivered on single/double systems, since 
the former always provide at least as much precision and often 
more than the latter. Trivial examples such as the C program 
above as well as more subtle programs based on the examples 
discussed below show that this wisdom is naive at best: some 
apparently portable programs, which are indeed portable across 
single/double systems, deliver incorrect results on 
extended-based systems precisely because the compiler and 
hardware conspire to occasionally provide more precision than 
the program expects.»


The example he refers to is laughable because it also checks for 
equality.


The notion that "error correction" can fix the inevitable 
degradation of accuracy with each floating-point calculation 
is just laughable.


Well, it is not laughable to computer scientists that accuracy 
depends on knowledge about precision and rounding... And I am a 
computer scientists, in case you have forgotten...


Computer scientists are no good if they don't know any science.

Re: Always false float comparisons


On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:
On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
wrote:

On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
In this case, not increasing precision gets the more accurate 
result, but other examples could be constructed that 
_heavily_ favor increasing precision.  In fact, almost any 
real-world, non-toy calculation would favor it.


Please stop saying this. It is very wrong.


I will keep saying it because it is _not_ wrong.


Can you then please read this paper in it's entirety before 
continuing saying it. Because changing precision breaks 
properties of the semantics of IEEE floating point.


What Every Computer Scientist Should Know About Floating-Point 
Arithmetic


https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377

«Conventional wisdom maintains that extended-based systems must 
produce results that are at least as accurate, if not more 
accurate than those delivered on single/double systems, since the 
former always provide at least as much precision and often more 
than the latter. Trivial examples such as the C program above as 
well as more subtle programs based on the examples discussed 
below show that this wisdom is naive at best: some apparently 
portable programs, which are indeed portable across single/double 
systems, deliver incorrect results on extended-based systems 
precisely because the compiler and hardware conspire to 
occasionally provide more precision than the program expects.»


And that is what _you_ need to stop saying: there's _nothing 
unpredictable_ about what D does.  You may find it unintuitive, 
but that's your problem.


No. It is not my problem as I would never use a system with this 
kind of semantics for anything numerical.


It is a problem for D. Not for me.


The notion that "error correction" can fix the inevitable 
degradation of accuracy with each floating-point calculation is 
just laughable.


Well, it is not laughable to computer scientists that accuracy 
depends on knowledge about precision and rounding... And I am a 
computer scientists, in case you have forgotten...

Re: Always false float comparisons

2016-05-19 Thread Ola Fosheim Grøstad via Digitalmars-d

On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
wrote:

On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
In this case, not increasing precision gets the more accurate 
result, but other examples could be constructed that _heavily_ 
favor increasing precision.  In fact, almost any real-world, 
non-toy calculation would favor it.


Please stop saying this. It is very wrong.


I will keep saying it because it is _not_ wrong.

Algorithms that need higher accuracy need error correction 
mechanisms, not unpredictable precision and rounding. 
Unpredictable precision and rounding makes adding error 
correction difficult so it does not improve accuracy, it harms 
accuracy when you need it.


And that is what _you_ need to stop saying: there's _nothing 
unpredictable_ about what D does.  You may find it unintuitive, 
but that's your problem.  The notion that "error correction" can 
fix the inevitable degradation of accuracy with each 
floating-point calculation is just laughable.

Re: Always false float comparisons


On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
In this case, not increasing precision gets the more accurate 
result, but other examples could be constructed that _heavily_ 
favor increasing precision.  In fact, almost any real-world, 
non-toy calculation would favor it.


Please stop saying this. It is very wrong.

Algorithms that need higher accuracy need error correction 
mechanisms, not unpredictable precision and rounding. 
Unpredictable precision and rounding makes adding error 
correction difficult so it does not improve accuracy, it harms 
accuracy when you need it.

Re: Always false float comparisons

2016-05-19 Thread Ola Fosheim Grøstad via Digitalmars-d


On Wednesday, 18 May 2016 at 22:16:44 UTC, jmh530 wrote:
On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
Wakeling wrote:

On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
I do not understand the tolerance for bad results in 
scientific, engineering, medical, or finance applications.


I don't think anyone has suggested tolerance for bad results 
in any of those applications.




I don't think its about tolerance for bad results, so much as 
the ability to make the trade-off between speed and precision 
when you need to.


It isn't only about speed. It is about correctness as well. 
Compilers should not change the outcome (including precision and 
rounding) unless the programmer has explicitly requested it, and 
if you do the effects should be well documented so that the 
effect is clear to the programmer. This is a well known and 
universally accepted principle in compiler design.


Take for instance the documentation page for a professional level 
compiler targeting embedded programming:


http://processors.wiki.ti.com/index.php/Floating_Point_Optimization

It gives the programmer explicit control over what kind of 
deviations the compiler can create.


If you look at Intel's compiler you'll see that they even turn 
off fused-multiply-add in strict mode because it skips a single 
rounding between the multiply and the add.


https://software.intel.com/sites/default/files/article/326703/fp-control-2012-08.pdf

Some languages allow constant folding of literals in 
_expressions_ to use infinite precision, but once you have bound 
it to a variable it should use the same precision, and you always 
have to use the same rounding mode. This means that if you should 
check the rounding mode before using precomputed values.


If you cannot emulate the computation then you can simply run the 
computations prior to entering main and put the "constant 
computations" in global variables.


That said, compilers may have some problematic settings enabled 
by default in order to look good in benchmarks/test suites.


In order to be IEEE compliant you cannot even optimize "0.0 - x" 
to "-x", you also cannot optimize "x - 0.0" to "x". Such issues 
make compiler vendors provide many different floating point 
options as command line flags.

Re: Always false float comparisons

2016-05-19 Thread Max Samukha via Digitalmars-d


On Thursday, 19 May 2016 at 07:09:30 UTC, Walter Bright wrote:

On 5/18/2016 10:10 AM, Timon Gehr wrote:

double kahan(double[] arr){
double sum = 0.0;
double c = 0.0;
foreach(x;arr){
double y=x-c;

  double y = roundToDouble(x - c);

double t=sum+y;

  double t = roundToDouble(sum + y);

c = (t-sum)-y;
sum=t;
}
return sum;
}


Those are the only two points in the program that depend on 
rounding. If you're implementing Kahan, you'd know that, so 
there shouldn't be anything difficult about it.


There's no reason to make the rest of the program suffer 
inaccuracies.


People are trying to get across that, if they wanted to maximize 
accuracy, they would request the most precise type explicitly. D 
has 'real' for that. This thread has shown unequivocally that the 
semantics you are insisting on is bound to cause endless 
confusion and complaints.

Re: Always false float comparisons

2016-05-19 Thread Walter Bright via Digitalmars-d


On 5/18/2016 10:10 AM, Timon Gehr wrote:

double kahan(double[] arr){
double sum = 0.0;
double c = 0.0;
foreach(x;arr){
double y=x-c;

  double y = roundToDouble(x - c);

double t=sum+y;

  double t = roundToDouble(sum + y);

c = (t-sum)-y;
sum=t;
}
return sum;
}


Those are the only two points in the program that depend on rounding. If you're 
implementing Kahan, you'd know that, so there shouldn't be anything difficult 
about it.


There's no reason to make the rest of the program suffer inaccuracies.

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
It's not just slightly worse, it can cut the number of useful 
bits in half or more! It is not unusual, I have actually run 
into those problems in the past, and it can break an algorithm 
that is in Phobos today!


I wouldn't call that broken.  Looking at the hex output by 
replacing %f with %A in writefln, it appears the only differences 
in all those results is the last byte in the significand.  As 
Don's talk pointed out, all floating-point calculations will see 
loss of precision starting there.


In this case, not increasing precision gets the more accurate 
result, but other examples could be constructed that _heavily_ 
favor increasing precision.  In fact, almost any real-world, 
non-toy calculation would favor it.


In any case, nobody should depend on the precision out that far 
being accurate or "reliable."

Re: Always false float comparisons

2016-05-18 Thread H. S. Teoh via Digitalmars-d

On Wed, May 18, 2016 at 04:28:13PM -0700, Walter Bright via Digitalmars-d wrote:
> On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:
> > On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:
> > > Now try the square root of 2. Or pi, e, etc. The irrational
> > > numbers are, by definition, not representable as a ratio.
> > 
> > Continued fraction? :-)
> 
> Somehow I don't think gcc is using Mathematica for constant folding.

http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/drl_ieee01.pdf

Side-note: continued fractions of quadratic irrationals (a + b*sqrt(c))
are periodic, so it's possible to do exact arithmetic with them using
only finite storage.


T

-- 
All problems are easy in retrospect.

Re: Always false float comparisons

2016-05-18 Thread H. S. Teoh via Digitalmars-d

On Wed, May 18, 2016 at 04:09:28PM -0700, Walter Bright via Digitalmars-d wrote:
[...]
> Now try the square root of 2. Or pi, e, etc. The irrational numbers
> are, by definition, not representable as a ratio.

This is somewhat tangential, but in certain applications it is perfectly
possible to represent certain irrationals exactly. For example, if
you're working with quadratic fractions of the form a + b*sqrt(r) where
r is fixed, you can exactly represent all such numbers as a pair of
rational coefficients. These numbers are closed under addition,
subtraction, multiplication, and division, so you have a nice closed
system that can handle numbers that contain square roots.

It is possible to have multiple square roots (e.g., a + b*sqrt(r) +
c*sqrt(s) + d*sqrt(r*s)), but the tuple gets exponentially long with
each different root, so it quickly becomes unwieldy (not to mention
slow).

Similar techniques can be used for exactly representing roots of cubic
equations (including cube roots) -- a single cubic root can be
represented as a 3-tuple -- or higher. Again, these are closed under +,
*, -, /, so you can do quite a lot of interesting things with them
without sacrificing exact arithmetic. Though again, things quickly
become unwieldy.  Higher order roots are also possible, though of
questionable value since storage and performance costs quickly become
too high.

Transcendentals like pi or e are out of reach by this method, though.
You'd need a variable-length vector to represent numbers that contain pi
or e as a factor, because their powers never become integral, so you
potentially need to represent an arbitrary power of pi in order to
obtain closure under + and *.

Of course, with increasing sophistication the computation and storage
costs increase accordingly, but the point is that if your application is
known to only need a certain subset of irrationals, it may well be
possible to represent them exactly within reasonable costs. (There's
also RealLib, that can exactly represent *all* computable reals, but may
not be feasible depending on what your application does.)

T

-- 
Let's eat some disquits while we format the biskettes.

Re: Always false float comparisons

2016-05-18 Thread Joseph Rushton Wakeling via Digitalmars-d


On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:

On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:

Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by
definition, not representable as a ratio.


Continued fraction? :-)


Somehow I don't think gcc is using Mathematica for constant folding.

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:
Now try the square root of 2. Or pi, e, etc. The irrational 
numbers are, by definition, not representable as a ratio.


Continued fraction? :-)

Re: Always false float comparisons


On 5/18/2016 1:22 PM, deadalnix wrote:

On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:

On 5/18/2016 4:48 AM, deadalnix wrote:

Typo: arbitrary precision FP. Meaning some soft float that grows as big as
necessary to not lose precision à la BitInt but for floats.


0.10 is not representable in a binary format regardless of precision.


You should ask the gcc guys how they do it, but you can surely represent this as
a fraction,


Right.


so I see no major blocker.


Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by 
definition, not representable as a ratio.

Re: Always false float comparisons

2016-05-18 Thread Joseph Rushton Wakeling via Digitalmars-d


On Wednesday, 18 May 2016 at 22:06:43 UTC, Era Scarecrow wrote:

On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:
Can you give me a source for this, or at least the name of the 
relevant op code? (I'm new to x86 assembly.)


http://www.mathemainzel.info/files/x86asmref.html#mul

http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html


Thanks.


[...]
Downloading the 64 intel manual on opcodes says the same thing, 
only the registers become RDX:RAX with 64bit instructions.


Quadword RAX r/m64 RDX:RAX


I will look into adding intrinsics for full-width multiply and 
combined division-modulus.

Re: Always false float comparisons

2016-05-18 Thread jmh530 via Digitalmars-d

On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
Wakeling wrote:

On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
I do not understand the tolerance for bad results in 
scientific, engineering, medical, or finance applications.


I don't think anyone has suggested tolerance for bad results in 
any of those applications.




I don't think its about tolerance for bad results, so much as the 
ability to make the trade-off between speed and precision when 
you need to.


Just thinking of finance: a market maker has to provide quotes on 
potentially thousands of instruments in real-time. This might 
involve some heavy calculations for options pricing. When dealing 
with real-time tick data (the highest frequency of financial 
data), sometimes you take shortcuts that you wouldn't be willing 
to do if you were working with lower frequency data. It's not 
that you don't care about precision. It's just that sometimes 
it's more important to be fast than accurate.


I'm not a market maker and don't work with high frequency data. I 
usually look at low enough frequency data so that I actually do 
generally care more about accurate results than speed. 
Nevertheless, sometimes with hefty simulations that take several 
hours or days to run, I might be willing to take some short cuts 
to get a general idea of the results. Then, when I implement the 
strategy, I might do something different.

Re: Always false float comparisons

2016-05-18 Thread Era Scarecrow via Digitalmars-d

On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:

On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:

On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
I agree that intrinsics for this would be nice. I doubt that
any current D platform is actually computing the full 128 bit
result for every 64 bit multiply though - that would waste
both power and performance, for most programs.

Except the 128 result is _already_ there for 0 cost (at least
for x86 instructions that I'm aware).

Can you give me a source for this, or at least the name of the
relevant op code? (I'm new to x86 assembly.)

http://www.mathemainzel.info/files/x86asmref.html#mul

http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html

There's div, idiv, mul, and imul which follow this exact
pattern. Although the instruction mentioned in the following
pages is meant for 32bit or less, the pattern used is no
different.

(mathemainzel.info)
Usage
MUL src
Modifies flags
CF OF (AF,PF,SF,ZF undefined)

Unsigned multiply of the accumulator by the source. If "src" is a
byte value, then AL is used as the other multiplicand and the
result is placed in AX. If "src" is a word value, then AX is
multiplied by "src" and DX:AX receives the result. If "src" is a
double word value, then EAX is multiplied by "src" and EDX:EAX
receives the result. The 386+ uses an early out algorithm which
makes multiplying any size value in EAX as fast as in the 8 or 16
bit registers.

(intel.com)
Downloading the 64 intel manual on opcodes says the same thing,
only the registers become RDX:RAX with 64bit instructions.

Quadword RAX r/m64 RDX:RAX

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
I do not understand the tolerance for bad results in 
scientific, engineering, medical, or finance applications.


I don't think anyone has suggested tolerance for bad results in 
any of those applications.


What _has_ been argued for is that in order to _prevent_ bad 
results it's necessary for the programmer to have control and 
clarity over the choice of precision as much as possible.


If I'm writing a numerical simulation or calculation using 
insufficient floating-point precision, I don't _want_ to be saved 
by under-the-hood precision increases -- I would like it to break 
because then I will be forced to improve either the 
floating-point precision or the choice of algorithm (or both).


To be clear: the fact that D makes it a priority to offer me the 
highest possible floating-point precision is awesome.  But 
precision is not the only factor in generating accurate 
scientific results.

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:

On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
I agree that intrinsics for this would be nice. I doubt that 
any current D platform is actually computing the full 128 bit 
result for every 64 bit multiply though - that would waste 
both power and performance, for most programs.


 Except the 128 result is _already_ there for 0 cost (at least 
for x86 instructions that I'm aware).


Can you give me a source for this, or at least the name of the 
relevant op code? (I'm new to x86 assembly.)


There's bound to be enough cases (say pseudo random number 
generation, encryption, or numerical processing above 64bits) 
I'd like access to it supported by the language and not having 
to inject instructions using the asm command.


Of course it would be useful to have in the language; I wasn't 
disputing that. I'd like to have as much support for 128-bit 
integers in the language as possible. Among other things, this 
would greatly simplify getting 128-bit floating-point working.


I'm just surprised that the CPU would really calculate the upper 
64 bits of a multiply without being explicitly asked to.

Re: Always false float comparisons


On 5/18/2016 4:27 AM, Manu via Digitalmars-d wrote:

The comparison was a 24bit fpu doing runtime work but where some
constant input data was calculated with a separate 32bit fpu. The
particulars were not ever intended to be relevant to the conversation,
except the fact that 2 differently precisioned float units were
producing output that then had to reconcile.

The analogy was to CTFE doing all its work at 80bits, and then the
processor doing work with the types explicitly stated by the
programmer; a runtime calculation compared against the same compile
time calculate is likely to be quite radically different. I don't care
about the precision, I just care that they're really different.
Ideally, CTFE would produce a result that is as similar to the runtime
result as reasonably possible, and I expect using the stated types to
do the calculations would get much much closer.
I don't know if a couple of least-significant bits of difference would
have caused problems for us, I suspect not, but I know that doing math
at radically different precisions (ie, 32bits vs 80bits) does lead to
radically different results, not just a couple of bits. That is my
concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit
fpu's.



"radically different results" means you were getting catastrophic loss of 
precision in the runtime results, which is just what doing the CTFE at a higher 
precision is attempting to avoid (and apparently successfully).


I get that the game did not care about the value produced, even if it was total 
garbage. I suspect that games are the only category of FP apps where there is no 
concern for the values computed. I do not understand the tolerance for bad 
results in scientific, engineering, medical, or finance applications.


The only way to *reliably* get the same results at compile time as at runtime, 
regardless of what technology is put into the language/compiler, is to actually 
run the code on the target hardware, save the results, and insert those values 
into the code. It isn't that hard to do, and will save you from endless ghosts 
popping up that waste your debugging time.

Re: Always false float comparisons

2016-05-18 Thread jmh530 via Digitalmars-d


On Wednesday, 18 May 2016 at 19:30:12 UTC, deadalnix wrote:


I'm confused as to why the compiler would be using soft floats 
instead of hard floats.


Cross compilation.


Ah, looking back on the discussion, I see the comments about 
cross compilation and soft floats. Making more sense now...


So if compiling on x86 for x86, you could just use hard floats, 
but if compiling on x86 for some other system, then use soft 
floats to mimic what the result would be as if you had compiled 
on that system. Correct?


But what if you are compiling for a system whose float behavior 
matches the system you're compiling on? So for instance, suppose 
you are only using 32bit floats and not allowing anything fancy 
like 80bit intermediate calculations. And you're compiling for a 
system that treats floats the same way. Then, you could 
theoretically use hard floats in the compiler and the results 
would be the same.

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:

On 5/18/2016 4:48 AM, deadalnix wrote:
Typo: arbitrary precision FP. Meaning some soft float that 
grows as big as

necessary to not lose precision à la BitInt but for floats.


0.10 is not representable in a binary format regardless of 
precision.


You should ask the gcc guys how they do it, but you can surely 
represent this as a fraction, so I see no major blocker.

Re: Always false float comparisons


On 5/18/2016 4:48 AM, deadalnix wrote:

Typo: arbitrary precision FP. Meaning some soft float that grows as big as
necessary to not lose precision à la BitInt but for floats.


0.10 is not representable in a binary format regardless of precision.

Re: Always false float comparisons

2016-05-18 Thread Era Scarecrow via Digitalmars-d


On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
I agree that intrinsics for this would be nice. I doubt that 
any current D platform is actually computing the full 128 bit 
result for every 64 bit multiply though - that would waste both 
power and performance, for most programs.


 Except the 128 result is _already_ there for 0 cost (at least 
for x86 instructions that I'm aware). There's bound to be enough 
cases (say pseudo random number generation, encryption, or 
numerical processing above 64bits) I'd like access to it 
supported by the language and not having to inject instructions 
using the asm command.


 This is also the same with the division where the number could 
be a 128bit number divided by a 64bit divisor. We could get the 
main benefit of the 128bit cent with very few instructions if we 
simply guarantee one of the two arguments smaller than 65 bits 
(although the dividend and remainder both need to be 64 bits or 
smaller)

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 11:46:37 UTC, Era Scarecrow wrote:

On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:

https://code.dlang.org/packages/checkedint
https://dlang.org/phobos/core_checkedint.html


 Glancing at the checkedInt I really don't see it as being the 
same as what I'm talking about. Overflow/carry for add perhaps, 
but unless it breaks down to a single instruction for the 
compiler to determine if it needs to do something, I see it as 
a failure (at best, a workaround).


That's just my thoughts. CheckedInt simply _doesn't_ cover what 
I was talking about.


The functions in druntime's `core.checkedint` are intrinsics that 
map directly to the hardware overflow/carry instructions.


The DUB package I linked provides various wrapper functions and 
data structures to make it easier to use the `core.checkedint` 
intrinsics (among other things). The performance cost of the 
wrappers is low with proper inlining, which GDC and LDC are able 
to provide. (DMD is another story...)


Obtaining the modulus for 0 cost/instructions after doing a 
division which is in the hardware's opcode side effects (unless 
the compiler recognizes the pattern and offers it as an 
optimization), or having the full result of a multiply on hand 
(that exceeds it's built-in size, long.max*long.max = 128bit 
result, which the hardware hands to you if you check the 
register it stores the other half of the result in).


I agree that intrinsics for this would be nice. I doubt that any 
current D platform is actually computing the full 128 bit result 
for every 64 bit multiply though - that would waste both power 
and performance, for most programs.

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d


On Wednesday, 18 May 2016 at 19:20:20 UTC, jmh530 wrote:

On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:


Do you have a link explaining GCC actually uses such a soft 
float?


I'm confused as to why the compiler would be using soft floats 
instead of hard floats.


Cross compilation.

Re: Always false float comparisons

2016-05-18 Thread jmh530 via Digitalmars-d


On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:


Do you have a link explaining GCC actually uses such a soft 
float?


I'm confused as to why the compiler would be using soft floats 
instead of hard floats.

Re: Always false float comparisons

2016-05-18 Thread Timon Gehr via Digitalmars-d


On 17.05.2016 21:31, deadalnix wrote:

On Tuesday, 17 May 2016 at 18:08:47 UTC, Timon Gehr wrote:

Right. Hence, the 80-bit CTFE results have to be converted to the
final precision at some point in order to commence the runtime
computation. This means that additional rounding happens, which was
not present in the original program. The additional total roundoff
error this introduces can exceed the roundoff error you would have
suffered by using the lower precision in the first place, sometimes
completely defeating precision-enhancing improvements to an algorithm.



WAT ? Is that really possible ?



Yes, I'm sorry, but this can and does happen.
Consider http://forum.dlang.org/post/nhi7m4$css$1...@digitalmars.com

You can build similar examples involving only CTFE. Refer to 
http://forum.dlang.org/post/nhi9gh$fa4$1...@digitalmars.com for an 
explanation of one case where this can happen. (I had actually written 
that post three days ago, and assumed that it had been posted to the 
newsgroup, but something went wrong, apparently.)

Re: Always false float comparisons

2016-05-18 Thread Timon Gehr via Digitalmars-d

I had written and sent this message three days ago, but it seemingly 
never showed up on the newsgroup. I'm sorry if it seemed that I didn't 
explain myself, I was operating under the assumption that this message 
had been made available to you.

On 14.05.2016 03:26, Walter Bright wrote:
> On 5/13/2016 5:49 PM, Timon Gehr wrote:
>> Nonsense. That might be true for your use cases. Others might actually
>> depend on
>> IEE 754 semantics in non-trivial ways. Higher precision for
>> temporaries does not
>> imply higher accuracy for the overall computation.
>
> Of course it implies it.
> ...

No, see below.

> An anecdote: a colleague of mine was once doing a chained calculation.
> At every step, he rounded to 2 digits of precision after the decimal
> point, because 2 digits of precision was enough for anybody. I carried
> out the same calculation to the max precision of the calculator (10
> digits). He simply could not understand why his result was off by a
> factor of 2, which was a couple hundred times his individual roundoff
> error.
> ...

Now assume that colleague of your was doing that chained calculation, 
and his calculator magically added the additional digits behind his back 
(it can do this by caching the last full-precision value for each number 
prefix). He wouldn't even notice that his rounding strategy does not 
work. Sometime later he might then use a calculator that does not do the 
magical enhancing.

>
>> E.g., correctness of double-double arithmetic is crucially dependent
>> on correct
>> rounding semantics for double:
>> 
https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic

>>
>
> Double-double has its own peculiar issues, and is not relevant to this
> discussion.
> ...

It is relevant to this discussion insofar that it can occur in 
algorithms that use double-precision floating-point arithmetic. It 
illustrates a potential issue with implicit enhancement of precision. 
For double-double, there are two values of type double that together 
represent a higher-precision value. (One of them has a shifted exponent, 
such that their mantissa bits do not overlap.)

You have mantissas like:

|--double1--| |--double2|

Now assume that the compiler instead uses extended precision, what you 
get is something we might call extended-extended of the form:

|-extended1-| |-extended2---|

Now those values are written back into 64-bit double storage, now 
observe which part of the double-double mantissa is lost:

|-extended1---xx| |-extended2-xx|

|
v

|--double1--| |--double2|

The middle part of the mantissa is thrown away, and we are left with 
single double-precision plus some noise. Implicitly using extended 
precision for some parts of the computation approximately cuts the 
number of accurate mantissa bits in half. I don't want to have to deal 
with this. Just give me what I ask for.

>
>> Also, it seems to me that for e.g.
>> https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
>> the result can actually be made less precise by adding casts to higher
>> precision
>> and truncations back to lower precision at appropriate places in the
>> code.
>
> I don't see any support for your claim there.
> 

It's using the same trick that double-double does. The above reasoning 
should apply.

>
>> And even if higher precision helps, what good is a "precision-boost"
>> that e.g.
>> disappears on 64-bit builds and then creates inconsistent results?
>
> That's why I was thinking of putting in 128 bit floats for the compiler
> internals.
> ...

Runtime should do the same as CTFE. Are you suggesting we use 128-bit 
soft-floats at run time for all float types?

>
>> Sometimes reproducibility/predictability is more important than maybe
>> making
>> fewer rounding errors sometimes. This includes reproducibility between
>> CTFE and
>> runtime.
>
> A more accurate answer should never cause your algorithm to fail.

It's not more accurate, just more precise, and it is only for some 
temporary computations, and you don't necessarily know which. The way 
the new roundoff errors propagate is chaotic, and might not be what the 
code anticipated.

> It's like putting better parts in your car causing the car to fail.
> ...

It's like (possibly repeatedly) interchanging "better" parts and "worse" 
parts while the engine is still running.

Anyway, it should be obvious that this kind of reasoning by analogy does 
not lead anywhere.

>
>> Just actually comply to the IEEE floating point standard when using 
their

>> terminology. There are algorithms that are designed for it and that
>> might stop
>> working if the language does not comply.
>
> Conjecture.

I have given a concrete example.

> I've written FP algorithms (from Cody+Waite, for example),
> and none of them degraded when using more precision.
> ...

For the entire computation or some random

Re: Always false float comparisons

2016-05-18 Thread Timon Gehr via Digitalmars-d


On 17.05.2016 23:07, Walter Bright wrote:

On 5/17/2016 11:08 AM, Timon Gehr wrote:

Right. Hence, the 80-bit CTFE results have to be converted to the final
precision at some point in order to commence the runtime computation.
This means
that additional rounding happens, which was not present in the
original program.
The additional total roundoff error this introduces can exceed the
roundoff
error you would have suffered by using the lower precision in the
first place,
sometimes completely defeating precision-enhancing improvements to an
algorithm.


I'd like to see an example of double rounding "completely defeating" an
algorithm,


I have given this example, and I have explained it.

However, let me provide one of the examples I have given before, in a 
more concrete fashion. Unfortunately, there is no way to use "standard" 
D to illustrate the problem, as there is no way to write an 
implementation that is guaranteed not to be broken, so let us assume 
hypothetically for now that we are using D', where all computations are 
performed at the specified precision.


I'm copying the code from:
https://en.wikipedia.org/wiki/Kahan_summation_algorithm


$ cat kahanDemo.d

module kahanDemo;

double sum(double[] arr){
double s=0.0;
foreach(x;arr) s+=x;
return s;
}

double kahan(double[] arr){
double sum = 0.0;
double c = 0.0;
foreach(x;arr){
double y=x-c;
double t=sum+y;
c = (t-sum)-y;
sum=t;
}
return sum;
}

double kahanBroken(double[] arr){
double sum = 0;
double c= 0.0;
foreach(x;arr){
real y=x-c;
real t=sum+y;
c = (t-sum)-y;
sum=t;
}
return sum;
}

void main(){
double[] data=[1e16,1,-9e15];
import std.stdio;
writefln("%f",sum(data)); // baseline
writefln("%f",kahan(data)); // kahan
writefln("%f",kahanBroken(data)); // broken kahan
}

In D, the compiler is in principle allowed to transform the non-broken 
version to the broken version. (And maybe, it will soon be allowed to 
transform the baseline version to the Kahan version. Who knows.)


Now let's see what DMD does:

$ dmd --version
DMD64 D Compiler v2.071.0
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright

$ dmd -m64 -run kahanDemo.d
1000.00
1001.00
1000.00

Nice, this is what I expect.

$ dmd -m64 -O -run kahanDemo.d
1000.00
1001.00
1000.00

Still great.

$ dmd -m32 -run kahanDemo.d
1000.00
1001.00
1000.00

Liking this.

$ dmd -m32 -O -run kahanDemo.d
1000.00
1000.00
1000.00

Screw you, DMD!

And suddenly, I need to compile and test my code with all combinations 
of compiler flags, and even then I am not sure the compiler is not 
intentionally screwing me over. How is this remotely acceptable?



and why an unusual case of producing a slightly worse


It's not just slightly worse, it can cut the number of useful bits in 
half or more! It is not unusual, I have actually run into those problems 
in the past, and it can break an algorithm that is in Phobos today!



answer trumps the usual case of producing better answers.
...


The 'usual case of producing better answers' /is not actually 
desirable/, because the compiler does not guarantee that it happens all 
the time! I don't want my code to rely on something to happen that might 
not always happen. I want to be sure that my code is correct. I cannot 
conveniently do so if you don't tell me in advance what it does, and/or 
if the behaviour has a lot of abstraction-breaking special cases.





There are other reasons why I think that this kind of
implementation-defined
behaviour is a terribly bad idea, eg.:

- it breaks common assumptions about code, especially how it behaves
under
seemingly innocuous refactorings, or with a different set of compiler
flags.


As pointed out, this already happens with just about every language. It
happens with all C/C++ compilers I'm aware of.


I'm not claiming those languages don't have broken floating point 
semantics. I have sometimes been using inline assembler in C++ to get 
the results I want. It's painful and unnecessary.



It happens as the default behavior of the x86.


I know. I don't care. It is a stupid idea. See above.


And as pointed out, refactoring (x+y)+z to x+(y+z)
often produces different results, and surprises a lot of people.
...


As far as I can tell, you are saying: "You think A is bad, but A is 
similar to B, and B is bad, but B is hard to fix, hence A is actually 
good." I disagree.



For floating point, '+' means: "add precisely, then round". It is indeed 
potentially surprising and not very helpful for generic code that 
floating point types and integral types use the same syntax for 
conceptually different things (i.e., the language commits operator 
overloading abuse), but we are stuck with that

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 15:42:56 UTC, Joakim wrote:
I see, so the fact that both the C++ and D specs say the same 
thing doesn't matter, and the fact that D also has the const 
float in your example as single-precision at runtime, contrary 
to your claims, none of that matters.


D doesn't even have a spec, so how can they possibly say the same 
thing?


However, quoting the Wikipedia page on IEEE floats:

«The IEEE 754-1985 allowed many variations in implementations 
(such as the encoding of some values and the detection of certain 
exceptions). IEEE 754-2008 has strengthened up many of these, but 
a few variations still remain (especially for binary formats). 
The reproducibility clause recommends that language standards 
should provide a means to write reproducible programs (i.e., 
programs that will produce the same result in all implementations 
of a language), and describes what needs to be done to achieve 
reproducible results.»


That's the map people who care about floating point follow.



No sane DSP programmer would write that like you did.


What kind of insult is that?  I've read lots of DSP code written 
by others.  I know what kind of programming it entails.


In fact, what I have described here are techniques picked up from 
state-of-the-art DSP code written by top-the-of-line DSP 
programmers.



Since the vast majority of tests will never use such 
compile-test constants, your opinion is not only wrong but 
irrelevant.


Oh... Not only am I wrong, but my opinion is irrelevant. Well, 
with this attitude D will remain irrelevant as well.


For good reasons.



Then don't use differently defined constants in different places


I don't, and I didn't. DMD did it.

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d


On Wednesday, 18 May 2016 at 15:30:42 UTC, Matthias Bentrup wrote:
On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
wrote:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

And yes, half-precision is only 10 bits.


Actually, it turns out that the mantissa is 11 bits. So it 
clearly plays louder than other floats. ;-)


The mantissa is 10 bits, but it has 11 bit precision, just as 
the float type has a 23 bit mantissa and 24 bit precision. 
AFAIK the only float format that stores the always-one-bit of 
the mantissa is the x87 80 bit format.


It turns out both 10bits and 11bits are "right" and that one 
should qualify it with «with/without the hidden bit», except that 
it shouldn't be called the «mantissa», but the «significand»:


https://en.wikipedia.org/wiki/Significand

Thanks, always fun with learning  ;-)

Re: Always false float comparisons

2016-05-18 Thread Joakim via Digitalmars-d

On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:

Welcome to the wonderful world of C++! :D

More seriously, it is well-defined for that implementation, 
you did not raise the issue of the spec till now.  In fact, 
you seemed not to care what the specs say.


Eh? All C/C++ compilers I have ever used coerce to single 
precision upon binding.


I care about what the spec says, why it says it and how it is 
used when targetting the specific hardware.


Or more generally, when writing libraries I target the language 
spec, when writing applications I target the platform (language 
+ compiler + hardware).


I see, so the fact that both the C++ and D specs say the same 
thing doesn't matter, and the fact that D also has the const 
float in your example as single-precision at runtime, contrary to 
your claims, none of that matters.


No, it has nothing to do with language semantics and 
everything to do with bad numerical programming.


No. You can rant all you want about bad numerical programming. 
But by your definition all DSP programmers are bad at numerics. 
Which _obviously_ is not true.


That is grasping for straws.

There is NO excuse for preventing programmers from writing 
reliable performant code that solves the problem at hand to the 
accuracy that is required by the application.


No sane DSP programmer would write that like you did.

Because floating-point is itself fuzzy, in so many different 
ways.  You are depending on exactly repeatable results with a 
numerical type that wasn't meant for it.


Floating point is not fuzzy. It is basically a random sample on 
an interval of potential solutions with a range that increase 
with each computation. Unfortunately, you have to use interval 
arithmetics to get that interval, as in regular floating point 
you loose the information about the interval. It is an 
approximation. IEEE 754-2008 makes it possible to get that 
interval, btw.


Or in the vernacular, fuzzy. :) Of course, this is true of all 
non-integer calculation.


Fuzzy is different. Fuzzy means that the range itself is the 
value. It does not represent an approximation. ;-)


It's all approximations.

You keep saying this: where did anyone mention unit tests not 
running with the same precision till you just brought it up 
out of nowhere?  The only prior mention was that compile-time 
calculation of constants that are then checked for bit-exact 
equality in the tests might have problems, but that's 
certainly not all tests and I've repeatedly pointed out you 
should never be checking for bit-exact equality.


I have stated time and time again that it is completely 
unacceptable if there is even a remote chance for anything in 
the unit test to evaluate at a higher precision than in the 
production code.


That is not a unit test, it is a sham.


Since the vast majority of tests will never use such compile-test 
constants, your opinion is not only wrong but irrelevant.


The point is that what you consider reliable will be less 
accurate, sometimes much less.


Define accurate. Accurate has no meaning without a desired 
outcome to reference.


I care about 4 oscillators having the same phase. THAT MEANS: 
they all have to use the exact same constants.


If not, they _will_ drift and phase cancellation _will_ occur.

I care about adding and removing the same constant. If not, 
(more) DC offsets will build up.


It is all about getting below the right tolerance threshold 
while staying real time. You can compensate by increasing the 
control rate (reduce the number of interpolated values).


Then don't use differently defined constants in different places 
and don't use floating point, simple.


The point is that there are _always_ bounds, so you can never 
check for the same value.  Almost any guessed bounds will be 
better than incorrectly checking for the bit-exact value.


No. Guessed bounds are not better. I have never said that 
anyone should _incorrectly_ do anything. I have said that they 
should _correctly_ understand what they are doing and that 
guessing bounds just leads to a worse problem:


Programs that intermittently fail in spectacular ways that are 
very difficult to debug.


You cannot even compute the bounds if the compiler can use 
higher precision. IEEE754-2008 makes it possible to accurately 
compute bounds.


Not supporting that is _very_ bad.


If you're comparing bit-exact equality, you're not using _any_ 
error bounds: that's the worst possible choice.


should always be thought about.  In the latter case, ie your 
f(x) example, it has nothing to do with error bounds, but that 
your f(x) is not only invalid at 2, but in a range around 2.


It isn't. For what typed number besides 2 is it invalid?


Zero is not the only number that screws up that calculation.


It is. Not only can it screw up the calculation. It can screw 
up the real time properties of the algorithm on a specific FPU.

Re: Always false float comparisons

2016-05-18 Thread Matthias Bentrup via Digitalmars-d

On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
wrote:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

And yes, half-precision is only 10 bits.


Actually, it turns out that the mantissa is 11 bits. So it 
clearly plays louder than other floats. ;-)


The mantissa is 10 bits, but it has 11 bit precision, just as the 
float type has a 23 bit mantissa and 24 bit precision. AFAIK the 
only float format that stores the always-one-bit of the mantissa 
is the x87 80 bit format.

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d

On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

And yes, half-precision is only 10 bits.


Actually, it turns out that the mantissa is 11 bits. So it 
clearly plays louder than other floats. ;-)

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d


On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:
Do you have a link explaining GCC actually uses such a soft 
float? For example 
https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says "This file should be rewritten to use an arbitrary precision..."


Alright, it seems that GCC doesn't use arbitrary precision float 
everywhere :)

Re: Always false float comparisons

2016-05-18 Thread Johannes Pfau via Digitalmars-d

Am Wed, 18 May 2016 11:48:49 +
schrieb deadalnix :

> On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:
> > On 5/18/2016 3:15 AM, deadalnix wrote:  
> >> On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:  
> >>> Trying to make D behave exactly like various C++ compilers 
> >>> do, with all their
> >>> semi-documented behavior and semi-documented switches that 
> >>> affect constant
> >>> folding behavior, is a hopeless task.
> >>>
> >>> I doubt various C++ compilers are this compatible, even if 
> >>> they follow the
> >>> same ABI.
> >>>  
> >>
> >> They aren't. For instance, GCC use arbitrary precision FB, and 
> >> LLVM uses 128
> >> bits soft floats in their innards.  
> >
> > Looks like LLVM had the same idea as myself.
> >
> > Anyhow, this pretty much destroys the idea that I have proposed 
> > some sort of cowboy FP that's going to wreck FP programs.
> >
> > (What is arbitrary precision FB?)  
> 
> Typo: arbitrary precision FP. Meaning some soft float that grows 
> as big as necessary to not lose precision à la BitInt but for 
> floats.
> 

Do you have a link explaining GCC actually uses such a soft float? For
example
https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20
still says "This file should be rewritten to use an arbitrary
precision..."

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:

Welcome to the wonderful world of C++! :D

More seriously, it is well-defined for that implementation, you 
did not raise the issue of the spec till now.  In fact, you 
seemed not to care what the specs say.


Eh? All C/C++ compilers I have ever used coerce to single 
precision upon binding.


I care about what the spec says, why it says it and how it is 
used when targetting the specific hardware.


Or more generally, when writing libraries I target the language 
spec, when writing applications I target the platform (language + 
compiler + hardware).


No, it has nothing to do with language semantics and everything 
to do with bad numerical programming.


No. You can rant all you want about bad numerical programming. 
But by your definition all DSP programmers are bad at numerics. 
Which _obviously_ is not true.


That is grasping for straws.

There is NO excuse for preventing programmers from writing 
reliable performant code that solves the problem at hand to the 
accuracy that is required by the application.


Because floating-point is itself fuzzy, in so many different 
ways.  You are depending on exactly repeatable results with a 
numerical type that wasn't meant for it.


Floating point is not fuzzy. It is basically a random sample on 
an interval of potential solutions with a range that increase 
with each computation. Unfortunately, you have to use interval 
arithmetics to get that interval, as in regular floating point 
you loose the information about the interval. It is an 
approximation. IEEE 754-2008 makes it possible to get that 
interval, btw.


Fuzzy is different. Fuzzy means that the range itself is the 
value. It does not represent an approximation. ;-)



You keep saying this: where did anyone mention unit tests not 
running with the same precision till you just brought it up out 
of nowhere?  The only prior mention was that compile-time 
calculation of constants that are then checked for bit-exact 
equality in the tests might have problems, but that's certainly 
not all tests and I've repeatedly pointed out you should never 
be checking for bit-exact equality.


I have stated time and time again that it is completely 
unacceptable if there is even a remote chance for anything in the 
unit test to evaluate at a higher precision than in the 
production code.


That is not a unit test, it is a sham.

The point is that what you consider reliable will be less 
accurate, sometimes much less.


Define accurate. Accurate has no meaning without a desired 
outcome to reference.


I care about 4 oscillators having the same phase. THAT MEANS: 
they all have to use the exact same constants.


If not, they _will_ drift and phase cancellation _will_ occur.

I care about adding and removing the same constant. If not, 
(more) DC offsets will build up.


It is all about getting below the right tolerance threshold while 
staying real time. You can compensate by increasing the control 
rate (reduce the number of interpolated values).



The point is that there are _always_ bounds, so you can never 
check for the same value.  Almost any guessed bounds will be 
better than incorrectly checking for the bit-exact value.


No. Guessed bounds are not better. I have never said that anyone 
should _incorrectly_ do anything. I have said that they should 
_correctly_ understand what they are doing and that guessing 
bounds just leads to a worse problem:


Programs that intermittently fail in spectacular ways that are 
very difficult to debug.


You cannot even compute the bounds if the compiler can use higher 
precision. IEEE754-2008 makes it possible to accurately compute 
bounds.


Not supporting that is _very_ bad.


should always be thought about.  In the latter case, ie your 
f(x) example, it has nothing to do with error bounds, but that 
your f(x) is not only invalid at 2, but in a range around 2.


It isn't. For what typed number besides 2 is it invalid?


Zero is not the only number that screws up that calculation.


It is. Not only can it screw up the calculation. It can screw up 
the real time properties of the algorithm on a specific FPU. 
Which is even worse.


f(x) and what isn't, that has nothing to do with D.  You want D 
to provide you a way to only check for 0.0, whereas my point is 
that there are many numbers in the neighborhood of 0.0 which 
will screw up your calculation, so really you should be using 
approxEqual.


What IEEE32 number besides 2 can cause problems?


It isn't changing your model, you can always use a very small


But it is!

If your point is that you're modeling artificial worlds that 
have nothing to do with reality, you can always change your 
threshold around 0.0 to be much smaller, and who cares if it 
can't go all the way to zero, it's all artificial, right? :)


Why would I have a threshold around 2, when only 2 is causing 
problems at the hardware level?



If you're modeling the real world, any function that

Re: Always false float comparisons

2016-05-18 Thread Manu via Digitalmars-d

On 18 May 2016 at 21:53, ixid via Digitalmars-d
 wrote:
> On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
>>
>> That's precisely the suggestion; that compile time execution of a
>> given type mirror the runtime, that is, matching precisions in this
>> case.
>> ...within reason; as Walter has pointed out consistently, it's very
>> difficult to be PERFECT for all the reasons he's been repeating, but
>> there's still a massive difference between the runtime executing a
>> bunch of float code, and the compile time executing it all promoted to
>> 80bits. Results will drift apart very quickly.
>
>
> What do you think can be productively done to improve the situation? I am
> beginning to think a Wiki-like structure would be better for these
> discussions where each major debater has their thoughts on a specific issue
> in a relevantly headed section so there is more clarity.

I've said so many times; I think it would be more useful if CTFE
operated on the *specified type*. That is all. That should produce
results as close as reasonably possible to the runtime.
If I want CTFE to operate at real precision (as is often the case),
float functions just take an 'isFloatingPoint!T' and users can execute
them at whatever precision they like. Most float functions already
have this form.

Re: Always false float comparisons

2016-05-18 Thread ixid via Digitalmars-d


On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
That's precisely the suggestion; that compile time execution of 
a
given type mirror the runtime, that is, matching precisions in 
this

case.
...within reason; as Walter has pointed out consistently, it's 
very
difficult to be PERFECT for all the reasons he's been 
repeating, but
there's still a massive difference between the runtime 
executing a
bunch of float code, and the compile time executing it all 
promoted to

80bits. Results will drift apart very quickly.


What do you think can be productively done to improve the 
situation? I am beginning to think a Wiki-like structure would be 
better for these discussions where each major debater has their 
thoughts on a specific issue in a relevantly headed section so 
there is more clarity.

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d


On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:

On 5/18/2016 3:15 AM, deadalnix wrote:

On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
Trying to make D behave exactly like various C++ compilers 
do, with all their
semi-documented behavior and semi-documented switches that 
affect constant

folding behavior, is a hopeless task.

I doubt various C++ compilers are this compatible, even if 
they follow the

same ABI.



They aren't. For instance, GCC use arbitrary precision FB, and 
LLVM uses 128

bits soft floats in their innards.


Looks like LLVM had the same idea as myself.

Anyhow, this pretty much destroys the idea that I have proposed 
some sort of cowboy FP that's going to wreck FP programs.


(What is arbitrary precision FB?)


Typo: arbitrary precision FP. Meaning some soft float that grows 
as big as necessary to not lose precision à la BitInt but for 
floats.

Re: Always false float comparisons

2016-05-18 Thread Era Scarecrow via Digitalmars-d


On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:

On Wednesday, 18 May 2016 at 08:38:07 UTC, Era Scarecrow wrote:
 try {}// Considers the result of 1 line of basic math to 
be caught by:

 carry {} //only activates if carry is set
 overflow  {} //if overflowed during some math
 modulus(m){} //get the remainder as m after a division 
operation
 mult(dx)  {} //get upper 32/64/whatever after a multiply and 
set as dx


Of course I'd understand if some hardware doesn't offer such 
support, so the else could be thrown in to allow a workaround 
code to detect such an event, or only allow it if it's a 
compliant architecture. Although workaround detection is 
always possible, just not as fast as hardware supplied.


https://code.dlang.org/packages/checkedint
https://dlang.org/phobos/core_checkedint.html


 Glancing at the checkedInt I really don't see it as being the 
same as what I'm talking about. Overflow/carry for add perhaps, 
but unless it breaks down to a single instruction for the 
compiler to determine if it needs to do something, I see it as a 
failure (at best, a workaround).


 That's just my thoughts. CheckedInt simply _doesn't_ cover what 
I was talking about. Obtaining the modulus for 0 
cost/instructions after doing a division which is in the 
hardware's opcode side effects (unless the compiler recognizes 
the pattern and offers it as an optimization), or having the full 
result of a multiply on hand (that exceeds it's built-in size, 
long.max*long.max = 128bit result, which the hardware hands to 
you if you check the register it stores the other half of the 
result in).


 Perhaps what I want is more limited to handling certain tasks 
(making software math libraries) but I'd still like/want to see 
access to the other effects of these opcodes.

Re: Always false float comparisons

2016-05-18 Thread Johannes Pfau via Digitalmars-d

Am Wed, 18 May 2016 04:11:08 -0700
schrieb Walter Bright :

> On 5/18/2016 3:15 AM, deadalnix wrote:
> > On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:  
> >> Trying to make D behave exactly like various C++ compilers do,
> >> with all their semi-documented behavior and semi-documented
> >> switches that affect constant folding behavior, is a hopeless task.
> >>
> >> I doubt various C++ compilers are this compatible, even if they
> >> follow the same ABI.
> >>  
> >
> > They aren't. For instance, GCC use arbitrary precision FB, and LLVM
> > uses 128 bits soft floats in their innards.  
> 
> Looks like LLVM had the same idea as myself.
> 
> Anyhow, this pretty much destroys the idea that I have proposed some
> sort of cowboy FP that's going to wreck FP programs.
> 
> (What is arbitrary precision FB?)

A claim from GMP, a library used by GCC:

> GMP is a free library for arbitrary precision arithmetic, operating on
> signed integers, rational numbers, and floating-point numbers. There
> is
> no practical limit to the precision except the ones implied by the
> available memory in the machine GMP runs on.

It's difficult to find reliable information, but I think GCC always uses
the target precision for constant folding:
https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gccint/Floating-Point.html#Floating-Point

> Because different representation systems may offer different amounts
> of
> range and precision, all floating point constants must be represented
> in the target machine's format. Therefore, the cross compiler cannot
> safely use the host machine's floating point arithmetic; it must
> emulate the target's arithmetic. To ensure consistency, GCC always
> uses
> emulation to work with floating point values, even when the host and
> target floating point formats are identical.

Re: Always false float comparisons

On Wednesday, 18 May 2016 at 11:12:16 UTC, Joseph Rushton 
Wakeling wrote:
I'm not sure that the `const float` vs `float` is the 
difference per se.  The difference is that in the examples 
you've given, the `const float` is being determined (and used) 
at compile time.


They both have to be determined at compile time... as there is 
only a cast involved. The real issue is that the const float 
binding is treated textually.


But a `const float` won't _always_ be determined or used at 
compile time, depending on the context and manner in which the 
value is set.


Which makes the problem worse, not better.

Re: Always false float comparisons

2016-05-18 Thread Manu via Digitalmars-d

On 18 May 2016 at 21:28, ixid via Digitalmars-d
 wrote:
> On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:
>>
>> On 5/18/2016 1:30 AM, Ethan Watson wrote:

 You're also asking for a mode where the compiler for one machine is
 supposed
 to behave like hand-coded assembler for another machine with a different
 instruction set.
>>>
>>>
>>> Actually, I'm asking for something exactly like the arch option for
>>> MSVC/-mfmath
>>> option for GCC/etc, and have it respect that for CTFE.
>>
>>
>>
>> MSVC doesn't appear to have a switch that does what you ask for:
>>
>>   https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx
>
>
> Apologies if this has been addressed in the thread, it's a difficult
> structure to follow for technical discussion. You seem positive about
> software implementations of float. What are your thoughts on having the
> compile time implementation of a given type mirror the behaviour of the
> runtime version?
>
> Fundamentally whatever rules are chosen it would seem better to have fewer
> rules for people  to remember.

That's precisely the suggestion; that compile time execution of a
given type mirror the runtime, that is, matching precisions in this
case.
...within reason; as Walter has pointed out consistently, it's very
difficult to be PERFECT for all the reasons he's been repeating, but
there's still a massive difference between the runtime executing a
bunch of float code, and the compile time executing it all promoted to
80bits. Results will drift apart very quickly.

Re: Always false float comparisons

2016-05-18 Thread Ethan Watson via Digitalmars-d


On Wednesday, 18 May 2016 at 11:17:14 UTC, Walter Bright wrote:
Again, even if the precision matches, the rounding will NOT 
match, and you will get different results randomly dependent on 
the exact operand values.


We've already been burned by middlewares/APIS toggling MMX flags 
on and off and not cleaning up after themselves, and as such we 
strictly control those flags going in to and out of such areas. 
We even have a little class with implementations for x87 
(thoroughly deprecated) and SSE that is used in a RAII manner, 
copying the MMX flag on construction and restoring it on 
destruction.


I appreciate that it sounds like I'm starting to stretch to hold 
to my point, but I imagine we'd also be able to control such 
things with the compiler - or at least know what flags it uses so 
that we can ensure consistent behaviour between compilation and 
runtime.

Re: Always false float comparisons

2016-05-18 Thread ixid via Digitalmars-d


On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

On 5/18/2016 1:30 AM, Ethan Watson wrote:
You're also asking for a mode where the compiler for one 
machine is supposed
to behave like hand-coded assembler for another machine with 
a different

instruction set.


Actually, I'm asking for something exactly like the arch 
option for MSVC/-mfmath

option for GCC/etc, and have it respect that for CTFE.



MSVC doesn't appear to have a switch that does what you ask for:

  https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx


Apologies if this has been addressed in the thread, it's a 
difficult structure to follow for technical discussion. You seem 
positive about software implementations of float. What are your 
thoughts on having the compile time implementation of a given 
type mirror the behaviour of the runtime version?


Fundamentally whatever rules are chosen it would seem better to 
have fewer rules for people  to remember.

Re: Always false float comparisons

2016-05-18 Thread Manu via Digitalmars-d

On 18 May 2016 at 18:21, Walter Bright via Digitalmars-d
 wrote:
> On 5/18/2016 12:56 AM, Ethan Watson wrote:
>>
>> > In any case, the problem Manu was having was with C++.
>> VU code was all assembly, I don't believe there was a C/C++ compiler for
>> it.
>
>
> The constant folding part was where, then?

The comparison was a 24bit fpu doing runtime work but where some
constant input data was calculated with a separate 32bit fpu. The
particulars were not ever intended to be relevant to the conversation,
except the fact that 2 differently precisioned float units were
producing output that then had to reconcile.

The analogy was to CTFE doing all its work at 80bits, and then the
processor doing work with the types explicitly stated by the
programmer; a runtime calculation compared against the same compile
time calculate is likely to be quite radically different. I don't care
about the precision, I just care that they're really different.
Ideally, CTFE would produce a result that is as similar to the runtime
result as reasonably possible, and I expect using the stated types to
do the calculations would get much much closer.
I don't know if a couple of least-significant bits of difference would
have caused problems for us, I suspect not, but I know that doing math
at radically different precisions (ie, 32bits vs 80bits) does lead to
radically different results, not just a couple of bits. That is my
concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit
fpu's.

Re: Always false float comparisons


On 5/18/2016 3:46 AM, Ola Fosheim Grøstad wrote:

On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:

Can you back that up statistically?  Try running this same operation 600
million times plot a graph for the result from each run for it so we can get
an idea of just how random or arbitrary it really is.


Huh? This isn't about statistics.


It is when you say 'random'. D's fp math is not random, it is completely 
deterministic, and I've told you so before. You've been pushing the 'random' 
notion here, and other people have picked it up and inferred they'd get random 
results every time they ran a D compiled program.


Please use correct meanings of words.

Re: Always false float comparisons

2016-05-18 Thread Joseph Rushton Wakeling via Digitalmars-d


On 5/18/2016 2:54 AM, Ethan Watson wrote:

On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

MSVC doesn't appear to have a switch that does what you ask for


I'm still not entirely sure what the /fp switch does for x64 builds. The
documentation is not clear in the slightest and I haven't been able to find any
concrete information. As near as I can tell it has no effect as the original
behaviour was tied to how it handles the x87 control words. But it might also be
possible that the SSE instructions emitted can differ depending on what
operation you're trying to do. I have not dug deep to see exactly how the code
gen differs. I can take a guess that /fp:precise was responsible for promoting
my float to a double to call CRT functions, but I have not tested that so that's
purely theoretical at the moment.

Of course, while this conversation has mostly been for compile time constant
folding, the example of passing a value from the EE and treating it as a
constant in the VU is still analagous to calculating a value at compile time in
D at higher precision than the instruction set the runtime code is compiled to
work with.

/arch:sse2 is the default with MSVC x64 builds (Xbox One defaults to /arch:avx),
and it sounds like the DMD has defaulted to sse2 for a long time. The exception
being the compile time behaviour. That compile time behaviour conforming to the
the runtime behaviour is an option I want, with the default being whatever is
decided in here. Executing code at compile time at a higher precision than what
SSE dictates is effectively undesired behaviour for our use cases.

And in cases where we compile code for another architecture on x64 (let's say
ARM code with NEON instructions, as it's the most common case thanks to iOS
development) then it would be forced to fallback to the default. Fine for most
use cases as well. It would be up to the user to compile their ARM code on an
ARM processor to get the code execution match if they need it.



Again, even if the precision matches, the rounding will NOT match, and you will 
get different results randomly dependent on the exact operand values.


If those differences matter, then you'll randomly be up all night debugging it. 
If you're willing to try the approach I mentioned, it'll cost you a bit more 
time up front, but may save a lot of agony later.

Re: Always false float comparisons

2016-05-18 Thread Joakim via Digitalmars-d

On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:

On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:
On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
wrote:

On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
There is nothing "random" about increasing precision till 
the end, it follows a well-defined rule.


Can you please quote that well-defined rule?


It appears to be "the compiler carries everything internally 
to 80 bit precision, even if they are typed as some other 
precision."

http://forum.dlang.org/post/nh59nt$1097$1...@digitalmars.com


"The compiler" means: implementation defined. That is the same 
as not being well-defined. :-)


Welcome to the wonderful world of C++! :D

More seriously, it is well-defined for that implementation, you 
did not raise the issue of the spec till now.  In fact, you 
seemed not to care what the specs say.


I don't understand why you're using const for one block and 
not the other, seems like a contrived example.  If the 
precision of such constants matters so much, I'd be careful to 
use the same const float everywhere.


Now, that is a contrived defense for brittle language 
semantics! :-)


No, it has nothing to do with language semantics and everything 
to do with bad numerical programming.


If matching such small deltas matters so much, I wouldn't be 
using floating-point in the first place.


Why not? The hardware gives the same delta. It only goes wrong 
if the compiler decides to "improve".


Because floating-point is itself fuzzy, in so many different 
ways.  You are depending on exactly repeatable results with a 
numerical type that wasn't meant for it.


It depends on the unit tests running with the exact same 
precision as the production code.


What makes you think they don't?


Because the language says that I cannot rely on it and the 
compiler implementation proves that to be correct.


You keep saying this: where did anyone mention unit tests not 
running with the same precision till you just brought it up out 
of nowhere?  The only prior mention was that compile-time 
calculation of constants that are then checked for bit-exact 
equality in the tests might have problems, but that's certainly 
not all tests and I've repeatedly pointed out you should never be 
checking for bit-exact equality.


D is doing it wrong because it makes it is thereby forcing 
programmers to use algorithms that are 10-100x slower to get 
reliable results.


That is _wrong_.


If programmers want to run their code 10-100x slower to get 
reliably inaccurate results, that is their problem.


Huh?


The point is that what you consider reliable will be less 
accurate, sometimes much less.


If you're so convinced it's exact for a few cases, then check 
exact equality there.  For most calculation, you should be 
using approxEqual.


I am sorry, but this is not a normative rule at all. The rule 
is that you check for the bounds required. If it is exact, it 
just means the bounds are the same value (e.g. tight).


It does not help to say that people should use "approxEqual", 
because it does not improve on correctness. Saying such things 
just means that non-expert programmers assume that guessing the 
bounds will be sufficient. Well, it isn't sufficient.


The point is that there are _always_ bounds, so you can never 
check for the same value.  Almost any guessed bounds will be 
better than incorrectly checking for the bit-exact value.


Since the real error bound is always larger than that, almost 
any error bound you pick will tend to be closer to the real 
error bound, or at least usually bigger and therefore more 
realistic, than checking for exact equality.


I disagree. It is much better to get extremely wrong results 
frequently and therefore detect the error in testing.


What you are saying is that is better to get extremely wrong 
results infrequently which usually leads to error passing 
testing and enter production.


In order to test well you also need to understand for input 
makes the algorithm unstable/fragile.


Nobody is talking about the general principle of how often you 
get wrong results or unit testing.  We were talking about a very 
specific situation: how should compile-time constants be checked 
and variables compared to constants, compile-time or not, to 
avoid exceptional situations.  My point is that both should 
always be thought about.  In the latter case, ie your f(x) 
example, it has nothing to do with error bounds, but that your 
f(x) is not only invalid at 2, but in a range around 2.


Now, both will lead to less "wrong results," but those are wrong 
results you _should_ be trying to avoid as early as possible.


The computer doesn't know that, so it will just plug that x in 
and keep cranking, till you get nonsense data out the end, if 
you don't tell it to check that x isn't too close to 2 and not 
just 2.


Huh? I am not getting nonsense data. I am getting what I am 
asking for, I only want

Re: Always false float comparisons

On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:
No. The "const float y" will not be coerced to 32 bit, but the 
"float y" will be coerced to 32 bit. So you get two different y 
values. (On a specific compiler, i.e. DMD.)


I'm not sure that the `const float` vs `float` is the difference 
per se.  The difference is that in the examples you've given, the 
`const float` is being determined (and used) at compile time.


But a `const float` won't _always_ be determined or used at 
compile time, depending on the context and manner in which the 
value is set.


Let's be clear about the problem -- compile time vs. runtime, 
rather than `const` vs non-`const`.

Re: Always false float comparisons

2016-05-18 Thread Ola Fosheim Grøstad via Digitalmars-d


On 5/18/2016 3:15 AM, deadalnix wrote:

On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:

Trying to make D behave exactly like various C++ compilers do, with all their
semi-documented behavior and semi-documented switches that affect constant
folding behavior, is a hopeless task.

I doubt various C++ compilers are this compatible, even if they follow the
same ABI.



They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128
bits soft floats in their innards.


Looks like LLVM had the same idea as myself.

Anyhow, this pretty much destroys the idea that I have proposed some sort of 
cowboy FP that's going to wreck FP programs.


(What is arbitrary precision FB?)

Re: Always false float comparisons


On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:
Can you back that up statistically?  Try running this same 
operation 600 million times plot a graph for the result from 
each run for it so we can get an idea of just how random or 
arbitrary it really is.


Huh? This isn't about statistics. It is about math. The magnitude 
of the difference depends on what you do with the constant. It 
can be exponentially boosted.

Re: Always false float comparisons