Re: float equality
== Quote from Sean Kelly (s...@invisibleduck.org)'s article > Andrei Alexandrescu wrote: > > On 2/21/11 6:08 PM, bearophile wrote: > >> Andrei Alexandrescu: > >> > >>> This is a long-standing myth. I worked on Wall Street and have friends > >>> who have been doing it for years. Everybody uses double. > >> > >> Unbrutal Python programmers are encouraged to avoid the float type to > >> manage money values, and use decimal instead: > >> http://docs.python.org/library/decimal.html > >> > >> Bye, > >> bearophile > > > > ... and nobody uses Python :o). > C/C++, Visual Basic, and SQL in my experience. Though in SQL they may use > the 'money' type. Using double really isn't an issue so long as rounding is > handled appropriately. It seems like a lot of the trouble stems from people > thinking values must be naturally exactly represented by the storage type. The semantics of floating point are complicated, and I think the human brain usually sees 'complicated' and 'unpredictable/unstable' as synonyms. As for me, I heard long ago that double wasn't used and I think I believed it because I'd like to believe that someone out there is doing the responsible thing. Now that I have a bit more experience I can see that floating point is reliable, even if it is unintuitive. The instinct is to avoid it where correctness is important but if you analyze the code you can predict a lot about what floating point will do even if not always everything. If you can keep the error to the sub-penny level you should be fine with either approach. But I think C taught a lot of people to be wary. C considers an amazingly large percentage of its allowed syntax to have undefined semantics. Which direction integers round, sizes of basic types, bit shifting with values greater than sizeof(int)-1, etc. Almost anything that varies from CPU to CPU is not defined by C, with the result that you need to build a mini-jail in your mind to write portable code. Floating point is part of this because many floating point expressions produce different results depending on choices made by the compiler while optimizing, e.g. when to shift numbers from real <-> double. The number of things a language defines in a 'do whatever is easy or quick for your platform' ends up being a coefficient of the difficulty of using a language to write (portably) correct code. As a result, almost all large C programs are non-portable and I'm convinced the majority have undefined behavior if you take a narrow view of what C guarantees about the size of int, the order of side effects, and (if you really want to play hardball) the memory hierarchy in a multithreaded program. Kevin
Re: float equality
Andrei Alexandrescu wrote: > On 2/21/11 6:08 PM, bearophile wrote: >> Andrei Alexandrescu: >> >>> This is a long-standing myth. I worked on Wall Street and have friends >>> who have been doing it for years. Everybody uses double. >> >> Unbrutal Python programmers are encouraged to avoid the float type to >> manage money values, and use decimal instead: >> http://docs.python.org/library/decimal.html >> >> Bye, >> bearophile > > ... and nobody uses Python :o). C/C++, Visual Basic, and SQL in my experience. Though in SQL they may use the 'money' type. Using double really isn't an issue so long as rounding is handled appropriately. It seems like a lot of the trouble stems from people thinking values must be naturally exactly represented by the storage type.
Re: float equality
On 02/21/2011 10:17 PM, so wrote: If one doesn't know what floating point is and insists on using it, it is his own responsibility to face the consequences. I don't buy this argument. Why not? A logical flaw on my part or the statement being somewhat harsh? Because i don't think it is the former, i will give an example for the latter. I am a self-taught programmer and i too made big mistakes when using FP, probably i still do since it is a strange beast to deal with even if you know all about it. For this reason it is somewhat understandable for people like me failing this kind of traps but can we say same thing for others? Do they have this excuse? Not knowing the fundamental thing about FP and use it? I understand your position. But I don't share it. The problem is binary floating point representations of numbers introduce a little set of traps due to the fact that they are implicitely supposed to be the type representing "fractional" or real numbers, but they cannot do that in a way that matches our (widely unconscious) intuitions in the domain, themselves (I guess) a by-product of years of manipulation at school. The simple notation "1.1" comes with a huge baggage of pre-existing knowledge by everyone (programmers). We should not blindly make abstraction of that fact, and put the burden on the users' shoulders, without even a honest trial in solving the issue. When a programmer writes: x = 1.1; or x = 1/3.0; the meaning is just that, and refers to all this knowledge. There is in most cases no intention (not even implicite) to use binary floating point numbers, instead to have a representation for the plain arithmetics values as written down. Hope I'm clear. Very few people (ones that have repetedly been caught by the conceptual traps) will have "alarm bell" ring when writing this, and consequently take all appropriate precautions required to avoid subsequenty falling into the traps and... have buggy code. This may be a bit different if the notation did not recall all of that knowledge: x = 1.1f; // alarm bell!? I would advocate that a general-purpose language should use a different representation for ordinary need of fractional/real numbers, one that does not introduce such traps and/or warns about them (via warnongs/errors, like Bearophile's proposal). One possibility may be to use fixed point with a /decimal/ scale factor (the mantissa beeing binary or decimal integer). Other possibilities: true rationals, decimal 'reals'. The issue is indeed any of those solutions has a cost (even more with modern machines having native float artithmetic). But numeric computation intensive apps would just use floats, /explicitely/ (and hopefully correctly). A similar issue is the one of signed <--> unsigned ints. I would just make as default the range of unsigned ints be a subset of signed ones. And let the full range available for people who really need it, to be used explicitely. Anyway, such solutions may not be good for D, beeing (also) a systems programming language in the C line. Still, I think it's worth exploring the topic instead of letting bug-prone features silently sit at the deep core of the language. Denis -- _ vita es estrany spir.wikidot.com
Re: float equality
"Andrei Alexandrescu" wrote in message news:ijuub9$tv5$1...@digitalmars.com... > On 2/21/11 4:48 AM, Jonathan M Davis wrote: >> On Monday 21 February 2011 01:55:28 Walter Bright wrote: >>> Kevin Bealer wrote: 1. To solve the basic problem the original poster was asking -- if you are working with simple decimals and arithmetic you can get completely accurate representations this way. For some cases like simple financial work this might work really well. e.g. where float would not be because of the slow leak of information with each operation. (I assume real professional financial work is already done using a (better) representation.) >>> >>> A reasonable way to do financial work is to use longs to represent >>> pennies. >>> After all, you don't have fractional cents in your accounts. >>> >>> Using floating point to represent money is a disaster in the making. >> >> Actually, depending on what you're doing, I'm not sure that you can >> legally >> represent money with floating point values. As I understand it, there are >> definite >> restrictions on banking software and the like with regards to that sort >> of thing >> (though I don't know exactly what they are). > > This is a long-standing myth. I worked on Wall Street and have friends who > have been doing it for years. Everybody uses double. > > Andrei > Now we know who to blame for the downfall of the financial sector :-) -=mike=-
Re: float equality
bearophile Wrote: > Andrei Alexandrescu: > > > This is a long-standing myth. I worked on Wall Street and have friends > > who have been doing it for years. Everybody uses double. > > Unbrutal Python programmers are encouraged to avoid the float type to manage > money values, and use decimal instead: > http://docs.python.org/library/decimal.html Interesting question is whether 5e-17 is a big error. If I want to divide a dollar into 3 parts, should it be 33 cents or 33.33...? A possible error here is 1%.
Re: float equality
Andrei Alexandrescu wrote: I worked on Wall Street and have friends who have been doing it for years. Everybody uses double. Makes sense. Not many people on earth has more than 4,503,599,627,370,496 pennies lying about. -- Simen
Re: float equality
On Monday 21 February 2011 15:58:03 Andrei Alexandrescu wrote: > On 2/21/11 4:48 AM, Jonathan M Davis wrote: > > On Monday 21 February 2011 01:55:28 Walter Bright wrote: > >> Kevin Bealer wrote: > >>> 1. To solve the basic problem the original poster was asking -- if you > >>> are working with simple decimals and arithmetic you can get completely > >>> accurate representations this way. For some cases like simple > >>> financial work this might work really well. e.g. where float would not > >>> be because of the slow leak of information with each operation. (I > >>> assume real professional financial work is already done using a > >>> (better) > >>> representation.) > >> > >> A reasonable way to do financial work is to use longs to represent > >> pennies. After all, you don't have fractional cents in your accounts. > >> > >> Using floating point to represent money is a disaster in the making. > > > > Actually, depending on what you're doing, I'm not sure that you can > > legally represent money with floating point values. As I understand it, > > there are definite restrictions on banking software and the like with > > regards to that sort of thing (though I don't know exactly what they > > are). > > This is a long-standing myth. I worked on Wall Street and have friends > who have been doing it for years. Everybody uses double. Hmm. Good to know. I do find that a bit scary though. I wonder what the level of error is in doing that and how much it affects your typical monetary calculation. - Jonathan M Davis
Re: float equality
On 2/21/11 6:08 PM, bearophile wrote: Andrei Alexandrescu: This is a long-standing myth. I worked on Wall Street and have friends who have been doing it for years. Everybody uses double. Unbrutal Python programmers are encouraged to avoid the float type to manage money values, and use decimal instead: http://docs.python.org/library/decimal.html Bye, bearophile ... and nobody uses Python :o). Andrei
Re: float equality
Andrei Alexandrescu: > This is a long-standing myth. I worked on Wall Street and have friends > who have been doing it for years. Everybody uses double. Unbrutal Python programmers are encouraged to avoid the float type to manage money values, and use decimal instead: http://docs.python.org/library/decimal.html Bye, bearophile
Re: float equality
On 2/21/11 4:48 AM, Jonathan M Davis wrote: On Monday 21 February 2011 01:55:28 Walter Bright wrote: Kevin Bealer wrote: 1. To solve the basic problem the original poster was asking -- if you are working with simple decimals and arithmetic you can get completely accurate representations this way. For some cases like simple financial work this might work really well. e.g. where float would not be because of the slow leak of information with each operation. (I assume real professional financial work is already done using a (better) representation.) A reasonable way to do financial work is to use longs to represent pennies. After all, you don't have fractional cents in your accounts. Using floating point to represent money is a disaster in the making. Actually, depending on what you're doing, I'm not sure that you can legally represent money with floating point values. As I understand it, there are definite restrictions on banking software and the like with regards to that sort of thing (though I don't know exactly what they are). This is a long-standing myth. I worked on Wall Street and have friends who have been doing it for years. Everybody uses double. Andrei
Re: float equality
so: > Why not? A system language has to allow the programmer to do everything the hardware is capable to do, including comparing floats in the usual way. But in general for a well designed language it's better to clearly denote dangerous/unsafe/bad things, and to make the safer routes the first ones you usually try. > where does it stop? I don't know, I am here to learn this too. You have languages like Assembly, SPARK, ATS. They are successful in their niche and for their purposes. They are at the (current) opposites for safety, control, freedom, power, performance, etc. In the area that has those tree languages as vertices (and there are many more possible vertices to choose from) there is a lot of design space for good languages. Bye, bearophile
Re: float equality
If one doesn't know what floating point is and insists on using it, it
is
his own responsibility to face the consequences.
I don't buy this argument.
Why not? A logical flaw on my part or the statement being somewhat harsh?
Because i don't think it is the former, i will give an example for the
latter.
I am a self-taught programmer and i too made big mistakes when using FP,
probably i still do since it is a strange beast to deal with even if you
know all about it.
For this reason it is somewhat understandable for people like me failing
this kind of traps but can we say same thing for others?
Do they have this excuse? Not knowing the fundamental thing about FP and
use it?
The little proposal I was thinking about is:
1) Turn the "==" operator into a compile-time syntax error if one or
both operands are floating point values.
if (x == y) { ==> syntax error
2) The semantic of the "==" is now done by the "is" operator.
So if you want exactly this C code:
if (x == y) {
you use:
if (x is y) {
3) Then another built-in function or semantics is added, that is similar
to:
some_syntax(x, y, an_approximation_level)
It takes tree arguments: x, y and a number of bits.
Turning the == into a syntax error is done to remember programmers that
== among FP values is tricky, and the "is" operator is added because
there are some situations where you want an exact comparison and you
know what you are doing.
This was my idea. I know it doesn't improve the current situation a
lot... so I don't expect this to be accepted. But I like to be
understood :-)
Bye,
bearophile
One thing i fail to understand here is, yes i agree "float == float"
probably a "code smell" and we can find a workaround for this.
But i have to ask, where does it stop? I mean i can just here find you
pages and pages of illogical results created with crafted examples using
not even floating points but integrals.
If you want to make it easier like the case in hand, ignore a fundamental
rule and you get tons of examples.
All i am trying to say is that if we are not in a position to solve this
huge problem with a syntax change, lets just keep it the way it is instead
of making yet another inconsistency.
Re: float equality
so:
> I still think "==" should mean the exact equality test and must be
> consistent in language.
Everyone in this thread has misunderstood what I have tried to say, so I will
try to explain again, see the bottom of this post.
My idea was to turn the "==" operator among FP values into a syntax error (plus
other added ideas).
> Making something like almostEqual default is far more catastrophic than
> its current form.
This problem doesn't exists in my idea.
> It doesn't solve the existing problem and create a basis for new form of
> problems.
The problems it creates are smaller.
> If one doesn't know what floating point is and insists on using it, it is
> his own responsibility to face the consequences.
I don't buy this argument.
> If only interval arithmetic would solve all the problems, i wouldn't
> hesitate dumping everything about FP.
> But no, it comes with its shortcomings.
I agree that interval arithmetic has its downsides. I have not proposed to
replace normal floating point values with intervals. I have suggested to add a
Phobos module for interval arithmetic because it's a useful thing to have.
-
The little proposal I was thinking about is:
1) Turn the "==" operator into a compile-time syntax error if one or both
operands are floating point values.
if (x == y) { ==> syntax error
2) The semantic of the "==" is now done by the "is" operator.
So if you want exactly this C code:
if (x == y) {
you use:
if (x is y) {
3) Then another built-in function or semantics is added, that is similar to:
some_syntax(x, y, an_approximation_level)
It takes tree arguments: x, y and a number of bits.
Turning the == into a syntax error is done to remember programmers that ==
among FP values is tricky, and the "is" operator is added because there are
some situations where you want an exact comparison and you know what you are
doing.
This was my idea. I know it doesn't improve the current situation a lot... so I
don't expect this to be accepted. But I like to be understood :-)
Bye,
bearophile
Re: float equality
On Sun, 20 Feb 2011 14:53:03 +0200, bearophile wrote: Walter: There's a total lack of evidence for that. MISRA-C standard disallows the equality among FP values. I presume they have some serious evidence for their choices, but I don't know such evidence. Today MISRA-C is one of the most serious attempts at fixing the holes in C language to write more reliable code for the industry. MISRA-C standard is sometimes extreme and I don't suggest to follow everything they say, but I suggest to not totally ignore what it says about many C features. 1. Roundoff error is not of a fixed magnitude. I meant to replace the nude FP equality with a function that has the shared number of mantissa bits as third required argument. And then perform the normal FP equality using the "is" operator. I still think "==" should mean the exact equality test and must be consistent in language. Making something like almostEqual default is far more catastrophic than its current form. It doesn't solve the existing problem and create a basis for new form of problems. If one doesn't know what floating point is and insists on using it, it is his own responsibility to face the consequences. Regarding FP rounding errors, eventually it will be good to add to Phobos2 a library for Interval FP arithmetic, with trigonometric functions too, etc: http://en.wikipedia.org/wiki/Interval_arithmetic Bye, bearophile If only interval arithmetic would solve all the problems, i wouldn't hesitate dumping everything about FP. But no, it comes with its shortcomings.
Re: float equality
On Sat, 19 Feb 2011 14:06:38 +0200, spir wrote:
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4);// pass
assert(-1.1 + 3.3 + 2.2 == 4.4);// fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
Denis
Strange no one mentioned this.
Problem is not the floating point format in your example. I can do the
same with integral numbers, how?
int(5) / int(2) => int(2) or int(3)? And why?
Answer depends on the rounding mode, if you don't know the given rounding
mode for a (machine interpreted) number system you can't say anything.
You know the answer because you know the rule. I can say same for the
floating points since i know how they work.
Re: float equality
On Mon, 21 Feb 2011 10:15:28 -0500, Kagamin wrote: bearophile Wrote: Jonathan M Davis: > The thing is, of course, that actual equality sucks for floating point values. So much, that some people have proposed to deprecate the normal FP equality. MISRA-C disallows them. When you see a == among FP values is often a code smell. Is it safe to assert that noone needs exact FP comparison? Nope. if(var == 0) is a common, important and valid test. To say nothing of testing for sentinel values.
Re: float equality
bearophile Wrote: > Jonathan M Davis: > > > The thing is, of course, that actual equality sucks for floating point > > values. > > So much, that some people have proposed to deprecate the normal FP equality. > MISRA-C disallows them. When you see a == among FP values is often a code > smell. > Is it safe to assert that noone needs exact FP comparison?
Re: float equality
spir wrote:
On 02/21/2011 05:32 AM, Walter Bright wrote:
Kevin Bealer wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
Kevin Bealer wrote:
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise.
It cannot represent irrational numbers accurately.
True but I did mention this a few lines later.
I guess I'm not seeing the point of representing numbers as ratios.
That works
only if you stick to arithmetic. As soon as you do logs, trig
functions, roots,
pi, etc., you're back to square one.
"Naturally" non-representable numbers (irrationals), or results of
"naturally" approximate operations (like trig), are not an issue because
they are expected to yield inaccuracy.
You keep talking about "inaccuracy" and "approximation", when I think
you really mean "rounding".
Actually irrationals are a *big* issue. You might expect sin(PI) == 0,
but it isn't. This isn't a problem with sin(), it's a problem with the
limited resolution of PI. sin(3.141592654) != 0.
This is different from numbers which are well definite in base ten, as
well as results of operations which should yield well definite results.
We think in base ten, thus for us 1.1 is exact. Two operations yielding
1.1 (including plain conversion to binary of the literal '1.1') may in
fact yield unequal numbers at the binary level; this is a trap, and just
wrong:
assert (1.1 != 3.3 - 2.2);// pass
Subsequent issue is (as I just discovered) that there is no remedy for
that, since there is no way to know, in the general case, how many
common magnitude bits are supposed to be shared by the numbers to be
compared, when the results are supposed to be correct. (Is this worse if
floating point format since the scale factor is variable?)
Decimal floating point formats exist, and have been implemented in
hardware in some cases. They don't suffer from the rounding-during-I/O
issue that you mention. Fortunately D supports the %a binary floating
point format, eg, 0x1.34p+46, which gives you what-you-see-is-what-you-get.
This lets me think that, for common use, fixed point with a single
/decimal/ scale factor (for instance 10^-6) may be a better solution.
While the underlying integer type may well be binary --this is
orthogonal-- we don't need using eg BCD. Using longs, this example would
allow representing numbers up to +/- (2^63)/(10^-9), equivalent to
having about 43 bits for the integral part (*), together with a
precision of 6 decimal digits for the fractional part. Seems correct for
common use cases, I guess.
Denis
(*) 10^6 ~ 2^20
(2^63)/(10^-6) > 9_000_000_000_000
Re: float equality
"Walter Bright" wrote: A reasonable way to do financial work is to use longs to represent pennies. After all, you don't have fractional cents in your accounts. I tend to favor using floating point, doing calculations in pennies, and explicitly rounding result to nearest full penny. This is almost the same as using longs except that division does not always truncate numbers. Fractional pennies are often useful in intermediate results. If there is some more appropriate type to use, I might be easily convinced otherwise. However, I don't think using some indefinite precision number type is a fits-for-all solution either, since... I.e. you have to know what you're doing :-) Touche! -- Jouko
Re: float equality
== Quote from Walter Bright (newshou...@digitalmars.com)'s article > Kevin Bealer wrote: > A reasonable way to do financial work is to use longs to represent pennies. > After all, you don't have fractional cents in your accounts. > Using floating point to represent money is a disaster in the making. ... > There's just no getting around needing to understand how computer arithmetic > works. ... > I.e. you have to know what you're doing :-) I'm sure that banks do what you are suggesting and have a policy (maybe a regulatorily required one) on who gets the fractional cents. This is easy because cents are a standard that everyone agrees is the 'bottom'. But there are lots of other cases that aren't standardized... i.e. is time represented in seconds, milliseconds, nanoseconds? Java, Windows and Linux use different versions. Some IBM computers measure time in 1/2^Nths of a second, where N is around 20 or 30. Is land measured in square feet or acres? Whatever you pick as the bottom may not be good enough in the future. If you introduce more exact measurements, now you have mixed representations. If you had a rational number of acres, on the other hand, you could start doing measurements in square feet and representing them as x/(square feet in an acre). Mixed representations are trouble because they invite usage errors. Rational doesn't change the facts of life but it simplifies some things because it makes the choice of base an arbitrary problem-domain kind of choice whereas in fixed point it is very difficult to revisit this decision. Every time I think about this, though, I think of "classroom" examples, so it's probably the case that my arguments are academic (in the bad sense)... and I should give it up. If this was more useful it would probably be used more. Kevin
Re: float equality
On Monday 21 February 2011 01:55:28 Walter Bright wrote: > Kevin Bealer wrote: > > 1. To solve the basic problem the original poster was asking -- if you > > are working with simple decimals and arithmetic you can get completely > > accurate representations this way. For some cases like simple financial > > work this might work really well. e.g. where float would not be because > > of the slow leak of information with each operation. (I assume real > > professional financial work is already done using a (better) > > representation.) > > A reasonable way to do financial work is to use longs to represent pennies. > After all, you don't have fractional cents in your accounts. > > Using floating point to represent money is a disaster in the making. Actually, depending on what you're doing, I'm not sure that you can legally represent money with floating point values. As I understand it, there are definite restrictions on banking software and the like with regards to that sort of thing (though I don't know exactly what they are). Regardless, if you want to deal with money correctly, you pretty much have to treat it as cents and use integer values. Anything else will cause too many errors. That's definitely the way I've seen it done at any company that I've work at which have had to deal with money in their software. - Jonathan M Davis
Re: float equality
spir wrote: "Naturally" non-representable numbers (irrationals), or results of "naturally" approximate operations (like trig), are not an issue because they are expected to yield inaccuracy. Huh, I regularly run into people who are befuddled by the trig functions not conforming to the usual trig identity equations. Worse, I also regularly run into crappy sloppy implementations of the transcendental functions that lose excessive bits of accuracy (I'm looking at you, FreeBSD and OSX). Don has been moving std.math to using our own implementations because of this, avoiding dependency on crappy C libraries written by people who do not understand or care about FP math.
Re: float equality
Kevin Bealer wrote: 1. To solve the basic problem the original poster was asking -- if you are working with simple decimals and arithmetic you can get completely accurate representations this way. For some cases like simple financial work this might work really well. e.g. where float would not be because of the slow leak of information with each operation. (I assume real professional financial work is already done using a (better) representation.) A reasonable way to do financial work is to use longs to represent pennies. After all, you don't have fractional cents in your accounts. Using floating point to represent money is a disaster in the making. 2. To explain why the 'simple' task of representing something like .1 wasn't as easy as it looks. In other words, why the people who designed float weren't just brain dead. I think they really knew what they were doing but it shocks most people at first that a modern computer can't do what they see as grade school arithmetic. There's just no getting around needing to understand how computer arithmetic works. I think for some purposes though, lossless domain specific representations can be a good tool -- if you can represent a problem in a way that is lossless you can maybe do better calculations over long series than working with 'double' and taking the accuracy hit. This is necessarily an application specific technique though. I.e. you have to know what you're doing :-)
Re: float equality
On 02/21/2011 08:52 AM, Kevin Bealer wrote: == Quote from Walter Bright (newshou...@digitalmars.com)'s article ... I do understand that if you have a full symbolic representation, you can do so with zero losses. But Kevin's proposal was not that, it was for a ratio representation. All it represents symbolically is division. There are plenty of other operations. I'm just answering the original poster's question. You're right though -- it's not a complete numerical system, (and I don't propose it for inclusion in the language or even necessarily the library.) I had two goals: 1. To solve the basic problem the original poster was asking -- if you are working with simple decimals and arithmetic you can get completely accurate representations this way. For some cases like simple financial work this might work really well. e.g. where float would not be because of the slow leak of information with each operation. (I assume real professional financial work is already done using a (better) representation.) The few financial libs/types I have met (it's not my glass of bier) all used plain integers ;-) (after all, 1.23 € or $ is a plain integer when counted in cents...), with a fixed or variable decimal scale factor. (what I talk about in another post, I just realise it) 2. To explain why the 'simple' task of representing something like .1 wasn't as easy as it looks. In other words, why the people who designed float weren't just brain dead. I think they really knew what they were doing but it shocks most people at first that a modern computer can't do what they see as grade school arithmetic. I think for some purposes though, lossless domain specific representations can be a good tool -- if you can represent a problem in a way that is lossless you can maybe do better calculations over long series than working with 'double' and taking the accuracy hit. This is necessarily an application specific technique though. I think some typical features, or solutions to implement given features, used in present programming languages, are historic traces from times when scientific computations where the only, then the typical, use of computers. I'm not far to think that fractional numbers (id est, numbers able to represent /measures/) form a domain-specific feature -- thus belong to library (lol). Denis -- _ vita es estrany spir.wikidot.com
Re: float equality
On 02/21/2011 05:32 AM, Walter Bright wrote:
Kevin Bealer wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
Kevin Bealer wrote:
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise.
It cannot represent irrational numbers accurately.
True but I did mention this a few lines later.
I guess I'm not seeing the point of representing numbers as ratios. That works
only if you stick to arithmetic. As soon as you do logs, trig functions, roots,
pi, etc., you're back to square one.
"Naturally" non-representable numbers (irrationals), or results of "naturally"
approximate operations (like trig), are not an issue because they are expected
to yield inaccuracy.
This is different from numbers which are well definite in base ten, as well as
results of operations which should yield well definite results. We think in
base ten, thus for us 1.1 is exact. Two operations yielding 1.1 (including
plain conversion to binary of the literal '1.1') may in fact yield unequal
numbers at the binary level; this is a trap, and just wrong:
assert (1.1 != 3.3 - 2.2); // pass
Subsequent issue is (as I just discovered) that there is no remedy for that,
since there is no way to know, in the general case, how many common magnitude
bits are supposed to be shared by the numbers to be compared, when the results
are supposed to be correct. (Is this worse if floating point format since the
scale factor is variable?)
This lets me think that, for common use, fixed point with a single /decimal/
scale factor (for instance 10^-6) may be a better solution. While the
underlying integer type may well be binary --this is orthogonal-- we don't need
using eg BCD. Using longs, this example would allow representing numbers up to
+/- (2^63)/(10^-9), equivalent to having about 43 bits for the integral part
(*), together with a precision of 6 decimal digits for the fractional part.
Seems correct for common use cases, I guess.
Denis
(*) 10^6 ~ 2^20
(2^63)/(10^-6) > 9_000_000_000_000
--
_
vita es estrany
spir.wikidot.com
Re: float equality
== Quote from Walter Bright (newshou...@digitalmars.com)'s article ... > I do understand that if you have a full symbolic representation, you can do so > with zero losses. But Kevin's proposal was not that, it was for a ratio > representation. > > All it represents symbolically is division. There are plenty of other > operations. I'm just answering the original poster's question. You're right though -- it's not a complete numerical system, (and I don't propose it for inclusion in the language or even necessarily the library.) I had two goals: 1. To solve the basic problem the original poster was asking -- if you are working with simple decimals and arithmetic you can get completely accurate representations this way. For some cases like simple financial work this might work really well. e.g. where float would not be because of the slow leak of information with each operation. (I assume real professional financial work is already done using a (better) representation.) 2. To explain why the 'simple' task of representing something like .1 wasn't as easy as it looks. In other words, why the people who designed float weren't just brain dead. I think they really knew what they were doing but it shocks most people at first that a modern computer can't do what they see as grade school arithmetic. I think for some purposes though, lossless domain specific representations can be a good tool -- if you can represent a problem in a way that is lossless you can maybe do better calculations over long series than working with 'double' and taking the accuracy hit. This is necessarily an application specific technique though. Kevin
Re: float equality
dsimcha wrote: You, my friend, have obviously never used a computer algebra system (my favorite is Maxima) at all seriously. I'm not saying it's a practical solution in the context of this discussion, but it is a possible solution in a lot of cases. Basically, if you can't compute something exactly, you do the math symbolically (but programmatically) for as long as possible. You convert to a floating point number only when it's explicitly asked for. I do understand that if you have a full symbolic representation, you can do so with zero losses. But Kevin's proposal was not that, it was for a ratio representation. All it represents symbolically is division. There are plenty of other operations.
Re: float equality
On 2/20/2011 11:32 PM, Walter Bright wrote:
Kevin Bealer wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
Kevin Bealer wrote:
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise.
It cannot represent irrational numbers accurately.
True but I did mention this a few lines later.
I guess I'm not seeing the point of representing numbers as ratios. That
works only if you stick to arithmetic. As soon as you do logs, trig
functions, roots, pi, etc., you're back to square one.
You, my friend, have obviously never used a computer algebra system (my
favorite is Maxima) at all seriously. I'm not saying it's a practical
solution in the context of this discussion, but it is a possible
solution in a lot of cases. Basically, if you can't compute something
exactly, you do the math symbolically (but programmatically) for as long
as possible. You convert to a floating point number only when it's
explicitly asked for.
Re: float equality
Kevin Bealer wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
Kevin Bealer wrote:
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise.
It cannot represent irrational numbers accurately.
True but I did mention this a few lines later.
I guess I'm not seeing the point of representing numbers as ratios. That works
only if you stick to arithmetic. As soon as you do logs, trig functions, roots,
pi, etc., you're back to square one.
Re: float equality
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
> Kevin Bealer wrote:
> > You could switch to this:
> >
> > struct {
> > BigInt numerator;
> > BigInt denominator;
> > };
> >
> > Bingo -- no compromise.
> It cannot represent irrational numbers accurately.
True but I did mention this a few lines later.
Kevin
Re: float equality
Walter: > MISRA is not suggesting having the language silently default to approximate > equality. Right. But I am not suggesting that, in what I have suggested there is nothing silent :-) Given: double x, y; I have suggested in D: x == y => syntax error. x is y => the same FP equality as today. some syntax(x, y, how_much_to_approximate) => approximate equality. Bye, bearophile
Re: float equality
spir wrote: Is there any plan for them in D2 in the future? If you want to work on a library module that supports them, please have a go!
Re: float equality
Kevin Bealer wrote: > == Quote from Jonathan M Davis (jmdavisp...@gmx.com)'s article > ... >> It may be that you would still end up with situations where two values that >> you >> would think would be the same aren't due to rounding error or whatnot. >> However, >> with a fixed point value, you wouldn't have the problem where a particular >> value >> could not be held in it even if it's within its range of precision. As I >> understand it, there are a number of values which cannot be held in a >> floating >> point and therefore end up being rounded up or down simply because of how >> floating points work and not because there the precision isn't high enough. >> >> It's definitely true however, that using fractions would be much more >> accurate >> for a lot of stuff. That wouldn't be particulary efficient though. Still, if >> you're >> doing a lot of math that needs to be accurate, that may be the way to go. >> >> - Jonathan M Davis > > Floating point numbers are a good compromise for a lot of purposes, but yeah > they > are limited. Here are some ways to open up the limits a bit... > > (First some math for those who haven't gone down this path...) > > The reason that fixed point does better than floating is that fixed point > classes > use base ten and floating point uses base 2. In base two, all fractions that > have > a denominator of a power of 2 can be represented (e.g. 3/16) exactly, and all > others > can't. > > Fixed point solves this for numbers like .1 or .0003 because the denominator > is a > power of ten. Ten is 5*2 and the way it works is that any denominator that > is a power > of two times a power of ten can be represented exactly. > Wrong. There is nothing that says fixed-point uses a power of ten. Actually, I do a lot of work on embedded platforms without a FPU and with slow integer division (and to a lesser extent multiplication). Non-integer numbers are represented in fixed point (to work around the lack of FPU) using base 2 (so that adjustments only require a bitwise shift). Of course, this means that 0.1 cannot be represented exactly... Jerome -- mailto:jeber...@free.fr http://jeberger.free.fr Jabber: jeber...@jabber.fr signature.asc Description: OpenPGP digital signature
Re: float equality
On 02/20/2011 09:43 PM, Walter Bright wrote: Jonathan M Davis wrote: It may be that you would still end up with situations where two values that you would think would be the same aren't due to rounding error or whatnot. However, with a fixed point value, you wouldn't have the problem where a particular value could not be held in it even if it's within its range of precision. As I understand it, there are a number of values which cannot be held in a floating point and therefore end up being rounded up or down simply because of how floating points work and not because there the precision isn't high enough. That happens with decimal representations, too, just with a different set of numbers. It's definitely true however, that using fractions would be much more accurate for a lot of stuff. That wouldn't be particulary efficient though. Still, if you're doing a lot of math that needs to be accurate, that may be the way to go. The set of irrational numbers cannot (by definition) be represented by a ratio. For example, the square root of 2. This is true, indeed. The difference with decimals and/or rationals is that the set of (exactly) representable numbers is the one of numbers that look like 'finite' to us, like 0.1 (which cannot be represented using binary floats, whatever the size). In addition, rationals allow carrying representations across computations without creating unnecessary rounding errors at each step. Is there any plan for them in D2 in the future? The question of irrationals exists, for sure, but it's not a trap since we cannot ignore/forget they are not representable. (same for infinite rationals like 1/3) Denis -- _ vita es estrany spir.wikidot.com
Re: float equality
Kevin Bealer wrote:
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise.
It cannot represent irrational numbers accurately.
Re: float equality
Jonathan M Davis wrote: It may be that you would still end up with situations where two values that you would think would be the same aren't due to rounding error or whatnot. However, with a fixed point value, you wouldn't have the problem where a particular value could not be held in it even if it's within its range of precision. As I understand it, there are a number of values which cannot be held in a floating point and therefore end up being rounded up or down simply because of how floating points work and not because there the precision isn't high enough. That happens with decimal representations, too, just with a different set of numbers. It's definitely true however, that using fractions would be much more accurate for a lot of stuff. That wouldn't be particulary efficient though. Still, if you're doing a lot of math that needs to be accurate, that may be the way to go. The set of irrational numbers cannot (by definition) be represented by a ratio. For example, the square root of 2.
Re: float equality
bearophile wrote: There's a total lack of evidence for that. MISRA-C standard disallows the equality among FP values. I presume they have some serious evidence for their choices, but I don't know such evidence. Today MISRA-C is one of the most serious attempts at fixing the holes in C language to write more reliable code for the industry. MISRA-C standard is sometimes extreme and I don't suggest to follow everything they say, but I suggest to not totally ignore what it says about many C features. MISRA is not suggesting having the language silently default to approximate equality.
Re: float equality
Jonathan M Davis wrote: There's way too much to worry about. I don't _want_ to have to be worrying about rounding error and that sort of thing. I generally want to do whatever calculations I need to do for a particular task and be done with it. Math doesn't interest me enough for me to want to deal with it more than I have to to solve a particular problem. I find the fact that you have to worry about losing precision with floating point values and that they can't even represent all of the values in their range horrific. It's a fact of life, and you have to deal with it, but it sure isn't pretty. But that's true of any calculation you do, once you get past symbolic math. It's like if you're cutting wood and do not take into account the width of the saw blade. The parts won't fit together right. I was once on a hot rod forum where a guy was making a mold in order to cast his own intake manifold. I suggested to him that he had to make the mold larger than the actual part, in order to account for the shrinkage of hot aluminum down to room temperature. I calculated by about 1/2 inch. He thought I was an idiot, and ignored me. His manifold came out too small by half an inch.
Re: float equality
== Quote from Jonathan M Davis (jmdavisp...@gmx.com)'s article
...
> It may be that you would still end up with situations where two values that
> you
> would think would be the same aren't due to rounding error or whatnot.
> However,
> with a fixed point value, you wouldn't have the problem where a particular
> value
> could not be held in it even if it's within its range of precision. As I
> understand it, there are a number of values which cannot be held in a floating
> point and therefore end up being rounded up or down simply because of how
> floating points work and not because there the precision isn't high enough.
>
> It's definitely true however, that using fractions would be much more accurate
> for a lot of stuff. That wouldn't be particulary efficient though. Still, if
> you're
> doing a lot of math that needs to be accurate, that may be the way to go.
>
> - Jonathan M Davis
Floating point numbers are a good compromise for a lot of purposes, but yeah
they
are limited. Here are some ways to open up the limits a bit...
(First some math for those who haven't gone down this path...)
The reason that fixed point does better than floating is that fixed point
classes
use base ten and floating point uses base 2. In base two, all fractions that
have
a denominator of a power of 2 can be represented (e.g. 3/16) exactly, and all
others
can't.
Fixed point solves this for numbers like .1 or .0003 because the denominator is
a
power of ten. Ten is 5*2 and the way it works is that any denominator that is
a power
of two times a power of ten can be represented exactly.
So you're good for .01 or .039028 because these are (1/100) and
(39028/100). But
neither fixed point nor floating can represent something like 1/3 or 1/11
exactly.
If you used base 3, then 1/3 would be perfectly representable but 1/2 would not
be.
I think this is why the ancients used 360 degrees in a circle... you can
represent
a lot of common fractions as parts of 360, including 1/2, 1/3, 1/4, 1/5, 1/6,
1/8.
But still not 1/7 or 1/11.
How to solve this problem?
You can define a type like this:
struct Rational {
// represents numerator over denominator
... (various arithmetic methods and operators)...
long numerator;
long denominator;
}
(Lots of people have written classes like this, maybe even for D. ;)
If you've taken high school math and CS 101 you can write the methods here, and
then
you can represent any fraction. This is a better way to go for a lot of
problems if
you need exactness, but its a bit slower.
This isn't perfect though ... if you try to add the numbers 1/2, 1/3
1/100
you will eventually find the denominator overflows -- you can check for this and
'reduce' the fraction (remove common divisors from top and bottom) periodically,
but eventually you will even get an irreducable fraction and you will need,
again,
to compromise and give away some precision. The simplest way to do this
compromise
is probably to divide the top and bottom number by 2, try to reduce again, and
then
try the operation again (repeat until the operation doesn't overflow). (Maybe
there
is a better way though.)
For some problems the precision loss will never happen -- for example if you are
adding dollars and cents represented as (x/100) then the denominator never needs
to get bigger than 100. If you need to compute mortgage payments it might get a
little bigger... but I'm not sure if it will ever overflow for this case.
Another source of compromise is when a number is really big or small, for
example
a number like 2^200 or 1/2^200 can be represented in a 'double' but would
overflow
a Rational as defined above.
You can delay the compromise a bit by using a struct like this:
struct Rational2 {
// represents the number numerator / (denominator ^ exponent)
// adjust integer types to taste
long numerator;
long denominator;
long exponent;
};
This lets you go much further, and represent truly astonishingly large and small
numbers. It complicates the math in the arithmetic operators a bit though.
Eventually, even this version will lose precision as the fraction still becomes
irreducable if there are a unique set of prime factors in the denominator. So
this can represent much larger ranges than double or real, but I think it still
can't represent 1/2*1/3*1/1000_000_000 exactly.
Now... what if you wanted to avoid the compromise completely?
You could switch to this:
struct {
BigInt numerator;
BigInt denominator;
};
Bingo -- no compromise. Two down sides to this:
1. You trade off memory for accuracy as the BigInts grow over time, assuming
the fraction is irreducable.
2. It's slower -- BigInt objects are essentially byte strings and even things
like addition require a function call and looping.
This is a pretty flexible solution though. However, numbers like pi and 'e'
can't be represented as a rational number at all, so you are still stuck if
you want to d
Re: float equality
On Sunday 20 February 2011 05:21:12 spir wrote: > On 02/20/2011 06:17 AM, Jonathan M Davis wrote: > > On Saturday 19 February 2011 20:46:50 Walter Bright wrote: > >> bearophile wrote: > >>> Walter: > That'll just trade one set of problems for another. > >>> > >>> But the second set of problems may be smaller :-) > >> > >> There's a total lack of evidence for that. Furthermore, > >> > >> 1. Roundoff error is not of a fixed magnitude. > >> > >> 2. A user may choose to ignore roundoff errors, but that is not the > >> prerogative of the language. > >> > >> 3. What is an acceptable amount of roundoff error is not decidable by > >> the language. > >> > >> 4. At Boeing doing design work, I've seen what happens when engineers > >> ignore roundoff errors. It ain't pretty. It ain't safe. It ain't > >> correct. > > > > Honestly, the more that I learn about and deal with floating point > > numbers, the more I wonder why we don't just use fixed point. Obviously > > that can be a bit limiting for the size of the number (on either side of > > the decimal) - particularly in 32-bit land - but with 64-bit numbers, it > > sounds increasingly reasonable given all of the issues with floating > > point values. Ideally, I suppose, you'd have both, but the CPU > > specifically supports floating point (I don't know about fixed point), > > and I don't think that I've ever used a language which really had fixed > > point values (unless you count integers as fixed point with no digits to > > the right of the decimal). > > I don't see how fixed point would solve common issues with floats. Would > you expand a bit on this? > > For me, the source of the issue is inaccurate and unintuitive > "translations" > > from/to decimal and binary. For instance (using python just for example): > >>> 0.1 > > 0.10001 > > >>> 0.7 > > 0.69996 > > >>> 0.3 > > 0.2 > > To solve this, one may use rationals (representing 0.1 by 1/10) or decimals > (representing decimal digits, each eg by half a byte). Both are costly, > indeed. I may ignore some other points, solved by fixed point arithmetics. It may be that you would still end up with situations where two values that you would think would be the same aren't due to rounding error or whatnot. However, with a fixed point value, you wouldn't have the problem where a particular value could not be held in it even if it's within its range of precision. As I understand it, there are a number of values which cannot be held in a floating point and therefore end up being rounded up or down simply because of how floating points work and not because there the precision isn't high enough. It's definitely true however, that using fractions would be much more accurate for a lot of stuff. That wouldn't be particulary efficient though. Still, if you're doing a lot of math that needs to be accurate, that may be the way to go. - Jonathan M Davis
Re: float equality
On 02/20/2011 06:17 AM, Jonathan M Davis wrote: On Saturday 19 February 2011 20:46:50 Walter Bright wrote: bearophile wrote: Walter: That'll just trade one set of problems for another. But the second set of problems may be smaller :-) There's a total lack of evidence for that. Furthermore, 1. Roundoff error is not of a fixed magnitude. 2. A user may choose to ignore roundoff errors, but that is not the prerogative of the language. 3. What is an acceptable amount of roundoff error is not decidable by the language. 4. At Boeing doing design work, I've seen what happens when engineers ignore roundoff errors. It ain't pretty. It ain't safe. It ain't correct. Honestly, the more that I learn about and deal with floating point numbers, the more I wonder why we don't just use fixed point. Obviously that can be a bit limiting for the size of the number (on either side of the decimal) - particularly in 32-bit land - but with 64-bit numbers, it sounds increasingly reasonable given all of the issues with floating point values. Ideally, I suppose, you'd have both, but the CPU specifically supports floating point (I don't know about fixed point), and I don't think that I've ever used a language which really had fixed point values (unless you count integers as fixed point with no digits to the right of the decimal). I don't see how fixed point would solve common issues with floats. Would you expand a bit on this? For me, the source of the issue is inaccurate and unintuitive "translations" from/to decimal and binary. For instance (using python just for example): 0.1 0.10001 0.7 0.69996 0.3 0.2 To solve this, one may use rationals (representing 0.1 by 1/10) or decimals (representing decimal digits, each eg by half a byte). Both are costly, indeed. I may ignore some other points, solved by fixed point arithmetics. Denis -- _ vita es estrany spir.wikidot.com
Re: float equality
Walter: > There's a total lack of evidence for that. MISRA-C standard disallows the equality among FP values. I presume they have some serious evidence for their choices, but I don't know such evidence. Today MISRA-C is one of the most serious attempts at fixing the holes in C language to write more reliable code for the industry. MISRA-C standard is sometimes extreme and I don't suggest to follow everything they say, but I suggest to not totally ignore what it says about many C features. > 1. Roundoff error is not of a fixed magnitude. I meant to replace the nude FP equality with a function that has the shared number of mantissa bits as third required argument. And then perform the normal FP equality using the "is" operator. Regarding FP rounding errors, eventually it will be good to add to Phobos2 a library for Interval FP arithmetic, with trigonometric functions too, etc: http://en.wikipedia.org/wiki/Interval_arithmetic Bye, bearophile
Re: float equality
On Sunday 20 February 2011 01:20:34 Walter Bright wrote: > Jonathan M Davis wrote: > > Regardless, I'm glad that I don't generally have to do much in the way of > > math- centric programming. > > Why not? I enjoyed it. There's way too much to worry about. I don't _want_ to have to be worrying about rounding error and that sort of thing. I generally want to do whatever calculations I need to do for a particular task and be done with it. Math doesn't interest me enough for me to want to deal with it more than I have to to solve a particular problem. I find the fact that you have to worry about losing precision with floating point values and that they can't even represent all of the values in their range horrific. It's a fact of life, and you have to deal with it, but it sure isn't pretty. - Jonathan M Davis
Re: float equality
Jonathan M Davis wrote: Regardless, I'm glad that I don't generally have to do much in the way of math- centric programming. Why not? I enjoyed it.
Re: float equality
On Saturday 19 February 2011 23:09:51 Walter Bright wrote: > Walter Bright wrote: > > Bottom line is, if you work with numbers, you've got to pay attention to > > roundoff error and other mechanical issues in deriving answers, whether > > you're using a calculator, a computer, a slide rule, by hand, whatever. > > If you don't, you are incompetent. > > I mean "you" generically, not anyone in particular. Sorry 'bout that. I figured as much. Regardless, I'm glad that I don't generally have to do much in the way of math- centric programming. - Jonathan M Davis
Re: float equality
Walter Bright wrote: Bottom line is, if you work with numbers, you've got to pay attention to roundoff error and other mechanical issues in deriving answers, whether you're using a calculator, a computer, a slide rule, by hand, whatever. If you don't, you are incompetent. I mean "you" generically, not anyone in particular. Sorry 'bout that.
Re: float equality
Jonathan M Davis wrote: Honestly, the more that I learn about and deal with floating point numbers, the more I wonder why we don't just use fixed point. Obviously that can be a bit limiting for the size of the number (on either side of the decimal) - particularly in 32-bit land - but with 64-bit numbers, it sounds increasingly reasonable given all of the issues with floating point values. Ideally, I suppose, you'd have both, but the CPU specifically supports floating point (I don't know about fixed point), and I don't think that I've ever used a language which really had fixed point values (unless you count integers as fixed point with no digits to the right of the decimal). Fixed point won't solve your problems. The engineers who got caught by roundoff errors at Boeing were using calculators, not computers. They'd also get caught with roundoff errors using slide rules, graphical methods, and hand calculation. At Caltech, where we used calculators, we would get hammered if we did not carry through all equations symbolically to the final one, and then calculate it. The reason is because often terms drop away doing that, and hence roundoff error is minimized. We'd also get hammered if the numeric answers were not "reasonable", i.e. coming up with negative mass or other nonsense. By hammered, I mean you got an automatic zero on the problem. Bottom line is, if you work with numbers, you've got to pay attention to roundoff error and other mechanical issues in deriving answers, whether you're using a calculator, a computer, a slide rule, by hand, whatever. If you don't, you are incompetent.
Re: float equality
On Saturday 19 February 2011 20:46:50 Walter Bright wrote: > bearophile wrote: > > Walter: > >> That'll just trade one set of problems for another. > > > > But the second set of problems may be smaller :-) > > There's a total lack of evidence for that. Furthermore, > > 1. Roundoff error is not of a fixed magnitude. > > 2. A user may choose to ignore roundoff errors, but that is not the > prerogative of the language. > > 3. What is an acceptable amount of roundoff error is not decidable by the > language. > > 4. At Boeing doing design work, I've seen what happens when engineers > ignore roundoff errors. It ain't pretty. It ain't safe. It ain't correct. Honestly, the more that I learn about and deal with floating point numbers, the more I wonder why we don't just use fixed point. Obviously that can be a bit limiting for the size of the number (on either side of the decimal) - particularly in 32-bit land - but with 64-bit numbers, it sounds increasingly reasonable given all of the issues with floating point values. Ideally, I suppose, you'd have both, but the CPU specifically supports floating point (I don't know about fixed point), and I don't think that I've ever used a language which really had fixed point values (unless you count integers as fixed point with no digits to the right of the decimal). - Jonathan M Davis
Re: float equality
bearophile wrote: Walter: That'll just trade one set of problems for another. But the second set of problems may be smaller :-) There's a total lack of evidence for that. Furthermore, 1. Roundoff error is not of a fixed magnitude. 2. A user may choose to ignore roundoff errors, but that is not the prerogative of the language. 3. What is an acceptable amount of roundoff error is not decidable by the language. 4. At Boeing doing design work, I've seen what happens when engineers ignore roundoff errors. It ain't pretty. It ain't safe. It ain't correct.
Re: float equality
Walter: > That'll just trade one set of problems for another. But the second set of problems may be smaller :-) This happens all the time. Bye, bearophile
Re: float equality
spir wrote: So, maybe, default '==' should perform approxEqual if needed? That'll just trade one set of problems for another.
Re: float equality
On 02/19/2011 04:40 PM, Don wrote:
spir wrote:
On 02/19/2011 01:21 PM, Jonathan M Davis wrote:
On Saturday 19 February 2011 04:06:38 spir wrote:
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4); // pass
assert(-1.1 + 3.3 + 2.2 == 4.4); // fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
== is completely consistent. The floating point values in question are _not_
equal. The problem is that floating point values are inexact. There are values
which a floating point _cannot_ hold, so it holds the closest value that it
can.
This can quickly and easily lead to having two floating points which seem like
they should be equal but which aren't. == checks that they're equal, which is
exactly what it should do (especially in a systems language).
I know about inexact representation, due to bin <--> dec conversion.
Well, about consistency, I meant cases 2 vs 3.
The thing is, of course, that actual equality sucks for floating point values.
What's really needed in something like approxEquals. So, approxEquals really
_should_ be used for comparing floating points and == should only be used on
them
when that is actually what you need (which would be extremely rare). However,
you can't really make == do approxEquals, because that means that == isn't
really exact equality anymore and because then it makes it hard to actually
compare floating point values for equality if that's what you actually want.
Making == do approxEquals instead would make == inconsistent, not consistent.
This is just one of those cases where the programmer needs to know what they're
doing and not use ==.
Right, that's what I thought. I guess what people mean in 99% cases is
"conceptual" equality, right?
They don't care about internal representation if they are not forced to. So,
maybe, default '==' should perform approxEqual if needed?
Approximate equality is not problem-free, either. You still need to know what
you're doing. Regardless of what tolerance you use, there will be values which
are just inside and just outside the tolerance, which in pure mathematics would
be equal to each other.
And we may have a binEqual just for
floats when someone really needs binary repr equality? (I cannot impagine a
use case, but well...)
There are actually a lot of use cases. Most obviously in unit tests, where
you're testing for exact rounding. There is only one correct answer.
What do you think?
I any case, what value should one use as tolerance range for approxEqual?
That's exactly the problem! There is no good universal default value.
My very first contribution to D was a floating point function called
approxEqual, but someone insisted that it be called 'feqrel'. Which has pretty
much ensured that everyone misses it. :(
It defines approximate equality in the correct way, in terms of numbers of
significant bits. Basically, every time you have a multiply, you lose one bit.
Addition can lose multiple bits.
Man, but why do people chose names like that? Isn't programming hard enough
that people think we must add difficulty on difficulty?
Right, anyway, I think that's what I'm looking for.
Doc says:
int feqrel(X)(X x, X y);
To what precision is x equal to y?
Returns:
the number of mantissa bits which are equal in x and y.
eg, 0x1.F8p+60 and 0x1.F1p+60 are equal to 5 bits of precision.
Special Values
x y feqrel(x, y)
-
x x real.mant_dig
x >= 2*x 0
x <= x/2 0
NAN any 0
any NAN 0
How is it intended to be used? Is the user supposed to count the number of
(expected) remaining correct bits, depending on kind and number of operations?
Note that I'm aware of the issues caused by binary representations. But I
thought there were well-known methods to deal with them; and that modern
languages would implement them ;-).
In particuliar, I imagined there would be a way to somehow detect (thank to
increased internal precision) that an intermediate or final result is actually
(exactly/mathematically) equal to a finite precision decimal number, while its
binary representation is approximate. (I know some languages use some heuristic
for that, intended for string output, so that one gets eg '0.1' instead of,
say, '0.07').
FLoats are an infinite source of mess... We definitely need decimals, don't we;
or maybe rationals?
(I will let floats alone, waiting for processor manufacturors to implement
decimal computation in hw.)
Denis
--
_
vita es estrany
spir.wikidot.com
Re: float equality
On 2/19/11 8:30 PM, spir wrote: Right, but then what are we supposed to do? Just check for strict equality if you are sure you need it, and use a fuzzy comparison in all other cases? Sorry, I don't quite get your problem… David
Re: float equality
On 02/19/2011 02:41 PM, bearophile wrote: spir: Well, about consistency, I meant cases 2 vs 3. Floating point values are not the real values you are used in mathematics. They are approximations, and generally for them associative and distributive properties don't hold: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems Here you find more than enough about FPs: http://cr.yp.to/2005-590/goldberg.pdf I guess what people mean in 99% cases is "conceptual" equality, right? D is designed to behave like C when you write code that's compatible with C syntax. So you need to turn the normal equality into a syntax error and add another one if you want to change something. They don't care about internal representation if they are not forced to. In a system language you are often supposed to care for the internal implementations too. And we may have a binEqual just for floats when someone really needs binary repr equality? (I cannot impagine a use case, but well...) There are few use cases, but they are not many. I any case, what value should one use as tolerance range for approxEqual? I presume there are no good defaults for the number of shared bits. Right, but then what are we supposed to do? For instance, if I have a whole binch of float actual results from a testsuite, how am I supoose to match them against expected results? (That's exactly the case which made me bump into the issue; thought my code was buggy, but the problem actually comes from float '=='.) How to solve that sensibly? Denis -- _ vita es estrany spir.wikidot.com
Re: float equality
spir wrote:
On 02/19/2011 01:21 PM, Jonathan M Davis wrote:
On Saturday 19 February 2011 04:06:38 spir wrote:
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4);// pass
assert(-1.1 + 3.3 + 2.2 == 4.4);// fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
== is completely consistent. The floating point values in question are
_not_
equal. The problem is that floating point values are inexact. There
are values
which a floating point _cannot_ hold, so it holds the closest value
that it can.
This can quickly and easily lead to having two floating points which
seem like
they should be equal but which aren't. == checks that they're equal,
which is
exactly what it should do (especially in a systems language).
I know about inexact representation, due to bin <--> dec conversion.
Well, about consistency, I meant cases 2 vs 3.
The thing is, of course, that actual equality sucks for floating point
values.
What's really needed in something like approxEquals. So, approxEquals
really
_should_ be used for comparing floating points and == should only be
used on them
when that is actually what you need (which would be extremely rare).
However,
you can't really make == do approxEquals, because that means that ==
isn't
really exact equality anymore and because then it makes it hard to
actually
compare floating point values for equality if that's what you actually
want.
Making == do approxEquals instead would make == inconsistent, not
consistent.
This is just one of those cases where the programmer needs to know
what they're
doing and not use ==.
Right, that's what I thought. I guess what people mean in 99% cases is
"conceptual" equality, right?
They don't care about internal
representation if they are not forced to. So, maybe, default '==' should
perform approxEqual if needed?
Approximate equality is not problem-free, either. You still need to know
what you're doing. Regardless of what tolerance you use, there will be
values which are just inside and just outside the tolerance, which in
pure mathematics would be equal to each other.
And we may have a binEqual just for
floats when someone really needs binary repr equality? (I cannot
impagine a use case, but well...)
There are actually a lot of use cases. Most obviously in unit tests,
where you're testing for exact rounding. There is only one correct answer.
What do you think?
I any case, what value should one use as tolerance range for approxEqual?
That's exactly the problem! There is no good universal default value.
My very first contribution to D was a floating point function called
approxEqual, but someone insisted that it be called 'feqrel'. Which has
pretty much ensured that everyone misses it. :(
It defines approximate equality in the correct way, in terms of numbers
of significant bits. Basically, every time you have a multiply, you lose
one bit. Addition can lose multiple bits.
Re: float equality
spir: > Well, about consistency, I meant cases 2 vs 3. Floating point values are not the real values you are used in mathematics. They are approximations, and generally for them associative and distributive properties don't hold: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems Here you find more than enough about FPs: http://cr.yp.to/2005-590/goldberg.pdf > I guess what people mean in 99% cases is "conceptual" equality, right? D is designed to behave like C when you write code that's compatible with C syntax. So you need to turn the normal equality into a syntax error and add another one if you want to change something. > They don't care about internal representation if they are not forced to. In a system language you are often supposed to care for the internal implementations too. > And we may have a binEqual just for floats when someone really needs > binary repr equality? (I cannot impagine a use case, but well...) There are few use cases, but they are not many. > I any case, what value should one use as tolerance range for approxEqual? I presume there are no good defaults for the number of shared bits. Bye, bearophile
Re: float equality
On 02/19/2011 01:21 PM, Jonathan M Davis wrote:
On Saturday 19 February 2011 04:06:38 spir wrote:
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4);// pass
assert(-1.1 + 3.3 + 2.2 == 4.4);// fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
== is completely consistent. The floating point values in question are _not_
equal. The problem is that floating point values are inexact. There are values
which a floating point _cannot_ hold, so it holds the closest value that it can.
This can quickly and easily lead to having two floating points which seem like
they should be equal but which aren't. == checks that they're equal, which is
exactly what it should do (especially in a systems language).
I know about inexact representation, due to bin <--> dec conversion.
Well, about consistency, I meant cases 2 vs 3.
The thing is, of course, that actual equality sucks for floating point values.
What's really needed in something like approxEquals. So, approxEquals really
_should_ be used for comparing floating points and == should only be used on
them
when that is actually what you need (which would be extremely rare). However,
you can't really make == do approxEquals, because that means that == isn't
really exact equality anymore and because then it makes it hard to actually
compare floating point values for equality if that's what you actually want.
Making == do approxEquals instead would make == inconsistent, not consistent.
This is just one of those cases where the programmer needs to know what they're
doing and not use ==.
Right, that's what I thought. I guess what people mean in 99% cases is
"conceptual" equality, right? They don't care about internal representation if
they are not forced to. So, maybe, default '==' should perform approxEqual if
needed? And we may have a binEqual just for floats when someone really needs
binary repr equality? (I cannot impagine a use case, but well...)
What do you think?
I any case, what value should one use as tolerance range for approxEqual?
Denis
--
_
vita es estrany
spir.wikidot.com
Re: float equality
Jonathan M Davis: > The thing is, of course, that actual equality sucks for floating point > values. So much, that some people have proposed to deprecate the normal FP equality. MISRA-C disallows them. When you see a == among FP values is often a code smell. Bye, bearophile
Re: float equality
On 19.02.2011 13:06, spir wrote:
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4); // pass
assert(-1.1 + 3.3 + 2.2 == 4.4); // fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
Denis
It is generally a bad idea to use "==" with floats, as most decimals
cant be represented exact in binary floating point. That means there
is no float for the value "1.1". Though there are exact floats for
0.25, 0.5, 42.125 and so on. The only reason the first two testcases
work is, that it is rounded the same way both sides of the "==" but
you should not rely on that.
Also note, that these calculations are probably done at compile-time,
and the compiler is allowed to use a higher precision than at
run-time, so you might get different result when you let the user
input the numbers.
- Krox
Re: float equality
On Saturday 19 February 2011 04:06:38 spir wrote:
> Hello,
>
> What do you think of this?
>
> unittest {
> assert(-1.1 + 2.2 == 1.1); // pass
> assert(-1.1 + 2.2 + 3.3 == 4.4);// pass
> assert(-1.1 + 3.3 + 2.2 == 4.4);// fail
> assert(-1.1 + 3.3 == 2.2); // fail
> }
>
> There is approxEquals in stdlib, right; but shouldn't builtin "==" be
> consistent anyway?
== is completely consistent. The floating point values in question are _not_
equal. The problem is that floating point values are inexact. There are values
which a floating point _cannot_ hold, so it holds the closest value that it
can.
This can quickly and easily lead to having two floating points which seem like
they should be equal but which aren't. == checks that they're equal, which is
exactly what it should do (especially in a systems language).
The thing is, of course, that actual equality sucks for floating point values.
What's really needed in something like approxEquals. So, approxEquals really
_should_ be used for comparing floating points and == should only be used on
them
when that is actually what you need (which would be extremely rare). However,
you can't really make == do approxEquals, because that means that == isn't
really exact equality anymore and because then it makes it hard to actually
compare floating point values for equality if that's what you actually want.
Making == do approxEquals instead would make == inconsistent, not consistent.
This is just one of those cases where the programmer needs to know what they're
doing and not use ==.
- Jonathan M Davis
float equality
Hello,
What do you think of this?
unittest {
assert(-1.1 + 2.2 == 1.1); // pass
assert(-1.1 + 2.2 + 3.3 == 4.4);// pass
assert(-1.1 + 3.3 + 2.2 == 4.4);// fail
assert(-1.1 + 3.3 == 2.2); // fail
}
There is approxEquals in stdlib, right; but shouldn't builtin "==" be
consistent anyway?
Denis
--
_
vita es estrany
spir.wikidot.com
