Paul Moore writes:
> OK, so to me, 1.0 / 0.0 *is* a "true infinity" in that sense. If you
> divide "one" by "zero" the only plausible interpretation - if you
> allow infinity in your number system - is that you get infinity as
> a result. Of course that just pushes the question back to whether
> you mean (mathematical) "one" and "zero" when you make that
> statement.
The answer is yes. I can say so authoritatively :-) because I
introduced the term "true infinity" into this thread to distinguish
those results of computations done in the extended real system result
in infinity from those results of computations in the extended real
system which result in finite values, but overflow the floating point
representation. Both kinds of values are represented by inf in the
Python (or IEEE 754) floating point number system.
This is not a philosophical question, as you seem to think. It's
purely terminological, an abbreviation for the long sentence above. I
thought that would be obvious, but I guess I have a Dutch ancestor I
didn't know about. ;-) I apologize for not being more careful.
> I get that Ben is actually saying "things that round to 1" and "things
> that round to 0", but at that point error propagation and similar
> details come into play, making even equality a somewhat vague
> concept.
But that's the genius of IEEE 754: they don't come into play. IEEE
754 is a system of arithmetic that is internally consistent. Equality
of IEEE 754 floats is well-defined and exact, as are all IEEE 754
calculations.
The value of IEEE 754 to Python programmers is that for a very wide
range of calculations the IEEE 754 value is tuned to be close to the
value of the calculation using (extended) reals (and cast to float at
the end of the calculation rather than the beginning). (For example,
"round to nearest, ties to even" usually increases that likelihood by
conserving precision.) It's true that once we want to take advantage
of "closeness" we may need to do error propagation analysis, and we
almost always want to avoid equality comparisons. But in many cases
(eg, one matrix multiplication, no matter how large the matrices)
there is no point in error analysis. You either get over/underflow,
or you don't, and you check that at the end.
On the other hand, this probability of closeness goes way down once
inf enters the calculation. The odds that an inf result in the IEEE
number system is not only finite in the extended real number system,
but in fact representable as an IEEE float, are unpleasantly high.
> > He's talking about the arithmetic of floating point numbers, which
> > involves rounding rules intended to model the extended real line.
>
> Again OK, but in the face of rounding rules, many "obvious"
> mathematical properties, such as associativity and distributivity,
> don't hold. So we have to be *very* careful when inferring
> equivalences like the one you use below, a * b = c => 1.0 / (a * b) =
> 1.0 / c.
But that's not a problem (unless maybe you're within a few ULP of
sys.float_info.max or sys.float_info.min). For the examples we're
talking about the problem is entirely that IEEE 754 defaults to
returning inf for a calculation where Python raises:
Division by zero: an operation on finite operands gives an exact
infinite result, e.g., 1/0 or log(0). By default, returns
±infinity.
> Note that I'm not saying here that Python's behaviour might not be
> inconsistent, but rather that it's not obvious (to me, at least) that
> there is an intuitively consistent set of rules
IEEE 754 is strong evidence that there is no such set of rules, at
least for fixed width float representations. IEEE 754 is internally
consistent, of course (I trust Kahan that far! ;-), but consistency
with intuition is as hard as predicting the future.
> so Python has chosen one particular way of being inconsistent,
But that's an important aspect of the examples Ben presents: Python
has chosen like half a dozen ways of being inconsistent, some of which
are not even monotonic (I forget the exact example, but there was a
case where as the argument value increases, you go from inf to raise
and then back to inf, or some such sequence).
That kind of thing means that if you are serious about doing large
numbers of computations you have to handle a number of different
exceptions during the loops, as well as cleaning up the inf/nan mess
after the loops.
> practical considerations like "raising an exception is more user
> friendly for people who don't understand IEEE-754".
The advocates haven't presented any such arguments for copying
math.inf (and I assume math.nan) to the builtins.
> > > 3. Can you give an example of 1.0/0.0 *not* being a "true infinity"? I
> > > have a feeling you're going to point to denormalised numbers close to
> > > zero, but why are you expressing them as "0.0"?
> >
> > Not subnormal numbers, but non-zero numbers that round to zero.
[examples omitted]
> See above. [presumably refers to issues of associativity failures
> and error propagation]
I don't understand your point. This is a completely unnecessary
inconsistency. It's not a matter of "error analysis" or failure of
associativity, it's a matter of following the IEEE rules and handling
the unrepresentable result one way when underflow and another way when
overflow.
>From a pragmatic point of view, it becomes very important to the
Python programmer to study the data (which they may not have yet!) and
decide to write 1/(x**2) as 1 / (x * x) or as (1 / x) * (1 / x).
Which one they choose determines whether they get infs (and fast
computation) or exceptions (and slow or no computation).
> > > 4. Can you provide the proofs you claim exist?
> >
> > Proofs of what? All of this is just straightforward calculations in
> > the extended real line, on the one hand, in IEEE 754 floats on the
> > other hand, and in Python, on the gripping hand.[1]
>
> You claimed that "1e1000 *provably* is not a true infinity". How would
> you prove that?
By assertion. It was my term in the first place. :-)
> Typing it into Python gives
>
> >>> 1e1000
> inf
> That's a proof that in Python, 1e1000 *is* a true infinity.
No, that's a demonstration that Python follows the IEEE rules and
*represents results that overflow the float representation as inf*.
My term, my rules. :-)
> On the extended real line, 1e1000 is not a true infinity because
> they are different points on the line. In IEEE 754 I'd have to
> check the details, but I think someone posted that the standard
> defines parsing rules such that 1e1000 must translate to infinity.
I think you're thinking of one of Steven d'Aprano's posts. Steven's
post can be read that way, but I'm pretty sure the standard doesn't
say "must", but provides returning inf as one option for handling the
overflow exception.
> So when I asked for your proof, I was trying to determine which set
> of rules you were using.
Mine. :-)
> > I think they're pretty disastrous. I'd be much happier with *either*
> > consistently raising on non-representable finite results, or
> > consistently returning -inf, 0.0, inf, or nan. That's not the Python
> > we have.
>
> I think we could have rules that are easier to reason with, but I
> doubt anyone but specialists are ever likely to care much.
Somebody who gets inf in a test calculation, adds an equality check
(that's why we need inf in the builtins, right?) and someday gets an
uncaught ZeroDivisionError on a production machine is gonna care. No?
> 1.0 / 2.0 is 0.5. No question there.
> 1 / 2 is *also* 0.5. So integer division can give float values. And
> should give "the same result" (mathematically).
> 1.0 / 0.0 is being debated, but let's say we agree that it should
> produce inf, because that's IEEE-754.
> Now, what about 1 / 0? Do we violate the principle that led to 1 / 2
> being 0.5? Or do we allow this to generate a floating infinity?
> 1 // 0 has to give ZeroDivisionError, as there's no possible *integer*
> result. But now I find myself uncomfortable that 1//0 raises an
> exception but 1/0 doesn't.
I don't have any problem with 1 // 0 raising and 1 / 0 returning inf
because Python is not Lisp[1]: x / y *always* gives a float result.
1 / 0 "should" obey float conventions. Users need to know that floats
are weird in a host of ways, so making users learn that 1 / 0 does
something different from 1 // 0 doesn't seem like a deal-breaker. If
I were designing Python numerics for Python 4.0 ... but I'm not.
<shudder/>
> And as soon as we start considering integer division, we're talking
> about breaking a *vast* amount of code.
Yeah, I'm ok with *not* breaking that code.
> But extended reals are only one thing you might be trying to model
> with Python's floating point. Non-specialists are more likely to have
> an (imperfect) mental model of a calculator. Which typically errors on
> divide by zero.
I'm fine with that ... but then I want overflow to error too. Those
users are going to be surprised that code with no zeros in it anywhere
generates ZeroDivisionError. Once again, I'm ok with *not* breaking
existing code, but this is an inconsistency that is not implied by
IEEE 754 -- we chose it for ourselves.
> Python floats exist in a context where they need to interact
> "consistently" with other types. So there's a tension between
> precisely modeling IEEE-754 and fitting with the rest of the
> language.
I've seen no evidence that there is any such tension. The tension
that exists is seems to all be based on backward compatibility with
past implementations of the float type, with the single exception of
integer float division, where the possibility that 1 // 0 raises and
1 / 0 returns inf would be in some tension.
> > [R]eturning inf in the case of overflow (and continuing the
> > computation), and raising on zero division (stopping the
> > computation and getting the attention of the user), [might be]
> > the pragmatic sweet spot.
>
> That's probably my main point, coupled with the points that "division,
> as a language built in operator, is very different from functions in
> the math module", and "division is polymorphic, and consistent
> behaviour across types is an extra consideration that math functions
> don't need to be concerned about".
I'm fine with the "pragmatic sweet spot", if that's what a vocal
minority wants and a passive majority shrugs off, or simply is needed
for backward compatibility.
But I still do not see what practical use there is in putting inf in
the builtins, except saving a few keystrokes for people who generally
will not know what they're doing. (Those who do will mostly use NumPy
and save themselves a ton of grief.) Specifically, unless your
calculations are *extremely* limited, if you worry about inf, you
should worry about nan. It is by definition useless to test for
equality with nan. So you need to import isnan if you're going to
check for overflows and underflows after a complicated computation.
isinf saves you a call to abs, or checking both inf and -inf. And
whatever you do about the special float values, you still need to trap
ZeroDivisionErrors.
Footnotes:
[1] Some Lisps (Common? Scheme?) don't distinguish 1.0 from 1, or 1/1
from 1, or 1.0 + 0.0j from 1, etc. In each case the value is
converted to integer if possible, or to a real ratio or real float if
the imaginary part is zero.
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/HJA3WLQSUF75YVMVKUMPOGZ2M3BMSOS7/
Code of Conduct: http://python.org/psf/codeofconduct/
