On Feb 2, 2010, at 6:32 PM, David Joyner wrote:
On Tue, Feb 2, 2010 at 9:26 PM, Dima Pasechnik <dimp...@gmail.com>
wrote:
...
sage: abs(M.determinant())
Expected:
712483534798848
Got:
7.12483534798848e14
I vaguely remember that M.det() is only well-defined up to sign.
You could rewrite the test to avoid abs, but I'm wondering if abs is
the
reason scientific notation is used here. Do you know?
I think using abs is the clearest for something that's only defined up
to sign. As for the scientific notation, what's it returning here? Try
putting parent(M.det()) or type (M.det()) into the doctest. The
absolute value of integers *should* return integers...
- Robert
sage: z = -3**500; z
-36360291795869936842385267079543319118023385026001623040346035832580600191583895484198508262979388783308179702534403855752855931517013066142992430916562025780021771247847643450125342836565813209972590371590152578728008385990139795377610001
sage: abs(z)
36360291795869936842385267079543319118023385026001623040346035832580600191583895484198508262979388783308179702534403855752855931517013066142992430916562025780021771247847643450125342836565813209972590371590152578728008385990139795377610001
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
