Below follow details about CELL and DCELL datatypes in GRASS. It would
be good to have them summarrised in GRASS raster intro IMHO; + FCELL
specific notes. I'm not competent - anybody please do.
questions by me, Glynn replies:
CELL maps are limited to 32-bit integers. There doesn't seem much
point in extending r.mapcalc to handle anything larger; too much work
for too little reward.
something bizzare to me:
$ r.mapcalc 'map=99999999999999999999999.0'
$ r.info -rt map
min=99999999999999991611392.000000
max=99999999999999991611392.000000
datatype=DCELL
A double has a precision of ~16 decimal digits, which matches what you
see above.
So processing any number with more than 16 decimal diggits in r.mapcalc
must yield such "strange" values?
The precision of a "double" corresponds to roughly 16 significant
decimal digits (it's exactly 52 binary digits).
The problem arises when something (in this case, r.info) tries to
print numbers where the number of digits to the left of the decimal
point exceeds the precision.
If you use exponential notation, you can limit the number of digits to
match the precision, so the problem doesn't arise.
And how many diggits after the decimal
separator are safe?
Floating point numbers have a fixed relative error rather than a fixed
absolute error. The issue is the number of significant digits. The
position of the decimal point doesn't matter.
Does that mean that actually in the raster the value is something else
than what r.info reports?
What is in the raster is a binary floating point value:
http://en.wikipedia.org/wiki/IEEE_floating-point
E.g. 3.40282e38 is stored as
(9007189542424620/(2^53))*(2^128)
= 9007189542424620*(2^(128-53))
= 340282000000000014192072600942972764160
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
