Jenny Turner wrote: > I have a raster (FCELL ) that has a minimum value of 0.1312103122
The data is stored in binary floating point, and 0.1312103122 isn't exactly representable in that format. The closest representable value is 0.0010cb7fe (hex) = 0.13121031224727630615234375 (decimal). FCELL (single-precision "float") has just under 7 significant digits of precision. More precisely, it has 24 binary digits, resulting in an "epsilon" of ~1.2e-7. The next representable value below the above value is 0.0010cb7fc (hex) = 0.1312102973461151123046875 (decimal), while the next value above it is 0.0010cb800 (hex) = 0.1312103271484375 (decimal) IOW, any value whose decimal expansion begins with 0.13121031 will have the same binary representation, and thus all such values will be indistinguishable. > and If I > try to have an image with just those values > r.mapcalc "minim= if(image==0.1312103122,10,0)" I get an all Zero image. This is to be expected. The decimal constant will be parsed as double precision then converted to single precision. The result may not be exactly equal to the value stored in the map. In general, you shouldn't perform equality comparisions between floating-point values unless you are certain that no rounding errors can be introduced. > Then I opened the minim metadata and i see that in comments I have: > if(clustered.1 == 0.13121031, 10, 0) > r.mapcalc has eliminated the last 2 digits. Floating-point constants are printed with "%.8g", which exceeds the precision of FCELL. Printing more digits would typically be pointless, as it's highly unlikely that the data will actually accurate to a relative error of 1e-7 in the first place. > How can I avoid this to happen? Replace the equality check with a range check, e.g.: if(abs(image-0.13121031) <= 1e-8,... -- Glynn Clements <gl...@gclements.plus.com> _______________________________________________ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user