#335: export floats and doubles with correct precision -----------------------+---------------------------------------------------- Reporter: hamish | Owner: grass-dev@… Type: task | Status: new Priority: critical | Milestone: 6.4.4 Component: Default | Version: svn-develbranch6 Keywords: precision | Platform: All Cpu: All | -----------------------+----------------------------------------------------
Comment(by hamish): > > %.15,7g was chosen because it's perfectly reproducible > > No, it is not. %.17,9g is reproducible. See IEEE 754 standard. I didn't mean reproducible as far as the binary stored value round-trip was concerned, I meant reproducible as far as getting the same ascii result on two different hardware platforms. for example, getting r.md5sum (addon to aid the test_suite) to give the same (portable) result when checksumming r.out.ascii output when the data is stored as floating point &/or not strictly real (2.5000000). My understanding, for what it is, is that the least sig. digits can flicker depending on the CPU arch, perhaps the compiler, and programming language's implementation too. I can't remember which right now, but one of the ogr/gdal/proj4 input streams was introducing .00000000001 style artifacts, it's probably worth digging that up as a real-world test case example as there are probably two classes of this problem with the same symptom: one to do with GRASS's use of %f and atof() in places, and another to do with the output of input values which were malformed before they came into grass. Another interesting corner-case example to try with this is the earthquake demo from the grass-promo/tutorials/ svn, where the heavily logarithmic data scale within a single dataset really stretches what fits well in the ieee FP model, and so something more linear like magnitude is used to store it instead of raw extajoules. > A perfectly valid range of double fp data is e.g. 1.0e-200 > to 1.0e-300, way smaller than GRASS_EPSILON. You do not want to > corrupt these data. (see the following post with the `2^1024` example where I corrected myself about that after seeing how you fixed the error in my earlier attempt to make sure the color min/max rule was bigger than the data range) regards, Hamish -- Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:26> GRASS GIS <http://grass.osgeo.org> _______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev