#475: r.stats: last bin always has a single cell -----------------------+---------------------------------------------------- Reporter: hamish | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-develbranch6 Resolution: | Keywords: r.stats Platform: All | Cpu: All -----------------------+---------------------------------------------------- Comment (by glynn):
Replying to [comment:3 hamish]: > request for help solving this critical bug: we are reporting bad statistical data from our core stats module. It is used by r.report, d.histogram, etc and on to users' work output. > > I'm horrified by this bug but unfamiliar with hash tables and don't know how to proceed with debugging. I think that you're looking in the wrong place. This appears to be a problem with the quantisation semantics. In your test case, r.stats calls {{{ G_quant_add_rule(&q, 1061.064087, 1846.743408, 1, 10) }}} It then assigns the quantisation rules to the input map. Cell values are obtained using G_get_c_raster_row(), which converts the map's FP values to integer CELL values via G_quant_get_cell_value(). Given the above quantisation rule, it converts FP values as follows: {{{ > print G_quant_get_cell_value(&q, 1846.743408) $15 = 10 > print G_quant_get_cell_value(&q, 1846.743407) $17 = 9 > print G_quant_get_cell_value(&q, 1846.743409) $18 = -2147483648 }}} Essentially, G_quant_add_rule() treats the ranges as half-open, i.e. the values range from low (inclusive) to high (exclusive). While half-open ranges are a common concept (e.g. floor() behaves the same way), the range of a GRASS raster is closed, i.e. both the low and high values are inclusive. I suggest changing r.stats to use cHigh=nsteps+1, and truncate the quantised values into range (fudging dHigh may seem simpler, but the maximum value may be positive, negative or zero, and the range may be 1e+300 or 1e-300, so choosing a suitable fudge factor isn't straightforward). -- Ticket URL: <http://trac.osgeo.org/grass/ticket/475#comment:4> GRASS GIS <http://grass.osgeo.org>
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev