stupid question alert.... has anyone run across this kind of behavior before in r.mapcalc?
i have these maps with absurdly large values and i am trying to pull out the pixels whose values are below some threshold. the original map has floating point values. i was/am using integer values for the theshold. for small thresholds (e.g., less than 2,000,000,000, i.e., 2 billion) everything seemed hunky-dory. however, when i moved up to 3 billion, it wouldn't pull any out. for some thresholds above that, it would generate strange results, others would just do the no-pixels thing. i have written a little script to show and reproduce in excruciating detail what the strange behavior is. here is the basic summary. when i do a "less than" comparison with the large integer, i get no pixels even though my threshold is well below the maximum value. when i do a "greater than" comparison (which should give me the complement), i get some pixels. actually, i get all of the non-null pixels regardless of whether they are above or below the threshold. when i do the "less than" using 3 billion point zero (3000000000.0) instead of 3 billion (3000000000), it pulls out some pixels. when i do the "greater than" using the float, it pulls out some other pixels. using the floating point versions seems to add up in that the number of greater-than pixels plus the number of less-than pixels adds up to the total number of non-null pixels. i'm guessing this has to do with some sort of overflow when i try to use an integer larger than the memory representation is intended to handle... one can observe the script that demonstrates this strange behavior at https://netfiles.uiuc.edu/rdrobert/www/bug_demo.sh the output can be seen at https://netfiles.uiuc.edu/rdrobert/www/bug_output.txt and if you are really a masochist, i could try to send copies of the data by request, but i doubt anyone is that bored... peace, ricky robertson _______________________________________________ grass-user mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-user
