George Hartzell writes: > I've reduced a problem that I'm having to the following test case. > > I insert two rows into an rtree table. In the first case when I > select chromStart and chromEnd I get the same values as I inserted. > In the second case the chromEnd is 1 greater than what I inserted. In > the third case chromStart is 1 les than what I inserted. > > I'm using a copy of sqlite-3.6.18 that I compiled myself, using > defaults except for rtree support, on a Suse SLES 10 system. > > It seems odd. Are the values large enought that they're causing > problems with the 32-bit floats? It almost seems like odd numbers are > rounded up/down depending on whether they're min or max? > > Any help would be greatly appreciated. > > Thanks, > > g. > > -- --------------- snip ---------------- > > create virtual table snp130_rtree > using rtree > (snp130_rowid, > chromMin, chromMax, > strandMin, strandMax, > chromStart, chromEnd); > > insert into snp130_rtree > (snp130_rowid, > chromMin, chromMax, > strandMin, strandMax, > chromStart, chromEnd) > values > (1, 1, 1, 1, 1, 1, 2); > insert into snp130_rtree > (snp130_rowid, > chromMin, chromMax, > strandMin, strandMax, > chromStart, chromEnd) > values > (2, 1, 1, 1, 1, 19201046, 19201047); > insert into snp130_rtree > (snp130_rowid, > chromMin, chromMax, > strandMin, strandMax, > chromStart, chromEnd) > values > (3, 1, 1, 1, 1, 19201045, 19201046); > > select * from snp130_rtree; > > -- --------------- snip ---------------- > > >>~/src/sqlite-3.6.18/sqlite3 ./bug.db < bug.sql > 1|1.0|1.0|1.0|1.0|1.0|2.0 > 2|1.0|1.0|1.0|1.0|19201046.0|19201048.0 > 3|1.0|1.0|1.0|1.0|19201044.0|19201046.0 > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > >
The following small C program seems to indicate that the problem is that my numbers are too large for a 32-bit float. Is this the correct interpretation? A quick glance through the source code suggests that it would be possible to build a version that stored coord's as either doubles or ints, but in the integer case it seems like care would need to be taking when computing areas and such. g. #include <stdio.h> int main(int argc, char **argv) { float f1; float f2; float f3; f1 = 19201045; f2 = 19201046; f3 = 19201047; printf("f1 is %f\n", f1); printf("f2 is %f\n", f2); printf("f3 is %f\n", f3); } ~>>./foo f1 is 19201044.000000 f2 is 19201046.000000 f3 is 19201048.000000 ~>> _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users