[sqlite] CONFIG_FPE_FASTFPE and the text representation of a float
L.S. Here's the patch for those interested: --- sqlite-3.4.2-orig/src/printf.c 2007-06-28 14:46:19.0 +0200 +++ sqlite-3.4.2/src/printf.c 2007-10-08 11:56:49.0 +0200 @@ -158,7 +158,11 @@ static int et_getdigit(LONGDOUBLE_TYPE *val, int *cnt){ int digit; LONGDOUBLE_TYPE d; +#ifdef SQLITE_FPE_FASTFPE + if( (*cnt)++ >= 10 ) return '0'; +#else if( (*cnt)++ >= 16 ) return '0'; +#endif /* SQLITE_FPE_FASTFPE */ digit = (int)*val; d = digit; digit += '0'; and diff -u -r sqlite-3.4.2-orig/src/vdbemem.c sqlite-3.4.2/src/vdbemem.c --- sqlite-3.4.2-orig/src/vdbemem.c 2007-06-28 14:46:19.0 +0200 +++ sqlite-3.4.2/src/vdbemem.c 2007-10-08 11:58:07.0 +0200 @@ -217,7 +217,11 @@ sqlite3_snprintf(NBFS, z, "%lld", pMem->u.i); }else{ assert( fg & MEM_Real ); +#ifdef SQLITE_FPE_FASTFPE +sqlite3_snprintf(NBFS, z, "%!.9g", pMem->r); +#else sqlite3_snprintf(NBFS, z, "%!.15g", pMem->r); +#endif /* SQLITE_FPE_FASTFPE */ } pMem->n = strlen(z); pMem->z = z; FYI, the (latest?) 2.6 kernels seem to have ditched FASTFPE completely, so the above is probably a 2.4 - issue only. -- Best, Frank. - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] CONFIG_FPE_FASTFPE and the text representation of a float
L.S. Ouch, bad copy&paste, obviously log(2^33) ~= 9.934 => 10 digits Drh, can I wrap up this stuff and send you the patch or did other locations come to mind that need checking? -- Best, Frank. - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] CONFIG_FPE_FASTFPE and the text representation of a float
L.S. As per earlier posts: I'm using Sqlite on an Arm architecture without FP-unit running a kernel that has FASTFPE built in. After solving the mixed-endian issue, it turns out there's yet another problem when one is looking at floating point numbers in a text representation.. A number like 1.24, which is not exactly representable, will be presented still by Sqlite as '1.24', but the underlying 'magic' fails in the situation of FASTFPE, simply because the assumptions made on accuracy are obviously 'wrong' then ;( * a regular 64 bit float has 52+1 bits for the mantissa and thus an estimated number of accurate digits of log(2^53) ~= 15.955 => 16 digits * a fastfpe float has 32+1 bits for the mantissa and thus an estimated number of accurate digits of log(2^33) ~= 15.955 => 10 digits So, I've been trying to find the locations for this 'magic' and came up with the following two: src/printf.c:158 contains et_getdigit() with line 161: if( (*cnt)++ >= 16 ) return '0'; in which I'd need to change the 16 into 10. src/vdbemem.c:201 contains sqlite3VbdeMemStringify() with line 220: sqlite3_snprintf(NBFS, z, "%!.15g", pMem->r); in which I'd need to change the 15 into 9 Changing these values seems to work out fine (and it returns '1.24' again instead of something like '1.2399962718'), but since I'm not thát known with sqlite's source, I'm wondering if there are more locations that have some hardcoded notion on float-accuracy... DRH, any hints...? I'll wrap the changes up in a patch using some define like SQLITE_FASTFPE or something, unless DRH has a different idea ;) -- Best, Frank. - To unsubscribe, send email to [EMAIL PROTECTED] -