This is very worrying since it means that the statement cannot be compiled on a
low memory device.
I am new to Sqlite, but I would guess that a precompiled query could be used,
where memory is low
and I also suppose that variable values could be bound to that precompiled
query.
Clive
Joe Wilson <[EMAIL PROTECTED]> on 05/06/2007 14:33:42
Please respond to sqlite-users@sqlite.org
To: sqlite-users@sqlite.org
cc: (bcc: clive/Emultek)
Subject: Re: [sqlite] Stack usage
--- [EMAIL PROTECTED] wrote:
> I'v read in change log that some stack allocted memory were moved to the heap,
but I think that
> there is still to much allocated memory on the stack.
> After creating a table with 2000 columns, jdbc driver created a query that run
out of stack.
> Default java's stack limit is low, but it wasn't hard to create simillar query
that crashed C
> application with default stack limit. And the fact that it crashed instead
repoting an error
> isn't really nice.
> The query created by the driver looks like that:
>
> select null as TABLE_CAT, null as TABLE_SCHEM, 'test' as TABLE_NAME, cn as
COLUMN_NAME, dt as
> DATA_TYPE, tn as TYPE_NAME, 2000000000 as COLUMN_SIZE, 2000000000 as
BUFFER_LENGTH, 10 as
> DECIMAL_DIGITS, 10 as NUM_PREC_RADIX, colnullable as NULLABLE, null as
REMARKS, null as
> COLUMN_DEF, 0 as SQL_DATA_TYPE, 0 as SQL_DATETIME_SUB, 2000000000 as
CHAR_OCTET_LENGTH,
> ordpos as ORDINAL_POSITION, (case colnullable when 0 then 'N' when 1 then 'Y'
else '' end) as
> IS_NULLABLE, null as SCOPE_CATLOG, null as SCOPE_SCHEMA, null as SCOPE_TABLE,
null as
> SOURCE_DATA_TYPE from (select 0 as ordpos, 1 as colnullable, 'col1' as cn,
'double' as tn, 8 as
> dt union all select 1 as ordpos, 1 as colnullable, 'col2' as cn, 'double' as
tn, 8 as dt union
> all select 2 as ordpos, 1 as colnullable, 'col3' as cn, 'double' as tn, 8 as
dt union all select
> 3 as ordpos, 1 as colnullable, 'col4' as cn, 'double' as tn, 8 as dt union all
select 4 as
> ordpos, 1 as colnullable, 'lastcol' as cn, 'double' as tn, 8 as dt);
>
> but uses more columns.
SQLite uses recursion to generate code for SELECT UNION chains.
** SELECT a FROM t1 UNION SELECT b FROM t2 UNION SELECT c FROM t3
**
** This statement is parsed up as follows:
**
** SELECT c FROM t3
** |
** `-----> SELECT b FROM t2
** |
** `------> SELECT a FROM t1
So for your example you will have a stack of 2000 nested calls
of sqlite3Select's (via multiSelect).
Using gcc -O2 on x86 I see that each level's stack is 480 bytes.
So for 2000 unions in a select, SQLite will consume at least
480 bytes/frame x 2000 frames = 960,000 bytes. I'm guessing that Java
limits the stack to 1M per thread in your case, hence your problem.
It is not easy to adapt SQLite's code to not use recursion for
code generation, although with enough time and effort anything is
possible. Conceivably, the processing of compound queries could
be turned into a for loop.
The SQLite authors have recently added a number of maximum limits
via #defines. Perhaps the SQLITE_MAX_EXPR_DEPTH limit could also
be applied here:
/* If there is are a sequence of queries, do the earlier ones first.
*/
if( p->pPrior ){
if( p->pRightmost==0 ){
Select *pLoop;
for(pLoop=p; pLoop; pLoop=pLoop->pPrior){
pLoop->pRightmost = p;
}
}
return multiSelect(pParse, p, eDest, iParm, aff);
}
____________________________________________________________________________________
It's here! Your new message!
Get new email alerts with the free Yahoo! Toolbar.
http://tools.search.yahoo.com/toolbar/features/mail/
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
