Re: [sqlite] Weird out of memory problem a prepare

[RSmith]

On 2014/03/04 22:05, Eduardo Morras wrote:


The tables have 4 rows each one, that's why I got suprised with the Out of 
Memory error. The biggest row has 12KB and with the join I do, shouldn't use 
more than 200KB.

Changing the ',' with the join you propose, gives Out of Memory too. It happens 
on prepare phase, before binding the ? with my data. The query didn't reach the 
step call.

Trying simple "SELECT r.name FROM resource AS r WHERE r.name = ?" gets "Out of 
memory" too calling preparev2.

Surely something is rotten on my development platform...


Yes, something is very rotten. I would start by looking for possible memory coruptors... an array being adjusted with an index out 
of the declared range, calling methods on an object via a reference that wasn't nulled after the actual object (via another ref) was 
disposed. The problem with these types of errors is the thing that breaks usually have nothing to do with the thing that causes the 
memory to go bad, so it is very hard to trace since we normally try to fix the thing that breaks and stares too long at the code of 
the thing that broke, with which there is rarely a problem.


Do you use any memory profiling tools and checkers? It might give you some 
hints.

If that query fails in an SQLite tool too, then maybe there is a problem, or if you use a custom altered version of the SQLite code. 
Barring that, you need to hunt down the corrupting code - Good luck!




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Dan Kennedy]

On 03/05/2014 03:05 AM, Eduardo Morras wrote:

On Tue, 4 Mar 2014 15:19:24 +
Simon Slavin  wrote:


On 4 Mar 2014, at 3:15pm, Simon Slavin  wrote:


On 4 Mar 2014, at 3:09pm, Eduardo Morras  wrote:


zSql= "SELECT r.name, s.content FROM resource AS r, static AS s
WHERE (r.ids = s.ids AND r.name = ?);";

[snip]

Error on query: out of memory

I think this might require comparing every row in resource with
every row in static.  Which is a lot of temporary data to hold in
memory.

You might try something like

SELECT r.name, s.content FROM resource AS r JOIN static AS s ON
s.ids = r.ids  WHERE r.name = ?

and, of course, an index

CREATE INDEX i1 on resource (name,ids)

will make it run extremely quickly.

I'll reply both answers here, if you don't mind.

Thanks Simon, for the answers.

The tables have 4 rows each one, that's why I got suprised with the Out of 
Memory error. The biggest row has 12KB and with the join I do, shouldn't use 
more than 200KB.

Changing the ',' with the join you propose, gives Out of Memory too. It happens 
on prepare phase, before binding the ? with my data. The query didn't reach the 
step call.

Trying simple "SELECT r.name FROM resource AS r WHERE r.name = ?" gets "Out of 
memory" too calling preparev2.


Does the "out of memory" error come from sqlite3_prepare_v2()
directly or from sqlite3_errmsg()? Is the sqlite3* pointer
a NULL?

Dan.



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Simon Slavin]

On 4 Mar 2014, at 8:05pm, Eduardo Morras  wrote:

> The tables have 4 rows each one,

Hahahaha.  Hah.  That changes things.

You have something wrong.  Either the database is corrupt (check it with a 
PRAGMA or make another one) or your code is messed up somehow.

As a test, open that database with the SQLite shell tool and execute exactly 
the same command that gave you the memory error in your code.  I bet it works 
fine.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[mm.w]

Hello,

what's your SharedSection value? [heap limitation]

Best Regards.


On Tue, Mar 4, 2014 at 12:16 PM, Clemens Ladisch  wrote:

> Eduardo Morras wrote:
> > Trying simple "SELECT r.name FROM resource AS r WHERE r.name = ?" gets
> "Out of memory" too calling preparev2.
>
> This has nothing to do with the query itself.
>
> If you aren't doing something funny with the memory allocator, it's likely
> that SQLite's database object got corrupt by some other buggy code.
>
>
> Regards,
> Clemens
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Clemens Ladisch]

Eduardo Morras wrote:
> Trying simple "SELECT r.name FROM resource AS r WHERE r.name = ?" gets "Out 
> of memory" too calling preparev2.

This has nothing to do with the query itself.

If you aren't doing something funny with the memory allocator, it's likely
that SQLite's database object got corrupt by some other buggy code.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Eduardo Morras]

On Tue, 4 Mar 2014 15:19:24 +
Simon Slavin  wrote:

> 
> On 4 Mar 2014, at 3:15pm, Simon Slavin  wrote:
> 
> > On 4 Mar 2014, at 3:09pm, Eduardo Morras  wrote:
> > 
> >> zSql= "SELECT r.name, s.content FROM resource AS r, static AS s
> >> WHERE (r.ids = s.ids AND r.name = ?);";
> >> 
> >> [snip]
> >> 
> >> Error on query: out of memory
> > 
> > I think this might require comparing every row in resource with
> > every row in static.  Which is a lot of temporary data to hold in
> > memory.
> > 
> > You might try something like
> > 
> > SELECT r.name, s.content FROM resource AS r JOIN static AS s ON
> > s.ids = r.ids  WHERE r.name = ?
> 
> and, of course, an index
> 
> CREATE INDEX i1 on resource (name,ids)
> 
> will make it run extremely quickly.

I'll reply both answers here, if you don't mind.

Thanks Simon, for the answers.

The tables have 4 rows each one, that's why I got suprised with the Out of 
Memory error. The biggest row has 12KB and with the join I do, shouldn't use 
more than 200KB.

Changing the ',' with the join you propose, gives Out of Memory too. It happens 
on prepare phase, before binding the ? with my data. The query didn't reach the 
step call.

Trying simple "SELECT r.name FROM resource AS r WHERE r.name = ?" gets "Out of 
memory" too calling preparev2.

Surely something is rotten on my development platform...

> 
> Simon.

Thanks
---   ---
Eduardo Morras 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Simon Slavin]

On 4 Mar 2014, at 3:15pm, Simon Slavin  wrote:

> On 4 Mar 2014, at 3:09pm, Eduardo Morras  wrote:
> 
>> zSql= "SELECT r.name, s.content FROM resource AS r, static AS s WHERE (r.ids 
>> = s.ids AND r.name = ?);";
>> 
>> [snip]
>> 
>> Error on query: out of memory
> 
> I think this might require comparing every row in resource with every row in 
> static.  Which is a lot of temporary data to hold in memory.
> 
> You might try something like
> 
> SELECT r.name, s.content FROM resource AS r JOIN static AS s ON s.ids = r.ids 
>  WHERE r.name = ?

and, of course, an index

CREATE INDEX i1 on resource (name,ids)

will make it run extremely quickly.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Weird out of memory problem a prepare

[Simon Slavin]

On 4 Mar 2014, at 3:09pm, Eduardo Morras  wrote:

>  zSql= "SELECT r.name, s.content FROM resource AS r, static AS s WHERE (r.ids 
> = s.ids AND r.name = ?);";
> 
> [snip]
> 
> Error on query: out of memory

I think this might require comparing every row in resource with every row in 
static.  Which is a lot of temporary data to hold in memory.

You might try something like

SELECT r.name, s.content FROM resource AS r JOIN static AS s ON s.ids = r.ids  
WHERE r.name = ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Weird out of memory problem a prepare

[Eduardo Morras]

Hi,

I have this code that fails always with the error output next: 

=
  zSql= "SELECT r.name, s.content FROM resource AS r, static AS s WHERE (r.ids 
= s.ids AND r.name = ?);";

  stmt = NULL;
  rc = sqlite3_prepare_v2(db, zSql, strlen(zSql), , NULL);

  if ( rc != SQLITE_OK ){
dprintf( log, "Error on query: %s\n", sqlite3_errmsg(db));
dprintf( log, "Query : %s\n", zSql);
dprintf( log, " at %s %d:\n", __func__, __LINE__);
exit(1);
=

Error on query: out of memory

Query : SELECT r.nombre, s.content FROM resource AS r, static AS s WHERE (r.ids 
= s.ids AND r.nombre = ?);
 at process_request 66:
=

The tables schema are:
"CREATE TABLE IF NOT EXISTS resource(\n"
"  rid INTEGER PRIMARY KEY NOT NULL,\n"   // Resource ID
"  type INTEGER NOT NULL,\n"  // Type of resource 
(music,video,+18)
"  ids INTEGER,\n"// FK Static Resource ID (if 
applicable)
"  sys INTEGER DEFAULT 0,\n"  // Is a system file (do not 
delete)
"  replicated INTEGER DEFAULT 0,\n"   // Is resource replicated 
somewhere (and safe)
"  nombre TEXT NOT NULL,\n"   // Resource name (filename or 
appropiate)
"  path TEXT\n"   // Path to resource
");

and

"CREATE TABLE IF NOT EXISTS static(\n"
"  ids INTEGER PRIMARY KEY,\n"// Static resource ID
"  desc TEXT,\n"  // Description
"  content BLOB\n"// Main content
");\n"

I don't know why I get an Out of memory preparing the query. Vars are 
initialized, db points to an open sqlite3 db, and stmt is the first time used 
int the code. I use a similar code on other projects abd works with no 
problems. 

Any clue?

Thanks

---   ---
Eduardo Morras 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users