Re: [sqlite] SQLITE3 Runs out of memory.
Thanks Michael and Jean-Christophe for your assistance, I change the cache_size
to 100k and it now performs correctly !
Thanks again
Stuart
-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org]
On Behalf Of Black, Michael (IS)
Sent: 17 November 2011 13:12
To: General Discussion of SQLite Database
Subject: Re: [sqlite] SQLITE3 Runs out of memory.
That's obviously not your real code...would be nice to see that as your problem
may elsewhere.
But...
#1 How much memory does your embedded device have AVAILABLE?
#2 Try reducing the cache_size (if #1 is less than 10's of megabytes) --
default cache_size is 1MB which is right around 33*30k -- reduce it to 100k
"pragma cache_size=100"'
#3 Your loop should be more like this:
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id = rowIndex")
while sqlite3_step()!=SQLITE_DONE
dostuff
end
sqlite3_finalize()
Michael D. Black
Senior Scientist
Advanced Analytics Directorate
Advanced GEOINT Solutions Operating Unit
Northrop Grumman Information Systems
From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on
behalf of Stuart Thomson [s_thom...@bitwisegroup.com]
Sent: Thursday, November 17, 2011 6:51 AM
To: sqlite-users@sqlite.org
Subject: EXT :[sqlite] SQLITE3 Runs out of memory.
Hi,
Before I start diving into the sqlite3 code, I was wondering if anyone had this
sort of problem before and could possibly point out to me where I might find
the problem.
I am running 3.7.9 on a embedded project using a database with one table
(structured (integer id, blobdata BLOB). Now the id goes from 0 to 61153 and
the blob data is off size 30K roughly.
When I do a loop of the following for every row in the table:
Loop for 61153 records start rowIndex at 0
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id = rowIndex")
sqlite3_step()
sqlite3_finialize()
End
The code will start to return 7 ( for NOMEM) after requesting the 34th row. Why
would it be running out of memory if I've completed the steps as per the sqlite
website for properly forming prepare, step then finalize?
Regards
Stuart T
BitWise Ltd - Crescent House, Carnegie Campus, Dunfermline, KY11 8GR, United
Kingdom
tel: +44 (0)1383 625151 -mob:
web: BitWise Group <http://www.bitwisegroup.com<http://www.bitwisegroup.com/>>
This e-mail may be confidential and privileged. Do not open it if you are in
any doubt that you are the intended recipient. You must scan this e-mail and
any attachments for the presence of viruses or any other unwelcome content.
This e-mail must be read in conjunction with the important legal notice at
BitWise Group/Legal <http://www.bitwisegroup.com/legal>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
BitWise Ltd - Crescent House, Carnegie Campus, Dunfermline, KY11 8GR, United
Kingdom
tel: +44 (0)1383 625151 -mob:
web: BitWise Group <http://www.bitwisegroup.com>
This e-mail may be confidential and privileged. Do not open it if you are in
any doubt that you are the intended recipient. You must scan this e-mail and
any attachments for the presence of viruses or any other unwelcome content.
This e-mail must be read in conjunction with the important legal notice at
BitWise Group/Legal <http://www.bitwisegroup.com/legal>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] SQLITE3 Runs out of memory.
Loop for 61153 records start rowIndex at 0
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id =
rowIndex")
sqlite3_step()
sqlite3_finialize()
End
Make that
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB")
Do
sqlite3_step() // handle errors somehow (busy, ...)
Until SQLITE_DONE
sqlite3_finalize()
You have primary key index, don't you? So you don't need ORDER BY.
Now 34 rows seem really few to cause memory shortage. Can you show the
relevant part of your code?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] SQLITE3 Runs out of memory.
That's obviously not your real code...would be nice to see that as your problem
may elsewhere.
But...
#1 How much memory does your embedded device have AVAILABLE?
#2 Try reducing the cache_size (if #1 is less than 10's of megabytes) --
default cache_size is 1MB which is right around 33*30k -- reduce it to 100k
"pragma cache_size=100"'
#3 Your loop should be more like this:
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id = rowIndex")
while sqlite3_step()!=SQLITE_DONE
dostuff
end
sqlite3_finalize()
Michael D. Black
Senior Scientist
Advanced Analytics Directorate
Advanced GEOINT Solutions Operating Unit
Northrop Grumman Information Systems
From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on
behalf of Stuart Thomson [s_thom...@bitwisegroup.com]
Sent: Thursday, November 17, 2011 6:51 AM
To: sqlite-users@sqlite.org
Subject: EXT :[sqlite] SQLITE3 Runs out of memory.
Hi,
Before I start diving into the sqlite3 code, I was wondering if anyone had this
sort of problem before and could possibly point out to me where I might find
the problem.
I am running 3.7.9 on a embedded project using a database with one table
(structured (integer id, blobdata BLOB). Now the id goes from 0 to 61153 and
the blob data is off size 30K roughly.
When I do a loop of the following for every row in the table:
Loop for 61153 records start rowIndex at 0
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id = rowIndex")
sqlite3_step()
sqlite3_finialize()
End
The code will start to return 7 ( for NOMEM) after requesting the 34th row. Why
would it be running out of memory if I've completed the steps as per the sqlite
website for properly forming prepare, step then finalize?
Regards
Stuart T
BitWise Ltd - Crescent House, Carnegie Campus, Dunfermline, KY11 8GR, United
Kingdom
tel: +44 (0)1383 625151 -mob:
web: BitWise Group <http://www.bitwisegroup.com<http://www.bitwisegroup.com/>>
This e-mail may be confidential and privileged. Do not open it if you are in
any doubt that you are the intended recipient. You must scan this e-mail and
any attachments for the presence of viruses or any other unwelcome content.
This e-mail must be read in conjunction with the important legal notice at
BitWise Group/Legal <http://www.bitwisegroup.com/legal>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] SQLITE3 Runs out of memory.
Hi,
Before I start diving into the sqlite3 code, I was wondering if anyone had this
sort of problem before and could possibly point out to me where I might find
the problem.
I am running 3.7.9 on a embedded project using a database with one table
(structured (integer id, blobdata BLOB). Now the id goes from 0 to 61153 and
the blob data is off size 30K roughly.
When I do a loop of the following for every row in the table:
Loop for 61153 records start rowIndex at 0
sqlite3_prepare_v2 ("SELECT id, blobdata FROM Table30KB Where id = rowIndex")
sqlite3_step()
sqlite3_finialize()
End
The code will start to return 7 ( for NOMEM) after requesting the 34th row. Why
would it be running out of memory if I've completed the steps as per the sqlite
website for properly forming prepare, step then finalize?
Regards
Stuart T
BitWise Ltd - Crescent House, Carnegie Campus, Dunfermline, KY11 8GR, United
Kingdom
tel: +44 (0)1383 625151 -mob:
web: BitWise Group
This e-mail may be confidential and privileged. Do not open it if you are in
any doubt that you are the intended recipient. You must scan this e-mail and
any attachments for the presence of viruses or any other unwelcome content.
This e-mail must be read in conjunction with the important legal notice at
BitWise Group/Legal
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
