Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
Thanks. I'll hold off on checking the result of every sqlite3_bind_* for now. It's unlikely I'm using a finalized statement otherwise I believe I would get a given crash much more often. The sqlite3_prepare_v2 statements that are failing only fail occasionally, not every time. But I'll double

Re: [sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Yuriy Kaminskiy
Gabriel Corneanu wrote: > I have the following scenario: I need to "clear"/"initialize" a db file > while potential readers are active (polling for data). > The "normal" way to do it is begin a transaction, drop all tables, recreate > tables, commit (vacuum to regain space). > > The biggest

Re: [sqlite] EXT :Re: Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Black, Michael (IS)
99% (or more) of the time this is going to be your code. I would move forward based on that assumption. Found this that might help you: http://stackoverflow.com/questions/1730180/is-this-kind-of-crash-report-useless Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Pavel Ivanov
On Thu, Jul 19, 2012 at 2:07 PM, Rick Maddy wrote: > Didn't mean to imply that failing to check the return value resulted in > memory corruption. I was wondering if it was possible that one of the many > calls to sqlite3_bind_* in my code may actually be causing some memory >

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Richard Hipp
On Thu, Jul 19, 2012 at 2:07 PM, Rick Maddy wrote: > Didn't mean to imply that failing to check the return value resulted in > memory corruption. I was wondering if it was possible that one of the many > calls to sqlite3_bind_* in my code may actually be causing some memory >

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
Didn't mean to imply that failing to check the return value resulted in memory corruption. I was wondering if it was possible that one of the many calls to sqlite3_bind_* in my code may actually be causing some memory corruption. I can envision some possible buffer overflows associated with

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Richard Hipp
On Thu, Jul 19, 2012 at 1:56 PM, Rick Maddy wrote: > What about checking all the sqlite3_bind_* methods? Is it possible any of > those could cause the problems I'm seeing? > Being busing with other issues, I have not been following this thread closely, so perhaps I have missed

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
What about checking all the sqlite3_bind_* methods? Is it possible any of those could cause the problems I'm seeing? Rick On Jul 19, 2012, at 11:42 AM, Simon Slavin wrote: > > On 19 Jul 2012, at 6:03pm, Rick Maddy wrote: > >> Thanks. Time to add checks to nearly 400

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
I should know better. I've been coding in C-based languages for over 25 years. I guess I just never thought a prepare statement needed checking. I figured once the query worked all was good. I already have a simple check method. I do check quite a few of the 'step' method result values, just

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Simon Slavin
On 19 Jul 2012, at 6:03pm, Rick Maddy wrote: > Thanks. Time to add checks to nearly 400 prepare and step statements. Sorry about that. But you should be able to make a tiny test function which just tests the result and makes sure it's SQLITE_OK. You are nowhere near the

Re: [sqlite] EXT :Re: Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Pavel Ivanov
There's no easy method of detecting memory corruption from inside your own application, especially the memory that your application don't control. You can use memory debuggers for that purpose. If you were on Linux I would recommend to use valgrind. But I don't know if there are any similar tools

Re: [sqlite] EXT :Re: Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
But that would flag valid state changes just as much as it would flag corrupted memory. Rick On Jul 19, 2012, at 11:15 AM, Black, Michael (IS) wrote: > It could easiliy be expanded to look at the whole structure... > > > > Or just do an XOR checksum on the bytes in sizeof(sqlite3) and

Re: [sqlite] EXT :Re: Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Black, Michael (IS)
Hmmm...somebody might need to point out what elements remain static. So may not be able to do the whole structure like that. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems

Re: [sqlite] EXT :Re: Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Black, Michael (IS)
It could easiliy be expanded to look at the whole structure... Or just do an XOR checksum on the bytes in sizeof(sqlite3) and compare that. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
Thanks but doesn't that code check to see if the database pointer has changed and not whether the memory it references has been corrupted? I guess that's a start though. Rick On Jul 19, 2012, at 11:02 AM, Black, Michael (IS) wrote: > Buffer overflow issues can cause problems at seemingly

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
Thanks. Time to add checks to nearly 400 prepare and step statements. Rick ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Black, Michael (IS)
Buffer overflow issues can cause problems at seemingly random points in a program. I've worked on some really nasty ones before when no debugging was available. If dbRef is being corrupted then put a dbRef check in your program in every function at beginning and end. int

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Simon Slavin
On 19 Jul 2012, at 5:46pm, Rick Maddy wrote: > With regard to memory issues, I don't see how this could be the case. One of > the places the app crashes (on rare occasions) is: > >sqlite3_stmt *load = nil; >sqlite3_prepare_v2(dbRef, "SELECT some_column FROM

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
Thanks. One of the users that reported one of the slqite3_prepare_v2 crashes also sent me their database. I just ran the 'pragma integrity_check' command on the database and it reported 'ok' though there was a journal file along with the database file. But there crash happened during a

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Simon Slavin
On 19 Jul 2012, at 5:09pm, Rick Maddy wrote: > Exception Type: SIGSEGV > Exception Codes: SEGV_ACCERR at 0x1a Those are segfaults. They indicate an attempt by your app to access memory which doesn't belong to it. This often but not always happens because the memory /did/

Re: [sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Pavel Ivanov
> For quite some time now I've been getting reports of crashes in my iOS app. > Specifically these are caused by crashes in sqlite3_prepare_v2 and > sqlite_step. The associated code works fine most of the time. So I'm looking > for thoughts on how to find and fix the problem since there seems

[sqlite] Getting occasional crashes with sqlite3_prepare_v2 and sqlite3_step in iOS app

2012-07-19 Thread Rick Maddy
For quite some time now I've been getting reports of crashes in my iOS app. Specifically these are caused by crashes in sqlite3_prepare_v2 and sqlite_step. The associated code works fine most of the time. So I'm looking for thoughts on how to find and fix the problem since there seems to be no

Re: [sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Black, Michael (IS)
Why don't you just attach another database and switch your user connections to that one? Then you can just delete the old file and not worry about vaccum at all. Sounds though like sqlite3 could use a "truncate" command like Oracle has which is the speedy way to zero out a table there.

Re: [sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Gabriel Corneanu
Rather than dropping all tables, why not just use DELETE FROM TABLE myTable with no clauses ? Also, I would guess that the thing which is taking most of the time is the VACUUM command. Do you >really need it ? Are you very short of filespace ? Are you about to make special backup

Re: [sqlite] select speed

2012-07-19 Thread Jay A. Kreibich
On Thu, Jul 19, 2012 at 01:35:23PM +0100, Simon Slavin scratched on the wall: > > On 19 Jul 2012, at 11:54am, Live Happy wrote: > > > wish one is faster to make select from table with 20 column and alot of > > records or to divide data in more than table and made join

Re: [sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Simon Slavin
On 19 Jul 2012, at 9:22am, Gabriel Corneanu wrote: > I have the following scenario: I need to "clear"/"initialize" a db file > while potential readers are active (polling for data). > The "normal" way to do it is begin a transaction, drop all tables, recreate >

Re: [sqlite] select speed

2012-07-19 Thread Simon Slavin
On 19 Jul 2012, at 11:54am, Live Happy wrote: > wish one is faster to make select from table with 20 column and alot of > records or to divide data in more than table and made join select Single table with 20 columns. Unless your data is very unusual. Simon.

Re: [sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Gabriel Corneanu
I poked through the sqlite source code and tried to force "sqlite3ResetOneSchema" or "sqlite3SchemaClear"... and found no clear way to do it... I found however an workaround: before clearing the file I get the table list and execute a rename (alter) for each table using fake names. Then

[sqlite] select speed

2012-07-19 Thread Live Happy
wish one is faster to make select from table with 20 column and alot of records or to divide data in more than table and made join select ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] force read schema after "delete from sqlite_master"

2012-07-19 Thread Gabriel Corneanu
Hello, I have the following scenario: I need to "clear"/"initialize" a db file while potential readers are active (polling for data). The "normal" way to do it is begin a transaction, drop all tables, recreate tables, commit (vacuum to regain space). The biggest problem is that dropping a "very