Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Simon Slavin
On 1 Feb 2012, at 7:41am, Guy Terreault wrote: > I have been reading your back and forth answers like the next 4 responce. > > And I am still waiting for the example that works in c code like the one that > Bill McCormick that is wrong. > > Why do you not include in your answer a working corre

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Marcus Grimm
On 01.02.2012 11:10, Guy Terreault wrote: On 12-02-01 04:32 AM, Marcus Grimm wrote: On 01.02.2012 09:32, Guy Terreault wrote: On 12-02-01 03:03 AM, Larry Brasfield wrote: I cannot answer why Simon does not write demo code at every opportunity, but I think I speak for more than just myself

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Guy Terreault
On 12-02-01 04:32 AM, Marcus Grimm wrote: On 01.02.2012 09:32, Guy Terreault wrote: On 12-02-01 03:03 AM, Larry Brasfield wrote: I cannot answer why Simon does not write demo code at every opportunity, but I think I speak for more than just myself by revealing that I have other work, and a l

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Marcus Grimm
On 01.02.2012 09:32, Guy Terreault wrote: On 12-02-01 03:03 AM, Larry Brasfield wrote: I cannot answer why Simon does not write demo code at every opportunity, but I think I speak for more than just myself by revealing that I have other work, and a life, and consider time a scarce resource.

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Guy Terreault
On 12-02-01 03:03 AM, Larry Brasfield wrote: Guy Terreault wrote: Hi, Simon. I have been reading your back and forth answers like the next 4 responce. And I am still waiting for the example that works in c code like the one that Bill McCormick that is wrong. Why do you not include in your a

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-02-01 Thread Larry Brasfield
Guy Terreault wrote: Hi, Simon. I have been reading your back and forth answers like the next 4 responce. And I am still waiting for the example that works in c code like the one that Bill McCormick that is wrong. Why do you not include in your answer a working correction of the c code so that

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Guy Terreault
Hi, Simon. I have been reading your back and forth answers like the next 4 responce. And I am still waiting for the example that works in c code like the one that Bill McCormick that is wrong. Why do you not include in your answer a working correction of the c code so that followers can lea

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Petite Abeille
On Jan 31, 2012, at 6:45 PM, Bill McCormick wrote: > Funny how SQL works; it really can't figure out what you REALLY want to do > :) Maybe somebody who is really smart could figure out how to add a feature > to do this? :) Try: pragma dwim = yes; _

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Simon Slavin
On 31 Jan 2012, at 5:16pm, Bill McCormick wrote: > So when I run an update command (to update some arbitrary record) from the > sqlite3 command line, do I need to do something to do a checkpoint? Iff you fail to open a transaction before you do a SQL command, SQLite will automatically wrap it

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Bill McCormick
Larry Brasfield wrote, On 1/31/2012 11:36 AM: Bill McCormick wrote: > < PRAGMA journal_mode=WAL; ... So when I run an update command (to update some arbitrary record) from the sqlite3 command line, do I need to do something to do a checkpoint? By "checkpoint", I presume you mean some p

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Larry Brasfield
Bill McCormick wrote: > < PRAGMA journal_mode=WAL; ... So when I run an update command (to update some arbitrary record) from the sqlite3 command line, do I need to do something to do a checkpoint? By "checkpoint", I presume you mean some previous state of the database to which you can

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Bill McCormick
Larry Brasfield wrote, On 1/31/2012 8:40 AM: < Either I have some incorrect version or don't understand how to use this: < < #include < < int main(int argc, char** argv) < { < sqlite3 *db; < < PRAGMA journal_mode=WAL; < < ... < < gcc [various errors] The PRAGMA is something you pas

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Larry Brasfield
< Either I have some incorrect version or don't understand how to use this: < < #include < < int main(int argc, char** argv) < { < sqlite3 *db; < < PRAGMA journal_mode=WAL; < < ... < < gcc [various errors] The PRAGMA is something you pass to the statement preparation API as if it wa

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Bill McCormick
Simon Slavin wrote, On 1/31/2012 8:34 AM: On 31 Jan 2012, at 2:32pm, Bill McCormick wrote: Where exactly to I put the PRAGMA journal_mode=WAL; line? You execute it as a SQL command, the same way you'd execute an INSERT command. You only need to do it once for the database file. The setting i

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Igor Tandetnik
Bill McCormick wrote: > Where exactly to I put the PRAGMA journal_mode=WAL; line? It's a statement. You execute it, just like you would, say, INSERT or UPDATE. -- Igor Tandetnik ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8

Re: [sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Simon Slavin
On 31 Jan 2012, at 2:32pm, Bill McCormick wrote: > Where exactly to I put the PRAGMA journal_mode=WAL; line? You execute it as a SQL command, the same way you'd execute an INSERT command. You only need to do it once for the database file. The setting is stored with the database and next time

[sqlite] PRAGMA journal_mode=WAL;

2012-01-31 Thread Bill McCormick
Where exactly to I put the PRAGMA journal_mode=WAL; line? Either I have some incorrect version or don't understand how to use this: #include int main(int argc, char** argv) { sqlite3 *db; PRAGMA journal_mode=WAL; ... gcc -g -Wall -lrt -I.. -I../../include -I../ini-parse -o prodd pr

Re: [sqlite] Is it possible to optimize the readperformanceofa C++ app using sqlite pragma journal_mode = wal

2011-05-29 Thread Michael Stephenson
n of SQLite Database Subject: Re: [sqlite] Is it possible to optimize the readperformanceofa C++ app using sqlite pragma journal_mode = wal On 30 May 2011, at 2:08am, Frank Chang wrote: > Thank you for your help with the PRAGMA page size question. Sometimes I see C++ programs which first iss

Re: [sqlite] Is it possible to optimize the readperformanceofa C++ app using sqlite pragma journal_mode = wal

2011-05-29 Thread Simon Slavin
On 30 May 2011, at 2:08am, Frank Chang wrote: > Thank you for your help with the PRAGMA page size question. Sometimes I see > C++ programs which first issue the PRAGMA page_size = 4096. Then these C++ > programs request a sqlite3_execute(Database,"VACUUM",callback,0,&Msg) which > takes about

Re: [sqlite] Is it possible to optimize the readperformanceofa C++ app using sqlite pragma journal_mode = wal

2011-05-29 Thread Frank Chang
Simon Slavin, Thank you for your help with the PRAGMA page size question. Sometimes I see C++ programs which first issue the PRAGMA page_size = 4096. Then these C++ programs request a sqlite3_execute(Database,"VACUUM",callback,0,&Msg) which takes about 4 minutes to complete.

Re: [sqlite] Is it possible to optimize the readperformanceof a C++ app using sqlite pragma journal_mode = wal

2011-05-28 Thread Michael Stephenson
/cvstrac/wiki?p=SqliteWrappers -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Frank Chang Sent: Saturday, May 28, 2011 9:38 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Is it possible to optimize the readperformanceof

Re: [sqlite] Is it possible to optimize the readperformanceof a C++ app using sqlite pragma journal_mode = wal

2011-05-28 Thread Frank Chang
Michael Stephenson, We want to thank you again for your excellent suggestions regarding how to improve the speed of our Sqlite WAL reads and our deduper prototype. We looked at the SQlite documentation for increasing the Sqlite page size to 4K and an excerpt of our code is shown below. If we c

Re: [sqlite] Is it possible to optimize the read performance of a C++ app using sqlite pragma journal_mode = wal & pragma wal_checkpoint?

2011-05-27 Thread Frank Chang
the read performance of a C++ app using sqlite pragma journal_mode = wal & pragma wal_checkpoint? Here's the download link promised. The build is for x86. Take some time to carefully read the explanations at top of the source code. The function you may want to test is typos().

Re: [sqlite] Is it possible to optimize the read performanceof a C++ app using sqlite pragma journal_mode = wal & pragma

2011-05-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/26/2011 10:41 PM, Frank Chang wrote: > > Roger Binns, Thank you for your reply. You never answered the important bit - is your concern about initial population of the database or about runtime later on. > Would you expect us to get faster

Re: [sqlite] Is it possible to optimize the read performanceof a C++ app using sqlite pragma journal_mode = wal & pragma

2011-05-26 Thread Frank Chang
Roger Binns, Thank you for your reply. I understand what you are saying that we should drop the sqlite3_wal_checkpoint_v2(Database,"main",SQLITE_CHECKPOINT_FULL, // &number1,&number2); after the commit transaction ReturnValue=sqlite3_prepare(Database,"COMMIT",-1,&Statement,0); status = sql

Re: [sqlite] Is it possible to optimize the read performanceof a C++ app using sqlite pragma journal_mode = wal & pragmawal_checkpoint?

2011-05-26 Thread Jean-Christophe Deschamps
> Simon Slavin, Thank you for your suggestion. Our deduper prototoype > uses fuzzy matching methods such as the Levenshtein Distance to > detect duplicates. We have found that these fuzzy matching methods > are best implemented in C++ for processing time requirements. > We would

Re: [sqlite] Is it possible to optimize the read performanceof a C++ app using sqlite pragma journal_mode = wal & pragmawal_checkpoint?

2011-05-26 Thread Simon Slavin
On 27 May 2011, at 2:05am, Frank Chang wrote: > Simon Slavin, Thank you for your suggestion. Our deduper prototoype uses > fuzzy matching methods such as the Levenshtein Distance to detect duplicates. > We have found that these fuzzy matching methods are best implemented in C++ > for processin

Re: [sqlite] Is it possible to optimize the read performanceof a C++ app using sqlite pragma journal_mode = wal & pragmawal_checkpoint?

2011-05-26 Thread Frank Chang
Simon Slavin, Thank you for your suggestion. Our deduper prototoype uses fuzzy matching methods such as the Levenshtein Distance to detect duplicates. We have found that these fuzzy matching methods are best implemented in C++ for processing time requirements. We would still like

Re: [sqlite] Is it possible to optimize the read performance of a C++ app using sqlite pragma journal_mode = wal & pragma wal_checkpoint?

2011-05-25 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/25/2011 07:04 PM, Frank Chang wrote: > While writing the records to to the sqlite database we do a commit every > 1 records. >We think we understand that we also need to do a sqlite pragma wal_checkpoint everytime >we do a sqlite database c

Re: [sqlite] Is it possible to optimize the read performance of a C++ app using sqlite pragma journal_mode = wal & pragma wal_checkpoint?

2011-05-25 Thread Simon Slavin
On 26 May 2011, at 3:04am, Frank Chang wrote: > In the second phase, we read the sqlite WAL database and try to find > out the duplicates in our input records. Here, we are only reading the sqlite > WAL database. We would like to find out how to optimize the read performance > of the sql

[sqlite] Is it possible to optimize the read performance of a C++ app using sqlite pragma journal_mode = wal & pragma wal_checkpoint?

2011-05-25 Thread Frank Chang
Good evening, We are trying to build a C++ deduper application using the latest sqlite release. Our deduper runs in two phases. In the first phase it reads the records to be deduped from a Microsoft DBF file and writes the records into sqlite wal database. While writing the records to to the s