Re: [sqlite] No journal vs WAL journal

2011-04-26 Thread Dan Kennedy
On 04/26/2011 08:13 AM, Nikolaus Rath wrote: Nikolaus Rathnikol...@rath.org writes: [ WAL Performance ] Really no one an idea of why this is happening? I tried to investigate this further myself and created a small example program (attached). It fills a table with 500,000 entries without

Re: [sqlite] No journal vs WAL journal

2011-04-25 Thread Nikolaus Rath
Nikolaus Rath nikol...@rath.org writes: [ WAL Performance ] Really no one an idea of why this is happening? I tried to investigate this further myself and created a small example program (attached). It fills a table with 500,000 entries without using explicit transactions. The final db is 24 MB

Re: [sqlite] No journal vs WAL journal

2011-04-08 Thread Nikolaus Rath
Nikolaus Rath nikolaus-bth8mxji...@public.gmane.org writes: I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Pavel Ivanov
Furthermore, if I turn off auto checkpointing, the WAL file grows to more than 5 GB without transactions, but only to about 922 MB with a transaction. Are the commit markers really taking that much space? WAL-journal is not some kind of change log with commit markers. It's a sequence of

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Simon Slavin
On 6 Apr 2011, at 3:15pm, Pavel Ivanov wrote: For each transaction SQLite must write all changed pages into WAL-journal. It would be at least one page per transaction. Two ? One for the page holding the table, another for the page holding the primary key ? Simon.

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Pavel Ivanov
Two ?  One for the page holding the table, another for the page holding the primary key ? If table has INTEGER PRIMARY KEY then it is the same page I believe. :) I.e. table storage is in fact an index on rowid (or its alias). Pavel On Wed, Apr 6, 2011 at 10:50 AM, Simon Slavin

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Nikolaus Rath
Igor Tandetnik itandetnik-fh9draxc...@public.gmane.org writes: On 4/5/2011 10:01 PM, Nikolaus Rath wrote: I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Nikolaus Rath
Pavel Ivanov paivanof-re5jqeeqqe8avxtiumw...@public.gmane.org writes: Furthermore, if I turn off auto checkpointing, the WAL file grows to more than 5 GB without transactions, but only to about 922 MB with a transaction. Are the commit markers really taking that much space? WAL-journal is not

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Simon Slavin
On 6 Apr 2011, at 5:06pm, Nikolaus Rath wrote: Another question though: what is the recommended way to insert lots of data with WAL enabled? Without a transaction, the WAL file stays small but the insertions take very long time. With a transaction, things are faster but the WAL file grows

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Nikolaus Rath
Simon Slavin slavins-drl2ohjjk6tg9huczpv...@public.gmane.org writes: On 6 Apr 2011, at 5:06pm, Nikolaus Rath wrote: Another question though: what is the recommended way to insert lots of data with WAL enabled? Without a transaction, the WAL file stays small but the insertions take very long

Re: [sqlite] No journal vs WAL journal

2011-04-06 Thread Simon Slavin
On 6 Apr 2011, at 8:14pm, Nikolaus Rath wrote: Simon Slavin slavins-drl2ohjjk6tg9huczpv...@public.gmane.org writes: On 6 Apr 2011, at 5:06pm, Nikolaus Rath wrote: Another question though: what is the recommended way to insert lots of data with WAL enabled? Without a transaction, the WAL

[sqlite] No journal vs WAL journal

2011-04-05 Thread Nikolaus Rath
Hello, I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54 seconds. With journal_mode = WAL (and still no transaction),

Re: [sqlite] No journal vs WAL journal

2011-04-05 Thread Igor Tandetnik
On 4/5/2011 10:01 PM, Nikolaus Rath wrote: I'm experimenting with creation of a 156 MB database from scratch. I've set synchronous = NORMAL and locking_mode = exlusive. With journal_mode = off and without wrapping the INSERTs into a transaction, creating the db takes 54 seconds. With