Paul Sanderson wrote
> Unlike a rollback journal a WAL file can have multiple copies of the same
> page.
>
> So from yor main loop, expanding the following code may help us
> understand.
>
> "insert all downloaded rows"
>
> If your inserted records is 5million separate insertions then each
> insertion could result in a table leaf page being written to the WAL
> file. Each insertion could potentially also result in one (or more)
> table interior pages being written to the WAL. Subsequent insertions
> could result in multiple copies of the same page.
Maybe you got me onto the right track...
I'll start with the record size estimation:
CREATE TABLE [discounttype]
(
[createdon] DATETIME NULL,
[description] NTEXT NULL,
[discounttypeid] UNIQUEIDENTIFIER NOT NULL CONSTRAINT
PK_discounttype
PRIMARY KEY ROWGUIDCOL DEFAULT (newid()),
[isamounttype] BIT NOT NULL DEFAULT(0),
[modifiedon] DATETIME NULL,
[name] NVARCHAR(200) NULL COLLATE NOCASE
);
The record will be rather small. It consists of
2xDATETIME - usu. 38 By
UNIQUEIDENTIFIER - 32 By
+ 2 rather short strings
Altogether I wouldn't be surprised if the page size (1024) is about 7x
larger than the average record length. This (together with your explanation)
would explain extreme WAL size.
---------------------------
Thanks to your post I discovered multiple-row inserts so that I now
understand what you asked.
I found this nice article
<https://www.simple-talk.com/sql/performance/comparing-multiple-rows-insert-vs-single-row-insert-with-three-data-load-methods/>
that (although for SQL server) proves that multi-inserts can greatly speed
up bulk inserts. I am quite confident that most of arguments brought by that
article will apply also to our case. (sqlite used from c#)
Will have to test it thoroughly and (if successful) modify our application
code.
--
View this message in context:
http://sqlite.1065341.n5.nabble.com/Huge-WAL-log-tp79991p80052.html
Sent from the SQLite mailing list archive at Nabble.com.
_______________________________________________
sqlite-users mailing list
[email protected]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
