Domingo Alvarez Duarte wrote:
Hello Simon !
I already did it without using "wal" and the result was the same.
And even for my surprise in one try I stopped at the middle performed an
"analyze" and the performance deteriorated a lot to a point that I needed to
delete the stats tables to get the better performance without "analyze".
I also tried with the lsm module and got a bit better performance but with an
irregular timing and a bigger disk usage (20%).
Also tested with lmdb with an astonishing insertion rate but with a lot more
disk usage and irregular timing.
Using LMDB the VACUUM command is supposed to be a no-op; at least that's how I
intended it. Since LMDB deletes records immediately instead of leaving
tombstones, there is nothing to vacuum.
Also tested with leveldb with a worse performance and almost twice disk space
usage.
The data distribution on some tables seem to fall into the worst corner cases
for btrees.
Cheers !
On 01/10/16 18:26, Simon Slavin wrote:
On 1 Oct 2016, at 10:18pm, Domingo Alvarez Duarte <mingo...@gmail.com> wrote:
About the vacuum I also understand the need to rewrite the whole database
but I'm not sure if it's really necessary to do almost 5 times the database
size in both reads and writes (also an equivalent amount of I/O happened
during insertions).
Can you try it without
db.exec_dml("PRAGMA wal_checkpoint(FULL);");
and see if that improves time ?
That's the only thing I can see. You're using a nested INSERT OR IGNORE
command I'm not familiar with.
Simon.
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
