> Will SQLite4 be a better solution for me then ? SQLite4 still in development phase. It is not production ready.
> Also @aris do you recommend BDB over Sqlite for 1-10 Billion records ? If you have many core of processors [and big RAM], then I recommend BDB Sql over Sqlite. Because you can have many processes or threads to write to a database concurrently. Because it use row or page level locking. If you can choose DBMS, other than SQLite, try to use DB that have storage engine optimized for write, for example LSM (hypertable), Fractal Tree (tokudb engine for mysql). On 11/4/13, Raheel Gupta <raheel...@gmail.com> wrote: > Will SQLite4 be a better solution for me then ? > > Also @aris do you recommend BDB over Sqlite for 1-10 Billion records ? > > > On Mon, Nov 4, 2013 at 8:03 AM, Aris Setyawan <aris.s...@gmail.com> wrote: > >> > I just looked, sophia is nothing special. See these microbench results. >> > http://pastebin.com/cFK1JsCN >> > >> > LMDB's codebase is still smaller and faster. Nothing else touches >> > LMDB's >> > read >> > speed. >> >> Focus to the write number. >> >> You are using SSD or HDD? >> >> On 11/4/13, Howard Chu <h...@symas.com> wrote: >> > Aris Setyawan wrote: >> >>> SQLightning replaces the SQLite backend with Symas' LMDB, which also >> >>> uses >> >>> MVCC >> >>> and thus supports high concurrency. It is also many times faster than >> >>> BerkeleyDB and vanilla SQLite. >> >> >> >> Your MVCC is different compared to InnoDB or BDB locking. Every one >> >> should carefully read each DB's doc, then test it before decide to use >> >> it. >> > >> > Yes, it's different. In LMDB writers never block readers and readers >> never >> > block writers. The original poster was complaining about SELECT taking >> > a >> > long >> > time and preventing other threads from making progress. That problem >> doesn't >> > >> > exist in LMDB. BDB locking *might* be able to avoid this in many cases, >> if >> > there are no hotspots, but is prone to deadlocks which require the >> calling >> > application to retry failed requests. >> > >> >> LMDB is storage engine optimized for read. Why you don't optimize it's >> >> write using cache oblivious data structure, for example LSM tree or >> >> create new, like in sophia (sphia.org) key value DB? >> > >> > I just looked, sophia is nothing special. See these microbench results. >> > http://pastebin.com/cFK1JsCN >> > >> > LMDB's codebase is still smaller and faster. Nothing else touches >> > LMDB's >> > read >> > speed. >> > >> > -- >> > -- Howard Chu >> > CTO, Symas Corp. http://www.symas.com >> > Director, Highland Sun http://highlandsun.com/hyc/ >> > Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > _______________________________________________ >> > sqlite-users mailing list >> > sqlite-users@sqlite.org >> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> > >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
