I'm not planning to develop a new SQL engine, nor am I proposing any changes to SQLite. I'm only looking for some pointers regarding the relevant issues in order to study them more closely. If something useful were to come as a result of doing so then I might be in a position to actually flesh out a proposal.
Because SQLite is a complex piece of code (as others have pointed out) I found it helpful to focus on just the following items: 1) maintaining shared lock state 2) efficiently detecting failed database processes The assumption being that a lock facility that can handle these issues is needed by any concurrency scheme (MVCC, shadow pages, etc) and so can be thought about independently. Does the ability of a client-server DB to support multiple writers follow solely from the fact that it centralizes the lock bookkeeping and that it can easily detect death of its children? If so, then I think one could get a similar result using a mmap'ed file and futexes among unrelated processes. This is the area I thought worth looking into provided that (1) and (2) really are at the heart of the matter. If not, course corrections are appreciated :) Regards, Iker Pavel Ivanov wrote: >> I'm hopeful that it's possible to devise a scheme that will let SQLite >> support >> multiple readers and writers while completely preserving all of its current >> benefits > > So the answer to my question is yes, you want to develop a new SQL engine... > Then first of all you have to describe one thing: how will you > implement ACID properties and most probably MVCC? What kind of OS and > programming tools will you use for this? How exactly you'll make sure > that multiple writers don't write to the same place in database and > readers don't read inconsistent data while writers are writing? > > As you explain your concepts in these topics we will be happy to > discuss it with you and help you make a great SQL engine. > > BTW, you can search list archives and find what DRH said about why > ACID cannot be implemented along with fine-grained locks using just > file system locking as SQLite does now. > > Pavel > > On Wed, Sep 9, 2009 at 4:09 PM, Iker Arizmendi <i...@research.att.com> wrote: >> I'm hopeful that it's possible to devise a scheme that will let SQLite >> support >> multiple readers and writers while completely preserving all of its current >> benefits (eg, serverless, efficient, zero-conf, simple API, small footprint). >> To that end, I'm trying to understand some of the "sub problems" that get in >> the way of multiple writers in order to take a stab at working them out. I >> can >> see that the overall problem is nontrivial and an acceptable solution has not >> been found. But this only proves that the problem is hard, not that it's >> impossible (*). >> >> Iker >> >> (*) Of course, if folks have actually shown that solving this problem >> amounts to squaring the circle then that's another story. >> >> Pavel Ivanov wrote: >>> I just keep wondering: do you want to write some new database engine >>> based on SQLite so that it will heed all these caveats? Otherwise this >>> discussion is useless because all these features are not implementable >>> on top of SQLite and are way nontrivial to implement inside SQLite... >>> >>> Pavel >> -- >> Iker Arizmendi >> AT&T Labs - Research >> Speech and Image Processing Lab >> >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >> > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- Iker Arizmendi AT&T Labs - Research Speech and Image Processing Lab e: i...@research.att.com w: http://research.att.com p: 973-360-8516 _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
