-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Emerson Clarke wrote: | For architectural reasons the above steps must be performed in that | order. This means that operations cant be separated or queued up in | the way that you suggested. Each step is dependent on the previous | step.
I was looking for parallelism expecting that indexing one document has no dependencies on other documents. Lets say that you have 3 documents A, B and C. Is it impossible to index B until after A's indexing is done? If that is the case then your indexing process is imposing serialization and there is nothing SQLite can do about that. If the indexing of A, B & C can be done in parallel, then you can have the indexing workers send updates to a single thread/process. That can aggregate information and then dump it all in a single transaction quickly. | Of course i know the simple answer, which is that the current api does | not support this. But im wondering why, and if there are any other | ways to achieve the desired performance. It is actually the internal design of SQLite that doesn't support this. ~ (Look at VDBE instructions for queries sometime). SQLite locks the whole database, rather than tables or rows. But the locks are held for the shortest time period possible. There are other database systems out there that have finer granularity locking, but that is more complicated. ~ There are also potential patent issues and SQLite deliberately uses old techniques to ensure no patent violation. SQLite's techniques have to work without centrally coordinating mechanism and within the confines of lowest common denominator functionality available on a variety of operating systems. Quite frankly if you cannot change the way you work with your data set then you may want to try a different database system such as Postgres. Of course that internally solves the problems by having a single process and doesn't have to worry about any other process/thread stepping on its toes. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFFlLG+mOOfHg372QQRAn+sAKCrImPgr0Ex1xo3XOXgWg0E7t3wWQCgtAAQ ZB5bs3fFntT4u3rABIGcOik= =c2pB -----END PGP SIGNATURE----- ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
