-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You never answered the bit about whether indexing A can be done at the same time as B. (I get the feeling you have a certain design in mind and insist that SQLite changes to meet that design, rather than change your own design around the constraints of SQLite).
Emerson Clarke wrote: | In general worker threads is not an efficient solution to the problem | even if parallelism is available. There is nothing to be gained by | having one thread handing off A to another set of worker threads | because you have to synchronise on whatever queue/list/pipe you use to | transfer the information between threads anyway. The synchronisation is only at the begining and end of a "job". The big efficiency gain is that you can write synchronous code in the threads/processes which is way easier to write and less likely to have bugs. It is correct that you can write the same thing using a single thread and using asynchronous I/O but that is way harder. | So you may as well | just have multiple threads all performing their A,B, and C operations | in parallel and they will naturally interleave their operations based | on the synchronisation that you use. You haven't said why that won't work in practise. | threads with mutexes cant do in the foreground and only introduces | overhead and delay in the queueing process and context switching. Sounds like premature optimization to me. Compared to any I/O you are going to have to do (all of which involves interrupts, DMA transfers and context switching), other context switching is going to be negligible. | On the second point, i cant see any reason why the VDBE design would | be a problem for what i want to do. Did you run explain? Quite simply SQLite is designed to keep the database locked for the minimum amount of time, rather than having multiple VDBE's having shared locks and doing lots of concurrent locking for longer periods of time. Fundamentally changing the design of SQLite is not a quick thing. | Taking out a table wide lock is | fine, so long as i can make changes to that table from multiple | threads using the same connection and lock. "Taking out a table lock is fine, as long as the table is not locked" :-) If you have actual empirical evidence that SQLite is a bottleneck, then there are two other approaches that may help: - - Use multiple databases (eg split on first letter of document name) and use ATTACH to have them all available at the same time - - Use the new virtual tables feature and user defined functions. You can have the code consult other tables, store up information and do almost any other workflow and locking scheme you want. You can copy batches of data from your virtual tables into the real ones, have virtual functions that look in virtual tables for updates, then fallback on original data or any other design that suits you. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFFlbQUmOOfHg372QQRAn/OAJwKH6O3nWmHDRdn4ZsF+wcTaV71VQCg1zjs AmBJS5ujkKDou83gWc+Inj8= =GQLS -----END PGP SIGNATURE----- ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
