Pavel, I do not have a requirement of persistence in my current design, but I expect that we might extend this shared-queue solution to more areas of the server and will require some sort of persistence then. That is one of the main reasons I do not want to use IPC queues (there are other reasons like fixed message sizes, minimal support for queue/message level metadata).
One of the main attractions of SQLite-based solution is to be able to perform all kind of queries on the queue itself (from the point of view of maintenance scripts/production support). In my experience, if there are lots of services sharing different types of messages over an IPC shared queue, sometimes you run into a situation where the queue starts backing up and there is no way for production support folks to determine which particular service is causing the backup (by sending messages too fast, or consuming them really slow). And, in the end the only solution is to bounce all the services (instead of just bouncing the culprit) and we never discover the root cause of the backup. If I use a SQLite-backed queue, I can simply use the command line shell and run queries like: select sender, receiver, count(*) from queue group by sender, receiver; Or any combination of message metadata to analyze the current state of the queue. Also, I can easily modify my queue APIs to just update a used flag, instead of deleting the message from the db. This way, I can analyze all the messages at the end of day and determine all kinds of statistics (like how long does a particular type of message sits in the queue). In short, using a SQLite-backed queue solution gives me a lot of options that a simple IPC based (and, for that matter, even a professional Messaging Product) does not give. Jay, I did think of implementing a VFS for the shared-memory, but as you mentioned a file-based DB with all syncs off might be a simpler trade-off. Alexey, As Simon said, having a socket based daemon solution is something I want to avoid because it adds another layer to the architecture. Thanks, Manuj On Mon, May 10, 2010 at 10:56 AM, Simon Slavin <slav...@bigfraud.org> wrote: > > On 10 May 2010, at 4:47pm, Alexey Pechnikov wrote: > > > TCP-socket listening daemon + SQLite in-memory database may be helpful. > > Yes. You can make one process, which handles all your SQLite transactions, > and receives its orders from other processes via inter-process calls or > TCP/IP. I've seen a few solutions which do this and they work fine. But > that process will itself become some sort of bottleneck if you have many > processes calling it. And I think that the original post in this thread > described a situation where that was not a good solution. > > Simon. > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
