> From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On > Behalf Of Simon Slavin
> Your description of your fault suggests that at least two of your > threads/processes are trying to use the same connection to the database at > the same time We have a bunch of reads happening at the same time on one connection. The writes were serialized with a write lock but the reads were just doing whatever they wanted. It seems like this is our mistake. > > We are trying to implement a system where all writes occur on one > connection ... Does it > sound like we are doing something fundamentally wrong by trying to share > connections across threads in this way? > > If you don't have your own mutex locks, then yes. Don't try to set up a > situation where two transactions are happening at the same time with the > same connection. The standard way to do that correctly is to give each > thread its own connection to the database and let SQLite do all the locking > necessary. However it's not difficult to make your own mutex system work, > it just seems like your own one isn't working properly. > Great info. I don't think we can afford to have a connection per thread. We have an arbitrarily large number of threads coming and going all the time and a combination of the amount of memory each connection is taking up and how long it takes to connect to the db it looks like limiting the number of connections would be the direction we go if we can get it to work. Maybe a pool of connections. We'll try serializing the reads on one connection and see what performance that gives us. It seems like we will be giving up a lot of parallelism, but as we are not sure yet exactly how the locking in the DB works that may not be as bad system impact as we fear. Thanks for the info Alex _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
