I'm not going to pretend to understand the SQLite source, but it seems like having a mutex per PCache1 (ie the param passed in to pcache1Fetch and other cache functions) would be a good approach instead of the global mutex. But that approach wasn't taken, and I've found everything to be very well thought out in SQLite so I'm wondering if anyone can comment on the why's.
Thanks Doug > -----Original Message----- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Pavel Ivanov > Sent: Monday, October 18, 2010 2:15 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Scaling of Cache > > Jeff, > > I can agree that on Windows mutex performance can be awful especially > in such frequently called place as pcache1Fetch. So you have only two > options to solve the problem: > 1) Split threads into different processes - make it one thread per > process. > 2) Make your own implementation of pcache. You can probably just take > SQLite's implementation and change usage of mutex to spin lock for > example, or make it work with TLS to make threads completely > independent. In latter case you have to be sure that each connection > is used solely in one thread though. > > > Pavel > > On Mon, Oct 18, 2010 at 1:24 PM, Powell, Jeff <jeff.pow...@sap.com> > wrote: > > I did some profiling of our current application through the Intel > Parallel Studio tools, and it identified the mutex in pcache1Fetch as > the primary source of waits. Each thread acts on its own, sharing > nothing with the other threads, so I would expect that there is zero > waiting. > > > > -Jeff > > > > -----Original Message----- > > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Pavel Ivanov > > Sent: Monday, October 18, 2010 12:03 PM > > To: General Discussion of SQLite Database > > Subject: Re: [sqlite] Scaling of Cache > > > >> In pcache1Fetch, sqlite mutexes around the cache handling, which > appears to be causing significant waits/scalability issues in my > application. If I disable this mutex, the application crashes. > > > > Why do you think that this mutex causes significant waits? > > Anyway ... > > > >> Is it possible to cache on a thread-by-thread basis such that no > mutexes are necessary? > > > > You can make your own pcache implementation which will work > > independently between threads and thus probably consume a lot more > > memory than it is now. > > > > > > Pavel > > > > On Mon, Oct 18, 2010 at 12:25 PM, Powell, Jeff <jeff.pow...@sap.com> > wrote: > >> I'm seeing some scaling issues (which I'm hoping someone else has > encountered before). I have an application which utilizes multiple > threads, each of which never writes to the database, and none of which > shares its cache. > >> > >> In pcache1Fetch, sqlite mutexes around the cache handling, which > appears to be causing significant waits/scalability issues in my > application. If I disable this mutex, the application crashes. > >> > >> Is it possible to cache on a thread-by-thread basis such that no > mutexes are necessary? > >> > >> -Jeff > >> > >> > >> _______________________________________________ > >> sqlite-users mailing list > >> sqlite-users@sqlite.org > >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > >> > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
