On Oct 19, 2010, at 9:01 AM, Doug wrote: > I'm not going to pretend to understand the SQLite source, but it seems > like having a mutex per PCache1 (ie the param passed in to > pcache1Fetch > and other cache functions) would be a good approach instead of the > global > mutex. But that approach wasn't taken, and I've found everything to > be > very well thought out in SQLite so I'm wondering if anyone can comment > on the why's.
It's so that page buffers can be managed more effectively. If SQLite reaches some internal limit (say the sqlite3_soft_heap_limit() limit), then when a page is required one connection, it can steal the buffer space from some other connections page cache. Dan. > Thanks > Doug > > >> -----Original Message----- >> From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- >> boun...@sqlite.org] On Behalf Of Pavel Ivanov >> Sent: Monday, October 18, 2010 2:15 PM >> To: General Discussion of SQLite Database >> Subject: Re: [sqlite] Scaling of Cache >> >> Jeff, >> >> I can agree that on Windows mutex performance can be awful especially >> in such frequently called place as pcache1Fetch. So you have only two >> options to solve the problem: >> 1) Split threads into different processes - make it one thread per >> process. >> 2) Make your own implementation of pcache. You can probably just take >> SQLite's implementation and change usage of mutex to spin lock for >> example, or make it work with TLS to make threads completely >> independent. In latter case you have to be sure that each connection >> is used solely in one thread though. >> >> >> Pavel >> >> On Mon, Oct 18, 2010 at 1:24 PM, Powell, Jeff <jeff.pow...@sap.com> >> wrote: >>> I did some profiling of our current application through the Intel >> Parallel Studio tools, and it identified the mutex in pcache1Fetch as >> the primary source of waits. Each thread acts on its own, sharing >> nothing with the other threads, so I would expect that there is zero >> waiting. >>> >>> -Jeff >>> >>> -----Original Message----- >>> From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- >> boun...@sqlite.org] On Behalf Of Pavel Ivanov >>> Sent: Monday, October 18, 2010 12:03 PM >>> To: General Discussion of SQLite Database >>> Subject: Re: [sqlite] Scaling of Cache >>> >>>> In pcache1Fetch, sqlite mutexes around the cache handling, which >> appears to be causing significant waits/scalability issues in my >> application. If I disable this mutex, the application crashes. >>> >>> Why do you think that this mutex causes significant waits? >>> Anyway ... >>> >>>> Is it possible to cache on a thread-by-thread basis such that no >> mutexes are necessary? >>> >>> You can make your own pcache implementation which will work >>> independently between threads and thus probably consume a lot more >>> memory than it is now. >>> >>> >>> Pavel >>> >>> On Mon, Oct 18, 2010 at 12:25 PM, Powell, Jeff <jeff.pow...@sap.com> >> wrote: >>>> I'm seeing some scaling issues (which I'm hoping someone else has >> encountered before). I have an application which utilizes multiple >> threads, each of which never writes to the database, and none of >> which >> shares its cache. >>>> >>>> In pcache1Fetch, sqlite mutexes around the cache handling, which >> appears to be causing significant waits/scalability issues in my >> application. If I disable this mutex, the application crashes. >>>> >>>> Is it possible to cache on a thread-by-thread basis such that no >> mutexes are necessary? >>>> >>>> -Jeff >>>> >>>> >>>> _______________________________________________ >>>> sqlite-users mailing list >>>> sqlite-users@sqlite.org >>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >>>> >>> _______________________________________________ >>> sqlite-users mailing list >>> sqlite-users@sqlite.org >>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >>> _______________________________________________ >>> sqlite-users mailing list >>> sqlite-users@sqlite.org >>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users >>> >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users@sqlite.org >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
