On 30 Sep 2010, at 09:15, Felix Meschberger wrote: >> >> >> I dont know if its my machine or the way of testing but at the moment I can >> only get a max of 600 request per seconds on a json properties GET, flat >> regardless of concurrency out of sling. If I test Tomcat 6 on a html page I >> get 6K/s and the throughput appears to have some relationship to the >> concurrency of the requests. > > I don't want to start finger-pointing, but ... > > It looks like Jackrabbit has some issues with concurrent reads. > > Setting up a test with a Java profiler allowing to measure thread > contention (synchronization stuff) would probably show a hint (IIRC > YourKit Profiler has such functionality). > > Probably we have such contentions in Sling, too (would not surprise me), > and we should work on that.
So Yourkit did help upto a point, but to find the problem I had to write some monitor code and add it to the ItemManager to see where sessions were shared between threads. It turns out that the SystemSession in all AccessControlProviders is shared between threads and so becomes single threaded when evaluating ACLs. 90% of the time in Sling no ACLs are evaluated because the AccessManager has an optimisation that assumes that the JCR item tree is 100% read granted and looks for deny nodes to check. As soon as there is one deny node, access to items in that subtree become single threaded. Some people using SLing OOTB might see single threaded behaviour on read, and will certainly see single threaded behaviour on write. Unfortunately for Nakamura, we cant make the 100% read granted assumption, so the entire server is single threaded....... which is a bit of a problem for us as typical deployments have upto 60K concurrent users. One solution is to rewrite/patch the entire Access Control Provider stack to make it multi threaded, which I will have to do. Ian
