I just committed the benchmark tool in http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/, if you can take a look.
To run the scenario: - install jdk8: [nxuser@nx0012 pderop]$ java -version java version "1.8.0_40" Java(TM) SE Runtime Environment (build 1.8.0_40-b26) Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode) - checkout the loadtest from http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/ - go the the "loadtest" directory and start the test, just like this: $ java -server -jar bin/felix.jar Welcome to Apache Felix Gogo g! Starting benchmarks (each tested bundle will add/remove 630 components during bundle activation). [Starting benchmarks with no processing done in components start methods] Benchmarking bundle: org.apache.felix.dependencymanager.benchmark.dependencymanager .................................................. -> results in nanos: [139,129,744 | 143,957,687 | 152,157,581 | 319,631,722 | 919,838,078] Benchmarking bundle: org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel . Here, the first "org.apache.felix.dependencymanager.benchmark.dependencymanager" test (single-threaded) passes OK. But the next one hangs (org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel). it uses a fork join pool with size=4. and when typing "log warn", we see: "log warn" 2015.05.14 13:56:10 ERROR - Bundle: org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel - [ForkJoinPool-1-worker-3] Error processing tasks - java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) at java.util.HashMap$KeyIterator.next(HashMap.java:1453) at java.util.AbstractCollection.addAll(AbstractCollection.java:343) at org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:245) at org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:212) at org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:189) at org.apache.felix.framework.ServiceRegistry.getServiceReferences(ServiceRegistry.java:269) at org.apache.felix.framework.Felix.getServiceReferences(Felix.java:3577) at org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:3655) at org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:434) at org.apache.felix.dm.tracker.ServiceTracker.getInitialReferences(ServiceTracker.java:422) at org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:375) at org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:319) at org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:295) at org.apache.felix.dm.impl.ServiceDependencyImpl.start(ServiceDependencyImpl.java:226) at org.apache.felix.dm.impl.ComponentImpl.startDependencies(ComponentImpl.java:657) at org.apache.felix.dm.impl.ComponentImpl.performTransition(ComponentImpl.java:535) at org.apache.felix.dm.impl.ComponentImpl.handleChange(ComponentImpl.java:492) at org.apache.felix.dm.impl.ComponentImpl.access$5(ComponentImpl.java:482) at org.apache.felix.dm.impl.ComponentImpl$3.run(ComponentImpl.java:227) at org.apache.felix.dm.impl.DispatchExecutor.runTask(DispatchExecutor.java:182) at org.apache.felix.dm.impl.DispatchExecutor.run(DispatchExecutor.java:165) at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) (I will investigate also in my code to check if the problem does not come from me ?) cheers; /Pierre On Thu, May 14, 2015 at 1:47 PM, Pierre De Rop <pierre.de...@gmail.com> wrote: > Hi David, > > I don't know if it's me (a bug in my benchmark tool) or if if there is a > regression somewhere in the framework, by my parallel test does not pass > anymore. > > The test first starts with a single-threaded scenario, which passes OK > (org.apache.felix.dependencymanager.benchmark.dependencymanager), then when > the parallel test starts > (org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel) > it suddenly hangs, and when I type "log warn" under the gogo shell, I see > the following exception: > > (I'm using java8): > > $ java -server -Xmx4g -Xms4g -jar bin/felix.jar > ____________________________ > Welcome to Apache Felix Gogo > > Benchmarking bundle: > org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel . > > (here, the dependencymanager.parallel test hangs and when I type "log > warn", I see this:) > > g! log warn > 2015.05.14 13:31:03 ERROR - Bundle: > org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel - > [ForkJoinPool-1-worker-3] Error processing tasks - > java.util.ConcurrentModificationException > at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) > at java.util.HashMap$KeyIterator.next(HashMap.java:1453) > at java.util.AbstractCollection.addAll(AbstractCollection.java:343) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:245) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:212) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:189) > at > org.apache.felix.framework.ServiceRegistry.getServiceReferences(ServiceRegistry.java:269) > at > org.apache.felix.framework.Felix.getServiceReferences(Felix.java:3577) > at > org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:3655) > at > org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:434) > at > org.apache.felix.dm.tracker.ServiceTracker.getInitialReferences(ServiceTracker.java:422) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:375) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:319) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:295) > at > org.apache.felix.dm.impl.ServiceDependencyImpl.start(ServiceDependencyImpl.java:226) > at > org.apache.felix.dm.impl.ComponentImpl.startDependencies(ComponentImpl.java:657) > at > org.apache.felix.dm.impl.ComponentImpl.performTransition(ComponentImpl.java:535) > at > org.apache.felix.dm.impl.ComponentImpl.handleChange(ComponentImpl.java:492) > at > org.apache.felix.dm.impl.ComponentImpl.access$5(ComponentImpl.java:482) > at > org.apache.felix.dm.impl.ComponentImpl$3.run(ComponentImpl.java:227) > at > org.apache.felix.dm.impl.DispatchExecutor.runTask(DispatchExecutor.java:182) > at > org.apache.felix.dm.impl.DispatchExecutor.run(DispatchExecutor.java:165) > at > java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) > at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) > at > java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) > at > java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689) > at > java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) > > (If I configure my threadpool to 1, I have no problems, but with > threadpool=4, then I have the problem) > > I will investigate, but Ideally, may be it would be helpful if you could > also run the test by yourself; so I will commit soon something to reproduce > the problem in my sandbox. > > cheers; > /Pierre > > On Thu, May 14, 2015 at 11:11 AM, David Bosschaert < > david.bosscha...@gmail.com> wrote: > >> I've committed this now in >> http://svn.apache.org/viewvc?view=revision&revision=1679327 >> >> Curious to see what others are measuring. My tests were focused on >> multiple bundles/threads obtaining the same service, as that's were I >> saw a bit of contention. >> >> Cheers, >> >> David >> >> On 13 May 2015 at 15:10, Pierre De Rop <pierre.de...@gmail.com> wrote: >> > Hi David, >> > >> > I'm looking forward to test your improvements using the >> dependencymanager >> > benchmark tool ([1]). >> > >> > >> > [1] >> > >> http://svn.apache.org/viewvc/felix/trunk/dependencymanager/org.apache.felix.dependencymanager.benchmark/ >> > >> > /Pierre >> > >> > On Wed, May 13, 2015 at 3:02 PM, David Bosschaert < >> > david.bosscha...@gmail.com> wrote: >> > >> >> I have implemented the performance improvements that I was thinking of >> >> using Java 5 concurrency tools, they can be viewed at [1]. >> >> >> >> I wrote a little performance test suite [2] that tests multithreaded >> >> service registry performance (10 threads) from single / multiple >> >> bundles with either singleton services and Prototype Service Factory >> >> services and the results are quite impressive. I'm getting performance >> >> improvements compared to the current trunk from 8 times better than >> >> the original (800%) to more than 30 times better (3000%). >> >> >> >> Carsten has already reviewed the code (thanks Carsten!) and I'm >> >> planning to commit it to Felix tomorrow if nobody objects. >> >> >> >> Cheers, >> >> >> >> David >> >> >> >> [1] >> >> >> https://github.com/bosschaert/felix/commit/e6a1b06c6e66d9c98e6d81b91ef7003c8e725450 >> >> [2] >> >> >> https://github.com/bosschaert/coderthoughts/tree/master/service-registry-perftest/srperf >> >> >> >> On 23 March 2015 at 15:39, Richard S. Hall <he...@ungoverned.org> >> wrote: >> >> > On 3/23/15 10:17 , David Bosschaert wrote: >> >> >> >> >> >> On 23 March 2015 at 13:39, Richard S. Hall <he...@ungoverned.org> >> >> wrote: >> >> >>> >> >> >>> On 3/23/15 03:55 , Guillaume Nodet wrote: >> >> >>>> >> >> >>>> There's a call to interrupt() in Felix#acquireBundleLock(), not >> sure >> >> if >> >> >>>> it >> >> >>>> can be the culprit though. >> >> >>>> Interrupts could also be caused by a bundle being shutdown while >> one >> >> of >> >> >>>> its >> >> >>>> thread is waiting for a service, which should is a valid use case >> >> imho. >> >> >>>> Anyway, I think sanely reacting to a thread being interrupted >> would be >> >> >>>> good. >> >> >>> >> >> >>> >> >> >>> Yes, threads can be interrupted if they are holding a bundle lock >> and >> >> the >> >> >>> global lock holder needs the bundle lock. >> >> >>> >> >> >>> I admit that I do not recall why we ignore the interrupt here, but >> >> didn't >> >> >>> we >> >> >>> implement service lookup so that a bundle lock wasn't necessary? I >> >> >>> thought >> >> >>> we just checked for the validity of the bundle context before >> returning >> >> >>> or >> >> >>> something. Perhaps we felt there was no reason to be interrupted in >> >> that >> >> >>> case. I really don't know. >> >> >> >> >> >> I think that the Service Registry could be rewritten to be >> completely >> >> >> free of synchronized blocks using the Java 5 concurrency libraries, >> >> > >> >> > >> >> > Well, that just moves the sync blocks to the library, but yeah sure. >> >> > >> >> >> which I think would really be a better approach. There is too much >> >> >> locking going on in the current SR implementation IMHO. >> >> > >> >> > >> >> > I don't really think there is too much, but it is complicated. >> >> > Unfortunately, it is complicated to make sure that locks aren't held >> >> while >> >> > do service lookups and this is complicated because you can run into >> >> cycles, >> >> > etc. >> >> > >> >> > But feel free to try to simplify it. >> >> > >> >> >> >> >> >> This brings the question: can we move to Java 5 (or Java 6) for the >> >> >> Framework codebase? AFAIK we're currently still JDK 1.4 compatible >> but >> >> >> I would be surprised if there is anyone who still needs a JDK that >> >> >> went end-of-life 7 years ago. >> >> > >> >> > >> >> > At this point, it doesn't really matter to me. >> >> > >> >> > -> richard >> >> > >> >> >> >> >> >> Best regards, >> >> >> >> >> >> David >> >> > >> >> > >> >> >> > >
