Hi Pierre, I'll take a look today.
Cheers, David On 14 May 2015 at 14:00, Pierre De Rop <pierre.de...@gmail.com> wrote: > I just committed the benchmark tool in > http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/, if you can > take a look. > > To run the scenario: > > - install jdk8: > > [nxuser@nx0012 pderop]$ java -version > java version "1.8.0_40" > Java(TM) SE Runtime Environment (build 1.8.0_40-b26) > Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode) > > - checkout the loadtest from > http://svn.apache.org/viewvc/felix/sandbox/pderop/loadtest/ > > - go the the "loadtest" directory and start the test, just like this: > > $ java -server -jar bin/felix.jar > Welcome to Apache Felix Gogo > > g! Starting benchmarks (each tested bundle will add/remove 630 components > during bundle activation). > > [Starting benchmarks with no processing done in components start > methods] > > Benchmarking bundle: > org.apache.felix.dependencymanager.benchmark.dependencymanager > .................................................. > -> results in nanos: [139,129,744 | 143,957,687 | 152,157,581 | 319,631,722 > | 919,838,078] > > Benchmarking bundle: > org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel . > > > Here, the first > "org.apache.felix.dependencymanager.benchmark.dependencymanager" test > (single-threaded) passes OK. But the next one hangs > (org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel). > it uses a fork join pool with size=4. > > and when typing "log warn", we see: > > "log warn" > > 2015.05.14 13:56:10 ERROR - Bundle: > org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel - > [ForkJoinPool-1-worker-3] Error processing tasks - > java.util.ConcurrentModificationException > at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) > at java.util.HashMap$KeyIterator.next(HashMap.java:1453) > at java.util.AbstractCollection.addAll(AbstractCollection.java:343) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:245) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:212) > at > org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:189) > at > org.apache.felix.framework.ServiceRegistry.getServiceReferences(ServiceRegistry.java:269) > at > org.apache.felix.framework.Felix.getServiceReferences(Felix.java:3577) > at > org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:3655) > at > org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:434) > at > org.apache.felix.dm.tracker.ServiceTracker.getInitialReferences(ServiceTracker.java:422) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:375) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:319) > at > org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:295) > at > org.apache.felix.dm.impl.ServiceDependencyImpl.start(ServiceDependencyImpl.java:226) > at > org.apache.felix.dm.impl.ComponentImpl.startDependencies(ComponentImpl.java:657) > at > org.apache.felix.dm.impl.ComponentImpl.performTransition(ComponentImpl.java:535) > at > org.apache.felix.dm.impl.ComponentImpl.handleChange(ComponentImpl.java:492) > at > org.apache.felix.dm.impl.ComponentImpl.access$5(ComponentImpl.java:482) > at > org.apache.felix.dm.impl.ComponentImpl$3.run(ComponentImpl.java:227) > at > org.apache.felix.dm.impl.DispatchExecutor.runTask(DispatchExecutor.java:182) > at > org.apache.felix.dm.impl.DispatchExecutor.run(DispatchExecutor.java:165) > at > java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) > at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) > at > java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) > at > java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689) > at > java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) > > > (I will investigate also in my code to check if the problem does not come > from me ?) > > cheers; > /Pierre > > > On Thu, May 14, 2015 at 1:47 PM, Pierre De Rop <pierre.de...@gmail.com> > wrote: > >> Hi David, >> >> I don't know if it's me (a bug in my benchmark tool) or if if there is a >> regression somewhere in the framework, by my parallel test does not pass >> anymore. >> >> The test first starts with a single-threaded scenario, which passes OK >> (org.apache.felix.dependencymanager.benchmark.dependencymanager), then when >> the parallel test starts >> (org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel) >> it suddenly hangs, and when I type "log warn" under the gogo shell, I see >> the following exception: >> >> (I'm using java8): >> >> $ java -server -Xmx4g -Xms4g -jar bin/felix.jar >> ____________________________ >> Welcome to Apache Felix Gogo >> >> Benchmarking bundle: >> org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel . >> >> (here, the dependencymanager.parallel test hangs and when I type "log >> warn", I see this:) >> >> g! log warn >> 2015.05.14 13:31:03 ERROR - Bundle: >> org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel - >> [ForkJoinPool-1-worker-3] Error processing tasks - >> java.util.ConcurrentModificationException >> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) >> at java.util.HashMap$KeyIterator.next(HashMap.java:1453) >> at java.util.AbstractCollection.addAll(AbstractCollection.java:343) >> at >> org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:245) >> at >> org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:212) >> at >> org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:189) >> at >> org.apache.felix.framework.ServiceRegistry.getServiceReferences(ServiceRegistry.java:269) >> at >> org.apache.felix.framework.Felix.getServiceReferences(Felix.java:3577) >> at >> org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:3655) >> at >> org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:434) >> at >> org.apache.felix.dm.tracker.ServiceTracker.getInitialReferences(ServiceTracker.java:422) >> at >> org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:375) >> at >> org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:319) >> at >> org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:295) >> at >> org.apache.felix.dm.impl.ServiceDependencyImpl.start(ServiceDependencyImpl.java:226) >> at >> org.apache.felix.dm.impl.ComponentImpl.startDependencies(ComponentImpl.java:657) >> at >> org.apache.felix.dm.impl.ComponentImpl.performTransition(ComponentImpl.java:535) >> at >> org.apache.felix.dm.impl.ComponentImpl.handleChange(ComponentImpl.java:492) >> at >> org.apache.felix.dm.impl.ComponentImpl.access$5(ComponentImpl.java:482) >> at >> org.apache.felix.dm.impl.ComponentImpl$3.run(ComponentImpl.java:227) >> at >> org.apache.felix.dm.impl.DispatchExecutor.runTask(DispatchExecutor.java:182) >> at >> org.apache.felix.dm.impl.DispatchExecutor.run(DispatchExecutor.java:165) >> at >> java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) >> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) >> at >> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) >> at >> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689) >> at >> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) >> >> (If I configure my threadpool to 1, I have no problems, but with >> threadpool=4, then I have the problem) >> >> I will investigate, but Ideally, may be it would be helpful if you could >> also run the test by yourself; so I will commit soon something to reproduce >> the problem in my sandbox. >> >> cheers; >> /Pierre >> >> On Thu, May 14, 2015 at 11:11 AM, David Bosschaert < >> david.bosscha...@gmail.com> wrote: >> >>> I've committed this now in >>> http://svn.apache.org/viewvc?view=revision&revision=1679327 >>> >>> Curious to see what others are measuring. My tests were focused on >>> multiple bundles/threads obtaining the same service, as that's were I >>> saw a bit of contention. >>> >>> Cheers, >>> >>> David >>> >>> On 13 May 2015 at 15:10, Pierre De Rop <pierre.de...@gmail.com> wrote: >>> > Hi David, >>> > >>> > I'm looking forward to test your improvements using the >>> dependencymanager >>> > benchmark tool ([1]). >>> > >>> > >>> > [1] >>> > >>> http://svn.apache.org/viewvc/felix/trunk/dependencymanager/org.apache.felix.dependencymanager.benchmark/ >>> > >>> > /Pierre >>> > >>> > On Wed, May 13, 2015 at 3:02 PM, David Bosschaert < >>> > david.bosscha...@gmail.com> wrote: >>> > >>> >> I have implemented the performance improvements that I was thinking of >>> >> using Java 5 concurrency tools, they can be viewed at [1]. >>> >> >>> >> I wrote a little performance test suite [2] that tests multithreaded >>> >> service registry performance (10 threads) from single / multiple >>> >> bundles with either singleton services and Prototype Service Factory >>> >> services and the results are quite impressive. I'm getting performance >>> >> improvements compared to the current trunk from 8 times better than >>> >> the original (800%) to more than 30 times better (3000%). >>> >> >>> >> Carsten has already reviewed the code (thanks Carsten!) and I'm >>> >> planning to commit it to Felix tomorrow if nobody objects. >>> >> >>> >> Cheers, >>> >> >>> >> David >>> >> >>> >> [1] >>> >> >>> https://github.com/bosschaert/felix/commit/e6a1b06c6e66d9c98e6d81b91ef7003c8e725450 >>> >> [2] >>> >> >>> https://github.com/bosschaert/coderthoughts/tree/master/service-registry-perftest/srperf >>> >> >>> >> On 23 March 2015 at 15:39, Richard S. Hall <he...@ungoverned.org> >>> wrote: >>> >> > On 3/23/15 10:17 , David Bosschaert wrote: >>> >> >> >>> >> >> On 23 March 2015 at 13:39, Richard S. Hall <he...@ungoverned.org> >>> >> wrote: >>> >> >>> >>> >> >>> On 3/23/15 03:55 , Guillaume Nodet wrote: >>> >> >>>> >>> >> >>>> There's a call to interrupt() in Felix#acquireBundleLock(), not >>> sure >>> >> if >>> >> >>>> it >>> >> >>>> can be the culprit though. >>> >> >>>> Interrupts could also be caused by a bundle being shutdown while >>> one >>> >> of >>> >> >>>> its >>> >> >>>> thread is waiting for a service, which should is a valid use case >>> >> imho. >>> >> >>>> Anyway, I think sanely reacting to a thread being interrupted >>> would be >>> >> >>>> good. >>> >> >>> >>> >> >>> >>> >> >>> Yes, threads can be interrupted if they are holding a bundle lock >>> and >>> >> the >>> >> >>> global lock holder needs the bundle lock. >>> >> >>> >>> >> >>> I admit that I do not recall why we ignore the interrupt here, but >>> >> didn't >>> >> >>> we >>> >> >>> implement service lookup so that a bundle lock wasn't necessary? I >>> >> >>> thought >>> >> >>> we just checked for the validity of the bundle context before >>> returning >>> >> >>> or >>> >> >>> something. Perhaps we felt there was no reason to be interrupted in >>> >> that >>> >> >>> case. I really don't know. >>> >> >> >>> >> >> I think that the Service Registry could be rewritten to be >>> completely >>> >> >> free of synchronized blocks using the Java 5 concurrency libraries, >>> >> > >>> >> > >>> >> > Well, that just moves the sync blocks to the library, but yeah sure. >>> >> > >>> >> >> which I think would really be a better approach. There is too much >>> >> >> locking going on in the current SR implementation IMHO. >>> >> > >>> >> > >>> >> > I don't really think there is too much, but it is complicated. >>> >> > Unfortunately, it is complicated to make sure that locks aren't held >>> >> while >>> >> > do service lookups and this is complicated because you can run into >>> >> cycles, >>> >> > etc. >>> >> > >>> >> > But feel free to try to simplify it. >>> >> > >>> >> >> >>> >> >> This brings the question: can we move to Java 5 (or Java 6) for the >>> >> >> Framework codebase? AFAIK we're currently still JDK 1.4 compatible >>> but >>> >> >> I would be surprised if there is anyone who still needs a JDK that >>> >> >> went end-of-life 7 years ago. >>> >> > >>> >> > >>> >> > At this point, it doesn't really matter to me. >>> >> > >>> >> > -> richard >>> >> > >>> >> >> >>> >> >> Best regards, >>> >> >> >>> >> >> David >>> >> > >>> >> > >>> >> >>> >> >>
