Hi David,
I don't know if it's me (a bug in my benchmark tool) or if if there is a
regression somewhere in the framework, by my parallel test does not pass
anymore.
The test first starts with a single-threaded scenario, which passes OK
(org.apache.felix.dependencymanager.benchmark.dependencymanager), then when
the parallel test starts
(org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel)
it suddenly hangs, and when I type "log warn" under the gogo shell, I see
the following exception:
(I'm using java8):
$ java -server -Xmx4g -Xms4g -jar bin/felix.jar
____________________________
Welcome to Apache Felix Gogo
Benchmarking bundle:
org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel .
(here, the dependencymanager.parallel test hangs and when I type "log
warn", I see this:)
g! log warn
2015.05.14 13:31:03 ERROR - Bundle:
org.apache.felix.dependencymanager.benchmark.dependencymanager.parallel -
[ForkJoinPool-1-worker-3] Error processing tasks -
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429)
at java.util.HashMap$KeyIterator.next(HashMap.java:1453)
at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
at
org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:245)
at
org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:212)
at
org.apache.felix.framework.capabilityset.CapabilitySet.match(CapabilitySet.java:189)
at
org.apache.felix.framework.ServiceRegistry.getServiceReferences(ServiceRegistry.java:269)
at
org.apache.felix.framework.Felix.getServiceReferences(Felix.java:3577)
at
org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:3655)
at
org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:434)
at
org.apache.felix.dm.tracker.ServiceTracker.getInitialReferences(ServiceTracker.java:422)
at
org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:375)
at
org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:319)
at
org.apache.felix.dm.tracker.ServiceTracker.open(ServiceTracker.java:295)
at
org.apache.felix.dm.impl.ServiceDependencyImpl.start(ServiceDependencyImpl.java:226)
at
org.apache.felix.dm.impl.ComponentImpl.startDependencies(ComponentImpl.java:657)
at
org.apache.felix.dm.impl.ComponentImpl.performTransition(ComponentImpl.java:535)
at
org.apache.felix.dm.impl.ComponentImpl.handleChange(ComponentImpl.java:492)
at
org.apache.felix.dm.impl.ComponentImpl.access$5(ComponentImpl.java:482)
at
org.apache.felix.dm.impl.ComponentImpl$3.run(ComponentImpl.java:227)
at
org.apache.felix.dm.impl.DispatchExecutor.runTask(DispatchExecutor.java:182)
at
org.apache.felix.dm.impl.DispatchExecutor.run(DispatchExecutor.java:165)
at
java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689)
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
(If I configure my threadpool to 1, I have no problems, but with
threadpool=4, then I have the problem)
I will investigate, but Ideally, may be it would be helpful if you could
also run the test by yourself; so I will commit soon something to reproduce
the problem in my sandbox.
cheers;
/Pierre
On Thu, May 14, 2015 at 11:11 AM, David Bosschaert <
david.bosscha...@gmail.com> wrote:
> I've committed this now in
> http://svn.apache.org/viewvc?view=revision&revision=1679327
>
> Curious to see what others are measuring. My tests were focused on
> multiple bundles/threads obtaining the same service, as that's were I
> saw a bit of contention.
>
> Cheers,
>
> David
>
> On 13 May 2015 at 15:10, Pierre De Rop <pierre.de...@gmail.com> wrote:
> > Hi David,
> >
> > I'm looking forward to test your improvements using the dependencymanager
> > benchmark tool ([1]).
> >
> >
> > [1]
> >
> http://svn.apache.org/viewvc/felix/trunk/dependencymanager/org.apache.felix.dependencymanager.benchmark/
> >
> > /Pierre
> >
> > On Wed, May 13, 2015 at 3:02 PM, David Bosschaert <
> > david.bosscha...@gmail.com> wrote:
> >
> >> I have implemented the performance improvements that I was thinking of
> >> using Java 5 concurrency tools, they can be viewed at [1].
> >>
> >> I wrote a little performance test suite [2] that tests multithreaded
> >> service registry performance (10 threads) from single / multiple
> >> bundles with either singleton services and Prototype Service Factory
> >> services and the results are quite impressive. I'm getting performance
> >> improvements compared to the current trunk from 8 times better than
> >> the original (800%) to more than 30 times better (3000%).
> >>
> >> Carsten has already reviewed the code (thanks Carsten!) and I'm
> >> planning to commit it to Felix tomorrow if nobody objects.
> >>
> >> Cheers,
> >>
> >> David
> >>
> >> [1]
> >>
> https://github.com/bosschaert/felix/commit/e6a1b06c6e66d9c98e6d81b91ef7003c8e725450
> >> [2]
> >>
> https://github.com/bosschaert/coderthoughts/tree/master/service-registry-perftest/srperf
> >>
> >> On 23 March 2015 at 15:39, Richard S. Hall <he...@ungoverned.org>
> wrote:
> >> > On 3/23/15 10:17 , David Bosschaert wrote:
> >> >>
> >> >> On 23 March 2015 at 13:39, Richard S. Hall <he...@ungoverned.org>
> >> wrote:
> >> >>>
> >> >>> On 3/23/15 03:55 , Guillaume Nodet wrote:
> >> >>>>
> >> >>>> There's a call to interrupt() in Felix#acquireBundleLock(), not
> sure
> >> if
> >> >>>> it
> >> >>>> can be the culprit though.
> >> >>>> Interrupts could also be caused by a bundle being shutdown while
> one
> >> of
> >> >>>> its
> >> >>>> thread is waiting for a service, which should is a valid use case
> >> imho.
> >> >>>> Anyway, I think sanely reacting to a thread being interrupted
> would be
> >> >>>> good.
> >> >>>
> >> >>>
> >> >>> Yes, threads can be interrupted if they are holding a bundle lock
> and
> >> the
> >> >>> global lock holder needs the bundle lock.
> >> >>>
> >> >>> I admit that I do not recall why we ignore the interrupt here, but
> >> didn't
> >> >>> we
> >> >>> implement service lookup so that a bundle lock wasn't necessary? I
> >> >>> thought
> >> >>> we just checked for the validity of the bundle context before
> returning
> >> >>> or
> >> >>> something. Perhaps we felt there was no reason to be interrupted in
> >> that
> >> >>> case. I really don't know.
> >> >>
> >> >> I think that the Service Registry could be rewritten to be completely
> >> >> free of synchronized blocks using the Java 5 concurrency libraries,
> >> >
> >> >
> >> > Well, that just moves the sync blocks to the library, but yeah sure.
> >> >
> >> >> which I think would really be a better approach. There is too much
> >> >> locking going on in the current SR implementation IMHO.
> >> >
> >> >
> >> > I don't really think there is too much, but it is complicated.
> >> > Unfortunately, it is complicated to make sure that locks aren't held
> >> while
> >> > do service lookups and this is complicated because you can run into
> >> cycles,
> >> > etc.
> >> >
> >> > But feel free to try to simplify it.
> >> >
> >> >>
> >> >> This brings the question: can we move to Java 5 (or Java 6) for the
> >> >> Framework codebase? AFAIK we're currently still JDK 1.4 compatible
> but
> >> >> I would be surprised if there is anyone who still needs a JDK that
> >> >> went end-of-life 7 years ago.
> >> >
> >> >
> >> > At this point, it doesn't really matter to me.
> >> >
> >> > -> richard
> >> >
> >> >>
> >> >> Best regards,
> >> >>
> >> >> David
> >> >
> >> >
> >>
>
