[ https://issues.apache.org/jira/browse/FELIX-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Felix Meschberger updated FELIX-3000: ------------------------------------- Description: We have a strange situation on a Framework 3.0.7 based system here which is not reproducible on all platforms. We can track down a system freeze/deadlock to three threads all contending for bundle locks and the global lock. See attached FELIX-3000-stacktrace.txt for the stack trace. Looking at the Framework source, particularly acquireBundleLock and acquireGlobalLock I cannot see where this "deadlock" can occur. The only hint I have is a note in the Felix.registerService: > // TODO: CONCURRENCY - Reconsider firing event here, outside of the > // bundle lock. I wonder whether this situation can be fixed with moving the service registration event ? Looking at the code it seems to have not been changed. Thus I report this against 3.0.7 where we saw this and 3.2.2 being the latest release. was: We have a strange situation on a Framework 3.0.7 based system here which is not reproducible on all platforms. We can track down a system freeze/deadlock to three threads all contending for bundle locks and the global lock: > "127.0.0.1 [1305717927375] GET /xxx.html HTTP/1.0" daemon prio=3 > tid=0x0000000106203000 nid=0x2c01b in Object.wait() [0x00007ffe939f7000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4712) > - locked <0x00007ffeabca9908> (a [Ljava.lang.Object;) > at > org.apache.felix.framework.Felix$FelixResolver.markBundleResolved(Felix.java:4220) > at > org.apache.felix.framework.Felix$FelixResolver.markResolvedModules(Felix.java:4200) > at > org.apache.felix.framework.Felix$FelixResolver.resolve(Felix.java:3999) > at org.apache.felix.framework.Felix.resolveBundle(Felix.java:3402) > at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1594) > at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:904) > at > org.apache.sling.commons.classloader.impl.PackageAdminClassLoader.findClass(PackageAdminClassLoader.java:150) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > - locked <0x00007ffecc57d040> (a > org.apache.sling.commons.classloader.impl.PackageAdminClassLoader) > at > org.apache.sling.commons.classloader.impl.PackageAdminClassLoader.loadClass(PackageAdminClassLoader.java:174) > at java.lang.ClassLoader.loadClass(ClassLoader.java:296) > - locked <0x00007ffecc58b260> (a > org.apache.sling.jcr.classloader.internal.DynamicRepositoryClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > at > org.apache.jsp.apps.wci.components.page.page.page_jsp._jspService(page_jsp.java:170) > at > org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) > at > org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:419) > at > org.apache.sling.scripting.jsp.JspServletWrapperAdapter.service(JspServletWrapperAdapter.java:59) > at > org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:173) > at > org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:84) > ... > > "ObservationManager" daemon prio=3 tid=0x000000010232e000 nid=0x30 in > Object.wait() [0x00007ffef24fe000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at org.apache.felix.framework.Felix.acquireGlobalLock(Felix.java:4785) > - locked <0x00007ffeabca9908> (a [Ljava.lang.Object;) > at org.apache.felix.framework.Felix.access$400(Felix.java:80) > at > org.apache.felix.framework.Felix$FelixResolver.resolve(Felix.java:4043) > at > org.apache.felix.framework.ModuleImpl.searchDynamicImports(ModuleImpl.java:1412) > at > org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:734) > at org.apache.felix.framework.ModuleImpl.access$400(ModuleImpl.java:71) > at > org.apache.felix.framework.ModuleImpl$ModuleClassLoader.loadClass(ModuleImpl.java:1768) > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > at > org.apache.felix.framework.ModuleImpl.getClassByDelegation(ModuleImpl.java:645) > at > org.apache.felix.framework.ServiceRegistrationImpl$ServiceReferenceImpl.isAssignableTo(ServiceRegistrationImpl.java:544) > at > org.apache.felix.framework.util.Util.isServiceAssignable(Util.java:275) > at > org.apache.felix.framework.Felix.getServiceReferences(Felix.java:2920) > at > org.apache.felix.framework.Felix.getAllowedServiceReferences(Felix.java:2970) > at > org.apache.felix.framework.BundleContextImpl.getServiceReferences(BundleContextImpl.java:309) > at > org.apache.felix.eventadmin.impl.handler.BlacklistingHandlerTasks.createHandlerTasks(BlacklistingHandlerTasks.java:102) > at > org.apache.felix.eventadmin.impl.EventAdminImpl.postEvent(EventAdminImpl.java:92) > at > org.apache.felix.eventadmin.impl.security.EventAdminSecurityDecorator.postEvent(EventAdminSecurityDecorator.java:77) > at > org.apache.sling.jcr.resource.internal.JcrResourceListener.sendOsgiEvent(JcrResourceListener.java:224) > at > org.apache.sling.jcr.resource.internal.JcrResourceListener.onEvent(JcrResourceListener.java:176) > at > org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:248) > at > org.apache.jackrabbit.core.observation.ObservationDispatcher.run(ObservationDispatcher.java:161) > at java.lang.Thread.run(Thread.java:662) > > "OsgiInstallerImpl" daemon prio=3 tid=0x0000000104f71800 nid=0x39 in > Object.wait() [0x00007ffef12fc000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:485) > at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4712) > - locked <0x00007ffeabca9908> (a [Ljava.lang.Object;) > at org.apache.felix.framework.Felix.registerService(Felix.java:2813) > at > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:449) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:460) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1009) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:333) > at > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:283) > at > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) > at > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:871) > at > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:733) > at > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:662) > at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:3735) > at org.apache.felix.framework.Felix.access$000(Felix.java:80) > at org.apache.felix.framework.Felix$2.serviceChanged(Felix.java:722) > at > org.apache.felix.framework.ServiceRegistry.registerService(ServiceRegistry.java:107) > at org.apache.felix.framework.Felix.registerService(Felix.java:2847) > at > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:449) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.registerComponentService(AbstractComponentManager.java:460) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager$Unsatisfied.activate(AbstractComponentManager.java:1009) > at > org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:333) > at > org.apache.felix.scr.impl.manager.DependencyManager.serviceAdded(DependencyManager.java:283) > at > org.apache.felix.scr.impl.manager.DependencyManager.serviceChanged(DependencyManager.java:170) > at > org.apache.felix.framework.util.EventDispatcher.invokeServiceListenerCallback(EventDispatcher.java:871) > at > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:733) > at > org.apache.felix.framework.util.EventDispatcher.fireServiceEvent(EventDispatcher.java:662) > at org.apache.felix.framework.Felix.fireServiceEvent(Felix.java:3735) > at org.apache.felix.framework.Felix.access$000(Felix.java:80) > at org.apache.felix.framework.Felix$2.serviceChanged(Felix.java:722) > at > org.apache.felix.framework.ServiceRegistry.registerService(ServiceRegistry.java:107) > at org.apache.felix.framework.Felix.registerService(Felix.java:2847) > at > org.apache.felix.framework.BundleContextImpl.registerService(BundleContextImpl.java:251) > at > org.apache.sling.commons.classloader.impl.Activator.registerManagerFactory(Activator.java:78) > at > org.apache.sling.commons.classloader.impl.Activator.bundleChanged(Activator.java:122) > at > org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:807) > at > org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:729) > at > org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:610) > at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:3724) > at org.apache.felix.framework.Felix.updateBundle(Felix.java:2030) > at org.apache.felix.framework.BundleImpl.update(BundleImpl.java:940) > at > org.apache.sling.installer.core.impl.tasks.BundleUpdateTask.execute(BundleUpdateTask.java:78) > at > org.apache.sling.installer.core.impl.OsgiInstallerImpl.executeTasks(OsgiInstallerImpl.java:522) > at > org.apache.sling.installer.core.impl.OsgiInstallerImpl.run(OsgiInstallerImpl.java:175) Looking at the Framework source, particularly acquireBundleLock and acquireGlobalLock I cannot see where this "deadlock" can occur. The only hint I have is a note in the Felix.registerService: > // TODO: CONCURRENCY - Reconsider firing event here, outside of the > // bundle lock. I wonder whether this situation can be fixed with moving the service registration event ? Looking at the code it seems to have not been changed. Thus I report this against 3.0.7 where we saw this and 3.2.2 being the latest release. > Move sending service registered event out of bundle lock > -------------------------------------------------------- > > Key: FELIX-3000 > URL: https://issues.apache.org/jira/browse/FELIX-3000 > Project: Felix > Issue Type: Improvement > Components: Framework > Affects Versions: framework-3.0.7, framework-3.2.2 > Reporter: Felix Meschberger > Attachments: FELIX-3000-stacktrace.txt, FELIX-3000.patch > > > We have a strange situation on a Framework 3.0.7 based system here which is > not reproducible on all platforms. > We can track down a system freeze/deadlock to three threads all contending > for bundle locks and the global lock. See attached FELIX-3000-stacktrace.txt > for the stack trace. > Looking at the Framework source, particularly acquireBundleLock and > acquireGlobalLock I cannot see where this "deadlock" can occur. > The only hint I have is a note in the Felix.registerService: > > // TODO: CONCURRENCY - Reconsider firing event here, outside of the > > // bundle lock. > I wonder whether this situation can be fixed with moving the service > registration event ? > Looking at the code it seems to have not been changed. Thus I report this > against 3.0.7 where we saw this and 3.2.2 being the latest release. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira