Thanks Miyuru, I will check the fix too..
On Fri, Nov 15, 2013 at 12:49 PM, Miyuru Wanninayaka <[email protected]>wrote: > > > > On Fri, Nov 15, 2013 at 12:29 PM, Miyuru Wanninayaka <[email protected]>wrote: > >> Seems once proxy service deploy/undploy lock in axis2 config instead of >> synapse config, no more deadlocks :) >> >> I have to >> >> - private AxisConfiguration axisConfiguration = null; >> + private AxisConfiguration axisConfiguration = new >> AxisConfiguration(); >> >> because in MultiXMLConfigurationBuilderTest, it creates a synapse config >> without axisConfig which will cause NPE in synchronization block. >> >> I'm currently running tests with these changes. >> >> >> On Fri, Nov 15, 2013 at 10:37 AM, Miyuru Wanninayaka <[email protected]>wrote: >> >>> May be for proxy service undeployment, synapse can lock on axis2 config >>> instead of synapse config, which will avoid deadlocking. >>> >>> >>> On Fri, Nov 15, 2013 at 10:22 AM, Miyuru Wanninayaka <[email protected]>wrote: >>> >>>> I have observed this few times even without patch0002. >>>> >>>> When analyzed deadloak flow it seems >>>> hot deployment of main, fault sequences and deleting ESB artifacts from >>>> admin services triggers in parallel. >>>> >>>> in hot deployment, axis2 thread first get a lock of axis2 configuration >>>> and then calls synapse deployer.undeploy which will lock on synapse >>>> configuration. >>>> during admin service call, it will directly call synapse to undeploy >>>> artifacts which will first lock on synapse config and in case of proxy, it >>>> will cause related axis2 service to undeploy and try to acquire lock of >>>> axis2 config which will become a deadlock. >>>> >>>> I have modified abstract ESBIntegrationTest.cleanup() method to wait >>>> few seconds to let main and fault sequence to hot update. Seems deadlock >>>> does not happens after that. >>>> >>>> >>>> On Thu, Nov 14, 2013 at 10:17 PM, Miyuru Wanninayaka >>>> <[email protected]>wrote: >>>> >>>>> Seems patch0002 causing deadlock. Once patch0002 removed from pack, >>>>> tests are running without any issue. >>>>> >>>>> >>>>> On Thu, Nov 14, 2013 at 9:31 PM, Krishantha Samaraweera < >>>>> [email protected]> wrote: >>>>> >>>>>> Integration tests hang at the early stage and following dead lock >>>>>> situation detected in ESB Alpha pack. >>>>>> >>>>>> Name: http-nio-9443-exec-19 >>>>>> State: BLOCKED on >>>>>> org.apache.axis2.engine.AxisConfiguration@510b02daowned by: >>>>>> pool-15-thread-1 >>>>>> Total blocked: 2 Total waited: 29 >>>>>> >>>>>> Stack trace: >>>>>> >>>>>> >>>>>> org.apache.axis2.engine.AxisConfiguration.removeService(AxisConfiguration.java:692) >>>>>> >>>>>> org.apache.synapse.config.SynapseConfiguration.removeProxyService(SynapseConfiguration.java:1037) >>>>>> - locked org.apache.synapse.config.SynapseConfiguration@73e64d34 >>>>>> >>>>>> org.wso2.carbon.proxyadmin.service.ProxyServiceAdmin.deleteProxyService(ProxyServiceAdmin.java:398) >>>>>> sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source) >>>>>> >>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>> java.lang.reflect.Method.invoke(Method.java:597) >>>>>> >>>>>> org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212) >>>>>> >>>>>> org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117) >>>>>> >>>>>> org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40) >>>>>> >>>>>> org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:110) >>>>>> org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>> >>>>>> org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172) >>>>>> >>>>>> org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146) >>>>>> >>>>>> org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java: >>>>>> >>>>>> >>>>>> --------------------- >>>>>> >>>>>> Name: pool-15-thread-1 >>>>>> State: BLOCKED on >>>>>> org.apache.synapse.config.SynapseConfiguration@73e64d34 owned by: >>>>>> http-nio-9443-exec-19 >>>>>> Total blocked: 1 Total waited: 17 >>>>>> >>>>>> Stack trace: >>>>>> >>>>>> >>>>>> org.apache.synapse.config.SynapseConfiguration.getDefinedSequences(SynapseConfiguration.java:289) >>>>>> >>>>>> org.apache.synapse.deployers.SequenceDeployer.updateSynapseArtifact(SequenceDeployer.java:120) >>>>>> >>>>>> org.wso2.carbon.sequences.SequenceDeploymentInterceptor.updateSynapseArtifact(SequenceDeploymentInterceptor.java:49) >>>>>> >>>>>> org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.resetDefaultSequence(AbstractSynapseArtifactDeployer.java:584) >>>>>> >>>>>> org.apache.synapse.deployers.SequenceDeployer.updatingFaultAndMainWithOriginal(SequenceDeployer.java:181) >>>>>> >>>>>> org.apache.synapse.deployers.SequenceDeployer.undeploy(SequenceDeployer.java:150) >>>>>> >>>>>> org.apache.axis2.deployment.DeploymentEngine.unDeploy(DeploymentEngine.java:909) >>>>>> - locked org.wso2.carbon.core.CarbonAxisConfigurator@5497eb53 >>>>>> >>>>>> org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:143) >>>>>> - locked org.wso2.carbon.core.CarbonAxisConfigurator@5497eb53 >>>>>> >>>>>> org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377) >>>>>> >>>>>> org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254) >>>>>> >>>>>> org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371) >>>>>> >>>>>> -- >>>>>> Krishantha Samaraweera >>>>>> Senior Technical Lead - Test Automation >>>>>> Mobile: +94 77 7759918 >>>>>> WSO2, Inc.; http://wso2.com/ >>>>>> lean . enterprise . middlewear. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Miyuru Wanninayaka >>>>> Technical Lead >>>>> WSO2 Inc. : http://wso2.com >>>>> >>>>> Mobile : +94 77 209 9788 >>>>> Blog : http://miyurudw.blogspot.com >>>>> Flickr : http://www.flickr.com/photos/miyuru_daminda >>>>> >>>> >>>> >>>> >>>> -- >>>> Miyuru Wanninayaka >>>> Technical Lead >>>> WSO2 Inc. : http://wso2.com >>>> >>>> Mobile : +94 77 209 9788 >>>> Blog : http://miyurudw.blogspot.com >>>> Flickr : http://www.flickr.com/photos/miyuru_daminda >>>> >>> >>> >>> >>> -- >>> Miyuru Wanninayaka >>> Technical Lead >>> WSO2 Inc. : http://wso2.com >>> >>> Mobile : +94 77 209 9788 >>> Blog : http://miyurudw.blogspot.com >>> Flickr : http://www.flickr.com/photos/miyuru_daminda >>> >> >> >> >> -- >> Miyuru Wanninayaka >> Technical Lead >> WSO2 Inc. : http://wso2.com >> >> Mobile : +94 77 209 9788 >> Blog : http://miyurudw.blogspot.com >> Flickr : http://www.flickr.com/photos/miyuru_daminda >> > > > > -- > Miyuru Wanninayaka > Technical Lead > WSO2 Inc. : http://wso2.com > > Mobile : +94 77 209 9788 > Blog : http://miyurudw.blogspot.com > Flickr : http://www.flickr.com/photos/miyuru_daminda > -- Krishantha Samaraweera Senior Technical Lead - Test Automation Mobile: +94 77 7759918 WSO2, Inc.; http://wso2.com/ lean . enterprise . middlewear.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
