Thanks for implementing this nice feature Imesh. This helps a lot in identifying the locking issues.
@Mari, Can we add this to our documentation under developer guide or troubleshoot section? So that it will help in debugging purpose. Thanks, Reka On Tue, Jan 20, 2015 at 10:26 PM, Dakshika Jayathilaka <daksh...@wso2.com> wrote: > Great Work imesh.. This will help to developers in many ways.. > > Better if we can add this to developer guide. > > *Dakshika Jayathilaka* > Software Engineer > WSO2, Inc. > lean.enterprise.middleware > 0771100911 > > On Tue, Jan 20, 2015 at 7:08 PM, Imesh Gunaratne <im...@apache.org> wrote: > >> Above issue is now fixed with commit revision: >> fe0e6f81a673d6db815a56a3592c41441c4be3ae >> >> On Tue, Jan 20, 2015 at 7:03 PM, Imesh Gunaratne <im...@apache.org> >> wrote: >> >>> Following JIRA explains the above problem found in 4.1.0-alpha: >>> https://issues.apache.org/jira/browse/STRATOS-1089 >>> >>> On Tue, Jan 20, 2015 at 6:50 PM, Imesh Gunaratne <im...@apache.org> >>> wrote: >>> >>>> Hi Devs, >>>> >>>> Currently we have significant amount of read write locks in Stratos >>>> codebase to synchronize data structure updates made by multiple threads. >>>> >>>> As I figured out recently there are several possibilities that these >>>> locks could cause problems if not properly implemented: >>>> >>>> *Problem 1: A thread trying to acquire a write lock while having a read >>>> lock.* >>>> *Problem 2: Locks acquired but not released* >>>> >>>> I just implemented a ReadWriteLock class in stratos common module which >>>> could detect these problems. This class make use >>>> of java.util.concurrent.locks.ReentrantReadWriteLock and implements >>>> additional logic to detect problems. By default problem 2 detection logic >>>> is disabled, it can be enabled by setting following system property: >>>> >>>> >>>> *read.write.lock.monitor.enabled=true* >>>> >>>> I have now updated all read write locks with the above class. >>>> Following is an example scenario where I detected an unreleased lock in >>>> the REST API: >>>> >>>> org.apache.stratos.common.exception.LockNotReleasedException >>>> at >>>> org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor.checkTimeout(ReadWriteLockMonitor.java:71) >>>> at >>>> org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor.run(ReadWriteLockMonitor.java:50) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>> at java.lang.Thread.run(Thread.java:745) >>>> [2015-01-20 18:30:12,952] ERROR >>>> {org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor} - System >>>> error, lock has not released for 30 seconds: [lock-name] topology >>>> [lock-type] Read [thread-id] 405 [thread-name] http-nio-9443-exec-21 >>>> [stack-trace] >>>> java.lang.Thread.getStackTrace(Thread.java:1589) >>>> >>>> org.apache.stratos.common.concurrent.locks.ReadWriteLock.acquireReadLock(ReadWriteLock.java:160) >>>> >>>> org.apache.stratos.messaging.message.receiver.topology.TopologyManager.acquireReadLockForCluster(TopologyManager.java:173) >>>> >>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addClustersInstancesToGroupInstanceBean(StratosApiV41Utils.java:1015) >>>> >>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.setSubGroupInstances(StratosApiV41Utils.java:1027) >>>> >>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addGroupsInstancesToApplicationInstanceBean(StratosApiV41Utils.java:979) >>>> >>>> org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.getApplicationRuntime(StratosApiV41Utils.java:963) >>>> >>>> org.apache.stratos.rest.endpoint.api.StratosApiV41.getApplicationRuntime(StratosApiV41.java:579) >>>> >>>> Thanks >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>> >>> >>> >>> -- >>> Imesh Gunaratne >>> >>> Technical Lead, WSO2 >>> Committer & PMC Member, Apache Stratos >>> >> >> >> >> -- >> Imesh Gunaratne >> >> Technical Lead, WSO2 >> Committer & PMC Member, Apache Stratos >> > > -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007
