Hi,
This seems to occur when metadata service too save properties in registry.
This occurs when multiple clients(cartridge agent) call metadata service to
publish property values. I did not see this error when only one client
talks to metadata service.
I guess h2 can not handle this. I will try with MySQL as Isuruh mentioned
since this is blocking for me now.
On Thu, May 7, 2015 at 5:12 PM, Isuru Haththotuwa <isu...@apache.org> wrote:
> This seems to be an issue with H2. There are many occasions that this
> error has been reported when using an H2 DB. Can you try:
>
> 1. replacing the currently used DB with a new H2 (from a fresh pack)
> 2. use a MySQL DB
>
> and see if the error is reproducible?
>
> On Thu, May 7, 2015 at 4:42 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> It looks like a database table lock is not being released until it
>> reaches the timeout:
>>
>> Timeout trying to lock table "REG_RESOURCE"; SQL statement:
>> SELECT REG_VERSION FROM REG_RESOURCE WHERE REG_PATH_ID=? AND REG_NAME=?
>> AND REG_TENANT_ID=? [50200-140]
>>
>> The concern is we do not handle any database locks in Stratos code, they
>> are handled by the Carbon Registry. Shall we try to narrow down the problem
>> by running it again without using the Metadata API?
>>
>> Thanks
>>
>>
>> On Thu, May 7, 2015 at 12:37 PM, Udara Liyanage <ud...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> I frequently get this exception when running Stratos for longtime (10+
>>> minutes). Please note that most failures occurs when metadata service also
>>> save properties in the registry (not all the time, this occurs when
>>> metadata is not persisting too).
>>>
>>>
>>> 2015-05-07 07:03:11,725] ERROR
>>> {org.apache.stratos.cloud.controller.registry.RegistryManager} - * Failed
>>> to persist resource in registry: /cloud.controller/topology*
>>> org.wso2.carbon.registry.core.exceptions.RegistryException: Failed to
>>> check the existence of the resource
>>> /_system/governance/cloud.controller/topology. Timeout trying to lock table
>>> "REG_RESOURCE"; SQL statement:
>>> SELECT REG_VERSION FROM REG_RESOURCE WHERE REG_PATH_ID=? AND REG_NAME=?
>>> AND REG_TENANT_ID=? [50200-140]
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.getVersion(JDBCResourceDAO.java:206)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.resourceExists(JDBCResourceDAO.java:147)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.resourceExists(JDBCResourceDAO.java:156)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.Repository.resourceExists(Repository.java:134)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.put(EmbeddedRegistry.java:688)
>>> at
>>> org.wso2.carbon.registry.core.caching.CacheBackedRegistry.put(CacheBackedRegistry.java:465)
>>> at
>>> org.wso2.carbon.registry.core.session.UserRegistry.put(UserRegistry.java:657)
>>> at
>>> org.apache.stratos.cloud.controller.registry.RegistryManager.persist(RegistryManager.java:73)
>>> at
>>> org.apache.stratos.cloud.controller.util.CloudControllerUtil.persistTopology(CloudControllerUtil.java:276)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.topology.TopologyManager.updateTopology(TopologyManager.java:103)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder.handleMemberStarted(TopologyBuilder.java:555)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.receiver.instance.status.InstanceStatusTopicReceiver$2.onEvent(InstanceStatusTopicReceiver.java:75)
>>> at
>>> org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42)
>>> at java.util.Observable.notifyObservers(Observable.java:159)
>>> at
>>> org.apache.stratos.messaging.event.EventObservable.notifyEventListeners(EventObservable.java:51)
>>> at
>>> org.apache.stratos.messaging.message.processor.instance.status.InstanceStatusMemberStartedMessageProcessor.process(InstanceStatusMemberStartedMessageProcessor.java:49)
>>> at
>>> org.apache.stratos.messaging.message.processor.instance.status.InstanceStatusMemberActivatedMessageProcessor.process(InstanceStatusMemberActivatedMessageProcessor.java:53)
>>> at
>>> org.apache.stratos.messaging.message.processor.MessageProcessorChain.process(MessageProcessorChain.java:61)
>>> at
>>> org.apache.stratos.messaging.message.receiver.instance.status.InstanceStatusEventMessageDelegator.run(InstanceStatusEventMessageDelegator.java:71)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:744)
>>> Caused by: org.h2.jdbc.JdbcSQLException: Timeout trying to lock table
>>> "REG_RESOURCE"; SQL statement:
>>> SELECT REG_VERSION FROM REG_RESOURCE WHERE REG_PATH_ID=? AND REG_NAME=?
>>> AND REG_TENANT_ID=? [50200-140]
>>> at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
>>> at org.h2.message.DbException.get(DbException.java:167)
>>> at org.h2.message.DbException.get(DbException.java:144)
>>> at org.h2.table.RegularTable.doLock(RegularTable.java:466)
>>> at org.h2.table.RegularTable.lock(RegularTable.java:404)
>>> at org.h2.table.TableFilter.lock(TableFilter.java:139)
>>> at org.h2.command.dml.Select.queryWithoutCache(Select.java:554)
>>> at org.h2.command.dml.Query.query(Query.java:241)
>>> at org.h2.command.CommandContainer.query(CommandContainer.java:80)
>>> at org.h2.command.Command.executeQuery(Command.java:132)
>>> at
>>> org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:96)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.getVersion(JDBCResourceDAO.java:197)
>>> ... 23 more
>>> [2015-05-07 07:03:11,726] FATAL
>>> {org.apache.stratos.cloud.controller.util.CloudControllerUtil} - Failed to
>>> persist the Topology in registry.
>>> org.wso2.carbon.registry.core.exceptions.RegistryException: Failed to
>>> persist resource in registry: /cloud.controller/topology
>>> at
>>> org.apache.stratos.cloud.controller.registry.RegistryManager.persist(RegistryManager.java:85)
>>> at
>>> org.apache.stratos.cloud.controller.util.CloudControllerUtil.persistTopology(CloudControllerUtil.java:276)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.topology.TopologyManager.updateTopology(TopologyManager.java:103)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.topology.TopologyBuilder.handleMemberStarted(TopologyBuilder.java:555)
>>> at
>>> org.apache.stratos.cloud.controller.messaging.receiver.instance.status.InstanceStatusTopicReceiver$2.onEvent(InstanceStatusTopicReceiver.java:75)
>>> at
>>> org.apache.stratos.messaging.listener.EventListener.update(EventListener.java:42)
>>> at java.util.Observable.notifyObservers(Observable.java:159)
>>> at
>>> org.apache.stratos.messaging.event.EventObservable.notifyEventListeners(EventObservable.java:51)
>>> at
>>> org.apache.stratos.messaging.message.processor.instance.status.InstanceStatusMemberStartedMessageProcessor.process(InstanceStatusMemberStartedMessageProcessor.java:49)
>>> at
>>> org.apache.stratos.messaging.message.processor.instance.status.InstanceStatusMemberActivatedMessageProcessor.process(InstanceStatusMemberActivatedMessageProcessor.java:53)
>>> at
>>> org.apache.stratos.messaging.message.processor.MessageProcessorChain.process(MessageProcessorChain.java:61)
>>> at
>>> org.apache.stratos.messaging.message.receiver.instance.status.InstanceStatusEventMessageDelegator.run(InstanceStatusEventMessageDelegator.java:71)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:744)
>>> Caused by: org.wso2.carbon.registry.core.exceptions.RegistryException:
>>> Failed to check the existence of the resource
>>> /_system/governance/cloud.controller/topology. Timeout trying to lock table
>>> "REG_RESOURCE"; SQL statement:
>>> SELECT REG_VERSION FROM REG_RESOURCE WHERE REG_PATH_ID=? AND REG_NAME=?
>>> AND REG_TENANT_ID=? [50200-140]
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.getVersion(JDBCResourceDAO.java:206)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.resourceExists(JDBCResourceDAO.java:147)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.resourceExists(JDBCResourceDAO.java:156)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.Repository.resourceExists(Repository.java:134)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.put(EmbeddedRegistry.java:688)
>>> at
>>> org.wso2.carbon.registry.core.caching.CacheBackedRegistry.put(CacheBackedRegistry.java:465)
>>> at
>>> org.wso2.carbon.registry.core.session.UserRegistry.put(UserRegistry.java:657)
>>> at
>>> org.apache.stratos.cloud.controller.registry.RegistryManager.persist(RegistryManager.java:73)
>>> ... 16 more
>>> Caused by: org.h2.jdbc.JdbcSQLException: Timeout trying to lock table
>>> "REG_RESOURCE"; SQL statement:
>>> SELECT REG_VERSION FROM REG_RESOURCE WHERE REG_PATH_ID=? AND REG_NAME=?
>>> AND REG_TENANT_ID=? [50200-140]
>>> at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
>>> at org.h2.message.DbException.get(DbException.java:167)
>>> at org.h2.message.DbException.get(DbException.java:144)
>>> at org.h2.table.RegularTable.doLock(RegularTable.java:466)
>>> at org.h2.table.RegularTable.lock(RegularTable.java:404)
>>> at org.h2.table.TableFilter.lock(TableFilter.java:139)
>>> at org.h2.command.dml.Select.queryWithoutCache(Select.java:554)
>>> at org.h2.command.dml.Query.query(Query.java:241)
>>> at org.h2.command.CommandContainer.query(CommandContainer.java:80)
>>> at org.h2.command.Command.executeQuery(Command.java:132)
>>> at
>>> org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:96)
>>> at
>>> org.wso2.carbon.registry.core.jdbc.dao.JDBCResourceDAO.getVersion(JDBCResourceDAO.java:197)
>>> ... 23 more
>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> web: http://udaraliyanage.wordpress.com
>>> phone: +94 71 443 6897
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Udara Liyanage
Software Engineer
WSO2, Inc.: http://wso2.com
lean. enterprise. middleware
web: http://udaraliyanage.wordpress.com
phone: +94 71 443 6897
