[
https://issues.apache.org/jira/browse/CLOUDSTACK-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15694529#comment-15694529
]
ASF GitHub Bot commented on CLOUDSTACK-9595:
--------------------------------------------
Github user serg38 commented on the issue:
https://github.com/apache/cloudstack/pull/1762
@rafaelweingartner Looks like the deadlocks 2 and 3 are the same. I
scanned our production log and since last December we had 6400 deadlocks. Out
of them close to 6000 were Deadlock 1
20 were Deadlock 2 and 700 of a different Deadlock 5. The other deadlocks
were in negligible numbers. I think if we figure out Deadlock 1 and Deadlock 5
this will be good start. I will try to find the source of transactions for
them. In production we run a commercial distribution based in most part on 4.7
branch of ACS.
Deadlock 5
*** (1) TRANSACTION:
TRANSACTION D518886F8, ACTIVE 2 sec fetching rows
mysql tables in use 4, locked 4
LOCK WAIT 24 lock struct(s), heap size 3112, 8 row lock(s), undo log
entries 17
MySQL thread id 29781, OS thread handle 0x7f9df36db700, query id 3625404021
ussclpdcsmgt012.autodesk.com 10.41.13.14 cloud Sorting result
SELECT user_ip_address.id, user_ip_address.account_id,
user_ip_address.domain_id, user_ip_address.public_ip_address,
user_ip_address.data_center_id, user_ip_address.source_n
at, user_ip_address.allocated, user_ip_address.vlan_db_id,
user_ip_address.one_to_one_nat, user_ip_address.vm_id, user_ip_address.state,
user_ip_address.mac_address, user_ip
_address.source_network_id, user_ip_address.network_id,
user_ip_address.uuid, user_ip_address.physical_network_id,
user_ip_address.is_system, user_ip_address.vpc_id, user_ip
_address.dnat_vmip, user_ip_address.is_portable, user_ip_address.display,
user_ip_address.removed, user_ip_address.created FROM user_ip_address INNER
JOIN vlan ON user_ip_a
ddress.vlan_db_id=vlan.id WHERE user_ip_address.data_center_id = 6 AND
user_ip_address.allocated IS NULL AND user_ip_address.vlan_db_id IN
(32,33,36,37,41,61,62,91,92,93,9
4,106,107,108,109,11
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
*** (2) TRANSACTION:
TRANSACTION D5188582B, ACTIVE 17 sec updating or deleting, thread declared
inside InnoDB 499
mysql tables in use 1, locked 1
25 lock struct(s), heap size 3112, 13 row lock(s), undo log entries 18
MySQL thread id 29820, OS thread handle 0x7fa35a868700, query id 3625417999
ussclpdcsmgt013.autodesk.com 10.41.13.15 cloud Updating
UPDATE user_ip_address SET user_ip_address.source_nat=0,
user_ip_address.is_system=0, user_ip_address.account_id=3309,
user_ip_address.allocated='2016-03-25 15:36:39',
user_ip_address.state='Allocated', user_ip_address.domain_id=335 WHERE
user_ip_address.id = 3284
*** (2) HOLDS THE LOCK(S):
> Transactions are not getting retried in case of database deadlock errors
> ------------------------------------------------------------------------
>
> Key: CLOUDSTACK-9595
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9595
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Affects Versions: 4.8.0
> Reporter: subhash yedugundla
> Fix For: 4.8.1
>
>
> Customer is seeing occasional error 'Deadlock found when trying to get lock;
> try restarting transaction' messages in their management server logs. It
> happens regularly at least once a day. The following is the error seen
> 2015-12-09 19:23:19,450 ERROR [cloud.api.ApiServer]
> (catalina-exec-3:ctx-f05c58fc ctx-39c17156 ctx-7becdf6e) unhandled exception
> executing api command: [Ljava.lang.String;@230a6e7f
> com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
> com.mysql.jdbc.JDBC4PreparedStatement@74f134e3: DELETE FROM
> instance_group_vm_map WHERE instance_group_vm_map.instance_id = 941374
> at com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1209)
> at sun.reflect.GeneratedMethodAccessor360.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy237.expunge(Unknown Source)
> at
> com.cloud.vm.UserVmManagerImpl$2.doInTransactionWithoutResult(UserVmManagerImpl.java:2593)
> at
> com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25)
> at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:57)
> at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
> at com.cloud.utils.db.Transaction.execute(Transaction.java:54)
> at
> com.cloud.vm.UserVmManagerImpl.addInstanceToGroup(UserVmManagerImpl.java:2575)
> at
> com.cloud.vm.UserVmManagerImpl.updateVirtualMachine(UserVmManagerImpl.java:2332)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
