Github user serg38 commented on the issue: https://github.com/apache/cloudstack/pull/1762 @rafaelweingartner You might be right that pod_vlan_map should be in the join. May be I didn't find the correct methods after all. @jburwell @rhtyd What do you think? I was able to find management serve log for Deadlock 1. Looks like one of transaction came from findAndUpdateDirectAgentToLoad method in HostDaoImpl which creates rather complex transaction: 2016-11-24 15:04:39,284 DEBUG [host.dao.HostDaoImpl] (ClusteredAgentManager Timer:ctx-a8e9449c) Resetting hosts suitable for reconnect 2016-11-24 15:04:39,320 DEBUG [db.Transaction.Transaction] (ClusteredAgentManager Timer:ctx-a8e9449c) Rolling back the transaction: Time = 36 Name = ClusteredAgentManager Timer; called by -TransactionLegacy.rollback:879-TransactionLegacy.removeUpTo:822-TransactionLegacy.close:646-TransactionContextInterceptor.invoke:36-ReflectiveMethodInvocation.proceed:161-ExposeInvocationInterceptor.invoke:91-ReflectiveMethodInvocation.proceed:172-JdkDynamicAopProxy.invoke:204-$Proxy48.findAndUpdateDirectAgentToLoad:-1-ClusteredAgentManagerImpl.scanDirectAgentToLoad:195-ClusteredAgentManagerImpl.runDirectAgentScanTimerTask:185-ClusteredAgentManagerImpl.access$100:99 2016-11-24 15:04:39,322 ERROR [agent.manager.ClusteredAgentManagerImpl] (ClusteredAgentManager Timer:ctx-a8e9449c) Unexpected exception DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@1e58727c: SELECT host.id, host.disconnected, host.name, host.status, host.type, host.private_ip_address, host.private_mac_address, host.private_netmask, host.public_netmask, host.public_ip_address, host.public_mac_address, host.storage_ip_address, host.cluster_id, host.storage_netmask, host.storage_mac_address, host.storage_ip_address_2, host.storage_netmask_2, host.storage_mac_address_2, host.hypervisor_type, host.proxy_port, host.resource, host.fs_type, host.available, host.setup, host.resource_state, host.hypervisor_version, host.update_count, host.uuid, host.data_center_id, host.pod_id, host.cpu_sockets, host.cpus, host.url, host.speed, host.ram, host.parent, host.guid, host.capabilities, host.total_size, host.last_ping, host.mgmt_server_id, host.dom0_memory, host.version, host.created, h ost.removed FROM host WHERE host.resource IS NOT NULL AND host.mgmt_server_id = 345048964870 AND host.last_ping <= 1445339907 AND host.cluster_id IS NOT NULL AND host.status IN ('Disconnected','Down','Alert') AND host.removed IS NULL FOR UPDATE Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction Beginning of second transaction was SELECT host.id, host.disconnected, host.name, host.status, host.type, host.private_ip_address, host.private_mac_address, host.private_netmask, host.public_netmask, host.public_ip_address, host.public_mac_address, host.storage_ip_address, host.cluster_id, host.storage_netmask, host.storage_mac_address, host.storage_ip_address_2, host.storage_netmask_2, host.storage_mac_address_2, host.hypervisor_type, host.proxy_port, host.resource, host.fs_type, host.available, host.setup, host.resource_state, host.hypervisor_version, host.update_count, host.uuid, host.data_center_id, host.pod_id, host.cpu_sockets, host.cpus, host.url, host.speed, host.ram, host.parent, host.guid, host.capabilities, host.total_size, host.last_ping, host.mgmt_server_id, host.dom0_memory, host.version, host.created, host.removed FROM host LEFT OUTER JOIN op_host_transfer ON host.id=op_host_transfer.id IN I will try to trace it to the ACS method.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---