[
https://issues.apache.org/jira/browse/CLOUDSTACK-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726265#comment-13726265
]
Sowmya Krishnan commented on CLOUDSTACK-3938:
---------------------------------------------
One of the issues i see so far in configuration is that to use a remote server
DB, it's not picking up db.properties.override. I also need to update
db.properties as well. This was not the case before.
Due to this issue, the MS was still pointing to the local db instead of the
remote DB causing issues.
I'll open a separate bug for this since the summary doesn't reflect the root
cause.
> Operation Timed out and Resource unreachable exceptions in clustered
> management server setup
> --------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3938
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3938
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Environment: 4.2, Using simulator to setup a load test env
> Reporter: Sowmya Krishnan
> Assignee: Koushik Das
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: vmops.log_MS1_3938.gz, vmops_MS2.log_3938.gz
>
>
> Set up:
> 2 Management servers, and mysql DB running in a remote server
> Using simulated hosts and resources.
> Deployed advanced zone with RVR, and setting up simulator hosts and storage
> pools.
> After deploying few of them, getting the following exceptions constantly:
> 2013-07-30 02:34:02,468 DEBUG [agent.transport.Request]
> (StatsCollector-2:null) Seq 2-197787660: Received: { Ans: , MgmtId:
> 206915885097283, via: 2, Ver: v1
> , Flags: 10, { GetStorageStatsAnswer } }
> 2013-07-30 02:34:02,472 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-35:null) Seq 1-1644953610: Response Received:
> 2013-07-30 02:34:02,472 DEBUG [agent.transport.Request]
> (StatsCollector-3:null) Seq 1-1644953610: Received: { Ans: , MgmtId:
> 206915885097283, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-07-30 02:34:02,476 DEBUG [agent.manager.ClusteredAgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Forwarding null to 206915885094132
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-15:null) Seq 3-2136014853: Routing from 206915885097283
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-15:null) Seq 3-2136014853: Link is closed
> 2013-07-30 02:34:02,478 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-15:null) Seq 3-2136014853: MgmtId 206915885097283: Req:
> Resource [Host:3] is unreachable: Host 3: Link is closed
> 2013-07-30 02:34:02,479 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-15:null) Seq 3--1: MgmtId 206915885097283: Req: Routing
> to peer
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-16:null) Seq 3--1: MgmtId 206915885097283: Req: Cancel
> request received
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-16:null) Seq 3-2136014853: Cancelling.
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Waiting some more time because this
> is the current command
> 2013-07-30 02:34:02,481 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Waiting some more time because this
> is the current command
> 2013-07-30 02:34:02,481 INFO [utils.exception.CSExceptionErrorCode]
> (StatsCollector-2:null) Could not find exception:
> com.cloud.exception.OperationTimedoutException in error code list for
> exceptions
> 2013-07-30 02:34:02,482 WARN [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Timed out on null
> 2013-07-30 02:34:02,482 DEBUG [agent.manager.AgentAttache]
> (StatsCollector-2:null) Seq 3-2136014853: Cancelling.
> 2013-07-30 02:34:02,482 DEBUG [cloud.storage.StorageManagerImpl]
> (StatsCollector-2:null) Unable to send storage pool command to
> Pool[2|NetworkFilesystem] via 3
> com.cloud.exception.OperationTimedoutException: Commands 2136014853 to Host 3
> timed out after 3600
> at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:430)
> at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:486)
> at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:439)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:977)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:428)
> at
> com.cloud.storage.StorageManagerImpl.sendToPool(StorageManagerImpl.java:442)
> at
> com.cloud.server.StatsCollector$StorageCollector.run(StatsCollector.java:562)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-30 02:34:02,483 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-34:null) Seq 2-197787661: Executing request
> 2013-07-30 02:34:02,488 DEBUG [agent.manager.ClusteredAgentAttache]
> (StatsCollector-2:null) Seq 4-1100414979: Forwarding null to 206915885094132
> 2013-07-30 02:34:02,489 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-34:null) Seq 2-197787661: Response Received:
> 2013-07-30 02:34:02,489 DEBUG [agent.transport.Request]
> (StatsCollector-3:null) Seq 2-197787661: Received: { Ans: , MgmtId:
> 206915885097283, via: 2, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-17:null) Seq 4-1100414979: Routing from 206915885097283
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentAttache]
> (AgentManager-Handler-17:null) Seq 4-1100414979: Link is closed
> 2013-07-30 02:34:02,490 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-17:null) Seq 4-1100414979: MgmtId 206915885097283: Req:
> Resource [Host:4] is unreachable: Host 4: Link is closed
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
