Hello Greg,
Check if any of your primary or secondary storage is in maintenance mode -
specific to the uuid mentioned in the error message -
Can't find volume:cb1b9a18-6fdb-4613-8153-9595c2f70f58","wait":0
Do you have the version related system template in place and have you
updated the template version in the global settings parameter?
With regards
Thomas
On Thu, 12 Dec 2019, 10:34 pm Greg Goodrich, <ggoodr...@ippathways.com>
wrote:
> We’ve been doing test runs in our staging environment of upgrading from
> 4.9.3 to 4.11, leveraging KVM on our 2 hosts. Recently, we downgraded back
> to 4.9.3 to re-run the processes. We restored the database, and the
> software on all machines, etc. However, now, the virtual routers are
> failing to start, and I’m having difficulties tracking down exactly what is
> going wrong. I’ll admit to not being adept at working with CloudStack yet.
> I’ve tried restarting the network, both with and without clean up. I’ve
> also tried starting the routers via the infrastructure -> routers area.
> After a decent amount of time, the status is updated to Stopped. The VMs
> never seem to come online, as I’ve checked virsh on both of our hosts, and
> the only VMs that seem to be running are the console proxy and secondary
> storage VMs. I also tried restarting the agents on both hosts, as well as
> the management node and libvirt.
>
> Here are some snippets from the logs that seem concerning to me, that may
> or may not help (these are not in order, and one is from restarting
> network, other from starting VR manually):
>
> 2019-12-12 18:00:34,557 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-10:null) (logid:) Seq 96-6863767307089348712:
> Processing: { Ans: , MgmtId: 345051498372, via: 96, Ver: v1, Flags: 110,
> [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException:
> com.cloud.utils.exception.CloudRuntimeException: Can't find
> volume:cb1b9a18-6fdb-4613-8153-9595c2f70f58","wait":0}}] }
>
> 2019-12-12 15:44:26,623 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-4:null) (logid:) Seq 96-6863767307089348508:
> Processing: { Ans: , MgmtId: 345051498372, via: 96, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.UnsupportedAnswer":{"result":false,"details":"Unsupported
> command issued: com.cloud.agent.api.StartCommand. Are you sure you got the
> right type of
> server?","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous
> failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
> by previous failure","wait":0}}] }
> 2019-12-12 15:44:26,623 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-20:ctx-a8bb9961 job-31381/job-31386 ctx-951bce3f)
> (logid:010e3619) Seq 96-6863767307089348508: Received: { Ans: , MgmtId:
> 345051498372, via: 96(labcloudkvm01.ipplab.corp), Ver: v1, Flags: 10, {
> UnsupportedAnswer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
> Answer } }
>
> Any hints on which path I should go down next would be greatly appreciated!
>
> --
> Greg Goodrich | IP Pathways
> Senior Developer
> 3600 109th Street | Urbandale, IA 50322
> p. 515.422.9346 | e. ggoodr...@ippathways.com<mailto:j...@ippathways.com>
>
>
