Maybe something changed in the database recently that I only picked up on today when I wiped my DB and started over because I no longer am seeing these exceptions.
Perhaps I was running with an older DB schema, but newer code and that was the problem here. On Fri, Jul 12, 2013 at 11:25 PM, Mike Tutkowski < [email protected]> wrote: > These are essentially the same stack trace, but here are some recent ones > (also, these seem to be in regards to CPVM instead of SSVM): > > ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Failed to > start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > INFO [xen.resource.CitrixResourceBase] (DirectAgent-42:) VM does not > exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Exception while trying to start console proxy > com.cloud.exception.AgentUnavailableException: Resource [Host:1] is > unreachable: Host 1: Unable to start instance due to Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:953) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > Caused by: com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > ... 20 more > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found > a stopped console proxy, bring it up to running pool. proxy vm id : 2 > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Unable to ssh to the VM: Can not ping System vm v-2-VMdue to:Timeout, > Unable to logon to 169.254.3.36 > INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) The guru did > not like the answers so stopping VM[ConsoleProxy|v-2-VM] > ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Failed to > start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > INFO [xen.resource.CitrixResourceBase] (DirectAgent-22:) VM does not > exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Exception while trying to start console proxy > com.cloud.exception.AgentUnavailableException: Resource [Host:1] is > unreachable: Host 1: Unable to start instance due to Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:953) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > Caused by: com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > ... 20 more > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found > a stopped console proxy, bring it up to running pool. proxy vm id : 2 > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-2:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-2:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > INFO [network.vpc.VpcManagerImpl] (VpcChecker-1:) Found 0 removed VPCs to > cleanup > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Unable to ssh to the VM: Can not ping System vm v-2-VMdue to:Timeout, > Unable to logon to 169.254.1.52 > INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) The guru did > not like the answers so stopping VM[ConsoleProxy|v-2-VM] > ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Failed to > start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > INFO [xen.resource.CitrixResourceBase] (DirectAgent-283:) VM does not > exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Exception while trying to start console proxy > com.cloud.exception.AgentUnavailableException: Resource [Host:1] is > unreachable: Host 1: Unable to start instance due to Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:953) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > Caused by: com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > ... 20 more > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found > a stopped console proxy, bring it up to running pool. proxy vm id : 2 > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-2:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-2:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-2:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-2:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-1:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-1:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Unable to ssh to the VM: Can not ping System vm v-2-VMdue to:Timeout, > Unable to logon to 169.254.3.90 > INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) The guru did > not like the answers so stopping VM[ConsoleProxy|v-2-VM] > ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Failed to > start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > INFO [xen.resource.CitrixResourceBase] (DirectAgent-337:) VM does not > exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Exception while trying to start console proxy > com.cloud.exception.AgentUnavailableException: Resource [Host:1] is > unreachable: Host 1: Unable to start instance due to Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:953) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > Caused by: com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > ... 20 more > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found > a stopped console proxy, bring it up to running pool. proxy vm id : 2 > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-2:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-2:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > INFO [cloud.ha.HighAvailabilityManagerImpl] (HA-2:) checking health of > usage server > WARN [apache.cloudstack.alerts] (HA-2:) alertType:: 13 // dataCenterId:: > 0 // podId:: 0 // clusterId:: null // message:: No usage server process > running > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Unable to ssh to the VM: Can not ping System vm v-2-VMdue to:Timeout, > Unable to logon to 169.254.2.58 > INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) The guru did > not like the answers so stopping VM[ConsoleProxy|v-2-VM] > ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Failed to > start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > INFO [xen.resource.CitrixResourceBase] (DirectAgent-433:) VM does not > exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea > WARN [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) > Exception while trying to start console proxy > com.cloud.exception.AgentUnavailableException: Resource [Host:1] is > unreachable: Host 1: Unable to start instance due to Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:953) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:566) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:559) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:680) > Caused by: com.cloud.utils.exception.ExecutionException: Unable to start > VM[ConsoleProxy|v-2-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:920) > ... 20 more > INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) Found > a stopped console proxy, bring it up to running pool. proxy vm id : 2 > > > > On Fri, Jul 12, 2013 at 10:50 PM, Prasanna Santhanam <[email protected]>wrote: > >> What stack traces do you see? >> >> On Fri, Jul 12, 2013 at 07:43:38PM -0600, Mike Tutkowski wrote: >> > Hi, >> > >> > I've noticed quite a few stack traces in my console as of late related >> to >> > SSVM and that the SSVM often gets deleted and re-created (in 4.2). >> > >> > Is there a known issue around this? >> > >> > Thanks! >> > >> > -- >> > *Mike Tutkowski* >> > *Senior CloudStack Developer, SolidFire Inc.* >> > e: [email protected] >> > o: 303.746.7302 >> > Advancing the way the world uses the >> > cloud<http://solidfire.com/solution/overview/?video=play> >> > *?* >> >> -- >> Prasanna., >> >> ------------------------ >> Powered by BigRock.com >> >> > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: [email protected] > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: [email protected] o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *™*
