Could it be possible the firewall is blocking port 9090 from the management-server?
2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable to ping management server at 10.97.38.10:9090 due to ConnectException java.net.ConnectException: Connection refused On Mon, Nov 3, 2014 at 9:02 AM, Nux! <n...@li.nux.ro> wrote: > BTW the agent logs should help clarify this. > The aforementioned changes must be done either in the current (broken) VR > or in the systemvm (after which you should redeploy the VR to pick up the > change). > > HTH > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > ----- Original Message ----- > > From: "Nux!" <n...@li.nux.ro> > > To: users@cloudstack.apache.org > > Sent: Monday, 3 November, 2014 13:53:10 > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > > Hello, > > > > I think you are hitting a problem I also had: > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl] > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2) > Failed to > > start instance VM[DomainRouter|r-4758-VM] > > com.cloud.utils.exception.ExecutionException: Unable to start > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying" > > > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781 > > > > Basically the systemvm template has some problems, you need to somehow > copy > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to > get it > > working. > > > > -- > > Sent from the Delta quadrant using Borg technology! > > > > Nux! > > www.nux.ro > > > > ----- Original Message ----- > >> From: "Adam" <adam.scarce...@gmail.com> > >> To: users@cloudstack.apache.org > >> Sent: Monday, 3 November, 2014 13:32:12 > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release > >> CentOS release 6.6 (Final) > >> [root@cs-east-prod1 ~]# uname -a > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC > >> 2014 x86_64 x86_64 x86_64 GNU/Linux > >> [root@cs-east-prod1 ~]# virsh --version > >> 0.10.2 > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g' > >> /etc/cloudstack/agent/log4j-cloud.xml > >> [root@cs-east-prod1 ~]# service cloudstack-management stop > >> Stopping cloudstack-management: [ OK ] > >> [root@cs-east-prod1 ~]# mv > >> /var/log/cloudstack/management/management-server.log > >> /var/log/cloudstack/management/management-server.bak > >> [root@cs-east-prod1 ~]# service cloudstack-management start > >> Starting cloudstack-management: [ OK ] > >> [root@cs-east-prod1 ~]# > >> > >> Please see attached ZIP of clean management-server.log in DEBUG mode. At > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached > >> management-server.log*) I tried starting the Virtual Router from the > >> Management UI and received this "Status" message in the UI. > >> > >> [image: Inline image 1] > >> > >> I can see in the management-server.log that while the > cloudstack-management > >> service is initializing, it throws the following ERROR at exactly > (*2014-11-03 > >> 08:14:36,813 on line 342 of the attached management-server.log*): > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) > Unable > >> to ping management server at 10.97.38.10:9090 due to ConnectException > >> java.net.ConnectException: Connection refused > >> at sun.nio.ch.Net.connect0(Native Method) > >> at sun.nio.ch.Net.connect(Net.java:465) > >> at sun.nio.ch.Net.connect(Net.java:457) > >> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > >> at > >> > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123) > >> at > >> > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092) > >> at > >> > com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170) > >> at > >> > com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057) > >> at > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114) > >> at > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153) > >> at > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110) > >> at > >> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56) > >> at > >> > org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167) > >> at > >> > org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) > >> at > >> > org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) > >> at > >> > org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) > >> at > >> > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) > >> at > >> > org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945) > >> at > >> > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) > >> at > >> > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > >> at > >> > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > >> at > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) > >> at > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57) > >> at > >> > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61) > >> at > >> > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) > >> at > >> > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210) > >> at > >> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709) > >> at > >> > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > >> at > >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > >> at > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) > >> at > >> > org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041) > >> at > >> > org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964) > >> at > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502) > >> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) > >> at > >> > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) > >> at > >> > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) > >> at > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) > >> at org.apache.catalina.core.StandardHost.start(StandardHost.java:722) > >> at > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) > >> at > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) > >> at > >> org.apache.catalina.core.StandardService.start(StandardService.java:516) > >> at > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710) > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >> at > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:606) > >> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) > >> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null) > >> Detected that another management node with the same IP 10.97.38.10 is > >> considered as running in DB, however it is not pingable, we will > continue > >> cluster initialization with this management server node > >> > >> I'm not sure what that error means, but more than likely that's the > reason > >> why the Virtual Router will not start. > >> > >> * PLEASE HELP ! * > >> > >> Best Regards, > >> > >> > >> > >> Adam Scarcella > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <n...@li.nux.ro> wrote: > >> > >>> Hi, > >>> > >>> More agent and management logs please, make sure they are in DEBUG[1] > mode. > >>> > >>> [1] - > >>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug > >>> > >>> -- > >>> Sent from the Delta quadrant using Borg technology! > >>> > >>> Nux! > >>> www.nux.ro > >>> > >>> ----- Original Message ----- > >>> > From: "Adam" <adam.scarce...@gmail.com> > >>> > To: users@cloudstack.apache.org > >>> > Sent: Sunday, 2 November, 2014 23:33:22 > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1 > >>> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed the > >>> > instructions perfectly, but my Virtual Router will not start, nor am > I > >>> able > >>> > to "upgrade" it. > >>> > > >>> > I tried destroying the Virtual Router, starting any VM Instance, and > >>> > allowing CS to recreate the VR. However that didn't help. The VR > still > >>> did > >>> > not start and it was still saying that it required to be upgraded. > >>> > > >>> > [image: Inline image 1] > >>> > [image: Inline image 2] > >>> > [image: Inline image 3] > >>> > > >>> > I know there are manual steps to finalize the systemvm upgrade, but > I did > >>> > those steps: > >>> > > >>> > > >>> > http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix > >>> > > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p > >>> > Enter password: > >>> > Welcome to the MySQL monitor. Commands end with ; or \g. > >>> > Your MySQL connection id is 174 > >>> > Server version: 5.1.73-log Source distribution > >>> > > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights > >>> reserved. > >>> > > >>> > Oracle is a registered trademark of Oracle Corporation and/or its > >>> > affiliates. Other names may be trademarks of their respective > >>> > owners. > >>> > > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input > >>> > statement. > >>> > > >>> > mysql> select id,name from cloud.vm_template where name = > >>> > 'systemvm-kvm-4.4'; > >>> > +-----+------------------+ > >>> >| id | name | > >>> > +-----+------------------+ > >>> >| 274 | systemvm-kvm-4.4 | > >>> > +-----+------------------+ > >>> > 1 row in set (0.00 sec) > >>> > > >>> > mysql> select type from cloud.vm_template where id='274'; > >>> > +--------+ > >>> >| type | > >>> > +--------+ > >>> >| SYSTEM | > >>> > +--------+ > >>> > 1 row in set (0.00 sec) > >>> > > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM'; > >>> > +----------------+ > >>> >| vm_template_id | > >>> > +----------------+ > >>> >| 274 | > >>> > +----------------+ > >>> > 1 row in set (0.00 sec) > >>> > > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM'; > >>> > +----------------+ > >>> >| vm_template_id | > >>> > +----------------+ > >>> >| 274 | > >>> > +----------------+ > >>> > 1 row in set (0.03 sec) > >>> > > >>> > mysql> select value from cloud.configuration where name = > >>> > 'router.template.kvm'; > >>> > +------------------+ > >>> >| value | > >>> > +------------------+ > >>> >| systemvm-kvm-4.4 | > >>> > +------------------+ > >>> > 1 row in set (0.00 sec) > >>> > > >>> > mysql> > >>> > > >>> > * PLEASE HELP ! * > >>> > > >>> > Best Regards, > >>> > > > >> > Adam Scarcella >
