[root@cs-east-prod1 management]# netstat -a|grep 9090 unix 3 [ ] STREAM CONNECTED 29090 [root@cs-east-prod1 management]# nmap localhost
Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST Nmap scan report for localhost (127.0.0.1) Host is up (0.000015s latency). Other addresses for localhost (not scanned): 127.0.0.1 Not shown: 990 closed ports PORT STATE SERVICE 22/tcp open ssh 25/tcp open smtp 80/tcp open http 111/tcp open rpcbind 199/tcp open smux 631/tcp open ipp 2049/tcp open nfs 3306/tcp open mysql 8080/tcp open http-proxy 9090/tcp open zeus-admin Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds [root@cs-east-prod1 management]# telnet localhost 9090 Trying ::1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. [root@cs-east-prod1 management]# nmap 10.97.38.10 Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10) Host is up (0.000015s latency). Not shown: 993 closed ports PORT STATE SERVICE 22/tcp open ssh 80/tcp open http 111/tcp open rpcbind 2049/tcp open nfs 3306/tcp open mysql 8080/tcp open http-proxy 9090/tcp open zeus-admin Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds [root@cs-east-prod1 management]# telnet 10.97.38.10 9090 Trying 10.97.38.10... Connected to 10.97.38.10. Escape character is '^]'. Connection closed by foreign host. [root@cs-east-prod1 management]# Also, I checked for the /opt/cloud/bin/ directory on my Management Host, but it's not there: [root@cs-east-prod1 management]# cd /opt/cloud/bin/ -bash: cd: /opt/cloud/bin/: No such file or directory [root@cs-east-prod1 management]# cd / [root@cs-east-prod1 /]# find . -name "createipAlias.sh" [root@cs-east-prod1 /]# find . -name "createIpAlias.sh" [root@cs-east-prod1 /]# Are you saying that the /opt/cloud/bin/createipAlias.sh script is inside the Virtual Router VM? If so, how could I possibly change that since it won't start? Best Regards, Adam Scarcella On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion <pdion...@apache.org> wrote: > Could it be possible the firewall is blocking port 9090 from the > management-server? > > 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) Unable > to ping management server at 10.97.38.10:9090 due to ConnectException > java.net.ConnectException: Connection refused > > On Mon, Nov 3, 2014 at 9:02 AM, Nux! <n...@li.nux.ro> wrote: > > > BTW the agent logs should help clarify this. > > The aforementioned changes must be done either in the current (broken) VR > > or in the systemvm (after which you should redeploy the VR to pick up the > > change). > > > > HTH > > Lucian > > > > -- > > Sent from the Delta quadrant using Borg technology! > > > > Nux! > > www.nux.ro > > > > ----- Original Message ----- > > > From: "Nux!" <n...@li.nux.ro> > > > To: users@cloudstack.apache.org > > > Sent: Monday, 3 November, 2014 13:53:10 > > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > > > > Hello, > > > > > > I think you are hitting a problem I also had: > > > "2014-11-03 08:16:17,024 ERROR [c.c.v.VirtualMachineManagerImpl] > > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 ctx-fcb072b2) > > Failed to > > > start instance VM[DomainRouter|r-4758-VM] > > > com.cloud.utils.exception.ExecutionException: Unable to start > > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, not retrying" > > > > > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781 > > > > > > Basically the systemvm template has some problems, you need to somehow > > copy > > > /opt/cloud/bin/createIpAlias.sh to /opt/cloud/bin/createipAlias.sh to > > get it > > > working. > > > > > > -- > > > Sent from the Delta quadrant using Borg technology! > > > > > > Nux! > > > www.nux.ro > > > > > > ----- Original Message ----- > > >> From: "Adam" <adam.scarce...@gmail.com> > > >> To: users@cloudstack.apache.org > > >> Sent: Monday, 3 November, 2014 13:32:12 > > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > > > > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release > > >> CentOS release 6.6 (Final) > > >> [root@cs-east-prod1 ~]# uname -a > > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 > UTC > > >> 2014 x86_64 x86_64 x86_64 GNU/Linux > > >> [root@cs-east-prod1 ~]# virsh --version > > >> 0.10.2 > > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g' > > >> /etc/cloudstack/agent/log4j-cloud.xml > > >> [root@cs-east-prod1 ~]# service cloudstack-management stop > > >> Stopping cloudstack-management: [ OK ] > > >> [root@cs-east-prod1 ~]# mv > > >> /var/log/cloudstack/management/management-server.log > > >> /var/log/cloudstack/management/management-server.bak > > >> [root@cs-east-prod1 ~]# service cloudstack-management start > > >> Starting cloudstack-management: [ OK ] > > >> [root@cs-east-prod1 ~]# > > >> > > >> Please see attached ZIP of clean management-server.log in DEBUG mode. > At > > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the attached > > >> management-server.log*) I tried starting the Virtual Router from the > > >> Management UI and received this "Status" message in the UI. > > >> > > >> [image: Inline image 1] > > >> > > >> I can see in the management-server.log that while the > > cloudstack-management > > >> service is initializing, it throws the following ERROR at exactly > > (*2014-11-03 > > >> 08:14:36,813 on line 342 of the attached management-server.log*): > > >> > > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] (main:null) > > Unable > > >> to ping management server at 10.97.38.10:9090 due to ConnectException > > >> java.net.ConnectException: Connection refused > > >> at sun.nio.ch.Net.connect0(Native Method) > > >> at sun.nio.ch.Net.connect(Net.java:465) > > >> at sun.nio.ch.Net.connect(Net.java:457) > > >> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > > >> at > > >> > > > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123) > > >> at > > >> > > > com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092) > > >> at > > >> > > > com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170) > > >> at > > >> > > > com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057) > > >> at > > >> > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114) > > >> at > > >> > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153) > > >> at > > >> > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110) > > >> at > > >> > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56) > > >> at > > >> > > > org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167) > > >> at > > >> > > > org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) > > >> at > > >> > > > org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) > > >> at > > >> > > > org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) > > >> at > > >> > > > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) > > >> at > > >> > > > org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945) > > >> at > > >> > > > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) > > >> at > > >> > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > > >> at > > >> > > > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > > >> at > > >> > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) > > >> at > > >> > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57) > > >> at > > >> > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61) > > >> at > > >> > > > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) > > >> at > > >> > > > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210) > > >> at > > >> > > org.apache.catalina.core.StandardContext.start(StandardContext.java:4709) > > >> at > > >> > > > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > > >> at > > >> > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > > >> at > > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) > > >> at > > >> > > > org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041) > > >> at > > >> > > > org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964) > > >> at > > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502) > > >> at > org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) > > >> at > > >> > > > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) > > >> at > > >> > > > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) > > >> at > > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) > > >> at > org.apache.catalina.core.StandardHost.start(StandardHost.java:722) > > >> at > > org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) > > >> at > > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) > > >> at > > >> > org.apache.catalina.core.StandardService.start(StandardService.java:516) > > >> at > > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710) > > >> at org.apache.catalina.startup.Catalina.start(Catalina.java:593) > > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > >> at > > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > >> at > > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > >> at java.lang.reflect.Method.invoke(Method.java:606) > > >> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) > > >> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) > > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] (main:null) > > >> Detected that another management node with the same IP 10.97.38.10 is > > >> considered as running in DB, however it is not pingable, we will > > continue > > >> cluster initialization with this management server node > > >> > > >> I'm not sure what that error means, but more than likely that's the > > reason > > >> why the Virtual Router will not start. > > >> > > >> * PLEASE HELP ! * > > >> > > >> Best Regards, > > >> > > >> > > >> > > >> Adam Scarcella > > >> > > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <n...@li.nux.ro> wrote: > > >> > > >>> Hi, > > >>> > > >>> More agent and management logs please, make sure they are in DEBUG[1] > > mode. > > >>> > > >>> [1] - > > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug > > >>> > > >>> -- > > >>> Sent from the Delta quadrant using Borg technology! > > >>> > > >>> Nux! > > >>> www.nux.ro > > >>> > > >>> ----- Original Message ----- > > >>> > From: "Adam" <adam.scarce...@gmail.com> > > >>> > To: users@cloudstack.apache.org > > >>> > Sent: Sunday, 2 November, 2014 23:33:22 > > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1 > > >>> > > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and followed > the > > >>> > instructions perfectly, but my Virtual Router will not start, nor > am > > I > > >>> able > > >>> > to "upgrade" it. > > >>> > > > >>> > I tried destroying the Virtual Router, starting any VM Instance, > and > > >>> > allowing CS to recreate the VR. However that didn't help. The VR > > still > > >>> did > > >>> > not start and it was still saying that it required to be upgraded. > > >>> > > > >>> > [image: Inline image 1] > > >>> > [image: Inline image 2] > > >>> > [image: Inline image 3] > > >>> > > > >>> > I know there are manual steps to finalize the systemvm upgrade, but > > I did > > >>> > those steps: > > >>> > > > >>> > > > >>> > > > http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix > > >>> > > > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p > > >>> > Enter password: > > >>> > Welcome to the MySQL monitor. Commands end with ; or \g. > > >>> > Your MySQL connection id is 174 > > >>> > Server version: 5.1.73-log Source distribution > > >>> > > > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights > > >>> reserved. > > >>> > > > >>> > Oracle is a registered trademark of Oracle Corporation and/or its > > >>> > affiliates. Other names may be trademarks of their respective > > >>> > owners. > > >>> > > > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the current input > > >>> > statement. > > >>> > > > >>> > mysql> select id,name from cloud.vm_template where name = > > >>> > 'systemvm-kvm-4.4'; > > >>> > +-----+------------------+ > > >>> >| id | name | > > >>> > +-----+------------------+ > > >>> >| 274 | systemvm-kvm-4.4 | > > >>> > +-----+------------------+ > > >>> > 1 row in set (0.00 sec) > > >>> > > > >>> > mysql> select type from cloud.vm_template where id='274'; > > >>> > +--------+ > > >>> >| type | > > >>> > +--------+ > > >>> >| SYSTEM | > > >>> > +--------+ > > >>> > 1 row in set (0.00 sec) > > >>> > > > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where > > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM'; > > >>> > +----------------+ > > >>> >| vm_template_id | > > >>> > +----------------+ > > >>> >| 274 | > > >>> > +----------------+ > > >>> > 1 row in set (0.00 sec) > > >>> > > > >>> > mysql> select distinct(vm_template_id) from cloud.vm_instance where > > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM'; > > >>> > +----------------+ > > >>> >| vm_template_id | > > >>> > +----------------+ > > >>> >| 274 | > > >>> > +----------------+ > > >>> > 1 row in set (0.03 sec) > > >>> > > > >>> > mysql> select value from cloud.configuration where name = > > >>> > 'router.template.kvm'; > > >>> > +------------------+ > > >>> >| value | > > >>> > +------------------+ > > >>> >| systemvm-kvm-4.4 | > > >>> > +------------------+ > > >>> > 1 row in set (0.00 sec) > > >>> > > > >>> > mysql> > > >>> > > > >>> > * PLEASE HELP ! * > > >>> > > > >>> > Best Regards, > > >>> > > > > >> > Adam Scarcella > > >
