Adam, I take it replacing the VR's disk did not work? Well, I'm hoping 4.4.2 will come with a fixed template.
Re downgrading this is how I did it: 1 - stop all services (mgmt and agents) and "yum downgrade cloudstack-\*" 2 - drop the DBs and import backups 3 - restart the downgraded services 4 - you'll also need to rebuild your VR, possibly "virsh destroy" the 4.4.1 one if it's still around Good luck -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro ----- Original Message ----- > From: "Adam" <adam.scarce...@gmail.com> > To: users@cloudstack.apache.org > Sent: Monday, 3 November, 2014 18:51:13 > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 > I've decided to revert back to 4.3.1 since I cannot wait any loner to get > 4.4.1 working. > > I have a complete backup of all three databases just before I attempted to > upgrade so I'm assuming all I would have to do is remove 4.4.1, reinstall > 4.3.1, restore the db and then start everything back up correct? > > Are there any official steps to roll back? > > Best Regards, > > > > Adam Scarcella > > On Mon, Nov 3, 2014 at 12:13 PM, Nux! <n...@li.nux.ro> wrote: > >> What I am saying - and excuse my poor explanations - is that the HV may >> have already cached the old template, so the VR still uses the old broken >> one. >> >> What I was suggesting is find out which /var/lib/libvirt/images/blah.qcow2 >> the VR is using and replace it with my template. >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> ----- Original Message ----- >> > From: "Adam" <adam.scarce...@gmail.com> >> > To: users@cloudstack.apache.org >> > Sent: Monday, 3 November, 2014 17:03:01 >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 >> >> >> >> >> Not sure if this operation that you performed replaces the current >> >> systemvm template and *especially if it replaces it also on the >> >> hypervisor where it is running.* >> >> >> > >> > That procedure is the one documented in the official CloudStack >> > documentation so I hope it actually replaces the systemvm template. The >> > final output from that command tells it was successful: >> > >> > Successfully installed system VM template to >> > /export/secondary/template/tmpl/1/274/ >> > >> > Also, as stated before, the VR is not running on any hypervisor / host >> > because I cannot get it to run, so I'm unsure what you mean when you >> > say "*especially >> > if it replaces it also on the hypervisor where it is running.*" >> > >> > One more thing I would try is to make sure the VR on the HV is also using >> >> this image. You can try to compare md5sums of the image on the secondary >> >> storage and also the backend file for the VR's disk image (qemu-img info >> >> will show it to you) on the HV. >> >> If the sums differ then I would try to replace the VR disk image with my >> >> template directly on the HV and see if that does it. >> > >> > >> > I really don't understand what you're asking me to do. >> > >> > Best Regards, >> > >> > >> > >> > Adam Scarcella >> > >> > On Mon, Nov 3, 2014 at 11:50 AM, Nux! <n...@li.nux.ro> wrote: >> > >> >> Adam, >> >> >> >> Not sure if this operation that you performed replaces the current >> >> systemvm template and especially if it replaces it also on the >> hypervisor >> >> where it is running. >> >> >> >> One more thing I would try is to make sure the VR on the HV is also >> using >> >> this image. You can try to compare md5sums of the image on the secondary >> >> storage and also the backend file for the VR's disk image (qemu-img info >> >> will show it to you) on the HV. >> >> If the sums differ then I would try to replace the VR disk image with my >> >> template directly on the HV and see if that does it. >> >> >> >> -- >> >> Sent from the Delta quadrant using Borg technology! >> >> >> >> Nux! >> >> www.nux.ro >> >> >> >> ----- Original Message ----- >> >> > From: "Adam" <adam.scarce...@gmail.com> >> >> > To: users@cloudstack.apache.org >> >> > Sent: Monday, 3 November, 2014 16:08:05 >> >> > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 >> >> >> >> > Thanks HTH, >> >> > >> >> > I decided to try to use your already modified qcow2 file: >> >> > >> >> > [root@cs-east-prod1 /]# >> >> > >> >> >> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt >> >> > -m /export/secondary -u >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2 >> >> > -h kvm -F >> >> > --2014-11-03 10:11:32-- >> >> http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2 >> >> > Resolving tmp.nux.ro... 217.19.15.108 >> >> > Connecting to tmp.nux.ro|217.19.15.108|:80... connected. >> >> > HTTP request sent, awaiting response... 200 OK >> >> > Length: 305498799 (291M) [application/x-bzip2] >> >> > Saving to: >> >> > >> >> >> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2” >> >> > >> >> > >> >> >> 100%[===================================================================================================================>] >> >> > 305,498,799 9.02M/s in 37s >> >> > >> >> > 2014-11-03 10:12:10 (7.82 MB/s) - >> >> > >> >> >> “/usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2” >> >> > saved [305498799/305498799] >> >> > >> >> > Uncompressing to >> >> > >> >> >> /usr/share/cloudstack-common/scripts/storage/secondary/00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2.tmp >> >> > (type bz2)...could take a long time >> >> > Moving to >> >> > >> >> >> /export/secondary/template/tmpl/1/274///00ccf4fc-fa94-4e82-8415-f8ac89fbbfaa.qcow2...could >> >> > take a while >> >> > Successfully installed system VM template to >> >> > /export/secondary/template/tmpl/1/274/ >> >> > [root@cs-east-prod1 /]# >> >> > >> >> > Then I went into the Admin GUI and tried to start the VR, but it >> failed >> >> > with the exact same message: >> >> > >> >> > [image: Inline image 1] >> >> > >> >> > So I destroyed the Virtual Router and stopped and started the >> following >> >> > services: >> >> > >> >> > [root@cs-east-prod1 /]# service iptables stop >> >> > iptables: Setting chains to policy ACCEPT: nat filter [ OK ] >> >> > iptables: Flushing firewall rules: [ OK ] >> >> > iptables: Unloading modules: [ OK ] >> >> > [root@cs-east-prod1 /]# service cloudstack-management stop >> >> > Stopping cloudstack-management: [ OK ] >> >> > [root@cs-east-prod1 /]# service cloudstack-agent stop >> >> > Stopping Cloud Agent: >> >> > [root@cs-east-prod1 /]# service cloudstack-usage stop >> >> > Stopping CloudStack Usage Monitor cloudstack-usage [ OK ] >> >> > [root@cs-east-prod1 /]# service cloudstack-management start >> >> > Starting cloudstack-management: [ OK ] >> >> > [root@cs-east-prod1 /]# service cloudstack-usage start >> >> > Starting CloudStack Usage Monitor cloudstack-usage [ OK ] >> >> > [root@cs-east-prod1 /]# service cloudstack-agent start >> >> > Starting Cloud Agent: >> >> > [root@cs-east-prod1 /]# >> >> > >> >> > I then tried starting any VM instance to have CS automatically >> re-create >> >> > the VR, but that failed as well. >> >> > >> >> > Motty, I also looked at trying to increase the RAM and CPU for the >> router >> >> > but only saw one option >> >> > >> >> > [image: Inline image 2] >> >> > >> >> > So I created a new Service Offering with twice as much resources: >> >> > >> >> > [image: Inline image 3] >> >> > >> >> > Then upgraded the service offering for the VR >> >> > >> >> > [image: Inline image 4] >> >> > >> >> > [image: Inline image 6] >> >> > >> >> > But it failed still with the exact same error: >> >> > >> >> > [image: Inline image 7] >> >> > >> >> > Best Regards, >> >> > >> >> > >> >> > >> >> > Adam Scarcella >> >> > >> >> > On Mon, Nov 3, 2014 at 10:23 AM, motty cruz <motty.c...@gmail.com> >> >> wrote: >> >> > >> >> >> Adam, >> >> >> what are the specifications of the VR? can you upgrade the RAM and >> CPU >> >> and >> >> >> try to restart that VR? I had similar issue and giving the VR more >> RAM >> >> and >> >> >> CPU will start the VR. >> >> >> >> >> >> Thanks, >> >> >> Motty >> >> >> >> >> >> On Mon, Nov 3, 2014 at 7:04 AM, Nux! <n...@li.nux.ro> wrote: >> >> >> >> >> >> > Adam, >> >> >> > >> >> >> > You can modify the qcow2 file that is in your secondary storage, >> i.e. >> >> you >> >> >> > should mount that storage somewhere and go to >> >> >> > /mount/point/template/tmpl/274 and modify the qcow2 file. >> >> >> > >> >> >> > Alternatively you could use my already modified copy to overwrite >> your >> >> >> > uuid.qcow2 with (at your own risk of course): >> >> >> > http://tmp.nux.ro/systemvm-4.4.1_jira7781.qcow2.bz2 >> >> >> > >> >> >> > HTH >> >> >> > >> >> >> > -- >> >> >> > Sent from the Delta quadrant using Borg technology! >> >> >> > >> >> >> > Nux! >> >> >> > www.nux.ro >> >> >> > >> >> >> > ----- Original Message ----- >> >> >> > > From: "Adam" <adam.scarce...@gmail.com> >> >> >> > > To: users@cloudstack.apache.org >> >> >> > > Sent: Monday, 3 November, 2014 14:58:58 >> >> >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 >> >> >> > >> >> >> > > [root@cs-east-prod1 management]# netstat -a|grep 9090 >> >> >> > > unix 3 [ ] STREAM CONNECTED 29090 >> >> >> > > [root@cs-east-prod1 management]# nmap localhost >> >> >> > > >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:47 EST >> >> >> > > Nmap scan report for localhost (127.0.0.1) >> >> >> > > Host is up (0.000015s latency). >> >> >> > > Other addresses for localhost (not scanned): 127.0.0.1 >> >> >> > > Not shown: 990 closed ports >> >> >> > > PORT STATE SERVICE >> >> >> > > 22/tcp open ssh >> >> >> > > 25/tcp open smtp >> >> >> > > 80/tcp open http >> >> >> > > 111/tcp open rpcbind >> >> >> > > 199/tcp open smux >> >> >> > > 631/tcp open ipp >> >> >> > > 2049/tcp open nfs >> >> >> > > 3306/tcp open mysql >> >> >> > > 8080/tcp open http-proxy >> >> >> > > 9090/tcp open zeus-admin >> >> >> > > >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds >> >> >> > > [root@cs-east-prod1 management]# telnet localhost 9090 >> >> >> > > Trying ::1... >> >> >> > > Connected to localhost. >> >> >> > > Escape character is '^]'. >> >> >> > > Connection closed by foreign host. >> >> >> > > [root@cs-east-prod1 management]# nmap 10.97.38.10 >> >> >> > > >> >> >> > > Starting Nmap 5.51 ( http://nmap.org ) at 2014-11-03 08:49 EST >> >> >> > > Nmap scan report for cs-east-prod1.na.tibco.com (10.97.38.10) >> >> >> > > Host is up (0.000015s latency). >> >> >> > > Not shown: 993 closed ports >> >> >> > > PORT STATE SERVICE >> >> >> > > 22/tcp open ssh >> >> >> > > 80/tcp open http >> >> >> > > 111/tcp open rpcbind >> >> >> > > 2049/tcp open nfs >> >> >> > > 3306/tcp open mysql >> >> >> > > 8080/tcp open http-proxy >> >> >> > > 9090/tcp open zeus-admin >> >> >> > > >> >> >> > > Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds >> >> >> > > [root@cs-east-prod1 management]# telnet 10.97.38.10 9090 >> >> >> > > Trying 10.97.38.10... >> >> >> > > Connected to 10.97.38.10. >> >> >> > > Escape character is '^]'. >> >> >> > > Connection closed by foreign host. >> >> >> > > [root@cs-east-prod1 management]# >> >> >> > > >> >> >> > > Also, I checked for the /opt/cloud/bin/ directory on my >> Management >> >> >> Host, >> >> >> > > but it's not there: >> >> >> > > >> >> >> > > [root@cs-east-prod1 management]# cd /opt/cloud/bin/ >> >> >> > > -bash: cd: /opt/cloud/bin/: No such file or directory >> >> >> > > [root@cs-east-prod1 management]# cd / >> >> >> > > [root@cs-east-prod1 /]# find . -name "createipAlias.sh" >> >> >> > > [root@cs-east-prod1 /]# find . -name "createIpAlias.sh" >> >> >> > > [root@cs-east-prod1 /]# >> >> >> > > >> >> >> > > Are you saying that the /opt/cloud/bin/createipAlias.sh script is >> >> >> inside >> >> >> > > the Virtual Router VM? If so, how could I possibly change that >> >> since it >> >> >> > > won't start? >> >> >> > > >> >> >> > > Best Regards, >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > Adam Scarcella >> >> >> > > >> >> >> > > On Mon, Nov 3, 2014 at 9:07 AM, Pierre-Luc Dion < >> >> pdion...@apache.org> >> >> >> > wrote: >> >> >> > > >> >> >> > >> Could it be possible the firewall is blocking port 9090 from the >> >> >> > >> management-server? >> >> >> > >> >> >> >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] >> >> (main:null) >> >> >> > Unable >> >> >> > >> to ping management server at 10.97.38.10:9090 due to >> >> ConnectException >> >> >> > >> java.net.ConnectException: Connection refused >> >> >> > >> >> >> >> > >> On Mon, Nov 3, 2014 at 9:02 AM, Nux! <n...@li.nux.ro> wrote: >> >> >> > >> >> >> >> > >> > BTW the agent logs should help clarify this. >> >> >> > >> > The aforementioned changes must be done either in the current >> >> >> > (broken) VR >> >> >> > >> > or in the systemvm (after which you should redeploy the VR to >> >> pick >> >> >> up >> >> >> > the >> >> >> > >> > change). >> >> >> > >> > >> >> >> > >> > HTH >> >> >> > >> > Lucian >> >> >> > >> > >> >> >> > >> > -- >> >> >> > >> > Sent from the Delta quadrant using Borg technology! >> >> >> > >> > >> >> >> > >> > Nux! >> >> >> > >> > www.nux.ro >> >> >> > >> > >> >> >> > >> > ----- Original Message ----- >> >> >> > >> > > From: "Nux!" <n...@li.nux.ro> >> >> >> > >> > > To: users@cloudstack.apache.org >> >> >> > >> > > Sent: Monday, 3 November, 2014 13:53:10 >> >> >> > >> > > Subject: Re: How to Upgrade Virtual Router from 4.3 to 4.4.1 >> >> >> > >> > >> >> >> > >> > > Hello, >> >> >> > >> > > >> >> >> > >> > > I think you are hitting a problem I also had: >> >> >> > >> > > "2014-11-03 08:16:17,024 ERROR >> >> [c.c.v.VirtualMachineManagerImpl] >> >> >> > >> > > (Work-Job-Executor-1:ctx-5c33d547 job-24555/job-24556 >> >> >> ctx-fcb072b2) >> >> >> > >> > Failed to >> >> >> > >> > > start instance VM[DomainRouter|r-4758-VM] >> >> >> > >> > > com.cloud.utils.exception.ExecutionException: Unable to >> start >> >> >> > >> > > VM[DomainRouter|r-4758-VM] due to error in finalizeStart, >> not >> >> >> > retrying" >> >> >> > >> > > >> >> >> > >> > > It's https://issues.apache.org/jira/browse/CLOUDSTACK-7781 >> >> >> > >> > > >> >> >> > >> > > Basically the systemvm template has some problems, you need >> to >> >> >> > somehow >> >> >> > >> > copy >> >> >> > >> > > /opt/cloud/bin/createIpAlias.sh to >> >> /opt/cloud/bin/createipAlias.sh >> >> >> > to >> >> >> > >> > get it >> >> >> > >> > > working. >> >> >> > >> > > >> >> >> > >> > > -- >> >> >> > >> > > Sent from the Delta quadrant using Borg technology! >> >> >> > >> > > >> >> >> > >> > > Nux! >> >> >> > >> > > www.nux.ro >> >> >> > >> > > >> >> >> > >> > > ----- Original Message ----- >> >> >> > >> > >> From: "Adam" <adam.scarce...@gmail.com> >> >> >> > >> > >> To: users@cloudstack.apache.org >> >> >> > >> > >> Sent: Monday, 3 November, 2014 13:32:12 >> >> >> > >> > >> Subject: Re: How to Upgrade Virtual Router from 4.3 to >> 4.4.1 >> >> >> > >> > > >> >> >> > >> > >> [root@cs-east-prod1 ~]# cat /etc/redhat-release >> >> >> > >> > >> CentOS release 6.6 (Final) >> >> >> > >> > >> [root@cs-east-prod1 ~]# uname -a >> >> >> > >> > >> Linux cs-east-prod1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 >> >> >> > 04:27:16 >> >> >> > >> UTC >> >> >> > >> > >> 2014 x86_64 x86_64 x86_64 GNU/Linux >> >> >> > >> > >> [root@cs-east-prod1 ~]# virsh --version >> >> >> > >> > >> 0.10.2 >> >> >> > >> > >> [root@cs-east-prod1 ~]# sed -i 's/INFO/DEBUG/g' >> >> >> > >> > >> /etc/cloudstack/agent/log4j-cloud.xml >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management stop >> >> >> > >> > >> Stopping cloudstack-management: >> [ >> >> >> OK ] >> >> >> > >> > >> [root@cs-east-prod1 ~]# mv >> >> >> > >> > >> /var/log/cloudstack/management/management-server.log >> >> >> > >> > >> /var/log/cloudstack/management/management-server.bak >> >> >> > >> > >> [root@cs-east-prod1 ~]# service cloudstack-management >> start >> >> >> > >> > >> Starting cloudstack-management: >> [ >> >> >> OK ] >> >> >> > >> > >> [root@cs-east-prod1 ~]# >> >> >> > >> > >> >> >> >> > >> > >> Please see attached ZIP of clean management-server.log in >> >> DEBUG >> >> >> > mode. >> >> >> > >> At >> >> >> > >> > >> exactly (*2014-11-03 08:15:35,700 on line 3025 of the >> attached >> >> >> > >> > >> management-server.log*) I tried starting the Virtual Router >> >> from >> >> >> > the >> >> >> > >> > >> Management UI and received this "Status" message in the UI. >> >> >> > >> > >> >> >> >> > >> > >> [image: Inline image 1] >> >> >> > >> > >> >> >> >> > >> > >> I can see in the management-server.log that while the >> >> >> > >> > cloudstack-management >> >> >> > >> > >> service is initializing, it throws the following ERROR at >> >> exactly >> >> >> > >> > (*2014-11-03 >> >> >> > >> > >> 08:14:36,813 on line 342 of the attached >> >> management-server.log*): >> >> >> > >> > >> >> >> >> > >> > >> 2014-11-03 08:14:36,813 ERROR [c.c.c.ClusterManagerImpl] >> >> >> > (main:null) >> >> >> > >> > Unable >> >> >> > >> > >> to ping management server at 10.97.38.10:9090 due to >> >> >> > ConnectException >> >> >> > >> > >> java.net.ConnectException: Connection refused >> >> >> > >> > >> at sun.nio.ch.Net.connect0(Native Method) >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:465) >> >> >> > >> > >> at sun.nio.ch.Net.connect(Net.java:457) >> >> >> > >> > >> at >> >> >> > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1123) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1092) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> com.cloud.cluster.ClusterManagerImpl.checkConflicts(ClusterManagerImpl.java:1170) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> com.cloud.cluster.ClusterManagerImpl.configure(ClusterManagerImpl.java:1057) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$3.with(CloudStackExtendedLifeCycle.java:114) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:153) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.configure(CloudStackExtendedLifeCycle.java:110) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:56) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:167) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:339) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:108) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> org.apache.catalina.core.StandardContext.start(StandardContext.java:4709) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> >> >> >> >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502) >> >> >> > >> > >> at >> >> >> > >> >> org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053) >> >> >> > >> > >> at >> >> >> > >> >> org.apache.catalina.core.StandardHost.start(StandardHost.java:722) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> >> >> >> >> org.apache.catalina.core.StandardService.start(StandardService.java:516) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> org.apache.catalina.core.StandardServer.start(StandardServer.java:710) >> >> >> > >> > >> at >> >> >> org.apache.catalina.startup.Catalina.start(Catalina.java:593) >> >> >> > >> > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> >> Method) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> >> > >> > >> at >> >> >> > >> > >> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> >> > >> > >> at java.lang.reflect.Method.invoke(Method.java:606) >> >> >> > >> > >> at >> >> >> > org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) >> >> >> > >> > >> at >> >> >> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) >> >> >> > >> > >> 2014-11-03 08:14:36,816 INFO [c.c.c.ClusterManagerImpl] >> >> >> > (main:null) >> >> >> > >> > >> Detected that another management node with the same IP >> >> >> 10.97.38.10 >> >> >> > is >> >> >> > >> > >> considered as running in DB, however it is not pingable, we >> >> will >> >> >> > >> > continue >> >> >> > >> > >> cluster initialization with this management server node >> >> >> > >> > >> >> >> >> > >> > >> I'm not sure what that error means, but more than likely >> >> that's >> >> >> the >> >> >> > >> > reason >> >> >> > >> > >> why the Virtual Router will not start. >> >> >> > >> > >> >> >> >> > >> > >> * PLEASE HELP ! * >> >> >> > >> > >> >> >> >> > >> > >> Best Regards, >> >> >> > >> > >> >> >> >> > >> > >> >> >> >> > >> > >> >> >> >> > >> > >> Adam Scarcella >> >> >> > >> > >> >> >> >> > >> > >> On Sun, Nov 2, 2014 at 8:20 PM, Nux! <n...@li.nux.ro> >> wrote: >> >> >> > >> > >> >> >> >> > >> > >>> Hi, >> >> >> > >> > >>> >> >> >> > >> > >>> More agent and management logs please, make sure they are >> in >> >> >> > DEBUG[1] >> >> >> > >> > mode. >> >> >> > >> > >>> >> >> >> > >> > >>> [1] - >> >> >> > >> > >>> >> >> >> > >> >> >> >> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug >> >> >> > >> > >>> >> >> >> > >> > >>> -- >> >> >> > >> > >>> Sent from the Delta quadrant using Borg technology! >> >> >> > >> > >>> >> >> >> > >> > >>> Nux! >> >> >> > >> > >>> www.nux.ro >> >> >> > >> > >>> >> >> >> > >> > >>> ----- Original Message ----- >> >> >> > >> > >>> > From: "Adam" <adam.scarce...@gmail.com> >> >> >> > >> > >>> > To: users@cloudstack.apache.org >> >> >> > >> > >>> > Sent: Sunday, 2 November, 2014 23:33:22 >> >> >> > >> > >>> > Subject: How to Upgrade Virtual Router from 4.3 to 4.4.1 >> >> >> > >> > >>> >> >> >> > >> > >>> > I've performed a complete upgrade from 4.3 to 4.4.1 and >> >> >> followed >> >> >> > >> the >> >> >> > >> > >>> > instructions perfectly, but my Virtual Router will not >> >> start, >> >> >> > nor >> >> >> > >> am >> >> >> > >> > I >> >> >> > >> > >>> able >> >> >> > >> > >>> > to "upgrade" it. >> >> >> > >> > >>> > >> >> >> > >> > >>> > I tried destroying the Virtual Router, starting any VM >> >> >> Instance, >> >> >> > >> and >> >> >> > >> > >>> > allowing CS to recreate the VR. However that didn't >> help. >> >> The >> >> >> VR >> >> >> > >> > still >> >> >> > >> > >>> did >> >> >> > >> > >>> > not start and it was still saying that it required to be >> >> >> > upgraded. >> >> >> > >> > >>> > >> >> >> > >> > >>> > [image: Inline image 1] >> >> >> > >> > >>> > [image: Inline image 2] >> >> >> > >> > >>> > [image: Inline image 3] >> >> >> > >> > >>> > >> >> >> > >> > >>> > I know there are manual steps to finalize the systemvm >> >> >> upgrade, >> >> >> > but >> >> >> > >> > I did >> >> >> > >> > >>> > those steps: >> >> >> > >> > >>> > >> >> >> > >> > >>> > >> >> >> > >> > >>> >> >> >> > >> > >> >> >> > >> >> >> >> > >> >> >> >> >> >> http://cloudstack-release-notes.readthedocs.org/en/latest/upgrade/upgrade-4.3.html#manual-hofix >> >> >> > >> > >>> > >> >> >> > >> > >>> > [root@cs-east-prod1 cloudstack-logs]# mysql -u root -p >> >> >> > >> > >>> > Enter password: >> >> >> > >> > >>> > Welcome to the MySQL monitor. Commands end with ; or >> \g. >> >> >> > >> > >>> > Your MySQL connection id is 174 >> >> >> > >> > >>> > Server version: 5.1.73-log Source distribution >> >> >> > >> > >>> > >> >> >> > >> > >>> > Copyright (c) 2000, 2013, Oracle and/or its affiliates. >> All >> >> >> > rights >> >> >> > >> > >>> reserved. >> >> >> > >> > >>> > >> >> >> > >> > >>> > Oracle is a registered trademark of Oracle Corporation >> >> and/or >> >> >> > its >> >> >> > >> > >>> > affiliates. Other names may be trademarks of their >> >> respective >> >> >> > >> > >>> > owners. >> >> >> > >> > >>> > >> >> >> > >> > >>> > Type 'help;' or '\h' for help. Type '\c' to clear the >> >> current >> >> >> > input >> >> >> > >> > >>> > statement. >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> select id,name from cloud.vm_template where name >> = >> >> >> > >> > >>> > 'systemvm-kvm-4.4'; >> >> >> > >> > >>> > +-----+------------------+ >> >> >> > >> > >>> >| id | name | >> >> >> > >> > >>> > +-----+------------------+ >> >> >> > >> > >>> >| 274 | systemvm-kvm-4.4 | >> >> >> > >> > >>> > +-----+------------------+ >> >> >> > >> > >>> > 1 row in set (0.00 sec) >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> select type from cloud.vm_template where >> id='274'; >> >> >> > >> > >>> > +--------+ >> >> >> > >> > >>> >| type | >> >> >> > >> > >>> > +--------+ >> >> >> > >> > >>> >| SYSTEM | >> >> >> > >> > >>> > +--------+ >> >> >> > >> > >>> > 1 row in set (0.00 sec) >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from >> >> cloud.vm_instance >> >> >> > where >> >> >> > >> > >>> > type='ConsoleProxy' and hypervisor_type = 'KVM'; >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> >| vm_template_id | >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> >| 274 | >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> > 1 row in set (0.00 sec) >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> select distinct(vm_template_id) from >> >> cloud.vm_instance >> >> >> > where >> >> >> > >> > >>> > type='SecondaryStorageVm' and hypervisor_type = 'KVM'; >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> >| vm_template_id | >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> >| 274 | >> >> >> > >> > >>> > +----------------+ >> >> >> > >> > >>> > 1 row in set (0.03 sec) >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> select value from cloud.configuration where name >> = >> >> >> > >> > >>> > 'router.template.kvm'; >> >> >> > >> > >>> > +------------------+ >> >> >> > >> > >>> >| value | >> >> >> > >> > >>> > +------------------+ >> >> >> > >> > >>> >| systemvm-kvm-4.4 | >> >> >> > >> > >>> > +------------------+ >> >> >> > >> > >>> > 1 row in set (0.00 sec) >> >> >> > >> > >>> > >> >> >> > >> > >>> > mysql> >> >> >> > >> > >>> > >> >> >> > >> > >>> > * PLEASE HELP ! * >> >> >> > >> > >>> > >> >> >> > >> > >>> > Best Regards, >> >> >> > >> > >>> > >> >> >> > >> > > >> > Adam Scarcella >> >> >> > >> > >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Thanks for your support, >> >> >> Motty
