RE: R: A Story of a Failed XenServer Upgrade
Hi Guys, I've taken a look at the article written by Geoff, it's actually referring to XenServer upgrading from 6.02 to 6.2 and from the date of the article CloudStack 4.3 Much has changed in that time. WRT to the manage.xenserver.pool.master=false setting... CloudStack's behaviour used to be that putting a host that was the pool master into maintenance would cause CloudStack to force an election for another host to become pool master - stopping you from then upgrading the active pool master first. There wasn't an 'unmanage' button at the time. I hope that explains Geoff's article. [ShapeBlue]<http://www.shapeblue.com> Paul Angus VP Technology , ShapeBlue d: +44 203 617 0528 | s: +44 203 603 0540 | m: +44 7711 418784 e: paul.an...@shapeblue.com | t: @cloudyangus<mailto:paul.an...@shapeblue.com%20|%20t:%20@cloudyangus> | w: www.shapeblue.com<http://www.shapeblue.com> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK [cid:imagea2d437.png@07b65bdb.429ef2f9] Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. -Original Message- From: Alessandro Caviglione [mailto:c.alessan...@gmail.com] Sent: Tuesday, January 12, 2016 11:55 AM To: users@cloudstack.apache.org Subject: Re: R: A Story of a Failed XenServer Upgrade Hi guys, I think that there are a little bit of confusion around this topic Pierre-Luc Dion posted an official documentation ( http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.6/hypervisor/xenserver.html#upgrading-xenserver-versions) that differs in many steps from that published by Geoff Higginbottom of Shapeblue and mentioned by Yiping Zhang... that is the same I've followed for my failed upgrade. In the Cloudstack document, the host do NOT GOES into Maintenance Mode and I've to run the following scripts: /opt/xensource/bin/cloud-clean-vlan.sh /opt/xensource/bin/cloud-prepare-upgrade.sh Since in the Cloudstack doc the host do not goes in MM, they says that I've to migrate all the VMs manually: "Live migrate all VMs on this host to other hosts. See the instructions for live migration in the Administrator’s Guide." Cloudstack doc says also that I've to copy the scripts from Pierre mentioned: /opt/xensource/sm/NFSSR.py /opt/xensource/bin/setupxenserver.sh /opt/xensource/bin/make_migratable.sh /opt/xensource/bin/cloud-clean-vlan.sh Point 6 and 7 are strange I've to upgrade the entire cluster and clear the host tag BEFORE connect again to Cloudstack: 6)Repeat these steps to upgrade every host in the cluster to the same version of XenServer. 7)Run the following command on one host in the XenServer cluster to clean up the host tag... Shapeblue says that I've to manage again the Cluster after the upgrade the Pool Master and I can upgrade the other hosts later... Another thing that is worring me is the script: /opt/xensource/bin/cloud-clean-vlan.sh So, what it does?? What means "clean-vlan"?? :o On Fri, Jan 8, 2016 at 10:52 PM, Yiping Zhang wrote: > Since I can’t use attachment, I just include the doc in the message. > Hopefully, the indentations and formats will go through properly. > > Yiping > > -- > XenServer pool manual upgrade from 6.2 to 6.5 using ISO Reference > article for upgrading XenServer pool used for Cloudstack > > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xe > nserver-cluster > > > Manual upgrade to XenServer from 6.2 to 6.5 using ISO > > > On CloudStack Management server > > * Edit the file /etc/cloudstack/managment/environment.properties to > include the following line at the end: > * manage.xenserver.pool.master=false > * Restart cloudstack-management service > * service cloudstack-management restart > > Pre-upgrade steps > > * Disable XenServer pool HA from XenCenter or CLI > * Backup XenServer resource pool co
Re: R: A Story of a Failed XenServer Upgrade
Hi guys, I think that there are a little bit of confusion around this topic Pierre-Luc Dion posted an official documentation ( http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.6/hypervisor/xenserver.html#upgrading-xenserver-versions) that differs in many steps from that published by Geoff Higginbottom of Shapeblue and mentioned by Yiping Zhang... that is the same I've followed for my failed upgrade. In the Cloudstack document, the host do NOT GOES into Maintenance Mode and I've to run the following scripts: /opt/xensource/bin/cloud-clean-vlan.sh /opt/xensource/bin/cloud-prepare-upgrade.sh Since in the Cloudstack doc the host do not goes in MM, they says that I've to migrate all the VMs manually: "Live migrate all VMs on this host to other hosts. See the instructions for live migration in the Administrator’s Guide." Cloudstack doc says also that I've to copy the scripts from Pierre mentioned: /opt/xensource/sm/NFSSR.py /opt/xensource/bin/setupxenserver.sh /opt/xensource/bin/make_migratable.sh /opt/xensource/bin/cloud-clean-vlan.sh Point 6 and 7 are strange I've to upgrade the entire cluster and clear the host tag BEFORE connect again to Cloudstack: 6)Repeat these steps to upgrade every host in the cluster to the same version of XenServer. 7)Run the following command on one host in the XenServer cluster to clean up the host tag... Shapeblue says that I've to manage again the Cluster after the upgrade the Pool Master and I can upgrade the other hosts later... Another thing that is worring me is the script: /opt/xensource/bin/cloud-clean-vlan.sh So, what it does?? What means "clean-vlan"?? :o On Fri, Jan 8, 2016 at 10:52 PM, Yiping Zhang wrote: > Since I can’t use attachment, I just include the doc in the message. > Hopefully, the indentations and formats will go through properly. > > Yiping > > -- > XenServer pool manual upgrade from 6.2 to 6.5 using ISO > Reference article for upgrading XenServer pool used for Cloudstack > > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster > > > Manual upgrade to XenServer from 6.2 to 6.5 using ISO > > > On CloudStack Management server > > * Edit the file /etc/cloudstack/managment/environment.properties to > include the following line at the end: > * manage.xenserver.pool.master=false > * Restart cloudstack-management service > * service cloudstack-management restart > > Pre-upgrade steps > > * Disable XenServer pool HA from XenCenter or CLI > * Backup XenServer resource pool configurations > * Take screen shot for pool network settings in XenCenter > * Take notes of Storage Repo mount points and NFS volumes. > > Inside CloudStack Web UI > > * Put pool master host into Maintenance (CLOUDSTACK ONLY!). This > should migrate all VM instances currently running on the pool master onto > other hosts > * Unmanage the cluster. This should make the hypervisors show as > disconnected in the UI. PLEASE MAKE SURE THAT YOU CLICK "Unmanage > Cluster", NOT "Disable Cluster"!!! > > On Pool Master > > * Connect to physical console using DRAC/iLO/equivalent > * Attach XenServer 6.5 ISO as virtual DVD > * Verify that this host uses Legacy BIOS rather than UEFI, as UEFI is > NOT supported by XenServer. (May not be needed, as XS 6.2 also requires > Legacy BIOS) > * Reboot physical server > * Once the server boots up off DVD image: > Note: we used an answer file to allow automated upgrade. You can just > manually do the upgrade > * At the first prompt, hit F2 to get the advanced menu, which won't > time out quickly > * Type: > * menu.c32 > * Hit Enter, then hit the Tab key. > * You will be presented with a boot line that looks similar to the > following: > * mboot.c32 /boot/xen.gz dom0_max_vcpus=2 dom0_mem=752M > com1=115200,8n1 console=com1,vga — /boot/vmlinuz xencons=hvc console=hvc0 > console=tty0 — /install.img > * You will need to edit this line to include the bold parts. > Nothing else in this line needs to be changed: > * mboot.c32 /boot/xen.gz dom0_max_vcpus=2 dom0_mem=752M > com1=115200,8n1 console=com1,vga — /boot/vmlinuz xencons=hvc console=hvc0 > console=tty0 answerfile=http://server_ip/path/to/answer-file.xml — > /install.img > * Hit Enter and watch the system upgrade itself. > * Once complete, eject the DVD image and reboot the host. > > Verify network and storage settings of upgraded host (in XenCenter): > > * Configure networks if necessary, just in case any additional NIC's > need to be configured > * Repair the HA SR, if necessary > > Inside CloudStack Web UI > > * Re-manage the cluster > * Wait for all hosts to be in the Up state (except the pool master, > which will stay as "disconnected") > * Wait for all SR are connected and online > * Take the pool master out of Maintenance mode > > On the CloudStack manag
Re: R: A Story of a Failed XenServer Upgrade
Since I can’t use attachment, I just include the doc in the message. Hopefully, the indentations and formats will go through properly. Yiping -- XenServer pool manual upgrade from 6.2 to 6.5 using ISO Reference article for upgrading XenServer pool used for Cloudstack http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster Manual upgrade to XenServer from 6.2 to 6.5 using ISO On CloudStack Management server * Edit the file /etc/cloudstack/managment/environment.properties to include the following line at the end: * manage.xenserver.pool.master=false * Restart cloudstack-management service * service cloudstack-management restart Pre-upgrade steps * Disable XenServer pool HA from XenCenter or CLI * Backup XenServer resource pool configurations * Take screen shot for pool network settings in XenCenter * Take notes of Storage Repo mount points and NFS volumes. Inside CloudStack Web UI * Put pool master host into Maintenance (CLOUDSTACK ONLY!). This should migrate all VM instances currently running on the pool master onto other hosts * Unmanage the cluster. This should make the hypervisors show as disconnected in the UI. PLEASE MAKE SURE THAT YOU CLICK "Unmanage Cluster", NOT "Disable Cluster"!!! On Pool Master * Connect to physical console using DRAC/iLO/equivalent * Attach XenServer 6.5 ISO as virtual DVD * Verify that this host uses Legacy BIOS rather than UEFI, as UEFI is NOT supported by XenServer. (May not be needed, as XS 6.2 also requires Legacy BIOS) * Reboot physical server * Once the server boots up off DVD image: Note: we used an answer file to allow automated upgrade. You can just manually do the upgrade * At the first prompt, hit F2 to get the advanced menu, which won't time out quickly * Type: * menu.c32 * Hit Enter, then hit the Tab key. * You will be presented with a boot line that looks similar to the following: * mboot.c32 /boot/xen.gz dom0_max_vcpus=2 dom0_mem=752M com1=115200,8n1 console=com1,vga — /boot/vmlinuz xencons=hvc console=hvc0 console=tty0 — /install.img * You will need to edit this line to include the bold parts. Nothing else in this line needs to be changed: * mboot.c32 /boot/xen.gz dom0_max_vcpus=2 dom0_mem=752M com1=115200,8n1 console=com1,vga — /boot/vmlinuz xencons=hvc console=hvc0 console=tty0 answerfile=http://server_ip/path/to/answer-file.xml — /install.img * Hit Enter and watch the system upgrade itself. * Once complete, eject the DVD image and reboot the host. Verify network and storage settings of upgraded host (in XenCenter): * Configure networks if necessary, just in case any additional NIC's need to be configured * Repair the HA SR, if necessary Inside CloudStack Web UI * Re-manage the cluster * Wait for all hosts to be in the Up state (except the pool master, which will stay as "disconnected") * Wait for all SR are connected and online * Take the pool master out of Maintenance mode On the CloudStack management server in a terminal window * Undo the change in /etc/cloudstack/management/environment.properties file * Restart cloudstack-management service and wait for the dust to settle on all servers/storage being visible. All hosts should be in Up state, including the pool master. Now for each slave hosts: In CloudStack web UI * Put slave host into Maintenance mode (CLOUDSTACK ONLY!), in order to evacuate all instances running on this host * On physical console, follow the same steps to attach DVD image and do upgrade as for pool master node * Perform the eject, reboot, and and host verification as listed above * Verify the networks and that it has properly rejoined the Xenserver Pool (in XenCenter) * Take the host out of Maintenance mode in the CloudStack UI Post-operation steps for each hosts: * Confirm XenCenter Licensing * You may need to remove the license from the cluster before applying a new one * Enable HA You are done !
Re: R: A Story of a Failed XenServer Upgrade
Maybe a Forum could be a better solution? It would be also a "old fashion" solution! :D On Fri, Jan 8, 2016 at 8:10 PM, Ahmad Emneina wrote: > The mailing list strips attachments you'll need to post it on an external > site. > > > > On Jan 8, 2016, at 11:00 AM, Yiping Zhang wrote: > > > > Hm I definitely had included the attachment (as I can see from my > Outlook’s sent folder that the message has an attachment). > > > > I am wondering if the mailing list requires any special privilege to > send messages with attachment. Does anyone know ? I can forward the doc to > someone who has the power to resend it to the list if anyone volunteers to > do so. > > > > To answer Nux!’s suggestion of doing a blog, but I am old fashioned guy > that I have not tried that yet :) > > > > Yiping > > > > > > > > > >> On 1/8/16, 10:35 AM, "Davide Pala" wrote: > >> > >> I think you've forget the attachment ... > >> > >> > >> > >> Inviato dal mio dispositivo Samsung > >> > >> > >> Messaggio originale > >> Da: Yiping Zhang > >> Data: 08/01/2016 18:44 (GMT+01:00) > >> A: users@cloudstack.apache.org > >> Oggetto: Re: A Story of a Failed XenServer Upgrade > >> > >> > >> See attached pdf document. This is the final procedure we adopted after > upgrading seven XenServer pools. > >> > >> Yiping > >> > >> > >> > >> > >> > >>> On 1/8/16, 2:20 AM, "Alessandro Caviglione" > wrote: > >>> > >>> Hi Yiping, > >>> yes, thank you very much!! > >>> Please share the doc so I can try again the upgrade process and see if > it > >>> was only a "unfortunate coincidence of events" or a wrong upgrade > process. > >>> > >>> Thanks! > >>> > >>>> On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: > >>>> > >>>> Yiping, > >>>> > >>>> Why not make a blog post about it so everyone can benefit? :) > >>>> > >>>> Lucian > >>>> > >>>> -- > >>>> Sent from the Delta quadrant using Borg technology! > >>>> > >>>> Nux! > >>>> www.nux.ro > >>>> > >>>> - Original Message - > >>>>> From: "Yiping Zhang" > >>>>> To: users@cloudstack.apache.org, aemne...@gmail.com > >>>>> Sent: Friday, 8 January, 2016 01:31:21 > >>>>> Subject: Re: A Story of a Failed XenServer Upgrade > >>>> > >>>>> Hi, Alessandro > >>>>> > >>>>> Late to the thread. Is this still an issue for you ? > >>>>> > >>>>> I went thru this process before and I have a step by step document > that > >>>> I can > >>>>> share if you still need it. > >>>>> > >>>>> Yiping > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>> On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: > >>>>>> > >>>>>> Hi Alessandro, > >>>>>> Without seeing the logs, or DB, it will be hard to diagnose the > issue. > >>>> I've > >>>>>> seen something similar in the past, where the XenServer host > version isnt > >>>>>> getting updated in the DB, as part of the XS upgrade process. That > caused > >>>>>> CloudStack to use the wrong hypervisor resource to try connecting > back to > >>>>>> the XenServers... ending up in failure. If you could share sanitized > >>>>>> versions of your log and db, someone here might be able to give you > the > >>>>>> necessary steps to get your cluster back under CloudStack control. > >>>>>> > >>>>>> On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < > >>>>>> c.alessan...@gmail.com> wrote: > >>>>>> > >>>>>>> No guys,as the article wrote, my first action was to put in > Maintenance > >>>>>>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > >>>> XenServer > >>>>>>> Pool Master first before any of the Slaves. To do so you need
Re: R: A Story of a Failed XenServer Upgrade
The mailing list strips attachments you'll need to post it on an external site. > On Jan 8, 2016, at 11:00 AM, Yiping Zhang wrote: > > Hm I definitely had included the attachment (as I can see from my Outlook’s > sent folder that the message has an attachment). > > I am wondering if the mailing list requires any special privilege to send > messages with attachment. Does anyone know ? I can forward the doc to > someone who has the power to resend it to the list if anyone volunteers to do > so. > > To answer Nux!’s suggestion of doing a blog, but I am old fashioned guy that > I have not tried that yet :) > > Yiping > > > > >> On 1/8/16, 10:35 AM, "Davide Pala" wrote: >> >> I think you've forget the attachment ... >> >> >> >> Inviato dal mio dispositivo Samsung >> >> >> Messaggio originale ---- >> Da: Yiping Zhang >> Data: 08/01/2016 18:44 (GMT+01:00) >> A: users@cloudstack.apache.org >> Oggetto: Re: A Story of a Failed XenServer Upgrade >> >> >> See attached pdf document. This is the final procedure we adopted after >> upgrading seven XenServer pools. >> >> Yiping >> >> >> >> >> >>> On 1/8/16, 2:20 AM, "Alessandro Caviglione" wrote: >>> >>> Hi Yiping, >>> yes, thank you very much!! >>> Please share the doc so I can try again the upgrade process and see if it >>> was only a "unfortunate coincidence of events" or a wrong upgrade process. >>> >>> Thanks! >>> >>>> On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: >>>> >>>> Yiping, >>>> >>>> Why not make a blog post about it so everyone can benefit? :) >>>> >>>> Lucian >>>> >>>> -- >>>> Sent from the Delta quadrant using Borg technology! >>>> >>>> Nux! >>>> www.nux.ro >>>> >>>> - Original Message - >>>>> From: "Yiping Zhang" >>>>> To: users@cloudstack.apache.org, aemne...@gmail.com >>>>> Sent: Friday, 8 January, 2016 01:31:21 >>>>> Subject: Re: A Story of a Failed XenServer Upgrade >>>> >>>>> Hi, Alessandro >>>>> >>>>> Late to the thread. Is this still an issue for you ? >>>>> >>>>> I went thru this process before and I have a step by step document that >>>> I can >>>>> share if you still need it. >>>>> >>>>> Yiping >>>>> >>>>> >>>>> >>>>> >>>>>> On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >>>>>> >>>>>> Hi Alessandro, >>>>>> Without seeing the logs, or DB, it will be hard to diagnose the issue. >>>> I've >>>>>> seen something similar in the past, where the XenServer host version isnt >>>>>> getting updated in the DB, as part of the XS upgrade process. That caused >>>>>> CloudStack to use the wrong hypervisor resource to try connecting back to >>>>>> the XenServers... ending up in failure. If you could share sanitized >>>>>> versions of your log and db, someone here might be able to give you the >>>>>> necessary steps to get your cluster back under CloudStack control. >>>>>> >>>>>> On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >>>>>> c.alessan...@gmail.com> wrote: >>>>>> >>>>>>> No guys,as the article wrote, my first action was to put in Maintenance >>>>>>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the >>>> XenServer >>>>>>> Pool Master first before any of the Slaves. To do so you need to >>>> empty the >>>>>>> Pool Master of all CloudStack VMs, and you do this by putting the Host >>>> into >>>>>>> Maintenance Mode within CloudStack to trigger a live migration of all >>>> VMs >>>>>>> to alternate Hosts" >>>>>>> >>>>>>> This is exactly what I've done and after the XS upgrade, no hosts was >>>> able >>>>>>> to communicate with CS and also with the upgraded host. >>>>>>> >>>>>>> Putting an host in Maint Mode within C
Re: R: A Story of a Failed XenServer Upgrade
Hm I definitely had included the attachment (as I can see from my Outlook’s sent folder that the message has an attachment). I am wondering if the mailing list requires any special privilege to send messages with attachment. Does anyone know ? I can forward the doc to someone who has the power to resend it to the list if anyone volunteers to do so. To answer Nux!’s suggestion of doing a blog, but I am old fashioned guy that I have not tried that yet :) Yiping On 1/8/16, 10:35 AM, "Davide Pala" wrote: >I think you've forget the attachment ... > > > >Inviato dal mio dispositivo Samsung > > > Messaggio originale >Da: Yiping Zhang >Data: 08/01/2016 18:44 (GMT+01:00) >A: users@cloudstack.apache.org >Oggetto: Re: A Story of a Failed XenServer Upgrade > > >See attached pdf document. This is the final procedure we adopted after >upgrading seven XenServer pools. > >Yiping > > > > > >On 1/8/16, 2:20 AM, "Alessandro Caviglione" wrote: > >>Hi Yiping, >>yes, thank you very much!! >>Please share the doc so I can try again the upgrade process and see if it >>was only a "unfortunate coincidence of events" or a wrong upgrade process. >> >>Thanks! >> >>On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: >> >>> Yiping, >>> >>> Why not make a blog post about it so everyone can benefit? :) >>> >>> Lucian >>> >>> -- >>> Sent from the Delta quadrant using Borg technology! >>> >>> Nux! >>> www.nux.ro >>> >>> - Original Message - >>> > From: "Yiping Zhang" >>> > To: users@cloudstack.apache.org, aemne...@gmail.com >>> > Sent: Friday, 8 January, 2016 01:31:21 >>> > Subject: Re: A Story of a Failed XenServer Upgrade >>> >>> > Hi, Alessandro >>> > >>> > Late to the thread. Is this still an issue for you ? >>> > >>> > I went thru this process before and I have a step by step document that >>> I can >>> > share if you still need it. >>> > >>> > Yiping >>> > >>> > >>> > >>> > >>> > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >>> > >>> >>Hi Alessandro, >>> >>Without seeing the logs, or DB, it will be hard to diagnose the issue. >>> I've >>> >>seen something similar in the past, where the XenServer host version isnt >>> >>getting updated in the DB, as part of the XS upgrade process. That caused >>> >>CloudStack to use the wrong hypervisor resource to try connecting back to >>> >>the XenServers... ending up in failure. If you could share sanitized >>> >>versions of your log and db, someone here might be able to give you the >>> >>necessary steps to get your cluster back under CloudStack control. >>> >> >>> >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >>> >>c.alessan...@gmail.com> wrote: >>> >> >>> >>> No guys,as the article wrote, my first action was to put in Maintenance >>> >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the >>> XenServer >>> >>> Pool Master first before any of the Slaves. To do so you need to >>> empty the >>> >>> Pool Master of all CloudStack VMs, and you do this by putting the Host >>> into >>> >>> Maintenance Mode within CloudStack to trigger a live migration of all >>> VMs >>> >>> to alternate Hosts" >>> >>> >>> >>> This is exactly what I've done and after the XS upgrade, no hosts was >>> able >>> >>> to communicate with CS and also with the upgraded host. >>> >>> >>> >>> Putting an host in Maint Mode within CS will trigger MM also on >>> XenServer >>> >>> host or just will move the VMs to other hosts? >>> >>> >>> >>> And again what's the best practices to upgrade a XS cluster? >>> >>> >>> >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < >>> rberg...@schubergphilis.com> >>> >>> wrote: >>> >>> >>> >>> > CloudStack should always do the migration of VM's not the Hypervisor. >>> >>> > >>> >>> > That's not true.
R: A Story of a Failed XenServer Upgrade
I think you've forget the attachment ... Inviato dal mio dispositivo Samsung Messaggio originale Da: Yiping Zhang Data: 08/01/2016 18:44 (GMT+01:00) A: users@cloudstack.apache.org Oggetto: Re: A Story of a Failed XenServer Upgrade See attached pdf document. This is the final procedure we adopted after upgrading seven XenServer pools. Yiping On 1/8/16, 2:20 AM, "Alessandro Caviglione" wrote: >Hi Yiping, >yes, thank you very much!! >Please share the doc so I can try again the upgrade process and see if it >was only a "unfortunate coincidence of events" or a wrong upgrade process. > >Thanks! > >On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: > >> Yiping, >> >> Why not make a blog post about it so everyone can benefit? :) >> >> Lucian >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> - Original Message - >> > From: "Yiping Zhang" >> > To: users@cloudstack.apache.org, aemne...@gmail.com >> > Sent: Friday, 8 January, 2016 01:31:21 >> > Subject: Re: A Story of a Failed XenServer Upgrade >> >> > Hi, Alessandro >> > >> > Late to the thread. Is this still an issue for you ? >> > >> > I went thru this process before and I have a step by step document that >> I can >> > share if you still need it. >> > >> > Yiping >> > >> > >> > >> > >> > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >> > >> >>Hi Alessandro, >> >>Without seeing the logs, or DB, it will be hard to diagnose the issue. >> I've >> >>seen something similar in the past, where the XenServer host version isnt >> >>getting updated in the DB, as part of the XS upgrade process. That caused >> >>CloudStack to use the wrong hypervisor resource to try connecting back to >> >>the XenServers... ending up in failure. If you could share sanitized >> >>versions of your log and db, someone here might be able to give you the >> >>necessary steps to get your cluster back under CloudStack control. >> >> >> >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >> >>c.alessan...@gmail.com> wrote: >> >> >> >>> No guys,as the article wrote, my first action was to put in Maintenance >> >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the >> XenServer >> >>> Pool Master first before any of the Slaves. To do so you need to >> empty the >> >>> Pool Master of all CloudStack VMs, and you do this by putting the Host >> into >> >>> Maintenance Mode within CloudStack to trigger a live migration of all >> VMs >> >>> to alternate Hosts" >> >>> >> >>> This is exactly what I've done and after the XS upgrade, no hosts was >> able >> >>> to communicate with CS and also with the upgraded host. >> >>> >> >>> Putting an host in Maint Mode within CS will trigger MM also on >> XenServer >> >>> host or just will move the VMs to other hosts? >> >>> >> >>> And again what's the best practices to upgrade a XS cluster? >> >>> >> >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < >> rberg...@schubergphilis.com> >> >>> wrote: >> >>> >> >>> > CloudStack should always do the migration of VM's not the Hypervisor. >> >>> > >> >>> > That's not true. You can safely migrate outside of CloudStack as the >> >>> power >> >>> > report will tell CloudStack where the vms live and the db gets >> updated >> >>> > accordingly. I do this a lot while patching and that works fine on >> 6.2 >> >>> and >> >>> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. >> >>> > >> >>> > Regards, Remi >> >>> > >> >>> > >> >>> > Sent from my iPhone >> >>> > >> >>> > On 02 Jan 2016, at 16:26, Jeremy Peterson > > >>> > jpeter...@acentek.net>> wrote: >> >>> > >> >>> > I don't use XenServer maintenance mode until after CloudStack has >> put the >> >>> > Host in maintenance mode. >> >>> > >> >>> > When you initiate maintenan
Re: A Story of a Failed XenServer Upgrade
See attached pdf document. This is the final procedure we adopted after upgrading seven XenServer pools. Yiping On 1/8/16, 2:20 AM, "Alessandro Caviglione" wrote: >Hi Yiping, >yes, thank you very much!! >Please share the doc so I can try again the upgrade process and see if it >was only a "unfortunate coincidence of events" or a wrong upgrade process. > >Thanks! > >On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: > >> Yiping, >> >> Why not make a blog post about it so everyone can benefit? :) >> >> Lucian >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> - Original Message - >> > From: "Yiping Zhang" >> > To: users@cloudstack.apache.org, aemne...@gmail.com >> > Sent: Friday, 8 January, 2016 01:31:21 >> > Subject: Re: A Story of a Failed XenServer Upgrade >> >> > Hi, Alessandro >> > >> > Late to the thread. Is this still an issue for you ? >> > >> > I went thru this process before and I have a step by step document that >> I can >> > share if you still need it. >> > >> > Yiping >> > >> > >> > >> > >> > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >> > >> >>Hi Alessandro, >> >>Without seeing the logs, or DB, it will be hard to diagnose the issue. >> I've >> >>seen something similar in the past, where the XenServer host version isnt >> >>getting updated in the DB, as part of the XS upgrade process. That caused >> >>CloudStack to use the wrong hypervisor resource to try connecting back to >> >>the XenServers... ending up in failure. If you could share sanitized >> >>versions of your log and db, someone here might be able to give you the >> >>necessary steps to get your cluster back under CloudStack control. >> >> >> >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >> >>c.alessan...@gmail.com> wrote: >> >> >> >>> No guys,as the article wrote, my first action was to put in Maintenance >> >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the >> XenServer >> >>> Pool Master first before any of the Slaves. To do so you need to >> empty the >> >>> Pool Master of all CloudStack VMs, and you do this by putting the Host >> into >> >>> Maintenance Mode within CloudStack to trigger a live migration of all >> VMs >> >>> to alternate Hosts" >> >>> >> >>> This is exactly what I've done and after the XS upgrade, no hosts was >> able >> >>> to communicate with CS and also with the upgraded host. >> >>> >> >>> Putting an host in Maint Mode within CS will trigger MM also on >> XenServer >> >>> host or just will move the VMs to other hosts? >> >>> >> >>> And again what's the best practices to upgrade a XS cluster? >> >>> >> >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < >> rberg...@schubergphilis.com> >> >>> wrote: >> >>> >> >>> > CloudStack should always do the migration of VM's not the Hypervisor. >> >>> > >> >>> > That's not true. You can safely migrate outside of CloudStack as the >> >>> power >> >>> > report will tell CloudStack where the vms live and the db gets >> updated >> >>> > accordingly. I do this a lot while patching and that works fine on >> 6.2 >> >>> and >> >>> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. >> >>> > >> >>> > Regards, Remi >> >>> > >> >>> > >> >>> > Sent from my iPhone >> >>> > >> >>> > On 02 Jan 2016, at 16:26, Jeremy Peterson > > >>> > jpeter...@acentek.net>> wrote: >> >>> > >> >>> > I don't use XenServer maintenance mode until after CloudStack has >> put the >> >>> > Host in maintenance mode. >> >>> > >> >>> > When you initiate maintenance mode from the host rather than >> CloudStack >> >>> > the db does not know where the VM's are and your UUID's get jacked. >> >>> > >> >>> > CS is your brains not the hypervisor. >&g
Re: A Story of a Failed XenServer Upgrade
Hi Alessandro, Sorry to step in late, did you follow current upgrade instruction [1] ? I think we still have to recopy 4 scripts from the cloudstack-management server to xenserver recently upgraded. /opt/xensource/sm/NFSSR.py /opt/xensource/bin/setupxenserver.sh /opt/xensource/bin/make_migratable.sh /opt/xensource/bin/cloud-clean-vlan.sh I don't see any wrong steps in your process, execpt the copy of the 4 files. Since you upgraded from 6.2 to 6.5 , I'm wondering if iptables from dom0 would have been changed and CloudStack would have lost connectivity to the freshly upgraded xenserver ? Also, once the pool-master upgraded, did you perform a "Force Reconnect" in cloudstack and look into the management-server.log to see what's wrong? I agree with you Davide, placing a node in maintenance mode in CloudStack must not place the xenserver in maintenance in xenserver pool because it could trigger a pool-master change which is not wanted during a maintenance such as applying hotfixes. [1] http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.6/hypervisor/xenserver.html#upgrading-xenserver-versions Regards, On Fri, Jan 8, 2016 at 5:20 AM, Alessandro Caviglione < c.alessan...@gmail.com> wrote: > Hi Yiping, > yes, thank you very much!! > Please share the doc so I can try again the upgrade process and see if it > was only a "unfortunate coincidence of events" or a wrong upgrade process. > > Thanks! > > On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: > > > Yiping, > > > > Why not make a blog post about it so everyone can benefit? :) > > > > Lucian > > > > -- > > Sent from the Delta quadrant using Borg technology! > > > > Nux! > > www.nux.ro > > > > - Original Message - > > > From: "Yiping Zhang" > > > To: users@cloudstack.apache.org, aemne...@gmail.com > > > Sent: Friday, 8 January, 2016 01:31:21 > > > Subject: Re: A Story of a Failed XenServer Upgrade > > > > > Hi, Alessandro > > > > > > Late to the thread. Is this still an issue for you ? > > > > > > I went thru this process before and I have a step by step document that > > I can > > > share if you still need it. > > > > > > Yiping > > > > > > > > > > > > > > > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: > > > > > >>Hi Alessandro, > > >>Without seeing the logs, or DB, it will be hard to diagnose the issue. > > I've > > >>seen something similar in the past, where the XenServer host version > isnt > > >>getting updated in the DB, as part of the XS upgrade process. That > caused > > >>CloudStack to use the wrong hypervisor resource to try connecting back > to > > >>the XenServers... ending up in failure. If you could share sanitized > > >>versions of your log and db, someone here might be able to give you the > > >>necessary steps to get your cluster back under CloudStack control. > > >> > > >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < > > >>c.alessan...@gmail.com> wrote: > > >> > > >>> No guys,as the article wrote, my first action was to put in > Maintenance > > >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > > XenServer > > >>> Pool Master first before any of the Slaves. To do so you need to > > empty the > > >>> Pool Master of all CloudStack VMs, and you do this by putting the > Host > > into > > >>> Maintenance Mode within CloudStack to trigger a live migration of all > > VMs > > >>> to alternate Hosts" > > >>> > > >>> This is exactly what I've done and after the XS upgrade, no hosts was > > able > > >>> to communicate with CS and also with the upgraded host. > > >>> > > >>> Putting an host in Maint Mode within CS will trigger MM also on > > XenServer > > >>> host or just will move the VMs to other hosts? > > >>> > > >>> And again what's the best practices to upgrade a XS cluster? > > >>> > > >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < > > rberg...@schubergphilis.com> > > >>> wrote: > > >>> > > >>> > CloudStack should always do the migration of VM's not the > Hypervisor. > > >>> > > > >>> > That's not true. You can safely migrate outside of CloudStack as > the > > &
Re: A Story of a Failed XenServer Upgrade
Hi Yiping, yes, thank you very much!! Please share the doc so I can try again the upgrade process and see if it was only a "unfortunate coincidence of events" or a wrong upgrade process. Thanks! On Fri, Jan 8, 2016 at 10:20 AM, Nux! wrote: > Yiping, > > Why not make a blog post about it so everyone can benefit? :) > > Lucian > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > - Original Message - > > From: "Yiping Zhang" > > To: users@cloudstack.apache.org, aemne...@gmail.com > > Sent: Friday, 8 January, 2016 01:31:21 > > Subject: Re: A Story of a Failed XenServer Upgrade > > > Hi, Alessandro > > > > Late to the thread. Is this still an issue for you ? > > > > I went thru this process before and I have a step by step document that > I can > > share if you still need it. > > > > Yiping > > > > > > > > > > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: > > > >>Hi Alessandro, > >>Without seeing the logs, or DB, it will be hard to diagnose the issue. > I've > >>seen something similar in the past, where the XenServer host version isnt > >>getting updated in the DB, as part of the XS upgrade process. That caused > >>CloudStack to use the wrong hypervisor resource to try connecting back to > >>the XenServers... ending up in failure. If you could share sanitized > >>versions of your log and db, someone here might be able to give you the > >>necessary steps to get your cluster back under CloudStack control. > >> > >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < > >>c.alessan...@gmail.com> wrote: > >> > >>> No guys,as the article wrote, my first action was to put in Maintenance > >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > XenServer > >>> Pool Master first before any of the Slaves. To do so you need to > empty the > >>> Pool Master of all CloudStack VMs, and you do this by putting the Host > into > >>> Maintenance Mode within CloudStack to trigger a live migration of all > VMs > >>> to alternate Hosts" > >>> > >>> This is exactly what I've done and after the XS upgrade, no hosts was > able > >>> to communicate with CS and also with the upgraded host. > >>> > >>> Putting an host in Maint Mode within CS will trigger MM also on > XenServer > >>> host or just will move the VMs to other hosts? > >>> > >>> And again what's the best practices to upgrade a XS cluster? > >>> > >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < > rberg...@schubergphilis.com> > >>> wrote: > >>> > >>> > CloudStack should always do the migration of VM's not the Hypervisor. > >>> > > >>> > That's not true. You can safely migrate outside of CloudStack as the > >>> power > >>> > report will tell CloudStack where the vms live and the db gets > updated > >>> > accordingly. I do this a lot while patching and that works fine on > 6.2 > >>> and > >>> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > >>> > > >>> > Regards, Remi > >>> > > >>> > > >>> > Sent from my iPhone > >>> > > >>> > On 02 Jan 2016, at 16:26, Jeremy Peterson >>> > jpeter...@acentek.net>> wrote: > >>> > > >>> > I don't use XenServer maintenance mode until after CloudStack has > put the > >>> > Host in maintenance mode. > >>> > > >>> > When you initiate maintenance mode from the host rather than > CloudStack > >>> > the db does not know where the VM's are and your UUID's get jacked. > >>> > > >>> > CS is your brains not the hypervisor. > >>> > > >>> > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > >>> > Upgrade. Reboot. Join Pool. Remove Maintenance starting at > hypervisor > >>> if > >>> > needed and then CS and move on to the next Host. > >>> > > >>> > CloudStack should always do the migration of VM's not the Hypervisor. > >>> > > >>> > Jeremy > >>> > > >>> > > >>> > -Original Message- > >>> > From: Da
Re: A Story of a Failed XenServer Upgrade
Yiping, Why not make a blog post about it so everyone can benefit? :) Lucian -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro - Original Message - > From: "Yiping Zhang" > To: users@cloudstack.apache.org, aemne...@gmail.com > Sent: Friday, 8 January, 2016 01:31:21 > Subject: Re: A Story of a Failed XenServer Upgrade > Hi, Alessandro > > Late to the thread. Is this still an issue for you ? > > I went thru this process before and I have a step by step document that I can > share if you still need it. > > Yiping > > > > > On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: > >>Hi Alessandro, >>Without seeing the logs, or DB, it will be hard to diagnose the issue. I've >>seen something similar in the past, where the XenServer host version isnt >>getting updated in the DB, as part of the XS upgrade process. That caused >>CloudStack to use the wrong hypervisor resource to try connecting back to >>the XenServers... ending up in failure. If you could share sanitized >>versions of your log and db, someone here might be able to give you the >>necessary steps to get your cluster back under CloudStack control. >> >>On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >>c.alessan...@gmail.com> wrote: >> >>> No guys,as the article wrote, my first action was to put in Maintenance >>> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer >>> Pool Master first before any of the Slaves. To do so you need to empty the >>> Pool Master of all CloudStack VMs, and you do this by putting the Host into >>> Maintenance Mode within CloudStack to trigger a live migration of all VMs >>> to alternate Hosts" >>> >>> This is exactly what I've done and after the XS upgrade, no hosts was able >>> to communicate with CS and also with the upgraded host. >>> >>> Putting an host in Maint Mode within CS will trigger MM also on XenServer >>> host or just will move the VMs to other hosts? >>> >>> And again what's the best practices to upgrade a XS cluster? >>> >>> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma >>> wrote: >>> >>> > CloudStack should always do the migration of VM's not the Hypervisor. >>> > >>> > That's not true. You can safely migrate outside of CloudStack as the >>> power >>> > report will tell CloudStack where the vms live and the db gets updated >>> > accordingly. I do this a lot while patching and that works fine on 6.2 >>> and >>> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. >>> > >>> > Regards, Remi >>> > >>> > >>> > Sent from my iPhone >>> > >>> > On 02 Jan 2016, at 16:26, Jeremy Peterson >> > jpeter...@acentek.net>> wrote: >>> > >>> > I don't use XenServer maintenance mode until after CloudStack has put the >>> > Host in maintenance mode. >>> > >>> > When you initiate maintenance mode from the host rather than CloudStack >>> > the db does not know where the VM's are and your UUID's get jacked. >>> > >>> > CS is your brains not the hypervisor. >>> > >>> > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. >>> > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor >>> if >>> > needed and then CS and move on to the next Host. >>> > >>> > CloudStack should always do the migration of VM's not the Hypervisor. >>> > >>> > Jeremy >>> > >>> > >>> > -Original Message- >>> > From: Davide Pala [mailto:davide.p...@gesca.it] >>> > Sent: Friday, January 1, 2016 5:18 PM >>> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >>> > Subject: R: A Story of a Failed XenServer Upgrade >>> > >>> > Hi alessandro. If u put in maintenance mode the master you force the >>> > election of a new pool master. Now when you have see the upgraded host as >>> > disconnected you are connected to the new pool master and the host (as a >>> > pool member) cannot comunicate with a pool master of an earliest version. >>> > The solution? Launche the upgrade on the pool master without enter in >>> > maintenance mode. And remember a consistent backup!!! >>> > >>> > >>> >
R: A Story of a Failed XenServer Upgrade
Hi Yiping. I'll do this early. If you can share this document i'll appreciate. Tnx Inviato dal mio dispositivo Samsung Messaggio originale Da: Yiping Zhang Data: 08/01/2016 02:32 (GMT+01:00) A: users@cloudstack.apache.org, aemne...@gmail.com Oggetto: Re: A Story of a Failed XenServer Upgrade Hi, Alessandro Late to the thread. Is this still an issue for you ? I went thru this process before and I have a step by step document that I can share if you still need it. Yiping On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >Hi Alessandro, >Without seeing the logs, or DB, it will be hard to diagnose the issue. I've >seen something similar in the past, where the XenServer host version isnt >getting updated in the DB, as part of the XS upgrade process. That caused >CloudStack to use the wrong hypervisor resource to try connecting back to >the XenServers... ending up in failure. If you could share sanitized >versions of your log and db, someone here might be able to give you the >necessary steps to get your cluster back under CloudStack control. > >On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >c.alessan...@gmail.com> wrote: > >> No guys,as the article wrote, my first action was to put in Maintenance >> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer >> Pool Master first before any of the Slaves. To do so you need to empty the >> Pool Master of all CloudStack VMs, and you do this by putting the Host into >> Maintenance Mode within CloudStack to trigger a live migration of all VMs >> to alternate Hosts" >> >> This is exactly what I've done and after the XS upgrade, no hosts was able >> to communicate with CS and also with the upgraded host. >> >> Putting an host in Maint Mode within CS will trigger MM also on XenServer >> host or just will move the VMs to other hosts? >> >> And again what's the best practices to upgrade a XS cluster? >> >> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma >> wrote: >> >> > CloudStack should always do the migration of VM's not the Hypervisor. >> > >> > That's not true. You can safely migrate outside of CloudStack as the >> power >> > report will tell CloudStack where the vms live and the db gets updated >> > accordingly. I do this a lot while patching and that works fine on 6.2 >> and >> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. >> > >> > Regards, Remi >> > >> > >> > Sent from my iPhone >> > >> > On 02 Jan 2016, at 16:26, Jeremy Peterson > > jpeter...@acentek.net>> wrote: >> > >> > I don't use XenServer maintenance mode until after CloudStack has put the >> > Host in maintenance mode. >> > >> > When you initiate maintenance mode from the host rather than CloudStack >> > the db does not know where the VM's are and your UUID's get jacked. >> > >> > CS is your brains not the hypervisor. >> > >> > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. >> > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor >> if >> > needed and then CS and move on to the next Host. >> > >> > CloudStack should always do the migration of VM's not the Hypervisor. >> > >> > Jeremy >> > >> > >> > -Original Message- >> > From: Davide Pala [mailto:davide.p...@gesca.it] >> > Sent: Friday, January 1, 2016 5:18 PM >> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >> > Subject: R: A Story of a Failed XenServer Upgrade >> > >> > Hi alessandro. If u put in maintenance mode the master you force the >> > election of a new pool master. Now when you have see the upgraded host as >> > disconnected you are connected to the new pool master and the host (as a >> > pool member) cannot comunicate with a pool master of an earliest version. >> > The solution? Launche the upgrade on the pool master without enter in >> > maintenance mode. And remember a consistent backup!!! >> > >> > >> > >> > Inviato dal mio dispositivo Samsung >> > >> > >> > Messaggio originale >> > Da: Alessandro Caviglione > > c.alessan...@gmail.com>> >> > Data: 01/01/2016 23:23 (GMT+01:00) >> > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >> > Oggetto: A Story of a Failed XenServer Upgrade >> > >> > Hi gu
Re: A Story of a Failed XenServer Upgrade
Hi, Alessandro Late to the thread. Is this still an issue for you ? I went thru this process before and I have a step by step document that I can share if you still need it. Yiping On 1/2/16, 4:43 PM, "Ahmad Emneina" wrote: >Hi Alessandro, >Without seeing the logs, or DB, it will be hard to diagnose the issue. I've >seen something similar in the past, where the XenServer host version isnt >getting updated in the DB, as part of the XS upgrade process. That caused >CloudStack to use the wrong hypervisor resource to try connecting back to >the XenServers... ending up in failure. If you could share sanitized >versions of your log and db, someone here might be able to give you the >necessary steps to get your cluster back under CloudStack control. > >On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < >c.alessan...@gmail.com> wrote: > >> No guys,as the article wrote, my first action was to put in Maintenance >> Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer >> Pool Master first before any of the Slaves. To do so you need to empty the >> Pool Master of all CloudStack VMs, and you do this by putting the Host into >> Maintenance Mode within CloudStack to trigger a live migration of all VMs >> to alternate Hosts" >> >> This is exactly what I've done and after the XS upgrade, no hosts was able >> to communicate with CS and also with the upgraded host. >> >> Putting an host in Maint Mode within CS will trigger MM also on XenServer >> host or just will move the VMs to other hosts? >> >> And again what's the best practices to upgrade a XS cluster? >> >> On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma >> wrote: >> >> > CloudStack should always do the migration of VM's not the Hypervisor. >> > >> > That's not true. You can safely migrate outside of CloudStack as the >> power >> > report will tell CloudStack where the vms live and the db gets updated >> > accordingly. I do this a lot while patching and that works fine on 6.2 >> and >> > 6.5. I use both CloudStack 4.4.4 and 4.7.0. >> > >> > Regards, Remi >> > >> > >> > Sent from my iPhone >> > >> > On 02 Jan 2016, at 16:26, Jeremy Peterson > > jpeter...@acentek.net>> wrote: >> > >> > I don't use XenServer maintenance mode until after CloudStack has put the >> > Host in maintenance mode. >> > >> > When you initiate maintenance mode from the host rather than CloudStack >> > the db does not know where the VM's are and your UUID's get jacked. >> > >> > CS is your brains not the hypervisor. >> > >> > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. >> > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor >> if >> > needed and then CS and move on to the next Host. >> > >> > CloudStack should always do the migration of VM's not the Hypervisor. >> > >> > Jeremy >> > >> > >> > -Original Message- >> > From: Davide Pala [mailto:davide.p...@gesca.it] >> > Sent: Friday, January 1, 2016 5:18 PM >> > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >> > Subject: R: A Story of a Failed XenServer Upgrade >> > >> > Hi alessandro. If u put in maintenance mode the master you force the >> > election of a new pool master. Now when you have see the upgraded host as >> > disconnected you are connected to the new pool master and the host (as a >> > pool member) cannot comunicate with a pool master of an earliest version. >> > The solution? Launche the upgrade on the pool master without enter in >> > maintenance mode. And remember a consistent backup!!! >> > >> > >> > >> > Inviato dal mio dispositivo Samsung >> > >> > >> > Messaggio originale >> > Da: Alessandro Caviglione > > c.alessan...@gmail.com>> >> > Data: 01/01/2016 23:23 (GMT+01:00) >> > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >> > Oggetto: A Story of a Failed XenServer Upgrade >> > >> > Hi guys, >> > I want to share my XenServer Upgrade adventure to understand if I did >> > domething wrong. >> > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs >> > has been upgraded I start the upgrade process of my XenServer hosts from >> > 6.2 to 6.5. >> >
Re: A Story of a Failed XenServer Upgrade
Hi Alessandro, Without seeing the logs, or DB, it will be hard to diagnose the issue. I've seen something similar in the past, where the XenServer host version isnt getting updated in the DB, as part of the XS upgrade process. That caused CloudStack to use the wrong hypervisor resource to try connecting back to the XenServers... ending up in failure. If you could share sanitized versions of your log and db, someone here might be able to give you the necessary steps to get your cluster back under CloudStack control. On Sat, Jan 2, 2016 at 1:27 PM, Alessandro Caviglione < c.alessan...@gmail.com> wrote: > No guys,as the article wrote, my first action was to put in Maintenance > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer > Pool Master first before any of the Slaves. To do so you need to empty the > Pool Master of all CloudStack VMs, and you do this by putting the Host into > Maintenance Mode within CloudStack to trigger a live migration of all VMs > to alternate Hosts" > > This is exactly what I've done and after the XS upgrade, no hosts was able > to communicate with CS and also with the upgraded host. > > Putting an host in Maint Mode within CS will trigger MM also on XenServer > host or just will move the VMs to other hosts? > > And again what's the best practices to upgrade a XS cluster? > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma > wrote: > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > That's not true. You can safely migrate outside of CloudStack as the > power > > report will tell CloudStack where the vms live and the db gets updated > > accordingly. I do this a lot while patching and that works fine on 6.2 > and > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > Regards, Remi > > > > > > Sent from my iPhone > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > jpeter...@acentek.net>> wrote: > > > > I don't use XenServer maintenance mode until after CloudStack has put the > > Host in maintenance mode. > > > > When you initiate maintenance mode from the host rather than CloudStack > > the db does not know where the VM's are and your UUID's get jacked. > > > > CS is your brains not the hypervisor. > > > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor > if > > needed and then CS and move on to the next Host. > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > Jeremy > > > > > > -Original Message- > > From: Davide Pala [mailto:davide.p...@gesca.it] > > Sent: Friday, January 1, 2016 5:18 PM > > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Subject: R: A Story of a Failed XenServer Upgrade > > > > Hi alessandro. If u put in maintenance mode the master you force the > > election of a new pool master. Now when you have see the upgraded host as > > disconnected you are connected to the new pool master and the host (as a > > pool member) cannot comunicate with a pool master of an earliest version. > > The solution? Launche the upgrade on the pool master without enter in > > maintenance mode. And remember a consistent backup!!! > > > > > > > > Inviato dal mio dispositivo Samsung > > > > > > Messaggio originale > > Da: Alessandro Caviglione > c.alessan...@gmail.com>> > > Data: 01/01/2016 23:23 (GMT+01:00) > > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Oggetto: A Story of a Failed XenServer Upgrade > > > > Hi guys, > > I want to share my XenServer Upgrade adventure to understand if I did > > domething wrong. > > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs > > has been upgraded I start the upgrade process of my XenServer hosts from > > 6.2 to 6.5. > > I do not already have PoolHA enabled so I followed this article: > > > > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ > > > > The cluster consists of n° 3 XenServer hosts. > > > > First of all I added manage.xenserver.pool.master=false > > to environment.properties file and restarted cloudstack-management > service. > > > > After that I put in Maintenance Mode Pool Master host and, after all VMs > > has been migrated, I Unmanaged the cluster. > > At this point all host appears as "Disconnected" from
Re: A Story of a Failed XenServer Upgrade
until we use "xe pool-designate-new-master", the master of the pool will not change. Unless, you have configured the HA features in your XenServer Pool/cluster, the reboot of the master may trigger a change of master server. Bottom line: today, the code does not change the master server of a XenServer pool. The maintenance process of ACS does not change the master of the cluster/XenServer pool On Sat, Jan 2, 2016 at 9:36 PM, Davide Pala wrote: > No. The upgrade must be done with a cold reboot without the maintenance. > XS pool master mist be the master again when it boot > > > > Inviato dal mio dispositivo Samsung > > > Messaggio originale > Da: Rafael Weingärtner > Data: 03/01/2016 00:25 (GMT+01:00) > A: users@cloudstack.apache.org > Oggetto: Re: A Story of a Failed XenServer Upgrade > > That is true it is not putting the host in maintenance. Not just the > master, but any XenServer host. > > The question is, should it? If so, we should open a Jira ticket. > > On Sat, Jan 2, 2016 at 9:18 PM, Davide Pala wrote: > > > So i dont know anything about cloud on xenserver and for this reason i > > think the cloudstack dont put in maintenance xenserver pool master > > > > > > > > Inviato dal mio dispositivo Samsung > > > > > > Messaggio originale ---- > > Da: Rafael Weingärtner > > Data: 02/01/2016 22:48 (GMT+01:00) > > A: users@cloudstack.apache.org > > Oggetto: Re: A Story of a Failed XenServer Upgrade > > > > Hi all, > > > > There is nothing better than looking at the source code. > > > > After the VM migration (or restart for LCX ?!), when the host is put in > > maintenance mode, for Xenserver it will remove a tag called “cloud”. > > > > On Sat, Jan 2, 2016 at 7:38 PM, Davide Pala > wrote: > > > > > i don't know what do the maintenance mode in CS but if it put in > > > maintenance also the pool master this article is wrong! > > > > > > > > > Davide Pala > > > Infrastructure Specialist > > > Gesca srl > > > > > > Via degli Olmetti, 18 > > > 00060 Formello (Roma) > > > Office: +39 06 9040661 > > > Fax: +39 06 90406666 > > > E-mail: davide.p...@gesca.it > > > Web:www.gesca.it<http://www.gesca.it> > > > > > > > > > > > > > > > > > > Da: Alessandro Caviglione [c.alessan...@gmail.com] > > > Inviato: sabato 2 gennaio 2016 22.27 > > > A: users@cloudstack.apache.org > > > Oggetto: Re: A Story of a Failed XenServer Upgrade > > > > > > No guys,as the article wrote, my first action was to put in Maintenance > > > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > > XenServer > > > Pool Master first before any of the Slaves. To do so you need to empty > > the > > > Pool Master of all CloudStack VMs, and you do this by putting the Host > > into > > > Maintenance Mode within CloudStack to trigger a live migration of all > VMs > > > to alternate Hosts" > > > > > > This is exactly what I've done and after the XS upgrade, no hosts was > > able > > > to communicate with CS and also with the upgraded host. > > > > > > Putting an host in Maint Mode within CS will trigger MM also on > XenServer > > > host or just will move the VMs to other hosts? > > > > > > And again what's the best practices to upgrade a XS cluster? > > > > > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < > > rberg...@schubergphilis.com> > > > wrote: > > > > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > > > > > That's not true. You can safely migrate outside of CloudStack as the > > > power > > > > report will tell CloudStack where the vms live and the db gets > updated > > > > accordingly. I do this a lot while patching and that works fine on > 6.2 > > > and > > > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > > > > > Regards, Remi > > > > > > > > > > > > Sent from my iPhone > > > > > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > > > > jpeter...@acentek.net>> wrote: > > > > > > > > I don't use XenServer maintenance mode until after CloudStack has put > > the > > > > Host in maintenance mode. >
R: A Story of a Failed XenServer Upgrade
No. The upgrade must be done with a cold reboot without the maintenance. XS pool master mist be the master again when it boot Inviato dal mio dispositivo Samsung Messaggio originale Da: Rafael Weingärtner Data: 03/01/2016 00:25 (GMT+01:00) A: users@cloudstack.apache.org Oggetto: Re: A Story of a Failed XenServer Upgrade That is true it is not putting the host in maintenance. Not just the master, but any XenServer host. The question is, should it? If so, we should open a Jira ticket. On Sat, Jan 2, 2016 at 9:18 PM, Davide Pala wrote: > So i dont know anything about cloud on xenserver and for this reason i > think the cloudstack dont put in maintenance xenserver pool master > > > > Inviato dal mio dispositivo Samsung > > > Messaggio originale > Da: Rafael Weingärtner > Data: 02/01/2016 22:48 (GMT+01:00) > A: users@cloudstack.apache.org > Oggetto: Re: A Story of a Failed XenServer Upgrade > > Hi all, > > There is nothing better than looking at the source code. > > After the VM migration (or restart for LCX ?!), when the host is put in > maintenance mode, for Xenserver it will remove a tag called “cloud”. > > On Sat, Jan 2, 2016 at 7:38 PM, Davide Pala wrote: > > > i don't know what do the maintenance mode in CS but if it put in > > maintenance also the pool master this article is wrong! > > > > > > Davide Pala > > Infrastructure Specialist > > Gesca srl > > > > Via degli Olmetti, 18 > > 00060 Formello (Roma) > > Office: +39 06 9040661 > > Fax: +39 06 9040 > > E-mail: davide.p...@gesca.it > > Web:www.gesca.it<http://www.gesca.it> > > > > > > > > > > ________________ > > Da: Alessandro Caviglione [c.alessan...@gmail.com] > > Inviato: sabato 2 gennaio 2016 22.27 > > A: users@cloudstack.apache.org > > Oggetto: Re: A Story of a Failed XenServer Upgrade > > > > No guys,as the article wrote, my first action was to put in Maintenance > > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > XenServer > > Pool Master first before any of the Slaves. To do so you need to empty > the > > Pool Master of all CloudStack VMs, and you do this by putting the Host > into > > Maintenance Mode within CloudStack to trigger a live migration of all VMs > > to alternate Hosts" > > > > This is exactly what I've done and after the XS upgrade, no hosts was > able > > to communicate with CS and also with the upgraded host. > > > > Putting an host in Maint Mode within CS will trigger MM also on XenServer > > host or just will move the VMs to other hosts? > > > > And again what's the best practices to upgrade a XS cluster? > > > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < > rberg...@schubergphilis.com> > > wrote: > > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > > > That's not true. You can safely migrate outside of CloudStack as the > > power > > > report will tell CloudStack where the vms live and the db gets updated > > > accordingly. I do this a lot while patching and that works fine on 6.2 > > and > > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > > > Regards, Remi > > > > > > > > > Sent from my iPhone > > > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > > jpeter...@acentek.net>> wrote: > > > > > > I don't use XenServer maintenance mode until after CloudStack has put > the > > > Host in maintenance mode. > > > > > > When you initiate maintenance mode from the host rather than CloudStack > > > the db does not know where the VM's are and your UUID's get jacked. > > > > > > CS is your brains not the hypervisor. > > > > > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > > > Upgrade. Reboot. Join Pool. Remove Maintenance starting at > hypervisor > > if > > > needed and then CS and move on to the next Host. > > > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > > > Jeremy > > > > > > > > > -Original Message- > > > From: Davide Pala [mailto:davide.p...@gesca.it] > > > Sent: Friday, January 1, 2016 5:18 PM > > > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > > Subject: R: A Story of a Failed XenServer Upgrade > > >
Re: A Story of a Failed XenServer Upgrade
That is true it is not putting the host in maintenance. Not just the master, but any XenServer host. The question is, should it? If so, we should open a Jira ticket. On Sat, Jan 2, 2016 at 9:18 PM, Davide Pala wrote: > So i dont know anything about cloud on xenserver and for this reason i > think the cloudstack dont put in maintenance xenserver pool master > > > > Inviato dal mio dispositivo Samsung > > > Messaggio originale > Da: Rafael Weingärtner > Data: 02/01/2016 22:48 (GMT+01:00) > A: users@cloudstack.apache.org > Oggetto: Re: A Story of a Failed XenServer Upgrade > > Hi all, > > There is nothing better than looking at the source code. > > After the VM migration (or restart for LCX ?!), when the host is put in > maintenance mode, for Xenserver it will remove a tag called “cloud”. > > On Sat, Jan 2, 2016 at 7:38 PM, Davide Pala wrote: > > > i don't know what do the maintenance mode in CS but if it put in > > maintenance also the pool master this article is wrong! > > > > > > Davide Pala > > Infrastructure Specialist > > Gesca srl > > > > Via degli Olmetti, 18 > > 00060 Formello (Roma) > > Office: +39 06 9040661 > > Fax: +39 06 9040 > > E-mail: davide.p...@gesca.it > > Web:www.gesca.it<http://www.gesca.it> > > > > > > > > > > ________________ > > Da: Alessandro Caviglione [c.alessan...@gmail.com] > > Inviato: sabato 2 gennaio 2016 22.27 > > A: users@cloudstack.apache.org > > Oggetto: Re: A Story of a Failed XenServer Upgrade > > > > No guys,as the article wrote, my first action was to put in Maintenance > > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the > XenServer > > Pool Master first before any of the Slaves. To do so you need to empty > the > > Pool Master of all CloudStack VMs, and you do this by putting the Host > into > > Maintenance Mode within CloudStack to trigger a live migration of all VMs > > to alternate Hosts" > > > > This is exactly what I've done and after the XS upgrade, no hosts was > able > > to communicate with CS and also with the upgraded host. > > > > Putting an host in Maint Mode within CS will trigger MM also on XenServer > > host or just will move the VMs to other hosts? > > > > And again what's the best practices to upgrade a XS cluster? > > > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma < > rberg...@schubergphilis.com> > > wrote: > > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > > > That's not true. You can safely migrate outside of CloudStack as the > > power > > > report will tell CloudStack where the vms live and the db gets updated > > > accordingly. I do this a lot while patching and that works fine on 6.2 > > and > > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > > > Regards, Remi > > > > > > > > > Sent from my iPhone > > > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > > jpeter...@acentek.net>> wrote: > > > > > > I don't use XenServer maintenance mode until after CloudStack has put > the > > > Host in maintenance mode. > > > > > > When you initiate maintenance mode from the host rather than CloudStack > > > the db does not know where the VM's are and your UUID's get jacked. > > > > > > CS is your brains not the hypervisor. > > > > > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > > > Upgrade. Reboot. Join Pool. Remove Maintenance starting at > hypervisor > > if > > > needed and then CS and move on to the next Host. > > > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > > > Jeremy > > > > > > > > > -Original Message- > > > From: Davide Pala [mailto:davide.p...@gesca.it] > > > Sent: Friday, January 1, 2016 5:18 PM > > > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > > Subject: R: A Story of a Failed XenServer Upgrade > > > > > > Hi alessandro. If u put in maintenance mode the master you force the > > > election of a new pool master. Now when you have see the upgraded host > as > > > disconnected you are connected to the new pool master and the host (as > a > > > pool member) cannot comunicate with a pool master of an earlies
R: A Story of a Failed XenServer Upgrade
So i dont know anything about cloud on xenserver and for this reason i think the cloudstack dont put in maintenance xenserver pool master Inviato dal mio dispositivo Samsung Messaggio originale Da: Rafael Weingärtner Data: 02/01/2016 22:48 (GMT+01:00) A: users@cloudstack.apache.org Oggetto: Re: A Story of a Failed XenServer Upgrade Hi all, There is nothing better than looking at the source code. After the VM migration (or restart for LCX ?!), when the host is put in maintenance mode, for Xenserver it will remove a tag called “cloud”. On Sat, Jan 2, 2016 at 7:38 PM, Davide Pala wrote: > i don't know what do the maintenance mode in CS but if it put in > maintenance also the pool master this article is wrong! > > > Davide Pala > Infrastructure Specialist > Gesca srl > > Via degli Olmetti, 18 > 00060 Formello (Roma) > Office: +39 06 9040661 > Fax: +39 06 9040 > E-mail: davide.p...@gesca.it > Web:www.gesca.it<http://www.gesca.it> > > > > > > Da: Alessandro Caviglione [c.alessan...@gmail.com] > Inviato: sabato 2 gennaio 2016 22.27 > A: users@cloudstack.apache.org > Oggetto: Re: A Story of a Failed XenServer Upgrade > > No guys,as the article wrote, my first action was to put in Maintenance > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer > Pool Master first before any of the Slaves. To do so you need to empty the > Pool Master of all CloudStack VMs, and you do this by putting the Host into > Maintenance Mode within CloudStack to trigger a live migration of all VMs > to alternate Hosts" > > This is exactly what I've done and after the XS upgrade, no hosts was able > to communicate with CS and also with the upgraded host. > > Putting an host in Maint Mode within CS will trigger MM also on XenServer > host or just will move the VMs to other hosts? > > And again what's the best practices to upgrade a XS cluster? > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma > wrote: > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > That's not true. You can safely migrate outside of CloudStack as the > power > > report will tell CloudStack where the vms live and the db gets updated > > accordingly. I do this a lot while patching and that works fine on 6.2 > and > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > Regards, Remi > > > > > > Sent from my iPhone > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > jpeter...@acentek.net>> wrote: > > > > I don't use XenServer maintenance mode until after CloudStack has put the > > Host in maintenance mode. > > > > When you initiate maintenance mode from the host rather than CloudStack > > the db does not know where the VM's are and your UUID's get jacked. > > > > CS is your brains not the hypervisor. > > > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor > if > > needed and then CS and move on to the next Host. > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > Jeremy > > > > > > -Original Message- > > From: Davide Pala [mailto:davide.p...@gesca.it] > > Sent: Friday, January 1, 2016 5:18 PM > > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Subject: R: A Story of a Failed XenServer Upgrade > > > > Hi alessandro. If u put in maintenance mode the master you force the > > election of a new pool master. Now when you have see the upgraded host as > > disconnected you are connected to the new pool master and the host (as a > > pool member) cannot comunicate with a pool master of an earliest version. > > The solution? Launche the upgrade on the pool master without enter in > > maintenance mode. And remember a consistent backup!!! > > > > > > > > Inviato dal mio dispositivo Samsung > > > > > > Messaggio originale > > Da: Alessandro Caviglione > c.alessan...@gmail.com>> > > Data: 01/01/2016 23:23 (GMT+01:00) > > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Oggetto: A Story of a Failed XenServer Upgrade > > > > Hi guys, > > I want to share my XenServer Upgrade adventure to understand if I did > > domething wrong. > > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs > > has been upgraded I start the upgrade process of my X
Re: A Story of a Failed XenServer Upgrade
Hi all, There is nothing better than looking at the source code. After the VM migration (or restart for LCX ?!), when the host is put in maintenance mode, for Xenserver it will remove a tag called “cloud”. On Sat, Jan 2, 2016 at 7:38 PM, Davide Pala wrote: > i don't know what do the maintenance mode in CS but if it put in > maintenance also the pool master this article is wrong! > > > Davide Pala > Infrastructure Specialist > Gesca srl > > Via degli Olmetti, 18 > 00060 Formello (Roma) > Office: +39 06 9040661 > Fax: +39 06 9040 > E-mail: davide.p...@gesca.it > Web:www.gesca.it > > > > > > Da: Alessandro Caviglione [c.alessan...@gmail.com] > Inviato: sabato 2 gennaio 2016 22.27 > A: users@cloudstack.apache.org > Oggetto: Re: A Story of a Failed XenServer Upgrade > > No guys,as the article wrote, my first action was to put in Maintenance > Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer > Pool Master first before any of the Slaves. To do so you need to empty the > Pool Master of all CloudStack VMs, and you do this by putting the Host into > Maintenance Mode within CloudStack to trigger a live migration of all VMs > to alternate Hosts" > > This is exactly what I've done and after the XS upgrade, no hosts was able > to communicate with CS and also with the upgraded host. > > Putting an host in Maint Mode within CS will trigger MM also on XenServer > host or just will move the VMs to other hosts? > > And again what's the best practices to upgrade a XS cluster? > > On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma > wrote: > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > That's not true. You can safely migrate outside of CloudStack as the > power > > report will tell CloudStack where the vms live and the db gets updated > > accordingly. I do this a lot while patching and that works fine on 6.2 > and > > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > > > Regards, Remi > > > > > > Sent from my iPhone > > > > On 02 Jan 2016, at 16:26, Jeremy Peterson > jpeter...@acentek.net>> wrote: > > > > I don't use XenServer maintenance mode until after CloudStack has put the > > Host in maintenance mode. > > > > When you initiate maintenance mode from the host rather than CloudStack > > the db does not know where the VM's are and your UUID's get jacked. > > > > CS is your brains not the hypervisor. > > > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor > if > > needed and then CS and move on to the next Host. > > > > CloudStack should always do the migration of VM's not the Hypervisor. > > > > Jeremy > > > > > > -Original Message- > > From: Davide Pala [mailto:davide.p...@gesca.it] > > Sent: Friday, January 1, 2016 5:18 PM > > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Subject: R: A Story of a Failed XenServer Upgrade > > > > Hi alessandro. If u put in maintenance mode the master you force the > > election of a new pool master. Now when you have see the upgraded host as > > disconnected you are connected to the new pool master and the host (as a > > pool member) cannot comunicate with a pool master of an earliest version. > > The solution? Launche the upgrade on the pool master without enter in > > maintenance mode. And remember a consistent backup!!! > > > > > > > > Inviato dal mio dispositivo Samsung > > > > > > Messaggio originale > > Da: Alessandro Caviglione > c.alessan...@gmail.com>> > > Data: 01/01/2016 23:23 (GMT+01:00) > > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > > Oggetto: A Story of a Failed XenServer Upgrade > > > > Hi guys, > > I want to share my XenServer Upgrade adventure to understand if I did > > domething wrong. > > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs > > has been upgraded I start the upgrade process of my XenServer hosts from > > 6.2 to 6.5. > > I do not already have PoolHA enabled so I followed this article: > > > > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ > > > > The cluster consists of n° 3 XenServer hosts. > > > > First of all I added manage.xenserver.pool.master=false > > to environmen
RE: A Story of a Failed XenServer Upgrade
i don't know what do the maintenance mode in CS but if it put in maintenance also the pool master this article is wrong! Davide Pala Infrastructure Specialist Gesca srl Via degli Olmetti, 18 00060 Formello (Roma) Office: +39 06 9040661 Fax: +39 06 9040 E-mail: davide.p...@gesca.it Web:www.gesca.it Da: Alessandro Caviglione [c.alessan...@gmail.com] Inviato: sabato 2 gennaio 2016 22.27 A: users@cloudstack.apache.org Oggetto: Re: A Story of a Failed XenServer Upgrade No guys,as the article wrote, my first action was to put in Maintenance Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer Pool Master first before any of the Slaves. To do so you need to empty the Pool Master of all CloudStack VMs, and you do this by putting the Host into Maintenance Mode within CloudStack to trigger a live migration of all VMs to alternate Hosts" This is exactly what I've done and after the XS upgrade, no hosts was able to communicate with CS and also with the upgraded host. Putting an host in Maint Mode within CS will trigger MM also on XenServer host or just will move the VMs to other hosts? And again what's the best practices to upgrade a XS cluster? On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma wrote: > CloudStack should always do the migration of VM's not the Hypervisor. > > That's not true. You can safely migrate outside of CloudStack as the power > report will tell CloudStack where the vms live and the db gets updated > accordingly. I do this a lot while patching and that works fine on 6.2 and > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > Regards, Remi > > > Sent from my iPhone > > On 02 Jan 2016, at 16:26, Jeremy Peterson jpeter...@acentek.net>> wrote: > > I don't use XenServer maintenance mode until after CloudStack has put the > Host in maintenance mode. > > When you initiate maintenance mode from the host rather than CloudStack > the db does not know where the VM's are and your UUID's get jacked. > > CS is your brains not the hypervisor. > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor if > needed and then CS and move on to the next Host. > > CloudStack should always do the migration of VM's not the Hypervisor. > > Jeremy > > > -Original Message- > From: Davide Pala [mailto:davide.p...@gesca.it] > Sent: Friday, January 1, 2016 5:18 PM > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > Subject: R: A Story of a Failed XenServer Upgrade > > Hi alessandro. If u put in maintenance mode the master you force the > election of a new pool master. Now when you have see the upgraded host as > disconnected you are connected to the new pool master and the host (as a > pool member) cannot comunicate with a pool master of an earliest version. > The solution? Launche the upgrade on the pool master without enter in > maintenance mode. And remember a consistent backup!!! > > > > Inviato dal mio dispositivo Samsung > > > ---- Messaggio originale > Da: Alessandro Caviglione c.alessan...@gmail.com>> > Data: 01/01/2016 23:23 (GMT+01:00) > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > Oggetto: A Story of a Failed XenServer Upgrade > > Hi guys, > I want to share my XenServer Upgrade adventure to understand if I did > domething wrong. > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs > has been upgraded I start the upgrade process of my XenServer hosts from > 6.2 to 6.5. > I do not already have PoolHA enabled so I followed this article: > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ > > The cluster consists of n° 3 XenServer hosts. > > First of all I added manage.xenserver.pool.master=false > to environment.properties file and restarted cloudstack-management service. > > After that I put in Maintenance Mode Pool Master host and, after all VMs > has been migrated, I Unmanaged the cluster. > At this point all host appears as "Disconnected" from CS interface and > this should be right. > Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a > in-place upgrade. > After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. > At this point I expected that, after click on Manage Cluster on CS, all > the hosts come back to "UP" and I could go ahead upgrading the other > hosts > > But, instead of that, all the hosts still appears as "Disconnected", I > tried a couple of cloudstack-management service restart without success. > &
Re: A Story of a Failed XenServer Upgrade
No guys,as the article wrote, my first action was to put in Maintenance Mode the Pool Master INSIDE CS; "It is vital that you upgrade the XenServer Pool Master first before any of the Slaves. To do so you need to empty the Pool Master of all CloudStack VMs, and you do this by putting the Host into Maintenance Mode within CloudStack to trigger a live migration of all VMs to alternate Hosts" This is exactly what I've done and after the XS upgrade, no hosts was able to communicate with CS and also with the upgraded host. Putting an host in Maint Mode within CS will trigger MM also on XenServer host or just will move the VMs to other hosts? And again what's the best practices to upgrade a XS cluster? On Sat, Jan 2, 2016 at 7:11 PM, Remi Bergsma wrote: > CloudStack should always do the migration of VM's not the Hypervisor. > > That's not true. You can safely migrate outside of CloudStack as the power > report will tell CloudStack where the vms live and the db gets updated > accordingly. I do this a lot while patching and that works fine on 6.2 and > 6.5. I use both CloudStack 4.4.4 and 4.7.0. > > Regards, Remi > > > Sent from my iPhone > > On 02 Jan 2016, at 16:26, Jeremy Peterson jpeter...@acentek.net>> wrote: > > I don't use XenServer maintenance mode until after CloudStack has put the > Host in maintenance mode. > > When you initiate maintenance mode from the host rather than CloudStack > the db does not know where the VM's are and your UUID's get jacked. > > CS is your brains not the hypervisor. > > Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. > Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor if > needed and then CS and move on to the next Host. > > CloudStack should always do the migration of VM's not the Hypervisor. > > Jeremy > > > -Original Message- > From: Davide Pala [mailto:davide.p...@gesca.it] > Sent: Friday, January 1, 2016 5:18 PM > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > Subject: R: A Story of a Failed XenServer Upgrade > > Hi alessandro. If u put in maintenance mode the master you force the > election of a new pool master. Now when you have see the upgraded host as > disconnected you are connected to the new pool master and the host (as a > pool member) cannot comunicate with a pool master of an earliest version. > The solution? Launche the upgrade on the pool master without enter in > maintenance mode. And remember a consistent backup!!! > > > > Inviato dal mio dispositivo Samsung > > > -------- Messaggio originale > Da: Alessandro Caviglione c.alessan...@gmail.com>> > Data: 01/01/2016 23:23 (GMT+01:00) > A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > Oggetto: A Story of a Failed XenServer Upgrade > > Hi guys, > I want to share my XenServer Upgrade adventure to understand if I did > domething wrong. > I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs > has been upgraded I start the upgrade process of my XenServer hosts from > 6.2 to 6.5. > I do not already have PoolHA enabled so I followed this article: > > http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ > > The cluster consists of n° 3 XenServer hosts. > > First of all I added manage.xenserver.pool.master=false > to environment.properties file and restarted cloudstack-management service. > > After that I put in Maintenance Mode Pool Master host and, after all VMs > has been migrated, I Unmanaged the cluster. > At this point all host appears as "Disconnected" from CS interface and > this should be right. > Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a > in-place upgrade. > After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. > At this point I expected that, after click on Manage Cluster on CS, all > the hosts come back to "UP" and I could go ahead upgrading the other > hosts > > But, instead of that, all the hosts still appears as "Disconnected", I > tried a couple of cloudstack-management service restart without success. > > So I opened XenCenter and connect to Pool Master I upgraded to 6.5 and it > appear in Maintenance Mode, so I tried to Exit from Maint Mode but I got > the error: The server is still booting > > After some investigation, I run the command "xe task-list" and this is the > result: > > uuid ( RO): 72f48a56-1d24-1ca3-aade-091f1830e2f1 > name-label ( RO): VM.set_memory_dynamic_range name-description ( RO): > status ( RO): pending > progress ( RO): 0.000 > > I tried a cou
Re: A Story of a Failed XenServer Upgrade
CloudStack should always do the migration of VM's not the Hypervisor. That's not true. You can safely migrate outside of CloudStack as the power report will tell CloudStack where the vms live and the db gets updated accordingly. I do this a lot while patching and that works fine on 6.2 and 6.5. I use both CloudStack 4.4.4 and 4.7.0. Regards, Remi Sent from my iPhone On 02 Jan 2016, at 16:26, Jeremy Peterson mailto:jpeter...@acentek.net>> wrote: I don't use XenServer maintenance mode until after CloudStack has put the Host in maintenance mode. When you initiate maintenance mode from the host rather than CloudStack the db does not know where the VM's are and your UUID's get jacked. CS is your brains not the hypervisor. Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor if needed and then CS and move on to the next Host. CloudStack should always do the migration of VM's not the Hypervisor. Jeremy -Original Message- From: Davide Pala [mailto:davide.p...@gesca.it] Sent: Friday, January 1, 2016 5:18 PM To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> Subject: R: A Story of a Failed XenServer Upgrade Hi alessandro. If u put in maintenance mode the master you force the election of a new pool master. Now when you have see the upgraded host as disconnected you are connected to the new pool master and the host (as a pool member) cannot comunicate with a pool master of an earliest version. The solution? Launche the upgrade on the pool master without enter in maintenance mode. And remember a consistent backup!!! Inviato dal mio dispositivo Samsung Messaggio originale Da: Alessandro Caviglione mailto:c.alessan...@gmail.com>> Data: 01/01/2016 23:23 (GMT+01:00) A: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> Oggetto: A Story of a Failed XenServer Upgrade Hi guys, I want to share my XenServer Upgrade adventure to understand if I did domething wrong. I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs has been upgraded I start the upgrade process of my XenServer hosts from 6.2 to 6.5. I do not already have PoolHA enabled so I followed this article: http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ The cluster consists of n° 3 XenServer hosts. First of all I added manage.xenserver.pool.master=false to environment.properties file and restarted cloudstack-management service. After that I put in Maintenance Mode Pool Master host and, after all VMs has been migrated, I Unmanaged the cluster. At this point all host appears as "Disconnected" from CS interface and this should be right. Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a in-place upgrade. After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. At this point I expected that, after click on Manage Cluster on CS, all the hosts come back to "UP" and I could go ahead upgrading the other hosts But, instead of that, all the hosts still appears as "Disconnected", I tried a couple of cloudstack-management service restart without success. So I opened XenCenter and connect to Pool Master I upgraded to 6.5 and it appear in Maintenance Mode, so I tried to Exit from Maint Mode but I got the error: The server is still booting After some investigation, I run the command "xe task-list" and this is the result: uuid ( RO): 72f48a56-1d24-1ca3-aade-091f1830e2f1 name-label ( RO): VM.set_memory_dynamic_range name-description ( RO): status ( RO): pending progress ( RO): 0.000 I tried a couple of reboot but nothing changes so I decided to shut down the server, force raise a slave host to master with emergency mode, remove old server from CS and reboot CS. After that, I see my cluster up and running again, so I installed XS 6.2SP1 on the "upgraded" host and added again to the cluster So after an entire day of work, I'm in the same situation! :D Anyone can tell me if I made something wrong?? Thank you very much!
RE: A Story of a Failed XenServer Upgrade
I don't use XenServer maintenance mode until after CloudStack has put the Host in maintenance mode. When you initiate maintenance mode from the host rather than CloudStack the db does not know where the VM's are and your UUID's get jacked. CS is your brains not the hypervisor. Maintenance in CS. All VM's will migrate. Maintenance in XenCenter. Upgrade. Reboot. Join Pool. Remove Maintenance starting at hypervisor if needed and then CS and move on to the next Host. CloudStack should always do the migration of VM's not the Hypervisor. Jeremy -Original Message- From: Davide Pala [mailto:davide.p...@gesca.it] Sent: Friday, January 1, 2016 5:18 PM To: users@cloudstack.apache.org Subject: R: A Story of a Failed XenServer Upgrade Hi alessandro. If u put in maintenance mode the master you force the election of a new pool master. Now when you have see the upgraded host as disconnected you are connected to the new pool master and the host (as a pool member) cannot comunicate with a pool master of an earliest version. The solution? Launche the upgrade on the pool master without enter in maintenance mode. And remember a consistent backup!!! Inviato dal mio dispositivo Samsung Messaggio originale Da: Alessandro Caviglione Data: 01/01/2016 23:23 (GMT+01:00) A: users@cloudstack.apache.org Oggetto: A Story of a Failed XenServer Upgrade Hi guys, I want to share my XenServer Upgrade adventure to understand if I did domething wrong. I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs has been upgraded I start the upgrade process of my XenServer hosts from 6.2 to 6.5. I do not already have PoolHA enabled so I followed this article: http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ The cluster consists of n° 3 XenServer hosts. First of all I added manage.xenserver.pool.master=false to environment.properties file and restarted cloudstack-management service. After that I put in Maintenance Mode Pool Master host and, after all VMs has been migrated, I Unmanaged the cluster. At this point all host appears as "Disconnected" from CS interface and this should be right. Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a in-place upgrade. After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. At this point I expected that, after click on Manage Cluster on CS, all the hosts come back to "UP" and I could go ahead upgrading the other hosts But, instead of that, all the hosts still appears as "Disconnected", I tried a couple of cloudstack-management service restart without success. So I opened XenCenter and connect to Pool Master I upgraded to 6.5 and it appear in Maintenance Mode, so I tried to Exit from Maint Mode but I got the error: The server is still booting After some investigation, I run the command "xe task-list" and this is the result: uuid ( RO): 72f48a56-1d24-1ca3-aade-091f1830e2f1 name-label ( RO): VM.set_memory_dynamic_range name-description ( RO): status ( RO): pending progress ( RO): 0.000 I tried a couple of reboot but nothing changes so I decided to shut down the server, force raise a slave host to master with emergency mode, remove old server from CS and reboot CS. After that, I see my cluster up and running again, so I installed XS 6.2SP1 on the "upgraded" host and added again to the cluster So after an entire day of work, I'm in the same situation! :D Anyone can tell me if I made something wrong?? Thank you very much!
R: A Story of a Failed XenServer Upgrade
Hi alessandro. If u put in maintenance mode the master you force the election of a new pool master. Now when you have see the upgraded host as disconnected you are connected to the new pool master and the host (as a pool member) cannot comunicate with a pool master of an earliest version. The solution? Launche the upgrade on the pool master without enter in maintenance mode. And remember a consistent backup!!! Inviato dal mio dispositivo Samsung Messaggio originale Da: Alessandro Caviglione Data: 01/01/2016 23:23 (GMT+01:00) A: users@cloudstack.apache.org Oggetto: A Story of a Failed XenServer Upgrade Hi guys, I want to share my XenServer Upgrade adventure to understand if I did domething wrong. I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs has been upgraded I start the upgrade process of my XenServer hosts from 6.2 to 6.5. I do not already have PoolHA enabled so I followed this article: http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ The cluster consists of n° 3 XenServer hosts. First of all I added manage.xenserver.pool.master=false to environment.properties file and restarted cloudstack-management service. After that I put in Maintenance Mode Pool Master host and, after all VMs has been migrated, I Unmanaged the cluster. At this point all host appears as "Disconnected" from CS interface and this should be right. Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a in-place upgrade. After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. At this point I expected that, after click on Manage Cluster on CS, all the hosts come back to "UP" and I could go ahead upgrading the other hosts But, instead of that, all the hosts still appears as "Disconnected", I tried a couple of cloudstack-management service restart without success. So I opened XenCenter and connect to Pool Master I upgraded to 6.5 and it appear in Maintenance Mode, so I tried to Exit from Maint Mode but I got the error: The server is still booting After some investigation, I run the command "xe task-list" and this is the result: uuid ( RO): 72f48a56-1d24-1ca3-aade-091f1830e2f1 name-label ( RO): VM.set_memory_dynamic_range name-description ( RO): status ( RO): pending progress ( RO): 0.000 I tried a couple of reboot but nothing changes so I decided to shut down the server, force raise a slave host to master with emergency mode, remove old server from CS and reboot CS. After that, I see my cluster up and running again, so I installed XS 6.2SP1 on the "upgraded" host and added again to the cluster So after an entire day of work, I'm in the same situation! :D Anyone can tell me if I made something wrong?? Thank you very much!
A Story of a Failed XenServer Upgrade
Hi guys, I want to share my XenServer Upgrade adventure to understand if I did domething wrong. I upgraded CS from 4.4.4 to 4.5.2 without any issues, after all the VRs has been upgraded I start the upgrade process of my XenServer hosts from 6.2 to 6.5. I do not already have PoolHA enabled so I followed this article: http://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ The cluster consists of n° 3 XenServer hosts. First of all I added manage.xenserver.pool.master=false to environment.properties file and restarted cloudstack-management service. After that I put in Maintenance Mode Pool Master host and, after all VMs has been migrated, I Unmanaged the cluster. At this point all host appears as "Disconnected" from CS interface and this should be right. Now I put XenServer 6.5 CD in the host in Maintenance Mode and start a in-place upgrade. After XS6.5 has been installed, I istalled the 6.5SP1 and reboot again. At this point I expected that, after click on Manage Cluster on CS, all the hosts come back to "UP" and I could go ahead upgrading the other hosts But, instead of that, all the hosts still appears as "Disconnected", I tried a couple of cloudstack-management service restart without success. So I opened XenCenter and connect to Pool Master I upgraded to 6.5 and it appear in Maintenance Mode, so I tried to Exit from Maint Mode but I got the error: The server is still booting After some investigation, I run the command "xe task-list" and this is the result: uuid ( RO): 72f48a56-1d24-1ca3-aade-091f1830e2f1 name-label ( RO): VM.set_memory_dynamic_range name-description ( RO): status ( RO): pending progress ( RO): 0.000 I tried a couple of reboot but nothing changes so I decided to shut down the server, force raise a slave host to master with emergency mode, remove old server from CS and reboot CS. After that, I see my cluster up and running again, so I installed XS 6.2SP1 on the "upgraded" host and added again to the cluster So after an entire day of work, I'm in the same situation! :D Anyone can tell me if I made something wrong?? Thank you very much!
