Hi David, with a bit of delay...
those steps need to be tested - I would skip the whole "environment.properties" file and see how it behaves today - the reason being that, though the article on shapeblue.com was old, it does mention both "Unmanage" cluster and Maintenace mode - so I'm not quite sure what is the difference today vs. how it behaved back in time of ACS 4.3 / XS 6.2 - the explanation that you've found on the mailing thread may not make sense, I mean specifically the "There wasn't an 'unmanage' button at the time", since the article clearly mentions it). There is also no need to manually migrate VM's away from a host (i.e. pool master) - simply put it into the Maintenance mode and it will move VMs away to other hosts. Assuming that putting the pool-master into Maintenance mode in ACS will, these days, NOT trigger a new host to become a master, your steps look fine. For the records, I've updated&&tested the official documentation, for XenServer 6.5+ : https://github.com/apache/cloudstack-documentation/pull/140 / https://acs-www.shapeblue.com/docs/WIP-PROOFING/pr140/installguide/hypervisor/xenserver.html#upgrading-xenserver-versions - i.e. removed unneeded steps and explained what each script is doing. This guide assumes a much longer management plane downtime, as the cluster is unmanaged while you update all the hosts in the pool. Works fine also for XCP-ng upgrades, etc. Either way, I prefer doing it based on the plan you laid out here. Regards, Andrija On Fri, 19 Jun 2020 at 23:24, David Merrill <david.merr...@otelco.com> wrote: > Hi All, > > I have a production deployment of ACS managing three XenServer Clusters > (XenServer pools of 6 hosts each) in two different Zones. I now find myself > in the position of needing to do a major version upgrade of those hosts. > Happily I have a ACS lab managing a XenServer cluster running the same > (old) version of XenServer that I can practice on. > > I have plenty of practice operating ACS to “quiesce things” for XenServer > patches (start with the pool master, move guest VMs off, put that host into > maintenance mode, unmanage the cluster, patch the host, then reverse & move > onto the next host with the same steps except we don’t bother > w/un-managing/re-managing the cluster), but as I understand a XenServer > version upgrade backs up the whole original XenServer installation to > another partition and makes a clean installation of XenServer on the > original partition (the problem there being that when the upgraded > XenServer boots up all the ACS provided/copied scripts are not there & ACS > can’t manage the host). > > So not much of an ask here (OK maybe at the end – have I missed something > obvious or doing anything foolish?), I wanted to share a bit research & lay > out a set of steps that I think will work to get a pool of XenServers in a > cluster upgraded and end up in a place where ACS is happy with them. > > Bear with me it’s a little long, > > > 1. In XenCenter – if HA is enabled for the XenServer pool, disable it > 2. Stop ACS management/usage services > 3. Do MySQL database backups > 4. Start ACS management/usage services > 5. Start with the pool master. > 6. In ACS – Migrate all guest VMs to other hosts in the cluster > 7. In ACS – Prepare to put the pool master into maintenance mode (so no > new guest VMs) > * A caveat here related to this item I found when researching – > https://www.shapeblue.com/how-to-upgrade-an-apache-cloudstack-citrix-xenserver-cluster/ > > i. A > recommendation is made here to edit > /etc/cloudstack/management/environment.properties > > ii. And > set manage.xenserver.pool.master=false > > iii. And > restart CloudStack management services > > iv. > BECAUSE if one didn’t I understand CloudStack WOULD force an election for > another host to become the pool master (which is “bad” as ASCs is > configured to speak to the currently configured pool master) > > * HOWEVER THIS MAY NOT BE NECESSARY > > i. > Found a thread titled “A Story of a failed XenServer Upgrade” here – > http://mail-archives.apache.org/mod_mbox/cloudstack-users/201601.mbox/browser > > ii. At > the end of the thread Paul Angus states that Geoff’s ShapeBlue blog article > was written in the ACS 4.3 era and that ACS’ behavior “used to be that > putting a host that was the pool master into maintenance would cause > CloudStack to force an election for another host to become pool master - > stopping you from then upgrading the active pool master first. There wasn't > an 'unmanage' button at the time.” > > iii. The > implication, (in my humble estimation) is that, today, one no longer needs > to make these edits to /etc/cloudstack/management/environment.properties > > 1. In ACS – Put the pool master into maintenance mode (so no new guest > VMs) > 2. In ACS – Un-manage the cluster > 3. NOW – Upgrade the XenServer pool master to the latest release > * Exercise left to the reader…(I’ll share what I end up doing) > 4. In XenServer – Do the following (found this nugget in a thread last > November about XenServer upgrades & how to re-setup the host – thanks > Richard Lawley!) > * Remove this host tag: xe host-param-remove uuid=HOSTUUID > param-name=tags > param-key=vmops-version-com.cloud.hypervisor.xenserver.resource.XenServer650R > * This makes management server set the host back up, presumably > since ACS has credentials to the host in the database it can copy all the > scripts back > 5. In ACS – Re-manage the cluster > 6. In ACS – Exit maintenance-mode for the newly upgraded host > 7. In ACS – Observe that the newly upgraded host is “Enabled” and “Up” > in the UI (Infrastructure --> Hosts) > 8. In ACS – Testing (e.g. move an existing router/VM to the upgraded > host, create new networks/VMs on the upgraded host) > 9. Rinse & repeat with the remaining XenServer pool members in the ACS > cluster. > * WITH THIS EXCEPTION: No need to manipulate management of the > cluster in ACS > 10. In XenCenter – if HA is required re-enable it > > Now all of this completely bypasses steps that are spelled out here (which > I *suspect* might now be “old”?): > > > * > http://docs.cloudstack.apache.org/en/latest/installguide/hypervisor/xenserver.html#upgrading-xenserver-versions > > which asks one to run a lot of scripts (cleaning VLANs, preparing for > upgrade etc.) in addition to copying files from the management server to > the XenServer host (to set it back up again for ACS). I’m really hoping > that step 11 saves me from that. > > So (asks after all): > > > 1. Is this a reasonable/viable plan? > 2. Is my take on step 7 correct? > > Thanks for taking a look, > David > > David Merrill > Senior Systems Engineer, > Managed and Private/Hybrid Cloud Services > OTELCO > 92 Oak Street, Portland ME 04101 > office 207.772.5678<callto:207.772.5678> > http://www.otelco.com/cloud-and-managed-services > > Confidentiality Message > The information contained in this e-mail transmission may be confidential > and legally privileged. If you are not the intended recipient, you are > notified that any dissemination, distribution, copying or other use of this > information, including attachments, is prohibited. If you received this > message in error, please call me at 207.772.5678<callto:207.772.5678> so > this error can be corrected. > > -- Andrija Panić
