Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
Hi David, 0. Good :) (needless to say, always drop the messed-up DBs, and then import the backup in, don't just try to import over the existing/messed-up DBs) 1. You are right, you HAVE TO use the **exact** name as it's stated in the upgrade notes (ACS code base is searching for a template with that name, otherwise DB upgrade will fail again) 2. Since it's not mentioned, just leave it as it is - it's not relevant (and i.e. for VMware/KVM it's ticked by default, if not mistaken) Cheers Andrija On Tue, 19 May 2020 at 20:58, David Merrill wrote: > Reporting in, I was able to roll-back the cloudstack packages and reload > the backup of the cloud database and get the UI going again. > > So that's good, but a couple questions on this page (and registering the > template in the UI): > > - > http://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade-4.11.html#update-system-vm-templates > > 1. I understand it's important to use the values specified, I assume > setting the "name" specifically as stated is what helps cloudstack "find" > the new templates when upgrading the database? > > 2. In the UI there's an HVM checkbox (that's defaulted to checked), but > the documentation (above) doesn't specifically refer to it. So for my > clarity (hopefully I just need to be remided), leave it checked or > unchecked? Does it matter in the context of the system VMs? > > Thanks! > David > > David Merrill > Senior Systems Engineer, > Managed and Private/Hybrid Cloud Services > OTELCO > 92 Oak Street, Portland ME 04101 > office 207.772.5678 > http://www.otelco.com/cloud-and-managed-services > > Confidentiality Message > The information contained in this e-mail transmission may be confidential > and legally privileged. If you are not the intended recipient, you are > notified that any dissemination, distribution, copying or other use of this > information, including attachments, is prohibited. If you received this > message in error, please call me at 207.772.5678 so > this error can be corrected. > > > On 5/18/20, 10:22 AM, "David Merrill" wrote: > > OK, got it, digging in... > > Thanks everyone, > David > > David Merrill > Senior Systems Engineer, > Managed and Private/Hybrid Cloud Services > OTELCO > 92 Oak Street, Portland ME 04101 > office 207.772.5678 > http://www.otelco.com/cloud-and-managed-services > > Confidentiality Message > The information contained in this e-mail transmission may be > confidential and legally privileged. If you are not the intended recipient, > you are notified that any dissemination, distribution, copying or other use > of this information, including attachments, is prohibited. If you received > this message in error, please call me at 207.772.5678 > so this error can be corrected. > > > On 5/18/20, 7:44 AM, "Andrija Panic" wrote: > > David, > > the procedure you laid out is correct - rollback DB, downgrade, etc > As Luis mentioned, make sure to also rollback the > "mysql-connector-java" in > case it was upgraded (if you left the mysql repo enabled during the > upgrade). > > There are ways to hack the DB, vm_template table and also use the > "cloud-install-sys-tmplt"... but a clean rollback (as it's very > easy in > your test env) is a much better way to proceed. > > Cheers > Andrija > > On Mon, 18 May 2020 at 10:06, Richard Lawley < > rich...@richardlawley.com> > wrote: > > > The database upgrade does not happen unless the systemVM > templates have > > been added, so nothing non-reversible has happened yet. You can > just use > > yum to downgrade to 4.11.2 and you'll be fine (we've also > accidentally done > > this at some point!). > > > > I'd recommend disabling your cloudstack yum repo so that it > doesn't happen > > again. > > > > On Sun, 17 May 2020 at 21:01, Luis > wrote: > > > > > Dont forget to downgrade the database connector or it will not > work > > > > > > Sent from Yahoo Mail on Android > > > > > > On Sun, May 17, 2020 at 3:47 PM, David Merrill< > > david.merr...@otelco.com> > > > wrote: Hi All, > > > > > > I've got a CloudStack 4.11.2 lab running on a single host & > made the > > > (dumb) mistake of running OS updates without updating the > system VMs > > > beforehand. The CloudStack packages upgraded to 4.11.3 just > fine but now > > > CloudStack management services wont start (see below). > > > > > > Here's my question: > > > > > > - Is it *at all* possible to rectify this (now, post package > updates) by > > > getting the new system templates in via the CLI tools > > > > > > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > > > - The
Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
Reporting in, I was able to roll-back the cloudstack packages and reload the backup of the cloud database and get the UI going again. So that's good, but a couple questions on this page (and registering the template in the UI): - http://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade-4.11.html#update-system-vm-templates 1. I understand it's important to use the values specified, I assume setting the "name" specifically as stated is what helps cloudstack "find" the new templates when upgrading the database? 2. In the UI there's an HVM checkbox (that's defaulted to checked), but the documentation (above) doesn't specifically refer to it. So for my clarity (hopefully I just need to be remided), leave it checked or unchecked? Does it matter in the context of the system VMs? Thanks! David David Merrill Senior Systems Engineer, Managed and Private/Hybrid Cloud Services OTELCO 92 Oak Street, Portland ME 04101 office 207.772.5678 http://www.otelco.com/cloud-and-managed-services Confidentiality Message The information contained in this e-mail transmission may be confidential and legally privileged. If you are not the intended recipient, you are notified that any dissemination, distribution, copying or other use of this information, including attachments, is prohibited. If you received this message in error, please call me at 207.772.5678 so this error can be corrected. On 5/18/20, 10:22 AM, "David Merrill" wrote: OK, got it, digging in... Thanks everyone, David David Merrill Senior Systems Engineer, Managed and Private/Hybrid Cloud Services OTELCO 92 Oak Street, Portland ME 04101 office 207.772.5678 http://www.otelco.com/cloud-and-managed-services Confidentiality Message The information contained in this e-mail transmission may be confidential and legally privileged. If you are not the intended recipient, you are notified that any dissemination, distribution, copying or other use of this information, including attachments, is prohibited. If you received this message in error, please call me at 207.772.5678 so this error can be corrected. On 5/18/20, 7:44 AM, "Andrija Panic" wrote: David, the procedure you laid out is correct - rollback DB, downgrade, etc As Luis mentioned, make sure to also rollback the "mysql-connector-java" in case it was upgraded (if you left the mysql repo enabled during the upgrade). There are ways to hack the DB, vm_template table and also use the "cloud-install-sys-tmplt"... but a clean rollback (as it's very easy in your test env) is a much better way to proceed. Cheers Andrija On Mon, 18 May 2020 at 10:06, Richard Lawley wrote: > The database upgrade does not happen unless the systemVM templates have > been added, so nothing non-reversible has happened yet. You can just use > yum to downgrade to 4.11.2 and you'll be fine (we've also accidentally done > this at some point!). > > I'd recommend disabling your cloudstack yum repo so that it doesn't happen > again. > > On Sun, 17 May 2020 at 21:01, Luis wrote: > > > Dont forget to downgrade the database connector or it will not work > > > > Sent from Yahoo Mail on Android > > > > On Sun, May 17, 2020 at 3:47 PM, David Merrill< > david.merr...@otelco.com> > > wrote: Hi All, > > > > I've got a CloudStack 4.11.2 lab running on a single host & made the > > (dumb) mistake of running OS updates without updating the system VMs > > beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now > > CloudStack management services wont start (see below). > > > > Here's my question: > > > > - Is it *at all* possible to rectify this (now, post package updates) by > > getting the new system templates in via the CLI tools > > > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > > - The error on the logs *seems* simple, if management could find the new > > template maybe things could proceed? > > > > My impression is no given research I've done so far which seems to amount > > to: > > > > - roll back to 4.11.2 > > - use the pre-upgrade dump of the DB (which I made) > > - start management services > > - get the new system VMs properly (like I should have in the first > place) > > - upgrade to 4.11.3 > > > > Humbly (& slightly embarrassed) yours, and thanks, > > David > > > > David Merrill > > Senior Systems Engineer, > > Managed and
Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
OK, got it, digging in... Thanks everyone, David David Merrill Senior Systems Engineer, Managed and Private/Hybrid Cloud Services OTELCO 92 Oak Street, Portland ME 04101 office 207.772.5678 http://www.otelco.com/cloud-and-managed-services Confidentiality Message The information contained in this e-mail transmission may be confidential and legally privileged. If you are not the intended recipient, you are notified that any dissemination, distribution, copying or other use of this information, including attachments, is prohibited. If you received this message in error, please call me at 207.772.5678 so this error can be corrected. On 5/18/20, 7:44 AM, "Andrija Panic" wrote: David, the procedure you laid out is correct - rollback DB, downgrade, etc As Luis mentioned, make sure to also rollback the "mysql-connector-java" in case it was upgraded (if you left the mysql repo enabled during the upgrade). There are ways to hack the DB, vm_template table and also use the "cloud-install-sys-tmplt"... but a clean rollback (as it's very easy in your test env) is a much better way to proceed. Cheers Andrija On Mon, 18 May 2020 at 10:06, Richard Lawley wrote: > The database upgrade does not happen unless the systemVM templates have > been added, so nothing non-reversible has happened yet. You can just use > yum to downgrade to 4.11.2 and you'll be fine (we've also accidentally done > this at some point!). > > I'd recommend disabling your cloudstack yum repo so that it doesn't happen > again. > > On Sun, 17 May 2020 at 21:01, Luis wrote: > > > Dont forget to downgrade the database connector or it will not work > > > > Sent from Yahoo Mail on Android > > > > On Sun, May 17, 2020 at 3:47 PM, David Merrill< > david.merr...@otelco.com> > > wrote: Hi All, > > > > I've got a CloudStack 4.11.2 lab running on a single host & made the > > (dumb) mistake of running OS updates without updating the system VMs > > beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now > > CloudStack management services wont start (see below). > > > > Here's my question: > > > > - Is it *at all* possible to rectify this (now, post package updates) by > > getting the new system templates in via the CLI tools > > > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > > - The error on the logs *seems* simple, if management could find the new > > template maybe things could proceed? > > > > My impression is no given research I've done so far which seems to amount > > to: > > > > - roll back to 4.11.2 > > - use the pre-upgrade dump of the DB (which I made) > > - start management services > > - get the new system VMs properly (like I should have in the first > place) > > - upgrade to 4.11.3 > > > > Humbly (& slightly embarrassed) yours, and thanks, > > David > > > > David Merrill > > Senior Systems Engineer, > > Managed and Private/Hybrid Cloud Services > > OTELCO > > 92 Oak Street, Portland ME 04101 > > office 207.772.5678 > > http://www.otelco.com/cloud-and-managed-services > > > > Confidentiality Message > > The information contained in this e-mail transmission may be confidential > > and legally privileged. If you are not the intended recipient, you are > > notified that any dissemination, distribution, copying or other use of > this > > information, including attachments, is prohibited. If you received this > > message in error, please call me at 207.772.5678 so > > this error can be corrected. > > > > > > On 5/17/20, 8:05 AM, "engineer...@reliablenetworks.com" < > > engineer...@reliablenetworks.com> wrote: > > > > 2020-05-17 07:35:15,659 INFO [o.e.j.s.AbstractConnector] > > (Thread-1:null) (logid:) Stopped ServerConnector@27808f31 > > {HTTP/1.1,[http/1.1]}{0.0.0.0:8080} > > 2020-05-17 07:35:15,664 INFO [o.e.j.s.session] (Thread-1:null) > > (logid:) Stopped scavenging > > 2020-05-17 07:35:15,665 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.s.h.MovedContextHandler@d6da883 > > {/,null,UNAVAILABLE} > > 2020-05-17 07:35:15,669 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.w.WebAppContext@7bc1a03d > > {/client,null,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} > > 2020-05-17 07:35:21,648 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Server configuration file found: > > /etc/cloudstack/management/server.properties > > 2020-05-17 07:35:21,655 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Initializing server daemon on null:8080, with > https.enabled=false, > >
Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
David, the procedure you laid out is correct - rollback DB, downgrade, etc As Luis mentioned, make sure to also rollback the "mysql-connector-java" in case it was upgraded (if you left the mysql repo enabled during the upgrade). There are ways to hack the DB, vm_template table and also use the "cloud-install-sys-tmplt"... but a clean rollback (as it's very easy in your test env) is a much better way to proceed. Cheers Andrija On Mon, 18 May 2020 at 10:06, Richard Lawley wrote: > The database upgrade does not happen unless the systemVM templates have > been added, so nothing non-reversible has happened yet. You can just use > yum to downgrade to 4.11.2 and you'll be fine (we've also accidentally done > this at some point!). > > I'd recommend disabling your cloudstack yum repo so that it doesn't happen > again. > > On Sun, 17 May 2020 at 21:01, Luis wrote: > > > Dont forget to downgrade the database connector or it will not work > > > > Sent from Yahoo Mail on Android > > > > On Sun, May 17, 2020 at 3:47 PM, David Merrill< > david.merr...@otelco.com> > > wrote: Hi All, > > > > I've got a CloudStack 4.11.2 lab running on a single host & made the > > (dumb) mistake of running OS updates without updating the system VMs > > beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now > > CloudStack management services wont start (see below). > > > > Here's my question: > > > > - Is it *at all* possible to rectify this (now, post package updates) by > > getting the new system templates in via the CLI tools > > > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > > - The error on the logs *seems* simple, if management could find the new > > template maybe things could proceed? > > > > My impression is no given research I've done so far which seems to amount > > to: > > > > - roll back to 4.11.2 > > - use the pre-upgrade dump of the DB (which I made) > > - start management services > > - get the new system VMs properly (like I should have in the first > place) > > - upgrade to 4.11.3 > > > > Humbly (& slightly embarrassed) yours, and thanks, > > David > > > > David Merrill > > Senior Systems Engineer, > > Managed and Private/Hybrid Cloud Services > > OTELCO > > 92 Oak Street, Portland ME 04101 > > office 207.772.5678 > > http://www.otelco.com/cloud-and-managed-services > > > > Confidentiality Message > > The information contained in this e-mail transmission may be confidential > > and legally privileged. If you are not the intended recipient, you are > > notified that any dissemination, distribution, copying or other use of > this > > information, including attachments, is prohibited. If you received this > > message in error, please call me at 207.772.5678 so > > this error can be corrected. > > > > > > On 5/17/20, 8:05 AM, "engineer...@reliablenetworks.com" < > > engineer...@reliablenetworks.com> wrote: > > > > 2020-05-17 07:35:15,659 INFO [o.e.j.s.AbstractConnector] > > (Thread-1:null) (logid:) Stopped ServerConnector@27808f31 > > {HTTP/1.1,[http/1.1]}{0.0.0.0:8080} > > 2020-05-17 07:35:15,664 INFO [o.e.j.s.session] (Thread-1:null) > > (logid:) Stopped scavenging > > 2020-05-17 07:35:15,665 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.s.h.MovedContextHandler@d6da883 > > {/,null,UNAVAILABLE} > > 2020-05-17 07:35:15,669 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.w.WebAppContext@7bc1a03d > > {/client,null,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} > > 2020-05-17 07:35:21,648 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Server configuration file found: > > /etc/cloudstack/management/server.properties > > 2020-05-17 07:35:21,655 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Initializing server daemon on null:8080, with > https.enabled=false, > > https.port=8443, context.path=/client > > 2020-05-17 07:35:21,666 INFO [o.e.j.u.log] (main:null) (logid:) > > Logging initialized @713ms to org.eclipse.jetty.util.log.Slf4jLog > > 2020-05-17 07:35:21,857 INFO [o.e.j.s.Server] (main:null) (logid:) > > jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T16:27:37-05:00, git > hash: > > 82b8fb23f757335bb3329d540ce37a2a2615f0a8 > > 2020-05-17 07:35:21,893 INFO [o.e.j.s.AbstractNCSARequestLog] > > (main:null) (logid:) Opened /var/log/cloudstack/management/access.log > > 2020-05-17 07:35:22,007 INFO [o.e.j.w.StandardDescriptorProcessor] > > (main:null) (logid:) NO JSP Support for /client, did not find > > org.eclipse.jetty.jsp.JettyJspServlet > > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > > DefaultSessionIdManager workerName=node0 > > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > > No SessionScavenger set, using defaults > > 2020-05-17 07:35:22,044 INFO [o.e.j.s.session] (main:null) (logid:) > > Scavenging every 60ms > > 2020-05-17 07:35:37,117 INFO > >
Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
The database upgrade does not happen unless the systemVM templates have been added, so nothing non-reversible has happened yet. You can just use yum to downgrade to 4.11.2 and you'll be fine (we've also accidentally done this at some point!). I'd recommend disabling your cloudstack yum repo so that it doesn't happen again. On Sun, 17 May 2020 at 21:01, Luis wrote: > Dont forget to downgrade the database connector or it will not work > > Sent from Yahoo Mail on Android > > On Sun, May 17, 2020 at 3:47 PM, David Merrill > wrote: Hi All, > > I've got a CloudStack 4.11.2 lab running on a single host & made the > (dumb) mistake of running OS updates without updating the system VMs > beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now > CloudStack management services wont start (see below). > > Here's my question: > > - Is it *at all* possible to rectify this (now, post package updates) by > getting the new system templates in via the CLI tools > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > - The error on the logs *seems* simple, if management could find the new > template maybe things could proceed? > > My impression is no given research I've done so far which seems to amount > to: > > - roll back to 4.11.2 > - use the pre-upgrade dump of the DB (which I made) > - start management services > - get the new system VMs properly (like I should have in the first place) > - upgrade to 4.11.3 > > Humbly (& slightly embarrassed) yours, and thanks, > David > > David Merrill > Senior Systems Engineer, > Managed and Private/Hybrid Cloud Services > OTELCO > 92 Oak Street, Portland ME 04101 > office 207.772.5678 > http://www.otelco.com/cloud-and-managed-services > > Confidentiality Message > The information contained in this e-mail transmission may be confidential > and legally privileged. If you are not the intended recipient, you are > notified that any dissemination, distribution, copying or other use of this > information, including attachments, is prohibited. If you received this > message in error, please call me at 207.772.5678 so > this error can be corrected. > > > On 5/17/20, 8:05 AM, "engineer...@reliablenetworks.com" < > engineer...@reliablenetworks.com> wrote: > > 2020-05-17 07:35:15,659 INFO [o.e.j.s.AbstractConnector] > (Thread-1:null) (logid:) Stopped ServerConnector@27808f31 > {HTTP/1.1,[http/1.1]}{0.0.0.0:8080} > 2020-05-17 07:35:15,664 INFO [o.e.j.s.session] (Thread-1:null) > (logid:) Stopped scavenging > 2020-05-17 07:35:15,665 INFO [o.e.j.s.h.ContextHandler] > (Thread-1:null) (logid:) Stopped o.e.j.s.h.MovedContextHandler@d6da883 > {/,null,UNAVAILABLE} > 2020-05-17 07:35:15,669 INFO [o.e.j.s.h.ContextHandler] > (Thread-1:null) (logid:) Stopped o.e.j.w.WebAppContext@7bc1a03d > {/client,null,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} > 2020-05-17 07:35:21,648 INFO [o.a.c.ServerDaemon] (main:null) > (logid:) Server configuration file found: > /etc/cloudstack/management/server.properties > 2020-05-17 07:35:21,655 INFO [o.a.c.ServerDaemon] (main:null) > (logid:) Initializing server daemon on null:8080, with https.enabled=false, > https.port=8443, context.path=/client > 2020-05-17 07:35:21,666 INFO [o.e.j.u.log] (main:null) (logid:) > Logging initialized @713ms to org.eclipse.jetty.util.log.Slf4jLog > 2020-05-17 07:35:21,857 INFO [o.e.j.s.Server] (main:null) (logid:) > jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T16:27:37-05:00, git hash: > 82b8fb23f757335bb3329d540ce37a2a2615f0a8 > 2020-05-17 07:35:21,893 INFO [o.e.j.s.AbstractNCSARequestLog] > (main:null) (logid:) Opened /var/log/cloudstack/management/access.log > 2020-05-17 07:35:22,007 INFO [o.e.j.w.StandardDescriptorProcessor] > (main:null) (logid:) NO JSP Support for /client, did not find > org.eclipse.jetty.jsp.JettyJspServlet > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > DefaultSessionIdManager workerName=node0 > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > No SessionScavenger set, using defaults > 2020-05-17 07:35:22,044 INFO [o.e.j.s.session] (main:null) (logid:) > Scavenging every 60ms > 2020-05-17 07:35:37,117 INFO > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > module context [bootstrap] from URL > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context.xml] > 2020-05-17 07:35:37,118 INFO > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > module context [bootstrap] from URL > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml] > 2020-05-17 07:35:37,298 DEBUG [c.c.u.c.EncryptionSecretKeyChecker] > (main:null) (logid:) Encryption Type: file > 2020-05-17 07:35:37,342 INFO [c.c.u.LogUtils] (main:null) (logid:)
Re: CloudStack - Ubuntu/KVM (all in one management-server/host) - OS Upgrade w/o updating System VMs
Dont forget to downgrade the database connector or it will not work Sent from Yahoo Mail on Android On Sun, May 17, 2020 at 3:47 PM, David Merrill wrote: Hi All, I've got a CloudStack 4.11.2 lab running on a single host & made the (dumb) mistake of running OS updates without updating the system VMs beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now CloudStack management services wont start (see below). Here's my question: - Is it *at all* possible to rectify this (now, post package updates) by getting the new system templates in via the CLI tools (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? - The error on the logs *seems* simple, if management could find the new template maybe things could proceed? My impression is no given research I've done so far which seems to amount to: - roll back to 4.11.2 - use the pre-upgrade dump of the DB (which I made) - start management services - get the new system VMs properly (like I should have in the first place) - upgrade to 4.11.3 Humbly (& slightly embarrassed) yours, and thanks, David David Merrill Senior Systems Engineer, Managed and Private/Hybrid Cloud Services OTELCO 92 Oak Street, Portland ME 04101 office 207.772.5678 http://www.otelco.com/cloud-and-managed-services Confidentiality Message The information contained in this e-mail transmission may be confidential and legally privileged. If you are not the intended recipient, you are notified that any dissemination, distribution, copying or other use of this information, including attachments, is prohibited. If you received this message in error, please call me at 207.772.5678 so this error can be corrected. On 5/17/20, 8:05 AM, "engineer...@reliablenetworks.com" wrote: 2020-05-17 07:35:15,659 INFO [o.e.j.s.AbstractConnector] (Thread-1:null) (logid:) Stopped ServerConnector@27808f31{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} 2020-05-17 07:35:15,664 INFO [o.e.j.s.session] (Thread-1:null) (logid:) Stopped scavenging 2020-05-17 07:35:15,665 INFO [o.e.j.s.h.ContextHandler] (Thread-1:null) (logid:) Stopped o.e.j.s.h.MovedContextHandler@d6da883{/,null,UNAVAILABLE} 2020-05-17 07:35:15,669 INFO [o.e.j.s.h.ContextHandler] (Thread-1:null) (logid:) Stopped o.e.j.w.WebAppContext@7bc1a03d{/client,null,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} 2020-05-17 07:35:21,648 INFO [o.a.c.ServerDaemon] (main:null) (logid:) Server configuration file found: /etc/cloudstack/management/server.properties 2020-05-17 07:35:21,655 INFO [o.a.c.ServerDaemon] (main:null) (logid:) Initializing server daemon on null:8080, with https.enabled=false, https.port=8443, context.path=/client 2020-05-17 07:35:21,666 INFO [o.e.j.u.log] (main:null) (logid:) Logging initialized @713ms to org.eclipse.jetty.util.log.Slf4jLog 2020-05-17 07:35:21,857 INFO [o.e.j.s.Server] (main:null) (logid:) jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8 2020-05-17 07:35:21,893 INFO [o.e.j.s.AbstractNCSARequestLog] (main:null) (logid:) Opened /var/log/cloudstack/management/access.log 2020-05-17 07:35:22,007 INFO [o.e.j.w.StandardDescriptorProcessor] (main:null) (logid:) NO JSP Support for /client, did not find org.eclipse.jetty.jsp.JettyJspServlet 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) DefaultSessionIdManager workerName=node0 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) No SessionScavenger set, using defaults 2020-05-17 07:35:22,044 INFO [o.e.j.s.session] (main:null) (logid:) Scavenging every 60ms 2020-05-17 07:35:37,117 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [bootstrap] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context.xml] 2020-05-17 07:35:37,118 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading module context [bootstrap] from URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml] 2020-05-17 07:35:37,298 DEBUG [c.c.u.c.EncryptionSecretKeyChecker] (main:null) (logid:) Encryption Type: file 2020-05-17 07:35:37,342 INFO [c.c.u.LogUtils] (main:null) (logid:) log4j configuration found at /etc/cloudstack/management/log4j-cloud.xml 2020-05-17 07:35:37,367 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded module context [bootstrap] in 250 ms 2020-05-17 07:35:37,368 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module Hierarchy: bootstrap 2020-05-17 07:35:37,368 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module Hierarchy: system 2020-05-17 07:35:37,368 INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet]