David, the procedure you laid out is correct - rollback DB, downgrade, etc As Luis mentioned, make sure to also rollback the "mysql-connector-java" in case it was upgraded (if you left the mysql repo enabled during the upgrade).
There are ways to hack the DB, vm_template table and also use the "cloud-install-sys-tmplt"... but a clean rollback (as it's very easy in your test env) is a much better way to proceed. Cheers Andrija On Mon, 18 May 2020 at 10:06, Richard Lawley <rich...@richardlawley.com> wrote: > The database upgrade does not happen unless the systemVM templates have > been added, so nothing non-reversible has happened yet. You can just use > yum to downgrade to 4.11.2 and you'll be fine (we've also accidentally done > this at some point!). > > I'd recommend disabling your cloudstack yum repo so that it doesn't happen > again. > > On Sun, 17 May 2020 at 21:01, Luis <lmartinez...@yahoo.com.invalid> wrote: > > > Dont forget to downgrade the database connector or it will not work > > > > Sent from Yahoo Mail on Android > > > > On Sun, May 17, 2020 at 3:47 PM, David Merrill< > david.merr...@otelco.com> > > wrote: Hi All, > > > > I've got a CloudStack 4.11.2 lab running on a single host & made the > > (dumb) mistake of running OS updates without updating the system VMs > > beforehand. The CloudStack packages upgraded to 4.11.3 just fine but now > > CloudStack management services wont start (see below). > > > > Here's my question: > > > > - Is it *at all* possible to rectify this (now, post package updates) by > > getting the new system templates in via the CLI tools > > > (/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt)? > > - The error on the logs *seems* simple, if management could find the new > > template maybe things could proceed? > > > > My impression is no given research I've done so far which seems to amount > > to: > > > > - roll back to 4.11.2 > > - use the pre-upgrade dump of the DB (which I made) > > - start management services > > - get the new system VMs properly (like I should have in the first > place) > > - upgrade to 4.11.3 > > > > Humbly (& slightly embarrassed) yours, and thanks, > > David > > > > David Merrill > > Senior Systems Engineer, > > Managed and Private/Hybrid Cloud Services > > OTELCO > > 92 Oak Street, Portland ME 04101 > > office 207.772.5678 <callto:207.772.5678> > > http://www.otelco.com/cloud-and-managed-services > > > > Confidentiality Message > > The information contained in this e-mail transmission may be confidential > > and legally privileged. If you are not the intended recipient, you are > > notified that any dissemination, distribution, copying or other use of > this > > information, including attachments, is prohibited. If you received this > > message in error, please call me at 207.772.5678 <callto:207.772.5678> so > > this error can be corrected. > > > > > > On 5/17/20, 8:05 AM, "engineer...@reliablenetworks.com" < > > engineer...@reliablenetworks.com> wrote: > > > > 2020-05-17 07:35:15,659 INFO [o.e.j.s.AbstractConnector] > > (Thread-1:null) (logid:) Stopped ServerConnector@27808f31 > > {HTTP/1.1,[http/1.1]}{0.0.0.0:8080} > > 2020-05-17 07:35:15,664 INFO [o.e.j.s.session] (Thread-1:null) > > (logid:) Stopped scavenging > > 2020-05-17 07:35:15,665 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.s.h.MovedContextHandler@d6da883 > > {/,null,UNAVAILABLE} > > 2020-05-17 07:35:15,669 INFO [o.e.j.s.h.ContextHandler] > > (Thread-1:null) (logid:) Stopped o.e.j.w.WebAppContext@7bc1a03d > > {/client,null,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} > > 2020-05-17 07:35:21,648 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Server configuration file found: > > /etc/cloudstack/management/server.properties > > 2020-05-17 07:35:21,655 INFO [o.a.c.ServerDaemon] (main:null) > > (logid:) Initializing server daemon on null:8080, with > https.enabled=false, > > https.port=8443, context.path=/client > > 2020-05-17 07:35:21,666 INFO [o.e.j.u.log] (main:null) (logid:) > > Logging initialized @713ms to org.eclipse.jetty.util.log.Slf4jLog > > 2020-05-17 07:35:21,857 INFO [o.e.j.s.Server] (main:null) (logid:) > > jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T16:27:37-05:00, git > hash: > > 82b8fb23f757335bb3329d540ce37a2a2615f0a8 > > 2020-05-17 07:35:21,893 INFO [o.e.j.s.AbstractNCSARequestLog] > > (main:null) (logid:) Opened /var/log/cloudstack/management/access.log > > 2020-05-17 07:35:22,007 INFO [o.e.j.w.StandardDescriptorProcessor] > > (main:null) (logid:) NO JSP Support for /client, did not find > > org.eclipse.jetty.jsp.JettyJspServlet > > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > > DefaultSessionIdManager workerName=node0 > > 2020-05-17 07:35:22,042 INFO [o.e.j.s.session] (main:null) (logid:) > > No SessionScavenger set, using defaults > > 2020-05-17 07:35:22,044 INFO [o.e.j.s.session] (main:null) (logid:) > > Scavenging every 600000ms > > 2020-05-17 07:35:37,117 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [bootstrap] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context.xml] > > 2020-05-17 07:35:37,118 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [bootstrap] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml] > > 2020-05-17 07:35:37,298 DEBUG [c.c.u.c.EncryptionSecretKeyChecker] > > (main:null) (logid:) Encryption Type: file > > 2020-05-17 07:35:37,342 INFO [c.c.u.LogUtils] (main:null) (logid:) > > log4j configuration found at /etc/cloudstack/management/log4j-cloud.xml > > 2020-05-17 07:35:37,367 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loaded > > module context [bootstrap] in 250 ms > > 2020-05-17 07:35:37,368 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: bootstrap > > 2020-05-17 07:35:37,368 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: system > > 2020-05-17 07:35:37,368 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: core > > 2020-05-17 07:35:37,369 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: allocator > > 2020-05-17 07:35:37,369 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: host-allocator-random > > 2020-05-17 07:35:37,369 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: planner > > 2020-05-17 07:35:37,369 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: api-planner > > 2020-05-17 07:35:37,369 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: baremetal-planner > > 2020-05-17 07:35:37,370 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: explicit-dedication > > 2020-05-17 07:35:37,370 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: host-affinity > > 2020-05-17 07:35:37,370 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: host-anti-affinity > > 2020-05-17 07:35:37,370 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: implicit-dedication > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-planner > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: skip-heurestics > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: user-concentrated-pod > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-allocator > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: api > > 2020-05-17 07:35:37,371 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: acl-dynamic-role-based > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: acl-static-role-based > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: cloudian > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ldap > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: md5 > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: metrics > > 2020-05-17 07:35:37,372 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: pbkdf2 > > 2020-05-17 07:35:37,373 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: plaintext > > 2020-05-17 07:35:37,373 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: prometheus > > 2020-05-17 07:35:37,373 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: quota > > 2020-05-17 07:35:37,374 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: rate-limit > > 2020-05-17 07:35:37,374 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: saml2 > > 2020-05-17 07:35:37,374 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-api > > 2020-05-17 07:35:37,374 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: sha256salted > > 2020-05-17 07:35:37,374 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: solidfire-intg-test > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vmware-sioc > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: api-config > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: backend > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ca > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: root-ca > > 2020-05-17 07:35:37,375 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: compute > > 2020-05-17 07:35:37,376 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: baremetal-compute > > 2020-05-17 07:35:37,376 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: hyperv-compute > > 2020-05-17 07:35:37,376 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: kvm-compute > > 2020-05-17 07:35:37,376 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ovm-compute > > 2020-05-17 07:35:37,376 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ovm3-compute > > 2020-05-17 07:35:37,377 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-alert-adapter-compute > > 2020-05-17 07:35:37,377 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-compute > > 2020-05-17 07:35:37,377 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-fencer > > 2020-05-17 07:35:37,377 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-investigator > > 2020-05-17 07:35:37,378 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vmware-compute > > 2020-05-17 07:35:37,378 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: xenserver-compute > > 2020-05-17 07:35:37,378 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: direct-download > > 2020-05-17 07:35:37,378 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: network > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: baremetal-network > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: bigswitch > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: cisco-vnmc > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: contrail > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: elb > > 2020-05-17 07:35:37,379 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: f5 > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: globodns > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: netscaler > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: nvp > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: opendaylight > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ovs > > 2020-05-17 07:35:37,380 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: paloalto > > 2020-05-17 07:35:37,381 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-network > > 2020-05-17 07:35:37,381 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: srx > > 2020-05-17 07:35:37,381 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ssp > > 2020-05-17 07:35:37,381 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vcs > > 2020-05-17 07:35:37,381 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vmware-network > > 2020-05-17 07:35:37,382 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vsp > > 2020-05-17 07:35:37,382 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vxlan > > 2020-05-17 07:35:37,382 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-alert-adapter-backend > > 2020-05-17 07:35:37,382 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage > > 2020-05-17 07:35:37,382 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: baremetal-storage > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-alert-adapter-storage > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-storage > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-template-adapter > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-allocator > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-image-default > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-image-s3 > > 2020-05-17 07:35:37,383 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-image-swift > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-volume-cloudbyte > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-volume-default > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: storage-volume-solidfire > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vmware-storage > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: discoverer > > 2020-05-17 07:35:37,384 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: baremetal-discoverer > > 2020-05-17 07:35:37,385 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: hyperv-discoverer > > 2020-05-17 07:35:37,385 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ovm-discoverer > > 2020-05-17 07:35:37,385 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ovm3-discoverer > > 2020-05-17 07:35:37,385 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: secondary-storage-discoverer > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: server-discoverer > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: vmware-discoverer > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: xenserver-discoverer > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: outofbandmanagement > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: ipmitool > > 2020-05-17 07:35:37,386 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Module > > Hierarchy: nested-cloudstack > > 2020-05-17 07:35:37,387 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-framework-config-system-context.xml] > > 2020-05-17 07:35:37,387 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-server-system-context.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-core-system-context.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-framework-db-system-context.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-engine-schema-system-checkers-context.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-framework-config-system-context-inheritable.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-core-system-context-inheritable.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/system/spring-contrail-system-context-inheritable.xml] > > 2020-05-17 07:35:37,388 INFO > > [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) (logid:) Loading > > module context [system] from URL > > > [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml] > > 2020-05-17 07:35:38,357 INFO [c.c.u.d.T.Transaction] (main:null) > > (logid:) Is Data Base High Availiability enabled? Ans : false > > 2020-05-17 07:35:38,428 DEBUG [c.c.u.d.DriverLoader] (main:null) > > (logid:) Successfully loaded DB driver com.mysql.jdbc.Driver > > 2020-05-17 07:35:38,447 DEBUG [c.c.u.d.DriverLoader] (main:null) > > (logid:) DB driver com.mysql.jdbc.Driver was already loaded. > > 2020-05-17 07:35:38,449 DEBUG [c.c.u.d.DriverLoader] (main:null) > > (logid:) DB driver com.mysql.jdbc.Driver was already loaded. > > 2020-05-17 07:35:39,037 DEBUG [c.c.u.d.ConnectionConcierge] > > (main:null) (logid:) Registering a database connection for LockMaster1 > > 2020-05-17 07:35:39,037 INFO [c.c.u.d.Merovingian2] (main:null) > > (logid:) Cleaning up locks for 121112988680429 > > 2020-05-17 07:35:39,040 INFO [c.c.u.d.Merovingian2] (main:null) > > (logid:) Released 0 locks for 121112988680429 > > 2020-05-17 07:35:39,387 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] > > (main:null) (logid:) Running system integrity checker > > com.cloud.upgrade.DatabaseUpgradeChecker@3935e9a8 > > 2020-05-17 07:35:39,389 INFO [c.c.u.DatabaseUpgradeChecker] > > (main:null) (logid:) Grabbing lock to check for database upgrade. > > 2020-05-17 07:35:39,491 DEBUG [c.c.u.d.VersionDaoImpl] (main:null) > > (logid:) Checking to see if the database is at a version before it was > the > > version table is created > > 2020-05-17 07:35:39,530 INFO [c.c.u.DatabaseUpgradeChecker] > > (main:null) (logid:) DB version = 4.11.2.0 Code Version = 4.11.3.0 > > 2020-05-17 07:35:39,531 INFO [c.c.u.DatabaseUpgradeChecker] > > (main:null) (logid:) Database upgrade must be performed from 4.11.2.0 to > > 4.11.3.0 > > 2020-05-17 07:35:39,532 DEBUG [c.c.u.DatabaseUpgradeChecker] > > (main:null) (logid:) Running upgrade Upgrade41120to41130 to upgrade from > > 4.11.2.0-4.11.3.0 to 4.11.3.0 > > 2020-05-17 07:35:39,534 DEBUG [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) Updating System Vm template IDs > > 2020-05-17 07:35:39,538 DEBUG [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) Updating LXC System Vms > > 2020-05-17 07:35:39,540 WARN [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) 4.11.3.0LXC SystemVm template not found. LXC > > hypervisor is not used, so not failing upgrade > > 2020-05-17 07:35:39,543 DEBUG [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) Updating Ovm3 System Vms > > 2020-05-17 07:35:39,544 WARN [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) 4.11.3.0Ovm3 SystemVm template not found. Ovm3 > > hypervisor is not used, so not failing upgrade > > 2020-05-17 07:35:39,545 DEBUG [c.c.u.d.Upgrade41120to41130] > > (main:null) (logid:) Updating KVM System Vms > > 2020-05-17 07:35:39,547 ERROR [c.c.u.DatabaseUpgradeChecker] > > (main:null) (logid:) Unable to upgrade the database > > com.cloud.utils.exception.CloudRuntimeException: 4.11.3.0KVM SystemVm > > template not found. Cannot upgrade system Vms > > at > > > com.cloud.upgrade.dao.Upgrade41120to41130.updateSystemVmTemplates(Upgrade41120to41130.java:203) > > at > > > com.cloud.upgrade.dao.Upgrade41120to41130.performDataMigration(Upgrade41120to41130.java:60) > > at > > > com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:586) > > at > > > com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:666) > > at > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65) > > at > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:55) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:183) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:52) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:358) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:159) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) > > at > > > org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:884) > > at > > > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > > at > > > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62) > > at > > > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:890) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:853) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344) > > at > > > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1515) > > at > > > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1477) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:785) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261) > > at > > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:273) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:115) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at org.eclipse.jetty.server.Server.start(Server.java:418) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at org.eclipse.jetty.server.Server.doStart(Server.java:385) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:200) > > at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:101) > > 2020-05-17 07:35:39,550 DEBUG [c.c.u.d.T.Transaction] (main:null) > > (logid:) Rolling back the transaction: Time = 18 Name = Upgrade; called > by > > > -TransactionLegacy.rollback:890-TransactionLegacy.removeUpTo:833-TransactionLegacy.close:657-DatabaseUpgradeChecker.upgrade:597-DatabaseUpgradeChecker.check:666-CloudStackExtendedLifeCycle.checkIntegrity:65-CloudStackExtendedLifeCycle.start:55-DefaultLifecycleProcessor.doStart:183-DefaultLifecycleProcessor.access$200:52-DefaultLifecycleProcessor$LifecycleGroup.start:358-DefaultLifecycleProcessor.startBeans:159-DefaultLifecycleProcessor.onRefresh:123 > > 2020-05-17 07:35:39,604 WARN > [o.a.c.s.m.c.ResourceApplicationContext] > > (main:null) (logid:) Exception encountered during context initialization > - > > cancelling refresh attempt: > > org.springframework.context.ApplicationContextException: Failed to start > > bean 'cloudStackLifeCycle'; nested exception is > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the > > database > > 2020-05-17 07:35:39,605 WARN [o.e.j.w.WebAppContext] (main:null) > > (logid:) Failed startup of context o.e.j.w.WebAppContext@7bc1a03d > > > {/client,file:///usr/share/cloudstack-management/webapp/,UNAVAILABLE}{/usr/share/cloudstack-management/webapp} > > org.springframework.context.ApplicationContextException: Failed to > > start bean 'cloudStackLifeCycle'; nested exception is > > com.cloud.utils.exception.CloudRuntimeException: Unable to upgrade the > > database > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:186) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:52) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:358) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:159) > > at > > > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) > > at > > > org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:884) > > at > > > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117) > > at > > > org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > > at > > > org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58) > > at > > > org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62) > > at > > > org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:890) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:853) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344) > > at > > > org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1515) > > at > > > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1477) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:785) > > at > > > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261) > > at > > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:273) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:115) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133) > > at org.eclipse.jetty.server.Server.start(Server.java:418) > > at > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107) > > at > > > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) > > at org.eclipse.jetty.server.Server.doStart(Server.java:385) > > at > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) > > at > org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:200) > > at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:101) > > Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to > > upgrade the database > > at > > > com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:595) > > at > > > com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:666) > > at > > > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65) > > ... 47 more > > Caused by: com.cloud.utils.exception.CloudRuntimeException: > > 4.11.3.0KVM SystemVm template not found. Cannot upgrade system Vms > > at > > > com.cloud.upgrade.dao.Upgrade41120to41130.updateSystemVmTemplates(Upgrade41120to41130.java:203) > > at > > > com.cloud.upgrade.dao.Upgrade41120to41130.performDataMigration(Upgrade41120to41130.java:60) > > at > > > com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:586) > > ... 49 more > > 2020-05-17 07:35:39,609 INFO [o.e.j.s.h.ContextHandler] (main:null) > > (logid:) Started o.e.j.s.h.MovedContextHandler@d6da883{/,null,AVAILABLE} > > 2020-05-17 07:35:39,658 INFO [o.e.j.s.AbstractConnector] (main:null) > > (logid:) Started ServerConnector@27808f31{HTTP/1.1,[http/1.1]}{ > > 0.0.0.0:8080} > > 2020-05-17 07:35:39,659 INFO [o.e.j.s.Server] (main:null) (logid:) > > Started @18708ms > > > > > > > > > -- Andrija Panić
