Hi friends, I'm having a very strange upgrade issue which I was wondering if anyone has encountered and whether someone can catch what is wrong. Basically I follow the steps outlined in the docs to the letter. When I restart the MS service it gets stuck at running the DB upgrade scripts. I checked catalina.out and noticed that the schema-430to440.sql script is trying to create tables that already exist or insert columns that already exit.
I commented out all the lines causing the issue and then restart the MS service but then I get a new error about the Listener. After Googling this error the only hints I got were about system templates but I made sure the new system template was installed before attempting the upgrade. I've repeated this no less than five time with the very same results. I'm miffed. Below is a snippet of the catalina.out log with the error. Can anyone spot the issue? TIA Osay ====== SNIP==== INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-core-4.5.2.jar!/META-INF/cloudstack/system/spring-core-syst em-context.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-engine-schema-4.5.2.jar!/META-INF/cloudstack/system/spring- engine-schema-system-checkers-context.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-framework-config-4.5.2.jar!/META-INF/cloudstack/system/spri ng-framework-config-system-context.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-framework-db-4.5.2.jar!/META-INF/cloudstack/system/spring-f ramework-db-system-context.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-server-4.5.2.jar!/META-INF/cloudstack/system/spring-server- system-context.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-core-4.5.2.jar!/META-INF/cloudstack/system/spring-core-syst em-context-inheritable.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-framework-config-4.5.2.jar!/META-INF/cloudstack/system/spri ng-framework-config-system-context-inheritable.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-network-contrail-4.5.2.jar!/META-INF/cloudstack/syst em/spring-contrail-system-context-inheritable.xml] INFO [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:null) Loading module context [system] from URL [jar:file:/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-core-4.5.2.jar!/META-INF/cloudstack/bootstrap/spring-bootst rap-context-inheritable.xml] INFO [c.c.u.d.T.Transaction] (main:null) Is Data Base High Availiability enabled? Ans : false INFO [c.c.u.d.Merovingian2] (main:null) Cleaning up locks for 7063056740127 INFO [c.c.u.d.Merovingian2] (main:null) Released 0 locks for 7063056740127 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Running system integrity checker com.cloud.upgrade.DatabaseUpgradeChecker@7fa02e81 INFO [c.c.u.DatabaseUpgradeChecker] (main:null) Grabbing lock to check for database upgrade. INFO [c.c.u.DatabaseUpgradeChecker] (main:null) DB version = 4.3.0 Code Version = 4.5.2 INFO [c.c.u.DatabaseUpgradeChecker] (main:null) Database upgrade must be performed from 4.3.0 to 4.5.2 Sep 04, 2015 11:58:32 AM org.apache.catalina.core.StandardContext start SEVERE: Error listenerStart Sep 04, 2015 11:58:32 AM org.apache.catalina.core.StandardContext start SEVERE: Context [/client] startup failed due to previous errors Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: A web application appears to have started a thread named [FileWatchdog] but has failed to stop it. This is very likely to create a memory leak. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: A web application appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: A web application appears to have started a thread named [ConnectionConcierge-1] but has failed to stop it. This is very likely to create a memory leak. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web a pplication was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63ba710]) and a value of type [com.cloud.utils.db.TransactionLegacy] (value [ : ]) but failed to remove it when the we b application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web a pplication was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Current AOP method invocation]) and a value of type [null] (value [null]) but failed to remove it when the web applicati on was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Prototype beans currently in creation]) and a value of type [null] (value [null]) but failed to remove it when the web a pplication was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:32 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63ba710]) and a value of type [com.cloud.utils.db.TransactionLegacy] (value [ : ]) but failed to remove it when the we b application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Sep 04, 2015 11:58:33 AM org.apache.coyote.http11.Http11NioProtocol start INFO: Starting Coyote HTTP/1.1 on http-8080 Sep 04, 2015 11:58:33 AM org.apache.jk.common.ChannelSocket init INFO: JK: ajp13 listening on /0.0.0.0:20400 Sep 04, 2015 11:58:33 AM org.apache.jk.server.JkMain start INFO: Jk running ID=0 time=0/42 config=null Sep 04, 2015 11:58:33 AM org.apache.catalina.core.StandardService start INFO: Starting service Catalina7080 Sep 04, 2015 11:58:33 AM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/6.0.24 Sep 04, 2015 11:58:33 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory webapps Sep 04, 2015 11:58:33 AM org.apache.catalina.startup.HostConfig deployDirectory INFO: Deploying web application directory awsapi Sep 04, 2015 11:58:33 AM org.apache.catalina.loader.WebappClassLoader validateJarFile INFO: validateJarFile(/usr/share/cloudstack-management/webapps7080/awsapi/WEB-INF/lib/geronimo-servlet_3.0_spec-1.0.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/lib/slf4j-jdk14-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/share/cloudstack-bridge/webapps/awsapi/WEB-INF/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory] Sep 04, 2015 11:58:34 AM org.apache.catalina.core.ApplicationContext log INFO: Initializing Spring root WebApplicationContext Sep 04, 2015 11:58:34 AM org.springframework.web.context.ContextLoader initWebApplicationContext
