This is what I can see in localhost log:
cat /var/log/cloudstack/management/localhost.2018-05-21.log
[...]
SEVERE: Exception sending context initialized event to listener instance
of class
org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener
java.lang.NullPointerException
at
com.cloud.vm.UserVmManagerImpl.loadVmDetailsInMapForExternalDhcpIp(UserVmManagerImpl.java:2019)
at
com.cloud.vm.UserVmManagerImpl.start(UserVmManagerImpl.java:1999)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle$1.with(CloudStackExtendedLifeCycle.java:75)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.with(CloudStackExtendedLifeCycle.java:154)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.startBeans(CloudStackExtendedLifeCycle.java:72)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycleStart.run(CloudStackExtendedLifeCycleStart.java:46)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$1.with(DefaultModuleDefinitionSet.java:105)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.startContexts(DefaultModuleDefinitionSet.java:97)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:80)
at
org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58)
at
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62)
at
org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5118)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5634)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at
org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1260)
at
org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2002)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
at java.lang.Thread.run(Thread.java:748)
El 21/05/2018 a las 9:21, Natalia Costas Lago escribió:
It has been working with 1.8 for long...
I switched to java 1.7, but java errors persist.
We recovered backups from a couple of days prior to the disaster, but
we are still unlucky, it seems the problem is also in the backups copy.
I installed a fresh cloudstack and copied a recent database and the
manager still gives and Exception but to a later point in the execution:
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.l.CloudStackExtendedLifeCycle] (localhost-startStop-1:null)
(logid:) Configuring CloudStack Components
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.l.CloudStackExtendedLifeCycle] (localhost-startStop-1:null)
(logid:) Done Configuring CloudStack Components
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
(localhost-startStop-1:null) (logid:) Loaded module context [ipmitool]
in 171 ms
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
(localhost-startStop-1:null) (logid:) Starting module [system]
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.l.CloudStackExtendedLifeCycle] (localhost-startStop-1:null)
(logid:) Starting CloudStack Components
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.l.CloudStackExtendedLifeCycle] (localhost-startStop-1:null)
(logid:) Done Starting CloudStack Components
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.m.m.i.DefaultModuleDefinitionSet]
(localhost-startStop-1:null) (logid:) Starting module [core]
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.l.CloudStackExtendedLifeCycle] (localhost-startStop-1:null)
(logid:) Starting CloudStack Components
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.h.m.HypervManagerImpl] (localhost-startStop-1:null) (logid:)
Cleanup mounted mount points used in previous session
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.u.DatabaseIntegrityChecker] (localhost-startStop-1:null) (logid:)
Grabbing lock to check for database integrity.
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.u.DatabaseIntegrityChecker] (localhost-startStop-1:null) (logid:)
Performing database integrity check
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.c.ClusterManagerImpl] (localhost-startStop-1:null) (logid:)
Starting Cluster manager, msid : 2199193518866
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.c.ClusterServiceServletContainer] (Thread-10:null) (logid:)
Cluster service servlet container listening on port 9090
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.c.ClusterManagerImpl] (localhost-startStop-1:null) (logid:)
Management server 2199193518866, runId 1526887063086 is being started
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.c.ClusterManagerImpl] (localhost-startStop-1:null) (logid:)
Management server (host id : 2) is being started at 10.38.3.18:9090
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.c.ClusterManagerImpl] (localhost-startStop-1:null) (logid:)
Cluster manager was started successfully
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.HighAvailabilityManagerImpl] (HA-Worker-1:null) (logid:)
Starting work
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.HighAvailabilityManagerImpl] (HA-Worker-4:null) (logid:)
Starting work
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.HighAvailabilityManagerImpl] (HA-Worker-3:null) (logid:)
Starting work
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.HighAvailabilityManagerImpl] (HA-Worker-0:null) (logid:)
Starting work
May 21 09:19:53 cldmanager.srv.cesga.es server: INFO
[c.c.h.HighAvailabilityManagerImpl] (HA-Worker-2:null) (logid:)
Starting work
May 21 09:19:55 cldmanager.srv.cesga.es server: log4j:WARN No
appenders could be found for logger
(com.cloud.utils.db.ConnectionConcierge).
May 21 09:19:55 cldmanager.srv.cesga.es server: log4j:WARN Please
initialize the log4j system properly.
May 21 09:19:55 cldmanager.srv.cesga.es server: log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
May 21 09:19:55 cldmanager.srv.cesga.es server: Exception in thread
"Thread-10" java.lang.NoClassDefFoundError:
org/apache/log4j/spi/ThrowableInformation
May 21 09:19:55 cldmanager.srv.cesga.es server: at
org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:165)
May 21 09:19:55 cldmanager.srv.cesga.es server: at
org.apache.log4j.Category.forcedLog(Category.java:391)
May 21 09:19:55 cldmanager.srv.cesga.es server: at
org.apache.log4j.Category.error(Category.java:322)
May 21 09:19:55 cldmanager.srv.cesga.es server: at
com.cloud.cluster.ClusterServiceServletContainer$ListenerThread.run(ClusterServiceServletContainer.java:168)
May 21 09:19:55 cldmanager.srv.cesga.es server: Caused by:
java.lang.ClassNotFoundException:
org.apache.log4j.spi.ThrowableInformation
May 21 09:19:55 cldmanager.srv.cesga.es server: at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1892)
May 21 09:19:55 cldmanager.srv.cesga.es server: at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1735)
May 21 09:19:55 cldmanager.srv.cesga.es server: ... 4 more
El 17/05/2018 a las 7:39, ilya musayev escribió:
You must be running java 1.7 - I assume Suresh was alluding to this.
We’ve seen this error with 1.8...
On Wed, May 16, 2018 at 10:36 PM Suresh Kumar Anaparti <
sureshkumar.anapa...@gmail.com> wrote:
Hi Natalia,
Any changes in the java version or system configuration?
-Suresh
On Thu, May 17, 2018 at 4:14 AM, Dag Sonstebo
<dag.sonst...@shapeblue.com>
wrote:
Hi Natalia,
My guess is you have some sort of corruption on your original host.
Since
the management service itself is stateless it’s generally much quicker
just
building a new one than trying to recover the old one.
So my suggestions would be:
- Recover your DB to a dedicated DB host if you haven’t done so
already.
- Build a new management server and just point this to the original
DB as
described in http://docs.cloudstack.apache.org/projects/cloudstack-
installation/en/4.9/management-server/index.html#
additional-management-servers
NOTE: do not use the “–deploy-as” option as this will write a new
blank
database. If you used encryption keys during the original setup you
also
need to specify these.
Once you have these two speaking you should in theory be back up and
running again.
Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue
dag.sonst...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
On 16/05/2018, 16:44, "Natalia Costas Lago" <nata...@cesga.es> wrote:
Dear all,
For some reason our cloudstack manager died and now we are not
able
to
make it work. Our cloudstack version is 4.9.0.
We tried to do a clean install of the manager and recover the
database
from the last backup, but we were not able to recover the
service.
At present we can see in the logs that it almost boot, but at
the end
we
get this error:
May 16 17:33:51 cldmanager.srv.cesga.es server: INFO
[o.a.c.e.o.NetworkOrchestrator] (localhost-startStop-1:null)
(logid:)
Network Manager will run the NetworkGarbageCollector every '600'
seconds.
May 16 17:33:51 cldmanager.srv.cesga.es server: INFO
[c.c.a.ApiServer]
(Thread-12:null) (logid:) ApiServer listening on port 8096
May 16 17:33:51 cldmanager.srv.cesga.es server: INFO
[o.a.c.s.SecondaryStorageManagerImpl]
(localhost-startStop-1:null)
(logid:) Start secondary storage vm manager
May 16 17:33:52 cldmanager.srv.cesga.es server: log4j:WARN No
appenders
could be found for logger
(com.cloud.utils.db.ConnectionConcierge).
May 16 17:33:52 cldmanager.srv.cesga.es server: log4j:WARN Please
initialize the log4j system properly.
May 16 17:33:52 cldmanager.srv.cesga.es server: log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
May 16 17:34:20 cldmanager.srv.cesga.es server: Exception in
thread
"CapacityChecker" java.lang.NoClassDefFoundError:
org/apache/cloudstack/managed/context/ManagedContextTimerTask$1
May 16 17:34:20 cldmanager.srv.cesga.es server: at
org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(
ManagedContextTimerTask.java:27)
May 16 17:34:20 cldmanager.srv.cesga.es server: at
java.util.TimerThread.mainLoop(Timer.java:555)
May 16 17:34:20 cldmanager.srv.cesga.es server: at
java.util.TimerThread.run(Timer.java:505)
May 16 17:34:20 cldmanager.srv.cesga.es server: Caused by:
java.lang.ClassNotFoundException:
org.apache.cloudstack.managed.context.ManagedContextTimerTask$1
May 16 17:34:20 cldmanager.srv.cesga.es server: at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
WebappClassLoaderBase.java:1892)
May 16 17:34:20 cldmanager.srv.cesga.es server: at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
WebappClassLoaderBase.java:1735)
May 16 17:34:20 cldmanager.srv.cesga.es server: ... 3 more
May 16 17:34:24 cldmanager.srv.cesga.es server: Exception in
thread
"Timer-1" java.lang.NoClassDefFoundError:
org/apache/cloudstack/managed/context/ManagedContextTimerTask$1
May 16 17:34:24 cldmanager.srv.cesga.es server: at
org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(
ManagedContextTimerTask.java:27)
May 16 17:34:24 cldmanager.srv.cesga.es server: at
java.util.TimerThread.mainLoop(Timer.java:555)
May 16 17:34:24 cldmanager.srv.cesga.es server: at
java.util.TimerThread.run(Timer.java:505)
Any help is appreciated..
Kind regards,
NOTA: Para cualquier incidencia/consulta/petición que requiera
seguimiento por favor enviar a helpdesk_comunicacio...@cesga.es
--
====================================================
Natalia Costas Lago
Senior Communications Technician
Galicia Supercomputing Centre (CESGA)
(CESGA on Twitter | CESGA on Facebook)
Avenida de Vigo, s/n (Campus Vida)
15705 Santiago de Compostela - SPAIN
E-mail: nata...@cesga.es
Cell: +34 981 56 98 10 (ext. 237)
Fax: +34 981 59 46 16
Web: https://www.cesga.es/
====================================================
[IMPORTANTE] La información contenida en este mensaje y
sus posibles documentos adjuntos es privada y confidencial
y está dirigida únicamente a su destinatario/a. Si usted no
es el/la destinatario/a original de este mensaje, por favor
elimínelo. La distribución o copia de este mensaje no está
autorizada.
NOTA: Para cualquier incidencia/consulta/petición que requiera
seguimiento por favor enviar a helpdesk_comunicacio...@cesga.es
NOTA: Para cualquier incidencia/consulta/petición que requiera
seguimiento por favor enviar a helpdesk_comunicacio...@cesga.es
--
====================================================
Natalia Costas Lago
Senior Communications Technician
Galicia Supercomputing Centre (CESGA)
(CESGA on Twitter | CESGA on Facebook)
Avenida de Vigo, s/n (Campus Vida)
15705 Santiago de Compostela - SPAIN
E-mail: nata...@cesga.es
Cell: +34 981 56 98 10 (ext. 237)
Fax: +34 981 59 46 16
Web: https://www.cesga.es/
====================================================
[IMPORTANTE] La información contenida en este mensaje y
sus posibles documentos adjuntos es privada y confidencial
y está dirigida únicamente a su destinatario/a. Si usted no
es el/la destinatario/a original de este mensaje, por favor
elimínelo. La distribución o copia de este mensaje no está
autorizada.
