On Sat, Jun 10, 2017 at 6:43 PM, Thomas Wakefield <[email protected]> wrote: > As a follow-up for future reference. Re-running engine-setup with the same > config file solved all my issues. I can now long into the website, and run > VM’s again.
"same config file" - the one passed to --config-append=? Interesting. It might be worthwhile to compare the setup logs created by these runs. Perhaps you can open a BZ bug, describe your flow, and attach the conf and log files? Thanks for the report! > > >> On Jun 5, 2017, at 5:12 AM, Thomas Wakefield <[email protected]> wrote: >> >> >>> On Jun 5, 2017, at 2:24 AM, Yedidyah Bar David <[email protected]> wrote: >>> >>> On Mon, Jun 5, 2017 at 4:44 AM, Thomas Wakefield <[email protected]> wrote: >>>> After a reboot I can’t access the management ovirt-engine webpage anymore. >>>> >>>> >>>> server.log line that looks bad: >>>> >>>> 2017-06-04 21:35:55,652-04 ERROR >>>> [org.jboss.as.controller.management-operation] (DeploymentScanner-threads >>>> - 2) WFLYCTL0190: Step handler >>>> org.jboss.as.server.deployment.DeploymentHandlerUtil$5@61b686ed for >>>> operation {"operation" => "undeploy","address" => [("deployment" => >>>> "engine.ear")],"owner" => [("subsystem" => >>>> "deployment-scanner"),("scanner" => "default")]} at address [("deployment" >>>> => "engine.ear")] failed handling operation rollback -- >>>> java.lang.IllegalStateException: WFLYCTL0345: Timeout after 5 seconds >>>> waiting for existing service service >>>> jboss.deployment.unit."engine.ear".contents to be removed so a new >>>> instance can be installed.: java.lang.IllegalStateException: WFLYCTL0345: >>>> Timeout after 5 seconds waiting for existing service service >>>> jboss.deployment.unit."engine.ear".contents to be removed so a new >>>> instance can be installed. >>>> at >>>> org.jboss.as.controller.OperationContextImpl$ContextServiceBuilder.install(OperationContextImpl.java:2107) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.server.deployment.PathContentServitor.addService(PathContentServitor.java:50) >>>> [wildfly-server-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.server.deployment.DeploymentHandlerUtil.doDeploy(DeploymentHandlerUtil.java:165) >>>> [wildfly-server-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.server.deployment.DeploymentHandlerUtil$5$1.handleResult(DeploymentHandlerUtil.java:333) >>>> [wildfly-server-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:644) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1329) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:400) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:222) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.ModelControllerImpl$3$1$1.run(ModelControllerImpl.java:756) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> org.jboss.as.controller.ModelControllerImpl$3$1$1.run(ModelControllerImpl.java:750) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> [rt.jar:1.8.0_131] >>>> at >>>> org.jboss.as.controller.ModelControllerImpl$3$1.run(ModelControllerImpl.java:750) >>>> [wildfly-controller-2.2.0.Final.jar:2.2.0.Final] >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> [rt.jar:1.8.0_131] >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> [rt.jar:1.8.0_131] >>>> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131] >>>> at org.jboss.threads.JBossThread.run(JBossThread.java:320) >>>> [jboss-threads-2.2.1.Final.jar:2.2.1.Final] >>>> >>>> 2017-06-04 21:36:00,654-04 ERROR >>>> [org.jboss.as.controller.management-operation] (DeploymentScanner-threads >>>> - 2) WFLYCTL0349: Timeout after [5] seconds waiting for service container >>>> stability while finalizing an operation. Process must be restarted. Step >>>> that first updated the service container was 'undeploy' at address >>>> '[("deployment" => "engine.ear")]' >>>> 2017-06-04 21:36:00,655-04 INFO [org.jboss.as.controller] >>>> (DeploymentScanner-threads - 2) WFLYCTL0183: Service status report >>>> WFLYCTL0186: Services which failed to start: service >>>> jboss.deployment.subunit."engine.ear"."bll.jar".component.InitBackendServicesOnStartupBean.START >>>> >>>> >>>> >>>> Similar issue in engine.log: >>>> >>>> 2017-06-04 21:30:50,076-04 ERROR >>>> [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] >>>> (ServerService Thread Pool -- 40) [38484164] Failed to initialize backend: >>>> org.jboss.weld.exceptions.WeldException: WELD-000049: Unable to invoke >>>> private void org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.init() >>>> on org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller@438212cb >>> >>> Seems somewhat similar to [1]. Sounds familiar? Adding Arik. >>> >>> Best, >>> >>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1345853 >> >> Seems possible, I was testing a new storage domain, and was adding and >> deleting it a couple times with similar names. And a VM with similar names >> for the tests, Any thoughts on how to recover? >> >> >> >>> >>>> at >>>> org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:100) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:81) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:126) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:70) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:742) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.bean.builtin.InstanceImpl.getBeanInstance(InstanceImpl.java:189) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.weld.bean.builtin.InstanceImpl.get(InstanceImpl.java:100) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.ovirt.engine.core.bll.ServiceLoader.load(ServiceLoader.java:34) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:65) >>>> [bll.jar:] >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> [rt.jar:1.8.0_131] >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> [rt.jar:1.8.0_131] >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> [rt.jar:1.8.0_131] >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> [rt.jar:1.8.0_131] >>>> at >>>> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96) >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:114) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:103) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) >>>> at >>>> org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> at >>>> org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28) >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:100) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:349) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:68) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.weld.injection.WeldInjectionContextInterceptor.processInvocation(WeldInjectionContextInterceptor.java:43) >>>> [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) >>>> [wildfly-ee-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) >>>> at >>>> org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) >>>> at >>>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) >>>> at >>>> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) >>>> at >>>> org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) >>>> at >>>> org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:134) >>>> at >>>> org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88) >>>> at >>>> org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:124) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:138) >>>> [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] >>>> at >>>> org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> [rt.jar:1.8.0_131] >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> [rt.jar:1.8.0_131] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> [rt.jar:1.8.0_131] >>>> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131] >>>> at org.jboss.threads.JBossThread.run(JBossThread.java:320) >>>> Caused by: java.lang.reflect.InvocationTargetException >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> [rt.jar:1.8.0_131] >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> [rt.jar:1.8.0_131] >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> [rt.jar:1.8.0_131] >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> [rt.jar:1.8.0_131] >>>> at >>>> org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:98) >>>> [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] >>>> ... 64 more >>>> Caused by: java.lang.NullPointerException >>>> at >>>> org.ovirt.engine.core.bll.storage.disk.image.BaseImagesCommand.isDataOperationsByHSM(BaseImagesCommand.java:128) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupCommand.isUsingSPDMFlow(CopyImageGroupCommand.java:105) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.storage.disk.image.CopyImageGroupCommand.getCallback(CopyImageGroupCommand.java:188) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.tasks.CommandsRepository.addToCallbackMap(CommandsRepository.java:69) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.initCommandExecutor(CommandCallbacksPoller.java:191) >>>> [bll.jar:] >>>> at >>>> org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.init(CommandCallbacksPoller.java:41) >>>> [bll.jar:] >>>> ... 69 more >>>> >>>> >>>> >>>> _______________________________________________ >>>> Users mailing list >>>> [email protected] >>>> http://lists.ovirt.org/mailman/listinfo/users >>> >>> >>> >>> -- >>> Didi >> >> _______________________________________________ >> Users mailing list >> [email protected] >> http://lists.ovirt.org/mailman/listinfo/users > > _______________________________________________ > Users mailing list > [email protected] > http://lists.ovirt.org/mailman/listinfo/users -- Didi _______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
