[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-16 Thread Strahil Nikolov
;> >>>> >> >>> >process status data >> >> >> >>>> >> >>> > >> >> >> >>>> >> >>> >Apr 08 22:48:27 ovirt-node

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Strahil Nikolov
t;>> >01:05:26.660+: 29307: error : >> >virNetSocketReadWire:1806 >> >> >: >> >> >>>> >End >> >> >>>> >> >of >> >> >>>> >> >>> >file >> >> >>>> &g

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Strahil Nikolov
2020-04-09 >> >>>> >> >>> >> >>>> >> >>> >> >>>> >> >> >>>> >> >> >>>> >> >>>> >> >> >>>>>08:07:31,438::broker::120::ovirt_hosted_engine_ha.broker.

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Strahil Nikolov
ead::INFO::2020-04-09 >>>> >> >>> >>>> >> >>> >>>> >> >>>> >> >>>> >>>> >>>>08:07:31,441::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) >

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Strahil Nikolov
2020-04-09 >> >> >>> >> >> >>> >> >> >> >> >> >> >>>>08:07:31,443::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) >> >> >>> >Loaded submon

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Shareef Jalloq
ad::INFO::2020-04-09 >> >> >>> >> >> >>> >> >> >> >> >> >> >>>08:07:31,443::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) >> >> >>> >Loaded submonitor mem-f

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-15 Thread Shareef Jalloq
>> > >> > > >>>08:07:31,443::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) > >> >>> >Loaded submonitor mem-free > >> >>> > > >> >>> >MainThread::INFO::2020-04-09 >

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-14 Thread Strahil Nikolov
>> >> >> >>>08:07:31,444::storage_backends::369::ovirt_hosted_engine_ha.lib.storage_backends::(connect) >> >>> >Connecting to VDSM >> >>> > >> >>> >MainThread::DEBUG::2020-04-09 >> >>> >> >>

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-14 Thread Shareef Jalloq
t;> >MainThread::INFO::2020-04-09 > >>> > >>> > > >>08:07:31,530::storage_backends::373::ovirt_hosted_engine_ha.lib.storage_backends::(connect) > >>> >Connecting the storage > >>> > > >>> >MainThread::INFO::2020-0

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-14 Thread Strahil Nikolov
;>08:07:32,199::storage_server::356::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) >>> >Connecting storage server >>> > >>> >MainThread::DEBUG::2020-04-09 >>> >08:07:32,199::stompclient::294::jsonrpc.Asyncor

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-14 Thread Shareef Jalloq
cf-a775-b4d0d47b26f2',)) >> > >> >MainThread::DEBUG::2020-04-09 >> >08:07:33,130::stompclient::294::jsonrpc.AsyncoreClient::(send) Sending >> >response >> > >> >MainThread::DEBUG::2020-04-09 >> >> >08:07:33,795::storage_backends

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-10 Thread Shareef Jalloq
> > > > > >The UUID it is moaning about is indeed the one that the HA sits on and > >is > >the one I listed the contents of in step 2 above. > > > > > >So why can't it see this domain? > > > > > >Thanks, Shareef. > > > >O

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-09 Thread eevans
v Sent: Thursday, April 9, 2020 12:57 PM To: Shareef Jalloq Cc: eev...@digitaldatatechs.com; Ovirt Users Subject: [ovirt-users] Re: ovirt-engine unresponsive - how to rescue? On April 9, 2020 11:12:30 AM GMT+03:00, Shareef Jalloq wrote: >OK, let's go through this. I'm looking at

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-09 Thread Strahil Nikolov
ing name >> >'vdsm-ovirtmgmt' >> > >> >Is this not referring to the interface name as the network is called >> >'ovirtmgnt'. >> > >> >On Wed, Apr 8, 2020 at 11:35 PM Shareef Jalloq > >> >wrote: >> > >

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-09 Thread Shareef Jalloq
econd > >host > >>> but my first host is still dead. > >>> > >>> First of all, what are these 56,317 .prob- files that get dumped to > >the > >>> NFS mounts? > >>> > >>> Secondly, why doesn't the node mount the NFS directories at boot? > &

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Strahil Nikolov
with this particular node? >>> >>> On Wed, Apr 8, 2020 at 11:12 PM wrote: >>> >>>> Did you try virsh list --inactive >>>> >>>> >>>> >>>> Eric Evans >>>> >>>> Digital Data Services LLC. >&g

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
ns >>> >>> Digital Data Services LLC. >>> >>> 304.660.9080 >>> >>> >>> >>> *From:* Shareef Jalloq >>> *Sent:* Wednesday, April 8, 2020 5:58 PM >>> *To:* Strahil Nikolov >>> *Cc:* Ovirt Users >>>

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
d you try virsh list --inactive >> >> >> >> Eric Evans >> >> Digital Data Services LLC. >> >> 304.660.9080 >> >> >> >> *From:* Shareef Jalloq >> *Sent:* Wednesday, April 8, 2020 5:58 PM >> *To:* Strahil Nikolov >> *Cc:

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
lov > *Cc:* Ovirt Users > *Subject:* [ovirt-users] Re: ovirt-engine unresponsive - how to rescue? > > > > I've now shut down the VMs on one host and rebooted it but the agent > service doesn't start. If I run 'hosted-engine --vm-status' I get: > >

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread eevans
Did you try virsh list --inactive Eric Evans Digital Data Services LLC. 304.660.9080 From: Shareef Jalloq Sent: Wednesday, April 8, 2020 5:58 PM To: Strahil Nikolov Cc: Ovirt Users Subject: [ovirt-users] Re: ovirt-engine unresponsive - how to rescue? I've now shut down th

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
I've now shut down the VMs on one host and rebooted it but the agent service doesn't start. If I run 'hosted-engine --vm-status' I get: The hosted engine configuration has not been retrieved from shared storage. Please ensure that ovirt-ha-agent is running and the storage server is reachable. an

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
Right, still down. I've run virsh and it doesn't know anything about the engine vm. I've restarted the broker and agent services and I still get nothing in virsh->list. In the logs under /var/log/ovirt-hosted-engine-ha I see lots of errors: broker.log: MainThread::INFO::2020-04-08 20:56:20,138

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread eevans
If you haven’t got this resolved, log into the host and use ‘saslpasswd’ without the quotes. Then virsh start and use the password you set on the local account. I’m not sure it will work, but has worked for regular vm’s. Eric Evans Digital Data Services LLC. 304.660.9080 From: S

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread eevans
: Maton, Brett Sent: Wednesday, April 8, 2020 12:09 PM To: Shareef Jalloq Cc: Ovirt Users Subject: [ovirt-users] Re: ovirt-engine unresponsive - how to rescue? First steps, on one of your hosts as root: To get information: hosted-engine --vm-status To start the engine: hosted-engine

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Strahil Nikolov
On April 8, 2020 7:47:20 PM GMT+03:00, "Maton, Brett" wrote: >On the host you tried to restart the engine on: > >Add an alias to virsh (authenticates with virsh_auth.conf) > >alias virsh='virsh -c >qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf' > >Then run virsh: > >virsh > >vi

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Maton, Brett
On the host you tried to restart the engine on: Add an alias to virsh (authenticates with virsh_auth.conf) alias virsh='virsh -c qemu:///system?authfile=/etc/ovirt-hosted-engine/virsh_auth.conf' Then run virsh: virsh virsh # list IdName State

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Shareef Jalloq
Thanks! The status hangs due to, I guess, the VM being down [root@ovirt-node-01 ~]# hosted-engine --vm-start VM exists and is down, cleaning up and restarting VM in WaitForLaunch but this doesn't seem to do anything. OK, after a while I get a status of it being barfed... --== Host ovirt-no

[ovirt-users] Re: ovirt-engine unresponsive - how to rescue?

2020-04-08 Thread Maton, Brett
First steps, on one of your hosts as root: To get information: hosted-engine --vm-status To start the engine: hosted-engine --vm-start On Wed, 8 Apr 2020 at 17:00, Shareef Jalloq wrote: > So my engine has gone down and I can't ssh into it either. If I try to > log into the web-ui of the node