On Tue, Jan 19, 2021 at 12:25 PM James Freeman <james.free...@a24.io> wrote:
>
> Thanks Didi
>
> Great pointer - I have just performed a fresh deploy - am in the
> hosted-engine VM, and in /var/log/ovirt-engine/engine-log, I can see the
> following 3 lines cycling over and over again:
>
> 2021-01-19 05:12:11,395-05 INFO
> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
> Reactor) [] Connecting to rhvh1.example.org/192.168.50.31
> 2021-01-19 05:12:11,399-05 ERROR
> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96)
> [] Unable to RefreshCapabilities: ConnectException: Connection refused
> 2021-01-19 05:12:11,401-05 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand]
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-96)
> [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = rhvh1.example.org,
> VdsIdAndVdsVDSCommandParametersBase:{hostId='12057f7e-a4cf-46ec-b563-c1037ba5c62d',
> vds='Host[rhvh1.example.org,12057f7e-a4cf-46ec-b563-c1037ba5c62d]'})'
> execution failed: java.net.ConnectException: Connection refused
>
> I can ping 192.168.50.31 and resolve rhvh1.example.org - however I note
> that firewalld on the hypervisor host (192.168.50.31) hasn't had
> anything allowed through it yet apart from SSH and Cockpit. Is this a
> problem, or a red herring?
Generally speaking, the deploy process connects first from the engine to
the host via ssh (22), then (also) configures firewalld to allow access
to vdsm (the oVirt host-side agent, port 54321), and later the engine
normally communicates with the host via vdsm.
Whether or not all of this worked, depends on exactly how you configured
your host's firewalld beforehand.
I suggest to start by not touching it, do the deployment, then see what
it does/did (and that it worked), then decide how you are going to adapt
your policy/tooling/whatever for later deployments, assuming you want to
harden your hosts before deploying.
>
> It seems that the hosted-engine is coming up and being installed and
> configured ok. The engine health page looks ok (as validated by
> Ansible). It looks like the hosted-engine is waiting for something to
> happen on the host itself, but this never completed - which I suspect it
> never will given that it cannot connect to the host.
The deploy process runs on the host, connects to the engine, asks it to
add the host, then waits until it sees the host in the engine with status
'Up'. It indeed does not try to further diagnose failures, nor fail more
quickly - if it's 'Up' it's quick, if it's not, it will wait for a timeout
(120 times * 10 seconds = 20 minutes).
>
> Am I on the right track?
You are :-).
Good luck and best regards,
>
> Yedidyah Bar David wrote on 19/01/2021 10:06:
> > On Tue, Jan 19, 2021 at 11:44 AM <james.free...@a24.io> wrote:
> >> Hi all
> >>
> >> I am in the process of migrating a RHV 4.3 setup to RHV 4.4 and struggling
> >> with the setup. I am installing on RHEL 8.3, using settings backed up from
> >> the RHV 4.3 install (via 'hosted-engine --deploy
> >> --restore-from-file=backup.bck').
> >>
> >> The install process always fails at the same point for me at the moment,
> >> and I can't figure out how to get past it. As far as install progress
> >> goes, the local hosted-engine comes up and runs on the node. I have been
> >> able to grep for local_vm_ip in the logs, and can SSH into it with the
> >> password I set during the setup phase.
> >>
> >> However the install playbooks always fail with:
> >> 2021-01-18 18:38:00,086-0500 ERROR otopi.plugins.gr_he_common.core.misc
> >> misc._terminate:167 Hosted Engine deployment failed: please check the logs
> >> for the issue, fix accordingly or re-deploy from scratch.
> >>
> >> Earlier in the logs, I note the following:
> >> 2021-01-18 18:34:51,258-0500 ERROR
> >> otopi.ovirt_hosted_engine_setup.ansible_utils
> >> ansible_utils._process_output:109 fatal: [localhost]: FAILED! =>
> >> {"changed": false, "msg": "Host is not up, please check logs, perhaps also
> >> on the engine machine"}
> >> 2021-01-18 18:37:16,661-0500 ERROR
> >> otopi.ovirt_hosted_engine_setup.ansible_utils
> >> ansible_utils._process_output:109 fatal: [localhost]: FAILED! =>
> >> {"changed": false, "msg": "The system may not be provisioned according to
> >> the playbook results: please check the logs for the issue, fix accordingly
> >> or re-deploy from scratch.\n"}
> >> Traceback (most recent call last):
> >> File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in
> >> _executeMethod
> >> method['method']()
> >> File
> >> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-ansiblesetup/core/misc.py",
> >> line 435, in _closeup
> >> raise RuntimeError(_('Failed executing ansible-playbook'))
> >> RuntimeError: Failed executing ansible-playbook
> >> 2021-01-18 18:37:18,996-0500 ERROR otopi.context
> >> context._executeMethod:154 Failed to execute stage 'Closing up': Failed
> >> executing ansible-playbook
> >> 2021-01-18 18:37:32,421-0500 ERROR
> >> otopi.ovirt_hosted_engine_setup.ansible_utils
> >> ansible_utils._process_output:109 fatal: [localhost]: UNREACHABLE! =>
> >> {"changed": false, "msg": "Failed to connect to the host via ssh: ssh:
> >> connect to host rhvm.example.org port 22: No route to host",
> >> "skip_reason": "Host localhost is unreachable", "unreachable": true}
> >>
> >> I find the unreachable message a bit odd, as at this stage all that has
> >> happened is that the local hosted-engine has been brought up to be
> >> configured, and so it is running on virbr0, not on my actual network. As a
> >> result, that DNS address will never resolve, and the IP it resolves to
> >> won't be up. I gave the installation script permission to modify the local
> >> /etc/hosts but this hasn't improved things.
> >>
> >> I presume I'm missing something in the install process, or earlier on in
> >> the logs, but I've been scanning for errors and possible clues to no avail.
> >>
> >> Any and all help greatly appreciated!
> > Please check/share, on the engine machine under /var/log/ovirt-engine,
> > or, if inaccessible, on the host, under
> > /var/log/ovirt-hosted-engine-setup/engine-logs-*:
> >
> > engine.log
> >
> > host-deploy/*
> >
> > Good luck and best regards,
>
--
Didi
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/W7U4MS6JOOR66474E2D4LZH6PVPMIJUT/
