On Mon, Feb 7, 2022 at 3:09 PM Gilboa Davara <gilb...@gmail.com> wrote:
> > > On Mon, Feb 7, 2022 at 4:03 PM Martin Perina <mper...@redhat.com> wrote: > >> >> >> On Mon, Feb 7, 2022 at 12:33 PM Gilboa Davara <gilb...@gmail.com> wrote: >> >>> Hello, >>> >>> On Mon, Feb 7, 2022 at 8:45 AM Yedidyah Bar David <d...@redhat.com> >>> wrote: >>> >>>> On Sun, Feb 6, 2022 at 5:09 PM Gilboa Davara <gilb...@gmail.com> wrote: >>>> > >>>> > Unlike my predecessor, I not only lost my vmengine, I also lost the >>>> vdsm services on all hosts. >>>> > All seem to be hitting the same issue - read, the certs under >>>> /etc/pki/vdsm/certs and /etc/pki/ovirt* all expired a couple of days ago. >>>> > As such, the hosted engine cannot go into global maintenance mode, >>>> >>>> What do you mean by that? What happens if you 'hosted-engine >>>> --set-maintenance --mode=global'? >>>> >>> >>> Failed, stating the cluster is not in global maintenance mode. >>> (Understandable, given two of 3 hosts were offline due to certificate >>> issues...) >>> >>> >>> >>>> >>>> > preventing engine-setup --offline from running. >>>> >>>> Actually just a few days ago I pushed a patch for: >>>> >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1700460 >>>> >>>> But: >>>> >>>> If you really have a problem that you can't set global maintenance, >>>> using this is a risk - HA might intervene in the middle and shutdown >>>> the VM. So either make sure global maintenance does work, or stop >>>> all HA services on all hosts. >>>> >>>> > Two questions: >>>> > 1. Is there any automated method to renew the vdsm certificates? >>>> >>>> You mean, without an engine? >>>> >>>> I think that if you have a functional engine one way or another, >>>> you can automate this somehow, didn't check. Try checking e.g. the >>>> python sdk examples - there might be there something you can base >>>> on. >>>> >>>> > 2. Assuming the previous answer is "no", assuming I'm somewhat versed >>>> in using openssl, how can I manually renew them? >>>> >>>> I'd rather not try to invent from memory how this is supposed to work, >>>> and doing this methodically and verifying before replying is quite >>>> an effort. >>>> >>>> If this is really what you want, I suggest something like: >>>> >>>> 1. Set up a test env with an engine and one host >>>> 2. Backup (or use git on) /etc on both >>>> 3. Renew the host cert from the UI >>>> 4. Check what changed >>>> >>>> You should find, IMO, that the key(s) on the host didn't >>>> change. I guess you might also find CSRs on one or both of them. >>>> So basically it should be something like: >>>> 1. Create a CSR on the host for the existing key (one or more, >>>> not sure). >>>> 2. Copy and sign this on the engine using pki-enroll-request.sh >>>> (I think you can find examples for it scattered around, perhaps >>>> even in the main guides) >>>> 3. Copy back the generated certs to the host >>>> 4. Perhaps restart one or more services there (vdsm, imageio?, >>>> ovn, etc.) >>>> >>>> You can check the code in >>>> /usr/share/ovirt-engine/ansible-runner-service-project/project >>>> to see how it's done when initiated from the UI. >>>> >>>> Good luck and best regards, >>>> >>> >>> I more of less found a document stating the above somewhere in the >>> middle of the night. >>> Tried it. >>> Got the WebUI working again. >>> However, for the life of me I couldn't get the hosts to work to talk to >>> the engine. (Even though I could use openssl s_client -showcerts -connect >>> host and got valid certs). >>> In the end, @around ~4am, I decided to take the brute force route, clean >>> the hosts, upgrade them to -streams, and redeploy the engine again (3'rd >>> attempt, after sufficient amount of coffee reminded me the qemu-6.1 is >>> broken, and needed to be downgraded before trying to deploy the HE...). >>> Either way, when I finish importing the VMs, I'll open a RFE to add >>> BIG-WARNING-IN-BOLD-LETTERS in the WebUI to notify the admin that the >>> certificates are about to expire. >>> >> >> We already have quite a lot of warnings/alters about certificates which >> are going to expire soon: >> >> >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/TMJVAJMH5MKUVRTSZG2BB46QKXYI6M2D/ >> >> So what exactly are you missing here? >> > > I don't know how, but the only errors I saw in the WebUI were update > related (failed to check updates on host). > That is not related to certificates errors used for engine <-> VDSM communication There was an error in engine-setup, but at this stage it was far, far too > late. > The warning/alerts mentioned above are stored in engine's audit log, which can be viewed within Events tab in webadmin, where you should see something like: Host ${VdsName} certification is about to expire at ${ExpirationDate}. Please renew the host's certification. or Engine's certification is about to expire at ${ExpirationDate}. Please renew the engine's certification. > - Gilboa > > > >> >>> Thanks for the help! >>> >>> - Gilboa >>> >>> >>> >>>> -- >>>> Didi >>>> >>>> _______________________________________________ >>> Users mailing list -- users@ovirt.org >>> To unsubscribe send an email to users-le...@ovirt.org >>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>> oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/CGSAB7NPWWOYON6WXIRJXPZASVWCPQJT/ >>> >> >> >> -- >> Martin Perina >> Manager, Software Engineering >> Red Hat Czech s.r.o. >> > -- Martin Perina Manager, Software Engineering Red Hat Czech s.r.o.
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZD4LHMJYHDD52KIUJNKXBRGJJ4GLZGM6/
