I found out the root cause was because of space issues in the nightly publisher run.
The 3 ovirt publishers ran in together and the jenkins dir did not have that capacity. Currently, I am restoring the ovirt-master snapshot and then 4.3 + 4.2, plus sent a patch [1] to ensure they will not run in parallel. [1]: https://gerrit.ovirt.org/111160 On Tue, Sep 8, 2020 at 12:13 PM Ehud Yonasi <[email protected]> wrote: > Hey, > > I am looking into this. > Will update with the results I will find. > > On Tue, Sep 8, 2020 at 12:00 PM Lev Veyde <[email protected]> wrote: > >> Hi Nir, >> >> It looks like the CI job broke it again, breaking 4.2, 4.3 and master >> snapshot release RPM. >> >> I guess it's the same issue we had last time. >> >> Thanks in advance, >> >> On Tue, Sep 8, 2020 at 11:45 AM Nir Soffer <[email protected]> wrote: >> >>> On Mon, Sep 7, 2020 at 2:30 PM Nir Soffer <[email protected]> wrote: >>> > >>> > This is the second time, last time there was an issue during a nightly >>> job that >>> > left the master snapshot repo broken. >>> >>> This is still broken today. >>> >>> In the past sending mail to infra-support created a bug, and someone >>> was handling >>> issue quickly. This seems to be broken now. >>> >>> We depend on >>> http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>> for development and CI. When the rpm is missing, CI jobs fail, and >>> developers cannot >>> update their setup. >>> >>> Please fix it as soon as possible. >>> >>> > I have 3 failure builds: >>> > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7744/ >>> > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7751/ >>> > https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7755/ >>> > >>> > All seems to fail in: >>> > >>> > [2020-09-07T10:00:56.163Z] + dnf install -y >>> > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>> > [2020-09-07T10:01:08.524Z] Status code: 403 for >>> > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm (IP: >>> > 66.187.230.40) >>> > >>> > Checking manually show: >>> > >>> > $ curl >>> https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>> > <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> >>> > <html><head> >>> > <title>403 Forbidden</title> >>> > </head><body> >>> > <h1>Forbidden</h1> >>> > <p>You don't have permission to access >>> /pub/yum-repo/ovirt-release-master.rpm >>> > on this server.</p> >>> > </body></html> >>> > >>> > Looking at: >>> > https://resources.ovirt.org/pub/yum-repo/ >>> > >>> > ovirt-elease-master.rpm does not exist. >>> > >>> > So we have 2 issues: >>> > - The file does not exist >>> > - We return "403 Forbidden" instead of "404 Not Found" >>> >>> >> >> -- >> >> Lev Veyde >> >> Senior Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> <https://red.ht/sig> >> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> >> _______________________________________________ >> Devel mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Privacy Statement: https://www.ovirt.org/privacy-policy.html >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/[email protected]/message/B7VMLC7JKYSU2LI6TRMEAJCXXM3NLSSG/ >> >
_______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/CA3DA7W36Z726EBH6LIQPDQ6NZVAPVFC/
