Hey Nir, Yeah, we need to work on a better mechanism. I will create a ticket and discuss it in the infra meeting.
Currently, the ovirt release master rpm is back, and soon the 4.3 + 4.2 will be restored. Thanks for the report! On Tue, Sep 8, 2020 at 2:42 PM Nir Soffer <nsof...@redhat.com> wrote: > On Tue, Sep 8, 2020 at 1:14 PM Ehud Yonasi <eyon...@redhat.com> wrote: > >> I found out the root cause was because of space issues in the nightly >> publisher run. >> >> The 3 ovirt publishers ran in together and the jenkins dir did not have >> that capacity. >> >> Currently, I am restoring the ovirt-master snapshot and then 4.3 + 4.2, >> > > Thanks! > > >> plus sent a patch [1] to ensure they will not run in parallel. >> >> [1]: https://gerrit.ovirt.org/111160 >> > > I probably don't understand the change, but it seems that all jobs > will run now at 00:00? > > I think we need to work on a better mechanism for updating the repository. > For example, we can have a symlink the to current repo: > > current -> 200908143754 > > 200908143754/ > repo files... > > 200908143830/ > repo files... > > When creating a new repo, create a new directory. Update the "current" > symlink only > if creating the new repo succeeded. Then delete the previous respo. > > With this the cases when the entire repo or some files are missing are not > possible. > > On Tue, Sep 8, 2020 at 12:13 PM Ehud Yonasi <eyon...@redhat.com> wrote: >> >>> Hey, >>> >>> I am looking into this. >>> Will update with the results I will find. >>> >>> On Tue, Sep 8, 2020 at 12:00 PM Lev Veyde <lve...@redhat.com> wrote: >>> >>>> Hi Nir, >>>> >>>> It looks like the CI job broke it again, breaking 4.2, 4.3 and master >>>> snapshot release RPM. >>>> >>>> I guess it's the same issue we had last time. >>>> >>>> Thanks in advance, >>>> >>>> On Tue, Sep 8, 2020 at 11:45 AM Nir Soffer <nsof...@redhat.com> wrote: >>>> >>>>> On Mon, Sep 7, 2020 at 2:30 PM Nir Soffer <nsof...@redhat.com> wrote: >>>>> > >>>>> > This is the second time, last time there was an issue during a >>>>> nightly job that >>>>> > left the master snapshot repo broken. >>>>> >>>>> This is still broken today. >>>>> >>>>> In the past sending mail to infra-support created a bug, and someone >>>>> was handling >>>>> issue quickly. This seems to be broken now. >>>>> >>>>> We depend on >>>>> http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>>>> for development and CI. When the rpm is missing, CI jobs fail, and >>>>> developers cannot >>>>> update their setup. >>>>> >>>>> Please fix it as soon as possible. >>>>> >>>>> > I have 3 failure builds: >>>>> > >>>>> https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7744/ >>>>> > >>>>> https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7751/ >>>>> > >>>>> https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/7755/ >>>>> > >>>>> > All seems to fail in: >>>>> > >>>>> > [2020-09-07T10:00:56.163Z] + dnf install -y >>>>> > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>>>> > [2020-09-07T10:01:08.524Z] Status code: 403 for >>>>> > http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>>>> (IP: >>>>> > 66.187.230.40) >>>>> > >>>>> > Checking manually show: >>>>> > >>>>> > $ curl >>>>> https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm >>>>> > <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> >>>>> > <html><head> >>>>> > <title>403 Forbidden</title> >>>>> > </head><body> >>>>> > <h1>Forbidden</h1> >>>>> > <p>You don't have permission to access >>>>> /pub/yum-repo/ovirt-release-master.rpm >>>>> > on this server.</p> >>>>> > </body></html> >>>>> > >>>>> > Looking at: >>>>> > https://resources.ovirt.org/pub/yum-repo/ >>>>> > >>>>> > ovirt-elease-master.rpm does not exist. >>>>> > >>>>> > So we have 2 issues: >>>>> > - The file does not exist >>>>> > - We return "403 Forbidden" instead of "404 Not Found" >>>>> >>>>> >>>> >>>> -- >>>> >>>> Lev Veyde >>>> >>>> Senior Software Engineer, RHCE | RHCVA | MCITP >>>> >>>> Red Hat Israel >>>> >>>> <https://www.redhat.com> >>>> >>>> l...@redhat.com | lve...@redhat.com >>>> <https://red.ht/sig> >>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> >>>> _______________________________________________ >>>> Devel mailing list -- de...@ovirt.org >>>> To unsubscribe send an email to devel-le...@ovirt.org >>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/de...@ovirt.org/message/B7VMLC7JKYSU2LI6TRMEAJCXXM3NLSSG/ >>>> >>>
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/TD4BLDJOLY3UE5KO2LVHE2M6LEJ3YPLB/
