Aha: while i386-linux-deb9-validate sets no extra XZ options, *nightly*-i386-linux-deb9-validate (the failing job) sets "XZ_OPT = 9".
A revert would fix the problem, but presumably so would tweaking that option. Does anyone have information that would lead to a better decision here? On Wed, Sep 28, 2022 at 2:02 PM Cheng Shao <cheng.s...@tweag.io> wrote: > Sure, in which case pls revert it. Apologies for the impact, though > I'm still a bit curious, the i386 job did pass in the original MR. > > On Wed, Sep 28, 2022 at 1:00 PM Bryan Richter <bryan@haskell.foundation> > wrote: > > > > Yep, it seems to mostly be xz that is running out of memory. (All recent > builds that I sampled, but not all builds through all time.) Thanks for > pointing it out! > > > > I can revert the change. > > > > On Wed, Sep 28, 2022 at 11:46 AM Cheng Shao <cheng.s...@tweag.io> wrote: > >> > >> Hi Bryan, > >> > >> This may be an unintended fallout of !8940. Would you try starting an > >> i386 pipeline with it reversed to see if it solves the issue, in which > >> case we should revert or fix it in master? > >> > >> On Wed, Sep 28, 2022 at 9:58 AM Bryan Richter via ghc-devs > >> <ghc-devs@haskell.org> wrote: > >> > > >> > Hi all, > >> > > >> > For the past week or so, nightly-i386-linux-deb9-validate has been > failing consistently. > >> > > >> > They show up on the failure dashboard because the logs contain the > phrase "Cannot allocate memory". > >> > > >> > I haven't looked yet to see if they always fail in the same place, > but I'll do that soon. The first example I looked at, however, has the line > "xz: (stdin): Cannot allocate memory", so it's not GHC (alone) causing the > problem. > >> > > >> > As a consequence of showing up on the dashboard, the jobs get > restarted. Since they fail consistently, they keep getting restarted. Since > the jobs keep getting restarted, the pipelines stay alive. When I checked > just now, there were 8 nightly runs still running. :) Thus I'm going to > cancel the still-running nightly-i386-linux-deb9-validate jobs and let the > pipelines die in peace. You can still find all examples of failed jobs on > the dashboard: > >> > > >> > > https://grafana.gitlab.haskell.org/d/167r9v6nk/ci-spurious-failures?orgId=2&from=now-90d&to=now&refresh=5m&var-types=cannot_allocate > >> > > >> > To prevent future problems, it would be good if someone could help me > look into this. Otherwise I'll just disable the job. :( > >> > _______________________________________________ > >> > ghc-devs mailing list > >> > ghc-devs@haskell.org > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
