Hey Giacomo, For the first two patches I can't access the failing Kokoro due to re-runs overwriting the log. In the last one, https://gem5-review.googlesource.com/c/public/gem5/+/56303/1, I'm not seeing a timeout issue. That appears to be my aforementioned issue, which I've come to guess is due to a failure of Kokoro to download a needed resource correctly (which has been known to happen from time-to-time, unfortunately). Of the ones that pass I can see them completing in the 5 to 6 hour mark, which is what I've come to expect for a normal run. If timeouts are happening, I suspect it's something to do with Kokoro running slower or stalling? I'm not trying to shift the blame here but I can't see any other explanation.
The Log4J checker code at the end is new to me: ``` [19:56:19] Loading Log4j Scanner Binary onto Build VM [19:56:21] Loading Log4j Scanner Wrapper Script onto Build VM [19:56:22] Running Scanner on VM. [ID: 5449648] Executing command via SSH: export KOKORO_BUILD_NUMBER="11020" export KOKORO_JOB_NAME="gem5/gcp_ubuntu/presubmit" source "$HOME/.kokoro_python_vars" ; "$KOKORO_PYTHON_COMMAND" /tmpfs/log4j_scanner.py Found no vulnerable JARs. [ID: 5449648] Build finished after 2 secs, exit value: 0 ``` I suspect Google has revamped this system at some point to, at least, add this checker. Perhaps that is what's been causing problems? Perhaps we've been pushing some patches while someone behind the scenes has been playing with the server? Kokoro is acting a bit odd elsewhere. In this chain https://gem5-review.googlesource.com/c/public/gem5/+/56203, for example, Kokoro just didn't start at all. -- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net On Fri, Feb 4, 2022 at 1:56 AM Giacomo Travaglini < giacomo.travagl...@arm.com> wrote: > > Thanks Bobby, > > > > Here are some examples: > > https://gem5-review.googlesource.com/c/public/gem5/+/56426 > > https://gem5-review.googlesource.com/c/public/gem5/+/55964/6 > > https://gem5-review.googlesource.com/c/public/gem5/+/56303/1 > > > > (If you check the verification history of the last patch you can see it first failed due to timeout. This has become very common in the past days; failing the first run, rebasing to re-trigger kokoro and hoping next time It won’t timeout) > > > > Kind Regards > > > > Giacomo > > > > From: Bobby Bruce <bbr...@ucdavis.edu> > Date: Thursday, 3 February 2022 at 19:52 > To: Giacomo Travaglini <giacomo.travagl...@arm.com> > Cc: gem5-dev@gem5.org <gem5-dev@gem5.org> > Subject: Re: Kokoro failures > > Are there examples of this timeout happening recently? I can't see any over the past week. > > > > There's a separate issue affecting one of Gabe's patches that I'm looking into (here: https://gem5-review.googlesource.com/c/public/gem5/+/56303) but these appear to be due to dynamic libraries not linking correctly. > > > > The only change in testing in the last month has been using clang-11 instead of clang-9 to do the clang compilation check. It's possible clang-11 takes longer than clang-9 did? In December I increase the timeout to 7 hours to give us more time to run tests as we were experiencing some timeouts, I think due to the inclusion of some more tests. > > > > We could move some tests to the Nightly run fairly easily. > > > > -- > > Dr. Bobby R. Bruce > Room 3050, > Kemper Hall, UC Davis > Davis, > CA, 95616 > > > > web: https://www.bobbybruce.net > > > > > > On Thu, Feb 3, 2022 at 7:00 AM Giacomo Travaglini < giacomo.travagl...@arm.com> wrote: > > Hi all, > > > > I am seeing a lot of failures on kokoro due to timeouts. > > Have we recently added some extra tests to the quick/pre-submit regressions? > > > > Kind Regards > > > > Giacomo > > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >
_______________________________________________ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
