Hey Giacomo,

For the first two patches I can't access the failing Kokoro due to re-runs
overwriting the log. In the last one,
https://gem5-review.googlesource.com/c/public/gem5/+/56303/1, I'm not
seeing a timeout issue. That appears to be my aforementioned issue, which
I've come to guess is due to a failure of Kokoro to download a needed
resource correctly (which has been known to happen from time-to-time,
unfortunately). Of the ones that pass I can see them completing in the 5 to
6 hour mark, which is what I've come to expect for a normal run. If
timeouts are happening, I suspect it's something to do with Kokoro running
slower or stalling? I'm not trying to shift the blame here but I can't see
any other explanation.

The Log4J checker code at the end is new to me:

```
[19:56:19] Loading Log4j Scanner Binary onto Build VM [19:56:21] Loading
Log4j Scanner Wrapper Script onto Build VM [19:56:22] Running Scanner on
VM. [ID: 5449648] Executing command via SSH: export
KOKORO_BUILD_NUMBER="11020" export
KOKORO_JOB_NAME="gem5/gcp_ubuntu/presubmit" source
"$HOME/.kokoro_python_vars" ; "$KOKORO_PYTHON_COMMAND"
/tmpfs/log4j_scanner.py Found no vulnerable JARs. [ID: 5449648] Build
finished after 2 secs, exit value: 0
```

I suspect Google has revamped this system at some point to, at least, add
this checker. Perhaps that is what's been causing problems? Perhaps we've
been pushing some patches while someone behind the scenes has been playing
with the server?

Kokoro is acting a bit odd elsewhere. In this chain
https://gem5-review.googlesource.com/c/public/gem5/+/56203, for example,
Kokoro just didn't start at all.

--
Dr. Bobby R. Bruce
Room 3050,
Kemper Hall, UC Davis
Davis,
CA, 95616

web: https://www.bobbybruce.net


On Fri, Feb 4, 2022 at 1:56 AM Giacomo Travaglini <
giacomo.travagl...@arm.com> wrote:
>
> Thanks Bobby,
>
>
>
> Here are some examples:
>
> https://gem5-review.googlesource.com/c/public/gem5/+/56426
>
> https://gem5-review.googlesource.com/c/public/gem5/+/55964/6
>
> https://gem5-review.googlesource.com/c/public/gem5/+/56303/1
>
>
>
> (If you check the verification history of the last patch you can see it
first failed due to timeout. This has become very common in the past days;
failing the first run, rebasing to re-trigger kokoro and hoping next time
It won’t timeout)
>
>
>
> Kind Regards
>
>
>
> Giacomo
>
>
>
> From: Bobby Bruce <bbr...@ucdavis.edu>
> Date: Thursday, 3 February 2022 at 19:52
> To: Giacomo Travaglini <giacomo.travagl...@arm.com>
> Cc: gem5-dev@gem5.org <gem5-dev@gem5.org>
> Subject: Re: Kokoro failures
>
> Are there examples of this timeout happening recently? I can't see any
over the past week.
>
>
>
> There's a separate issue affecting one of Gabe's patches that I'm looking
into (here: https://gem5-review.googlesource.com/c/public/gem5/+/56303) but
these appear to be due to dynamic libraries not linking correctly.
>
>
>
> The only change in testing in the last month has been using clang-11
instead of clang-9 to do the clang compilation check. It's possible
clang-11 takes longer than clang-9 did? In December I increase the timeout
to 7 hours to give us more time to run tests as we were experiencing some
timeouts, I think due to the inclusion of some more tests.
>
>
>
> We could move some tests to the Nightly run fairly easily.
>
>
>
> --
>
> Dr. Bobby R. Bruce
> Room 3050,
> Kemper Hall, UC Davis
> Davis,
> CA, 95616
>
>
>
> web: https://www.bobbybruce.net
>
>
>
>
>
> On Thu, Feb 3, 2022 at 7:00 AM Giacomo Travaglini <
giacomo.travagl...@arm.com> wrote:
>
> Hi all,
>
>
>
> I am seeing a lot of failures on kokoro due to timeouts.
>
> Have we recently added some extra tests to the quick/pre-submit
regressions?
>
>
>
> Kind Regards
>
>
>
> Giacomo
>
> IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
>
> IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.

>
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

Reply via email to