Yeah, I'm with you on your interpretation of that error. I forgot a negation in my original reply to this email thread. I meant to say I'm _not_ very confident in the wait solution, and the more I think it through, I'm even less so, as the error occurs after the docker image is pulled, so I'd assume the service is actually working just fine. That being said, I see no harm in trying.
I've added this patchset: https://gem5-review.googlesource.com/c/public/gem5/+/45999. It adds a wait and a couple of other changes that should help us gather more information on this bug. At the least, it'll do no harm. -- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net On Tue, May 25, 2021 at 12:16 PM Gabe Black <gabe.bl...@gmail.com> wrote: > Yeah, I'm hesitant to blame the docker image too, since as far as I know > it hasn't really changed much recently, and this problem has only really > started recently, although I've hit it a lot in the last few days. Some > sort of timing issue seems like a reasonable hypothesis. Could you please > add the sleep? I think even 1s is likely to help, or maybe some sort of > wait-and-check loop? > > I think this: > > bash: : command not found > > looks strange at first, but what I think is missing is a command between > the two :s, or in other words this is really "bash: ${CMD}: command not > found", and given that this is bash, I'd be willing to bet there's some > variable which hasn't been set and which is being used in a command, and > rather than complaining that variable BLAH_BLAH isn't set, it just expands > to nothing and creates an invalid command line. > > I would also suspect that the command line in question has no arguments, > or the arguments are built into the variable, since otherwise the first > argument would become the command and it would say something like "bash: > -l: command not found" or similar. > > Gabe > > On Tue, May 25, 2021 at 9:59 AM Bobby Bruce <bbr...@ucdavis.edu> wrote: > >> Correction: The command is "sleep", not "wait" :). >> >> -- >> Dr. Bobby R. Bruce >> Room 3050, >> Kemper Hall, UC Davis >> Davis, >> CA, 95616 >> >> web: https://www.bobbybruce.net >> >> >> On Tue, May 25, 2021 at 9:46 AM Bobby Bruce <bbr...@ucdavis.edu> wrote: >> >>> It is annoying when this happens, and doubly so that it's clearly flakey >>> and hard to reproduce. However, I've only ever seen this on Kokoro so I >>> think the blame is somewhere there. Keep in mind, we run nightly tests >>> using the same image and never seem to run into this issue, I'm quite >>> confident the images are good.. >>> >>> I'm very confident in this theory, but we do stop and start the Docker >>> service just before running the docker command (in >>> `tests/jenkins/presubmit.sh`). It could be the Jenkins service is still >>> figuring stuff out and needs a few more nanoseconds before we jump in and >>> booting a container. We could put a "wait 5" between starting the service >>> and running our tests to see if that helps. >>> >>> -- >>> Dr. Bobby R. Bruce >>> Room 3050, >>> Kemper Hall, UC Davis >>> Davis, >>> CA, 95616 >>> >>> web: https://www.bobbybruce.net >>> >>> >>> On Mon, May 24, 2021 at 11:21 PM Gabe Black via gem5-dev < >>> gem5-dev@gem5.org> wrote: >>> >>>> Hi folks. I've noticed kokoro runs occasionally failing with: >>>> >>>> bash: : command not found >>>> >>>> This is after the docker command runs it looks like, so I'm assuming >>>> this is a problem with our docker image? Any ideas? >>>> >>>> >>>> https://source.cloud.google.com/results/invocations/563a3724-49d8-4136-b30f-e19f85db8332/targets/gem5%2Fgcp_ubuntu%2Fpresubmit/log >>>> _______________________________________________ >>>> gem5-dev mailing list -- gem5-dev@gem5.org >>>> To unsubscribe send an email to gem5-dev-le...@gem5.org >>>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s >>> >>>
_______________________________________________ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
