Hi Jason, The kokoro environment provides many installed software including clang toolchain. If you need a more complete list of available SW, LMK.
/usr/bin $ ls -l clang* lrwxrwxrwx 1 root root 25 Feb 12 2015 clang-3.5 -> ../lib/llvm-3.5/bin/clang lrwxrwxrwx 1 root root 27 Feb 12 2015 clang++-3.5 -> ../lib/llvm-3.5/bin/clang++ lrwxrwxrwx 1 root root 25 Jul 23 2018 clang-3.6 -> ../lib/llvm-3.6/bin/clang lrwxrwxrwx 1 root root 27 Jul 23 2018 clang++-3.6 -> ../lib/llvm-3.6/bin/clang++ lrwxrwxrwx 1 root root 25 Jul 18 2017 clang-3.8 -> ../lib/llvm-3.8/bin/clang lrwxrwxrwx 1 root root 27 Jul 18 2017 clang++-3.8 -> ../lib/llvm-3.8/bin/clang++ lrwxrwxrwx 1 root root 25 Jul 26 2017 clang-3.9 -> ../lib/llvm-3.9/bin/clang lrwxrwxrwx 1 root root 27 Jul 26 2017 clang++-3.9 -> ../lib/llvm-3.9/bin/clang++ lrwxrwxrwx 1 root root 44 Feb 12 2015 clang-apply-replacements-3.5 -> ../lib/llvm-3.5/bin/clang-apply-replacements lrwxrwxrwx 1 root root 44 Jul 23 2018 clang-apply-replacements-3.6 -> ../lib/llvm-3.6/bin/clang-apply-replacements lrwxrwxrwx 1 root root 44 Jul 18 2017 clang-apply-replacements-3.8 -> ../lib/llvm-3.8/bin/clang-apply-replacements lrwxrwxrwx 1 root root 44 Jul 26 2017 clang-apply-replacements-3.9 -> ../lib/llvm-3.9/bin/clang-apply-replacements lrwxrwxrwx 1 root root 31 Feb 12 2015 clang-check-3.5 -> ../lib/llvm-3.5/bin/clang-check lrwxrwxrwx 1 root root 31 Jul 23 2018 clang-check-3.6 -> ../lib/llvm-3.6/bin/clang-check lrwxrwxrwx 1 root root 31 Jul 18 2017 clang-check-3.8 -> ../lib/llvm-3.8/bin/clang-check lrwxrwxrwx 1 root root 31 Jul 26 2017 clang-check-3.9 -> ../lib/llvm-3.9/bin/clang-check lrwxrwxrwx 1 root root 28 Jul 18 2017 clang-cl-3.8 -> ../lib/llvm-3.8/bin/clang-cl lrwxrwxrwx 1 root root 28 Jul 26 2017 clang-cl-3.9 -> ../lib/llvm-3.9/bin/clang-cl lrwxrwxrwx 1 root root 39 Jul 26 2017 clang-include-fixer-3.9 -> ../lib/llvm-3.9/bin/clang-include-fixer lrwxrwxrwx 1 root root 31 Feb 12 2015 clang-query-3.5 -> ../lib/llvm-3.5/bin/clang-query lrwxrwxrwx 1 root root 31 Jul 23 2018 clang-query-3.6 -> ../lib/llvm-3.6/bin/clang-query lrwxrwxrwx 1 root root 31 Jul 18 2017 clang-query-3.8 -> ../lib/llvm-3.8/bin/clang-query lrwxrwxrwx 1 root root 31 Jul 26 2017 clang-query-3.9 -> ../lib/llvm-3.9/bin/clang-query lrwxrwxrwx 1 root root 32 Jul 23 2018 clang-rename-3.6 -> ../lib/llvm-3.6/bin/clang-rename lrwxrwxrwx 1 root root 32 Jul 18 2017 clang-rename-3.8 -> ../lib/llvm-3.8/bin/clang-rename lrwxrwxrwx 1 root root 32 Jul 26 2017 clang-rename-3.9 -> ../lib/llvm-3.9/bin/clang-rename lrwxrwxrwx 1 root root 32 Feb 12 2015 clang-tblgen-3.5 -> ../lib/llvm-3.5/bin/clang-tblgen lrwxrwxrwx 1 root root 32 Jul 23 2018 clang-tblgen-3.6 -> ../lib/llvm-3.6/bin/clang-tblgen lrwxrwxrwx 1 root root 32 Jul 26 2017 clang-tblgen-3.9 -> ../lib/llvm-3.9/bin/clang-tblgen lrwxrwxrwx 1 root root 30 Feb 12 2015 clang-tidy-3.5 -> ../lib/llvm-3.5/bin/clang-tidy lrwxrwxrwx 1 root root 30 Jul 23 2018 clang-tidy-3.6 -> ../lib/llvm-3.6/bin/clang-tidy FYI, kokoro can use custom VM images or docker (via GCR) if there is need to change environment. I can grant limited users (only from gem5-admins ACL) access to a test-debug VM to check the kokoro GCP Ubuntu platform/env. Please email me your ssh public key. Thanks, Rahul. On Fri, May 3, 2019 at 9:53 AM Jason Lowe-Power <ja...@lowepower.com> wrote: > Thanks for the info, Rahul! > > Is there any way to specify a different compiler (e.g., clang) when > building on kokoro? > > Thanks, > Jason > > On Fri, May 3, 2019 at 9:12 AM Rahul Thakur <rjtha...@google.com> wrote: > >> Hi Giacomo, Jason, >> >> Sorry for late reply. >> Re: tool chain - scons and python are available in the test environment >> AFAIK. >> Re: test throughput - How long does kokoro take to finish current >> presubmit test? >> >> If you have much longer tests to run at periodic frequency, say once in a >> few days, on HEAD, kokoro also has a periodic continuous build mode. Such >> test will send out email report, serving as a pulse check for ToT. >> Presubmit jobs will not be influenced. I can look in to periodic builds if >> it fits for gem5 long regression use case. >> >> RT >> >> On Fri, May 3, 2019 at 6:17 AM Giacomo Travaglini < >> giacomo.travagl...@arm.com> wrote: >> >>> Hi Jason, >>> >>> I have seen patches being under review for a long time, and IMHO adding >>> an extra 10-30 mins is not the real bottleneck. >>> I'd rather wait a little bit more but being sure I am not breaking >>> anything... >>> >>> about ruby protocols, let me say that we (in arm) are relatively happy >>> with the current setup: >>> >>> 1) Quick regressions are run on a commit base (your presubmit.sh). This >>> means it won't take a lot of time/computation >>> 2) Long regressions (ruby protocols are tested here) are run every night >>> on a batch of MERGED commits. >>> Which means at a certain point we just checkout origin/HEAD and we run >>> long regressions. >>> >>> This is the setup I would actually recommend: a sanity suite (quick) >>> being run on a commit base, and more serious tests >>> (long) being run periodically. If you are scared about overloading the >>> framework, you can always scale down our ambition >>> and run long regressions every two nights. >>> Running less frequently is better than not running at all 🙂 >>> >>> Please let me know what you think about this; other devs are welcome to >>> comment as well, >>> >>> Giacomo >>> >>> >>> ------------------------------ >>> *From:* Jason Lowe-Power <ja...@lowepower.com> >>> *Sent:* 02 May 2019 22:59 >>> *To:* Giacomo Travaglini; Rahul Thakur >>> *Cc:* gem5 Developer List >>> *Subject:* Re: [gem5-dev] Continuous integration is live! >>> >>> Hi Giacomo, >>> >>> In tests/main.py we call scons and use the current environment defaults >>> to build gem5. I don't know if the kokoro infrastructure supports other >>> compilers. This might be something that Rahul can address. >>> >>> I'm also not sure if we can find a way to run more compilations in >>> parallel on Kokoro. I'm happy to refactor the test scripts to do this. >>> However, as it is, we are currently compiling at least 4 binaries mostly >>> sequentially, which is making the testing take a significant amount of >>> time. If we add more compilers (and more Ruby protocols), this is going to >>> begin to get out of hand. It would also be good to compile .fast, .opt, and >>> .debug, but I believe we're only compiling .opt right now. >>> >>> Cheers, >>> Jason >>> >>> On Thu, May 2, 2019 at 5:53 AM Giacomo Travaglini < >>> giacomo.travagl...@arm.com> wrote: >>> >>> Hi Jason, >>> >>> I understand; Another thing I would like to ask: >>> >>> Which script is building gem5 in jenkins? Ideally it would be nice to >>> build with BOTH gcc and clang (so that we avoid >>> periodic "fix clang build" patches. I would also make the version >>> configurable/visible from the script so that >>> we can track changes in compiler support and people can compare failures >>> in case they managed to build >>> seamlessly on their local workspace >>> >>> Giacomo >>> ------------------------------ >>> *From:* Jason Lowe-Power <ja...@lowepower.com> >>> *Sent:* 26 April 2019 17:49 >>> *To:* Giacomo Travaglini >>> *Cc:* gem5 Developer List >>> *Subject:* Re: [gem5-dev] Continuous integration is live! >>> >>> Hi Giacomo, >>> >>> You *do* have permission :). Anyone can modify >>> tests/jenkins/presubmit.cfg and presubmit.sh. In fact, if you look at the >>> history of the presubmit.sh, it *was* running the old tests. See >>> https://gem5-review.googlesource.com/c/testing/jenkins-gem5-prod/+/18028, >>> for instance. >>> >>> The problem is that we can't distribute most of the binaries (e.g., SPEC >>> binaries). We could probably upload them to a private location on the >>> Google Cloud and have jenkins consume them that way, but I believe that >>> will be more work than it's worth. >>> >>> I personally believe that putting effort into porting tests is more >>> worth everyone's time than trying to get the old tests to run, but that's >>> just my opinion. I'm happy to merge changes to run the old tests. I >>> personally believe we should only merge tests into the verification tester >>> which everyone can run locally, but I'm open to proprietary tests, >>> especially in the short term if we have a plan to make them not proprietary. >>> >>> Cheers, >>> Jason >>> >>> On Fri, Apr 26, 2019 at 9:36 AM Giacomo Travaglini < >>> giacomo.travagl...@arm.com> wrote: >>> >>> Hi Jason, >>> >>> It's really amazing that we have a testing framework in place, thanks >>> for your effort! >>> At the moment as far as I can tell we are only running tests registered >>> within the new >>> testing library. >>> >>> I was wondering if we could temporarily enable the system to run legacy >>> quick regressions as well, >>> while waiting for porting those to the new library. I guess it is >>> something that shouldn't require a lot of work >>> >>> (just calling .util/regress I guess) >>> >>> I am saying this since a patch recently merged broke some syscall >>> emulation tests and I think it would >>> be beneficial for us to run the entire test suite straightaway while >>> porting tests manually. >>> I could even handle it myself if I had permission to configure the >>> system. >>> >>> Let me know your thoughts, >>> >>> Giacomo >>> >>> ------------------------------ >>> *From:* gem5-dev <gem5-dev-boun...@gem5.org> on behalf of Jason >>> Lowe-Power <ja...@lowepower.com> >>> *Sent:* 16 April 2019 16:30 >>> *To:* gem5 Developer List; Rahul Thakur >>> *Subject:* [gem5-dev] Continuous integration is live! >>> >>> Hi all, >>> >>> We now have initial support for continuous integration testing! We should >>> all thank Google for donating the CPU time and infrastructure to run >>> these >>> tests. Specifically, Rahul Thakur has been incredibly helpful for the >>> past >>> two years in getting this off the ground. Thanks, Rahul and the rest of >>> the >>> team at Google who has been helping us set this up! >>> >>> Now, if you submit a patch to gerrit and receive a maintainer +1, >>> "kokoro" >>> will kick off a build / test of gem5. Once that is complete, you will >>> receive a verified +1. If it fails, you will receive a verified -1. The >>> logs can be viewed by anyone once the job is completed by following the >>> link posted by kokoro (the https://source.cloud.google.com, not the >>> sponge >>> link). You can see an example on a patch I recently submitted here: >>> https://gem5-review.googlesource.com/c/public/gem5/+/18068. Note that >>> the >>> tests take a couple of hours to run. However, I believe there is no limit >>> to the number of different changes that can be tested at the same time. >>> >>> Soon, we are going to enable commit gating with the verified +1 tag. >>> I.e., >>> you will have to pass the continuous integration tests before you can >>> commit your code. >>> >>> Note that this is using the "new" testing infrastructure. You can run >>> this >>> locally by running "./main.py" in the tests directory. More information >>> about how to run tests and add tests can be found in the TESTING.md file. >>> If there are any questions/issues do not hesitate to contact me or the >>> list. The documentation for the new infrastructure can still be improved. >>> Right now, we're running about 30 tests. You can find the tests that we >>> are >>> running in the tests/gem5 directory. >>> >>> We are looking for volunteers to help us port more of the old tests to >>> the >>> new infrastructure and to expand the coverage of our tests. I'm happy to >>> help anyone get started on this and point out which tests still need to >>> be >>> migrated, where our biggest coverage holes are, etc. >>> >>> Cheers, >>> Jason >>> _______________________________________________ >>> gem5-dev mailing list >>> gem5-dev@gem5.org >>> http://m5sim.org/mailman/listinfo/gem5-dev >>> IMPORTANT NOTICE: The contents of this email and any attachments are >>> confidential and may also be privileged. If you are not the intended >>> recipient, please notify the sender immediately and do not disclose the >>> contents to any other person, use it for any purpose, or store or copy the >>> information in any medium. Thank you. >>> >>> IMPORTANT NOTICE: The contents of this email and any attachments are >>> confidential and may also be privileged. If you are not the intended >>> recipient, please notify the sender immediately and do not disclose the >>> contents to any other person, use it for any purpose, or store or copy the >>> information in any medium. Thank you. >>> >>> IMPORTANT NOTICE: The contents of this email and any attachments are >>> confidential and may also be privileged. If you are not the intended >>> recipient, please notify the sender immediately and do not disclose the >>> contents to any other person, use it for any purpose, or store or copy the >>> information in any medium. Thank you. >>> >> _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
