On Mon, 2019-07-08 at 14:35 +0200, Severin Gehwolf wrote: > Hi Andrew, > > On Fri, 2019-06-28 at 14:45 +0100, Andrew John Hughes wrote: > > On 28/06/2019 10:52, Severin Gehwolf wrote: > > > Hi Andrew, > > > > > > On Thu, 2019-06-27 at 17:36 +0100, Andrew John Hughes wrote: > > > > On 22/05/2019 17:34, Severin Gehwolf wrote: > > > > > Hi, > > > > > > > > > > Could I please get reviews for this minimal implementation of a tier1- > > > > > like test set for JDK 8u? The implementation is rather barebones as I > > > > > don't think it's worth rewriting the build system just for a command > > > > > that runs a certain set of tests across a select set of repositories. > > > > > I've re-used existing work in Makefiles as much as possible. After > > > > > this > > > > > patch one can do: > > > > > > > > > > $ make test TEST="tier1" > > > > > > > > > > Inspiration came from JDK 11u's tier1. As for prior art to this, I've > > > > > only found "make test" to be working for JDK 8u from the top level. > > > > > Yet, it doesn't run any hotspot tests, exits with a zero code on test > > > > > failures and doesn't present a summary at the end. Overall not a nice > > > > > developer experience. > > > > > > > > > > This patch makes it easier for a developers tests. It presents a > > > > > summary at the end, returns non-zero on test failures so this can get > > > > > used in CI and runs hotspot tests. > > > > > > > > > > As a follow-on we can work on fixing/excluding tests so that we always > > > > > have a passing set of tests for developers to run before a checkin. > > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8222737 > > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/ > > > > > (includes changes to top/hotspot/jdk/langtools repos) > > > > > > > > > > Example excerpt from a run: > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/02/example_output.txt > > > > > > > > > > Thoughts? > > > > > > > > > > Thanks, > > > > > Severin > > > > > > > > > > > > > Is there a reason for creating new tier definitions here rather than > > > > backporting the existing ones? > > > > > > Yes. The tests in JDK 8 are significantly different from JDK 11. Most > > > notably the number of tests available and defined test groups. > > > Backporting the JDK 11 changes seems like fitting a square peg into a > > > round hole. > > > > > > The JDK 8u "tier1" could only be a rough approximation of the JDK 11 > > > "tier1" at best. There is also no intention to make the test output > > > 100% identical to JDK 11 as bringing back those changes would be too > > > invasive. > > > > > > I believe the proposed patch is a reasonable minimal patch which aids > > > 8u developer's testing. > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8075543 > > > > > > > > The subtasks also cover nashorn & jaxp which are missed here. jaxp would > > > > need JDK-8065673, JDK-8051540 and friends to convert its tests to jtreg. > > > > > > Sure. We can consider adding later tiers in upcoming work. That's not > > > the priority, though. Note that tier1 tests in JDK 11 only run > > > jdk/lantools/hotspot tests. The intention of this initial patch would > > > be to have some testing "baseline" covering a reasonable set of repos > > > and tests. > > > > > > Thanks, > > > Severin > > > > > > > I'm not talking about JDK 11 changes. These are changes in JDK 9. > > > > For example, > > > > https://hg.openjdk.java.net/jdk9/dev/jdk/rev/cd4aea326e89 > > > > adds groups to two tiers, all of which exist in 8u as well. It's not > > clear where the list in your patch comes from and it appears to add a > > bunch of arbitrary other tests beyond the tier 1 defined in the JDK 9 > > change. > > Other changes to the JDK tier1 definition were taken from current JDK > 11u. > > > It would make more sense to me to backport JDK-8075544 & JDK-8075573 as > > pre-requisites rather than creating completely new definitions. > > Done now. I've added fix-request comments/labels to the above bugs and > rebased on top of them. New jdk changeset: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8222737/04/jdk/webrev/ > > Test groups definition is the JDK 9 set plus :jdk_jdi test set (part of > JDK-8198551 in later JDKs). This seems a reasonable test set. Modulo > added intrinsics testing only relevant for 9+, see JDK-8132855 and JDK- > 8132854. > > OK to push?
Any more thoughts? Thanks, Severin
