Hi Leonid,
If there are specific unstable SA tests then they should be
problem-listed and/or excluded from tier1 and put in a later tier. Lets
not remove all SA testing in tier1 just because of a handful of issues.
Thanks,
David
On 10/12/2018 5:21 pm, Leonid Mesnik wrote:
David, Jini
I understand your concerns. But the original idea of tiered testing is
that tier1 failures are treated as urgent issues and to resolve. [1]
Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I
am not able to provide a link here) Please note that all SA tests are
excluded on Solaris and MacosX already.
1 compiler/aot/calls/fromAot/AotInvokeSpecial2AotTest.java
2 serviceability/sa/ClhsdbFindPC.java
3 serviceability/sa/TestPrintMdo.java
4 serviceability/sa/ClhsdbJstack.java
5 serviceability/sa/ClhsdbJdis.java
6 compiler/c2/Test8004741.java
7 runtime/handshake/HandshakeWalkSuspendExitTest.java
8 runtime/handshake/HandshakeWalkSuspendExitTest.java
9 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java
10 runtime/handshake/HandshakeWalkExitTest.java
11 runtime/handshake/HandshakeWalkSuspendExitTest.java
12 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java
13 serviceability/sa/ClhsdbRegionDetailsScanOopsForG1.java
14 compiler/aot/calls/fromAot/AotInvokeVirtual2AotTest.java
The failures in of 'runtime/handshake/' are relatively caused by
https://bugs.openjdk.java.net/browse/JDK-8214174 but should be also
fixed/excluded. SA tests are also unstable and there are no plans to fix
them soon.
So it means that we are going to have tier1 tests unstable for a long time.
The possible way to make tier1 more stable would be to run only some
very basic sanity SA tests in tier1. Might be to develop new sanity
test which have some failover for existing SA bugs.
Leonid
[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-March/001991.html
On Dec 9, 2018, at 7:19 PM, Jini George <jini.geo...@oracle.com
<mailto:jini.geo...@oracle.com>> wrote:
Hi Leonid,
I agree with David. I am also concerned about us not detecting SA
breakages (which could happen along with hotspot changes) soon enough.
(Which was the primary reason to get these tests in).
Thank you,
Jini.
On 12/8/2018 4:48 PM, David Holmes wrote:
Hi Leonid,
My concern here, if we care about keeping the SA operational, is that
in tier3 these tests will not be covered by the jdk/submit testing
process.
David
On 8/12/2018 3:53 pm, Leonid Mesnik wrote:
Hi
Could you please review following fix which moves SA tests from
tier1 to tier3. There are some bugs which cause intermittent
failures of any test. SA tests fail intermittently are not stable
enough for tier1.
However failures are not very frequent. Also I don't think that
putting all test in Problemlist.txt is very good idea because it
left SA without any testing at all.
So now all SA tests which are included in hotspot_tier3_runtime group.
webrev: http://cr.openjdk.java.net/~lmesnik/8215042/webrev.00/
bug: https://bugs.openjdk.java.net/browse/JDK-8215042
Leonid
