Hi Dan and Chris,
Problem-listing it for Xcomp only looks right to me.
Thank you for taking care about it!
Thanks,
Serguei
On 6/12/20 14:20, Chris Plummer wrote:
On 6/12/20 1:59 PM, Daniel D. Daugherty wrote:
On 6/12/20 4:48 PM, Chris Plummer wrote:
On 6/12/20 12:13 PM, Daniel D. Daugherty wrote:
On 6/12/20 2:58 PM, Chris Plummer wrote:
On 6/12/20 11:52 AM, Daniel D. Daugherty wrote:
On 6/12/20 2:49 PM, Chris Plummer wrote:
Hi Dan,
What's the criteria for "noise".
There is no specific criteria that I'm aware of.
It popped up in today's JDK15 testing so it got on my radar (again).
I don't consider the failures for this test as noisy. I only see
3 in mach5 CI testing for all of JDK 15. JDK 14 does appear to
have been somewhat noisy, possibly enough so that it looks like
maybe something changed to reduce the number of failures in 15.
In any case, do you plan on backporting to 14?
This failure has been around in one form or another since JDK7.
If someone
decides to fix it, then they can un-ProblemList it.
I'm planning to push it to JDK15 and JDK16. Those two releases
are the focus
of my CI noise reduction efforts. I don't monitor the JDK14u CI...
May I proceed with the ProblemListing?
I just don't feel if we problem list tests with this failure rate
that in the long run it is a productive or good thing to do. 3
failures during an entire 6 month CI test cycle seems rather low
to me. I'd like to get opinions from others.
It's not just the failure rate. It's the fact that this bug has sat
for
years without being fixed. I have tracked this bug for a very long
time
since I'm the guy that filed both bugs.
Mach5 is showing 54 sightings of 8205957 and here's the linking
distribution:
$ sort /tmp/fred | uniq -c | sort -rn
20 daniel.daughe...@oracle.com
10 rahul.v.ragha...@oracle.com
7 martin.thomp...@oracle.com
4 leonid.mes...@oracle.com
3 jesper.wilhelms...@oracle.com
3 chris.plum...@oracle.com
2 mikael.vidst...@oracle.com
1 tobias.hartm...@oracle.com
1 sangheon....@oracle.com
1 kim.barr...@oracle.com
1 daniil.x.ti...@oracle.com
1 calvin.che...@oracle.com
As you can see, I've observed and linked this bug a lot.
I'm tired of it.
I still think that what is most relevant is how often it reproduces
with CI with the current release, and for that the # is 3 times in 6
months. In our current test history it's failed 6 out of 21390 runs,
so you are disabling a test that passes 99.97% of the time. My
concern is that if a bug is introduced that makes it start failing
every run, or at least very frequently, it will be missed. We need
to carefully weigh the annoyance of failure noise with the
importance of test coverage. I don't think the balance is right for
this test to justify problem listing it.
What you might want to consider is disabling it in the mode where it
seems to be failing. The failures all seem to be with -Xcomp. Maybe
you should just problem list it in ProblemList-Xcomp.txt.
I didn't notice that this is an -Xcomp only failure. I was able to
verify
that fact for 46 of the 54 sightings. For the 8 oldest sightings, the
task
name has been lost to the dustbin of time so I can't confirm those.
I can move the entry from test/hotspot/jtreg/ProblemList.txt to
test/hotspot/jtreg/ProblemList-Xcomp.txt.
Here's the context diff:
$ hg diff
diff -r 015533451f4c test/hotspot/jtreg/ProblemList-Xcomp.txt
--- a/test/hotspot/jtreg/ProblemList-Xcomp.txt Fri Jun 12 09:31:08
2020 -0700
+++ b/test/hotspot/jtreg/ProblemList-Xcomp.txt Fri Jun 12 16:58:18
2020 -0400
@@ -27,3 +27,4 @@
#
#############################################################################
+vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java
8205957 generic-all
Is this acceptable to you?
Yes, that works for me.
Chris
Dan
Chris
Dan
Chris
Dan
thanks,
Chris
On 6/12/20 9:46 AM, Daniel D. Daugherty wrote:
Greetings,
It's time to reduce the noise in the CI so I'm ProblemListing
tests.
Here's the bug for failure:
JDK-8205957 setfldw001/TestDescription.java fails with bad
field value
https://bugs.openjdk.java.net/browse/JDK-8205957
and here's the bug for the ProblemListing:
JDK-8247495 ProblemList
vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java
https://bugs.openjdk.java.net/browse/JDK-8247495
I'm considering this a trivial change so I need a single
(R)eviewer.
Here's the context diff for the change:
$ hg diff
diff -r 015533451f4c test/hotspot/jtreg/ProblemList.txt
--- a/test/hotspot/jtreg/ProblemList.txt Fri Jun 12 09:31:08
2020 -0700
+++ b/test/hotspot/jtreg/ProblemList.txt Fri Jun 12 12:40:17
2020 -0400
@@ -141,6 +141,7 @@
vmTestbase/nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/TestDescription.java
8219652 aix-ppc64
vmTestbase/nsk/jvmti/scenarios/jni_interception/JI06/ji06t001/TestDescription.java
8219652 aix-ppc64
vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab001/TestDescription.java
8219652 aix-ppc64
+vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java
8205957 generic-all
vmTestbase/gc/lock/jni/jnilock002/TestDescription.java
8208243,8192647 generic-all
This issue is actually much older than JDK-8205957 would indicate
(first sighting in JDK11 for that bug ID). The older version of
the test is covered by
https://bugs.openjdk.java.net/browse/JDK-6528079
and that failures first sighting is in JDK7.
Thanks, in advance, for any comments, questions, or suggestions.
Dan
