the test is actually crashing the JVM, I am not quite sure why.
I noticed this while trying to add an action which displays test reports
on github. At first I thought it started when we enabled the xserver,
however, i just tested it while turning the xserver off again (temporary
in a PR) and it was still crashing the jvm.
So I went backwards through
https://github.com/apache/netbeans/commits/master and tried to find
"errors: 1" in the logs. And it is already happening for >24 days (thats
when i gave up looking at logs).
stopped at this commit:
https://github.com/apache/netbeans/commit/5b822a305e61bdb4ae3cb7177a2a21a8209c357b
I believe this is proof that most (including me) are not looking at log
output of unreliable tests when they fail before restarting or ignoring
them. That is basically as if CI wouldn't run them in the first place.
2022-06-24T19:36:52.4792402Z [junit] Testsuite:
org.netbeans.core.validation.ValidateLayerConsistencyTest
2022-06-24T19:36:52.4906892Z [junit] Tests run: 1, Failures: 0, Errors: 1,
Skipped: 0, Time elapsed: 0 sec
2022-06-24T19:36:52.4907315Z [junit]
2022-06-24T19:36:52.4908198Z [junit] Testcase:
org.netbeans.core.validation.ValidateLayerConsistencyTest:testActionInstancesOnlyInActionsFolder:
Caused an ERROR
2022-06-24T19:36:52.4908858Z [junit] Forked Java VM exited abnormally.
Please note the time in the report does not reflect the time until the VM exit.
2022-06-24T19:36:52.4909459Z [junit] junit.framework.AssertionFailedError:
Forked Java VM exited abnormally. Please note the time in the report does not
reflect the time until the VM exit.
2022-06-24T19:36:52.4910056Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4910660Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4911206Z [junit] at
java.base/java.util.Vector.forEach(Vector.java:1388)
2022-06-24T19:36:52.4911716Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4912366Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4912984Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4913634Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4914270Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4914927Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4916082Z [junit] at
java.base/java.util.Vector.forEach(Vector.java:1388)
2022-06-24T19:36:52.4916699Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4917293Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4917891Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4918462Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4919023Z [junit] at
jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
2022-06-24T19:36:52.4920176Z [junit] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-24T19:36:52.4920629Z [junit]
2022-06-24T19:36:52.4921187Z [junit]
2022-06-24T19:36:52.5374861Z [junit] Test
org.netbeans.core.validation.ValidateLayerConsistencyTest FAILED (crashed)
here is my +1 for turning this test off again. I see no benefit in
running this right now.
-mbien
On 29.05.22 19:19, Matthias Bläsing wrote:
Hi all,
for me the testing of NetBeans shows its ugly dark side and I'm
seriously thinking about killing the ValidateLayerConsistencyTest. It
is flaky and causes untraceable errors.
This is the PR:
https://github.com/apache/netbeans/pull/4058
The flakiness can be observed when comparing the commit validation
runs. 1 fails whiel 3 work ok. In this instance the failing instance is
JDK8, but I saw others.
I use this PR to try to fix this:
https://github.com/apache/netbeans/tree/try_fix_commit_validation2
One problem could be tracked down, that the maven module has a entry in
the configuration, that is a template, but is checked for instablility.
Fixed here:
https://github.com/apache/netbeans/commit/40a60e311000977b490935c6bf762464d3ddaa96
Currently it fails with an error that also seems to be a race condition
(line 23766-2769):
https://pipelines.actions.githubusercontent.com/serviceHosts/3cf2747f-b5d6-4e8e-a999-150ff8ae61e1/_apis/pipelines/1/runs/14001/signedlogcontent/7?urlExpires=2022-05-29T17%3A16%3A59.2143803Z&urlSigningMethod=HMACV1&urlSignature=f%2B%2BJVlE0n3fHCqA4wJkq4iKPdD%2B502hC2csZJFs5keE%3D
2022-05-29T17:06:31.9092439Z [junit] java.lang.AssertionError: Has to be
NbRepository: org.openide.filesystems.Repository@744b325b
2022-05-29T17:06:31.9093232Z [junit] at
org.netbeans.core.startup.Main.start(Main.java:298)
2022-05-29T17:06:31.9094120Z [junit] at
org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:98)
2022-05-29T17:06:31.9094571Z [junit] at
java.lang.Thread.run(Thread.java:750)
2022-05-29T17:06:31.9095235Z [junit]
-------------------------------------------------------------------------------
2022-05-29T17:06:31.9095602Z [junit] >Log Session: Sunday, May 29, 2022
5:06:31 PM UTC
2022-05-29T17:06:31.9095875Z [junit] >System Info:
2022-05-29T17:06:31.9096375Z [junit] Product Version = Apache
NetBeans Platform Dev (Build dev-40a60e311000977b490935c6bf762464d3ddaa96)
Given that I changed code in the JS area, this does not seem to related
to my changes.
So at this point this Test does harm and so is a candidate to be
removed.
I would like to fix this tough, but I put literally days into this and
am at the end of my ideas. Anyone wants to chime in?
Greetings
Matthias
---------------------------------------------------------------------
To unsubscribe, e-mail:[email protected]
For additional commands, e-mail:[email protected]
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
