Removing unstable tests from CI is proper attitude.
There is a value in `ValidateLayerConsistencyTest` - the problems it detects
(when working properly) would manifest themselves in incorrect runtime
behavior.
Up to everyone to judge what is more valuable - stable CI or stable runtime?
In this case I'd just restart the CI job, but I understand your frustration,
Matthias.
> 2022-05-29T17:06:31.9092439Z [junit] java.lang.AssertionError: Has to be
> NbRepository: org.openide.filesystems.Repository@744b325b
> 2022-05-29T17:06:31.9093232Z [junit] at
> org.netbeans.core.startup.Main.start(Main.java:298)
> 2022-05-29T17:06:31.9094120Z [junit] at
> org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:98)
> 2022-05-29T17:06:31.9094571Z [junit] at
> java.lang.Thread.run(Thread.java:750) 2022-05-29T17:06:31.9095235Z
> [junit]
Isn't this caused by some race condition? Temporarily adding `this.when = new
Exception("initialized")` into `Repository` constructor and printing the
stacktraces when the `AssertionError` happens might tell us who's responsible
for the misconfiguration of the test.
-jt
Dne neděle 29. května 2022 19:19:42 CEST, Matthias Bläsing napsal(a):
> Hi all,
>
> for me the testing of NetBeans shows its ugly dark side and I'm
> seriously thinking about killing the ValidateLayerConsistencyTest. It
> is flaky and causes untraceable errors.
>
> This is the PR:
>
> https://github.com/apache/netbeans/pull/4058
>
> The flakiness can be observed when comparing the commit validation
> runs. 1 fails whiel 3 work ok. In this instance the failing instance is
> JDK8, but I saw others.
>
> I use this PR to try to fix this:
>
> https://github.com/apache/netbeans/tree/try_fix_commit_validation2
>
> One problem could be tracked down, that the maven module has a entry in
> the configuration, that is a template, but is checked for instablility.
> Fixed here:
>
> https://github.com/apache/netbeans/commit/40a60e311000977b490935c6bf762464d3
> ddaa96
>
> Currently it fails with an error that also seems to be a race condition
> (line 23766-2769):
>
> https://pipelines.actions.githubusercontent.com/serviceHosts/3cf2747f-b5d6-4
> e8e-a999-150ff8ae61e1/_apis/pipelines/1/runs/14001/signedlogcontent/7?urlExp
> ires=2022-05-29T17%3A16%3A59.2143803Z&urlSigningMethod=HMACV1&urlSignature=f
> %2B%2BJVlE0n3fHCqA4wJkq4iKPdD%2B502hC2csZJFs5keE%3D
>
> 2022-05-29T17:06:31.9092439Z [junit] java.lang.AssertionError: Has to be
> NbRepository: org.openide.filesystems.Repository@744b325b
> 2022-05-29T17:06:31.9093232Z [junit] at
> org.netbeans.core.startup.Main.start(Main.java:298)
> 2022-05-29T17:06:31.9094120Z [junit] at
> org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:98)
> 2022-05-29T17:06:31.9094571Z [junit] at
> java.lang.Thread.run(Thread.java:750) 2022-05-29T17:06:31.9095235Z
> [junit]
> ---------------------------------------------------------------------------
> ---- 2022-05-29T17:06:31.9095602Z [junit] >Log Session: Sunday, May 29,
> 2022 5:06:31 PM UTC 2022-05-29T17:06:31.9095875Z [junit] >System Info:
> 2022-05-29T17:06:31.9096375Z [junit] Product Version = Apache
> NetBeans Platform Dev (Build dev-40a60e311000977b490935c6bf762464d3ddaa96)
>
> Given that I changed code in the JS area, this does not seem to related
> to my changes.
>
> So at this point this Test does harm and so is a candidate to be
> removed.
>
> I would like to fix this tough, but I put literally days into this and
> am at the end of my ideas. Anyone wants to chime in?
>
> Greetings
>
> Matthias
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
