Mickael Istria created SUREFIRE-1776:
----------------------------------------
Summary: DefaultReportFactory doesn't report errors when engine
fails
Key: SUREFIRE-1776
URL: https://issues.apache.org/jira/browse/SUREFIRE-1776
Project: Maven Surefire
Issue Type: Bug
Components: Maven Surefire Plugin
Affects Versions: 2.22.0
Reporter: Mickael Istria
We face an issue with tycho-surefire-plugin reporting "No tests found" when a
test is actually found but the engine fails (for a tycho-specific reason) when
preparing the test.
tycho-surefire-plugin basically builds an OSGi container as defined in the
usual Tycho way and uses Surefire API to run the tests inside that container
and report it as a plain surefire does regularly.
To do so, it does mainly uses `RunResult result =
ProviderFactory.invokeProvider(...)`. However, it can happen that an error
happen in the engine, before any test method starts (although the test method
is scanned and detected and an executionRequest is emitted for it). The cases
for it are relatively probable with Tycho/OSGi.
In such case, even if the engine faced an error, we get a RunResult mentioning
no error, actually reporting no test execution, but no error, usually making
the test report say everything is fine or telling "No tests found" (which is
also incorrect because test is found, it could not just be started).
I could dig a bit more and I think the
DefaultTestReporter.mergeTestHistoryResult() method just fails at handling the
cases of errors when the error do not happen on the test method, because it
wrongly assumes that `listener.getTestMethodStats()` is exhaustive.
More specifically, when debugging this method, I do see in debugger that
`this.listeners[0].detailsForThis` has `completedCount==1` and `errors==1` but
the return value says completedCount==0 and errors=0; its reportEntries show
one error entry, which has for source `<unrooted>` so they're not returned by
`listener.getTestMethodStats()`.
I believe the DefaultReportFactory should handle the case to verify before
returning whether such unrooted error wasn't missed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
