I forgot, there's one more issue that should really get into Surefire 2.4:
SUREFIRE-303 http://jira.codehaus.org/browse/SUREFIRE-303 "Ignored/Skipped
tests are not reported"
Currently there's no standard on the ground for how JUnit 4 @Ignores
should be handled. TestNG has a reasonable example (they put <skipped>
elements into the JUnit XML); we should, I suppose, follow their lead,
though I'm afraid the Ant/JUnit teams will decide to call it <ignored>
instead...
Still, I think we'd better clarify this before we try to support something
non-standard in Surefire 2.4.
-Dan
Dan Fabulich wrote:
As you should all know by now, I've been on a hard sprint most of this week
to get a reasonable-looking Surefire 2.4 out the door. I think we're
basically where I want it to be.
Just three main areas remain.
1) useSystemClassLoader issues
2) Snapshot dependencies
3) Measuring code coverage
and finally a remark 4) Dan takes a breather
1) useSystemClassLoader issues
Notably a lot of the very trickiest Surefire bugs we've gotten have all been
"Not Reproducible" or "Won't Fix", because all you have to do is set
useSystemClassLoader=true and things "just work." I'd love to make that the
default setting for Surefire, but there's some stuff in the way that worries
me about doing that.
SUREFIRE-334 (targeted for 2.3.1) points out that we're depending on a good
chunk of plexus stuff in Surefire, and that this would infect the AUT if we
just set useSystemClassLoader=true. 332 and 347 are probably dupes of this
issue, but these things are subtle so I haven't closed them yet.
Jason suggested that we might use the maven-shade-plugin to shade our plexus
dependencies (repackaging them in different package names and updating
dependencies); it's a good idea, but maven-shade-plugin fails to work
properly on the plexus archiver (it doesn't translate interfaces correctly,
apparently?).
It would be great to resolve this issue and make useSystemClassLoader the
default BEFORE 2.4 ships (it *is* targeted for 2.3.1). I think Brett has
claimed these. (The other 2.3.1 bug 335 is a weird one, suggesting that we
might somehow useSystemClassLoader without forking; that sounds like nonsense
to me and I'd be curious to hear someone [Brett] comment on it.)
2) Snapshot dependencies (SUREFIRE-394)
2.4 depends on plexus-utils 1.4.8-SNAPSHOT; it should be released before we
release Surefire 2.4.
Up until last week or so we used to depend on plexus-archiver
alpha-10-SNAPSHOT, but that created ICCEs (see SUREFIRE-389) and so I
downgraded to alpha-7. If we DO need alpha-10-SNAPSHOT, then we'll need to
figure out the dependency issue (perhaps by fixing SUREFIRE-334 and applying
maven-shade-plugin?). jdcasey apparently knew why we needed alpha-10, but I
still don't; my integration tests are all passing without it.
3) Measuring code coverage
I wrote all these tests! How good are they? Measuring the coverage of
out-of-proc integration tests seems to be pretty tricky, though not
impossible. Any ideas?
4) Dan takes a breather
I've been focused on getting this far all week, but I'll be basically useless
for most of the next week, maybe two. I'll try to be available to answer
e-mail and chat, but this next week I won't be doing a lot of coding.
I'd appreciate it if somebody non-me were to take up the baton and help get
this release out sometime this week (we're so close)! :-)
OK, that's it. Let me know if you have questions. I'm off to bed!
-Dan
