Fwd: Backwards incompatibility with 2.0.8?
Guys, This is the issue I was referring to - looks like just the classpath ordering, and so can just be documented. Or is that overlooking a potential issue with 2.3.1 on 2.0.6? Cheers, Brett -- Forwarded message -- From: [EMAIL PROTECTED] <[EMAIL PROTECTED]> Date: 10 Dec 2007 16:45 Subject: Re: Backwards incompatibility with 2.0.8? To: Maven Users List <[EMAIL PROTECTED]> Cc: Maven Users List <[EMAIL PROTECTED]> I've worked out what happened. Surefire 2.3.1 is not in the central repository yet but apparantly it has been tagged (http://jira.codehaus.org/browse/MNG-3118). Last Friday, I've downloaded maven source and build it on my local machine (and added apache snapshot into Artifactory). As a result, surefire 2.3.1-SNAPSHOT got into our Artifactory. This is what I've found. Maven 2.0.6 + surefire 2.3 works. Maven 2.0.6 + surefire 2.3.1-SNAPSHOT doesn't work. Maven 2.0.8 + surefire 2.3 doesn't work Maven 2.0.8 + surefire 2.3.1-SNAPSHOT works. I'm not sure what differences there are in surefire 2.3.1-SNAPSHOT and 2.3.1. I can't find surefire 2.3.1 anywhere so I'll wait and see. Thanks for your help. Cheers, rOnn c. "Brett Porter" <[EMAIL PROTECTED]> 12/10/2007 11:24 AM Please respond to "Maven Users List" <[EMAIL PROTECTED]> To "Maven Users List" <[EMAIL PROTECTED]> cc Subject Re: Backwards incompatibility with 2.0.8? Your act of upgrading certainly can't have broken his environment :) There are two possibilities: a) something changed in a surefire snapshot. This is possible as its under active development, though I'm not sure if new snapshots are being deployed regularly. b) a test was added to your environment that relied on the classpath ordering. b) still sounds like the most likely based on your original message - note that 2.0.6 always had main-first ordering, while 2.0.8 has test-first ordering. If you have a sample case where 2.0.6 + surefire 2.3 succeeds and 2.0.6 + 2.3.1 fails, please file it in the surefire JIRA. Otherwise, I would look at the test cases as above. Note you can also use the enforcer to force a Maven version to require people to upgrade to 2.0.8 - consistency is likely to give you less headaches, though I can understand that this is not something you would want to jump to quite so soon perhaps given it is a recent release. I'm still using 2.0.7 myself :) Cheers, Brett On 10/12/2007, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Ok, but our problem is not a simple backward compatibility. > > I was working on project A on my machine. I upgraded to 2.0.8. Nothing > breaks here but it it break then fair enough. > > A colleauge was working on project B and still using 2.0.6. > > Some how my action of upgrading to 2.0.8 in my machine has cause his build > environment to break in his machine. > > Both build machine uses surefire 2.3.1-SNAPSHOT. So perhaps surefure 2.3.1 > doesn't work with Maven 2.0.6 ? > > > rOnn c. > > > > > > > "Brett Porter" <[EMAIL PROTECTED]> > 12/10/2007 10:42 AM > Please respond to > "Maven Users List" <[EMAIL PROTECTED]> > > > To > "Maven Users List" <[EMAIL PROTECTED]> > cc > > Subject > Re: Backwards incompatibility with 2.0.8? > > > > > > > No, it's because 2.0.8 is not backwards compatible with 2.0.6, as > documented in the release notes. > > On 10/12/2007, Michael McCallum <[EMAIL PROTECTED]> wrote: > > its just coincidental that surefire 2.3.1 was released at the same time > as > > 2.0.8... > > > > if you specify surefire 2.3 or less in you pluginManagement then the > tests > > will start working again... of course its best practice to specify > plugin > > versions anyway so magic upgrades don't break things > > > > On Mon, 10 Dec 2007 12:30:25 [EMAIL PROTECTED] wrote: > > > We were using maven 2.0.6 in a team with Artifactory as a proxy. > > > > > > I then upgrade my environment to 2.0.8. A whole bunch of new plugins > get > > > fetched. > > > > > > Other people were using 2.0.6 but it appears that they also get a > whole > > > heap of new plugins. > > > > > > As a result, test cases in another project which had worked before now > > > failed for people running 2.0.6. It appears that the problem is with > > > classpath ordering - for some reason with 2.0.6 and the new bunch of > > > plugins, src/main/resources gets picked up before src/test/resources > and > > > so it doesn't load the resources for test cases. > > > > > > So what happens now is that everyone has to abandon 2.0.6 to 2.0.8. > > > > > > Can someone offer any explanation how this could happen? > > > > > > It concerns me that someone running 2.0.6 would now be forced to move > to > > > 2.0.8 simply because I decided to upgrade to 2.0.8 in my own > environment > > > and project? Is the automatic plugin update a problem here? > > > > > > I don't understand how automatic plugin update works but how would one > > > guarantee repeatable builds ? esp with the releases that have been > tagged > > > and potentailly checked out for build later. > >
Re: Running a single test in a reactor
I'd go with (1). I expect it will be clear enough why it's failing so people will pick up the new usage fast enough. Can you mention the "failIfNoTests=false" flag in the error message? (I assume the true is a typo below?) (3) is a possibility, but you can't really detect if tests are run at "some point" as you say other than externally gathering the info in surefire by aggregating the results. Sounds like more work than its worth. - Brett On 11/12/2007, at 7:02 AM, Dan Fabulich wrote: SUREFIRE-350 suggests that "if test parameter is provided, and no match is found, an error should occur, not a successful build with 0 tests." That made sense to me, so I checked in a fix in revision 597952. However, I discovered that this broke a certain maybe-standard usage: if you've got a parent aggregator POM and two children "foo" and "bar" running in a reactor, you used to be able to say - Dtest=BarTest; we'd compile foo and run no tests in foo, then run BarTest in bar. This usage very convenient, because you don't have to know that BarTest is in the bar module; you can just kick off the tests from the root module and run them in whatever module happens to contain them. You could also run multiple individual tests separated by commas in multiple projects. Now, if you try to run that line from the parent, you'll get a build failure in foo, because BarTest isn't defined in foo. I can think of a number of different possible options here. 1) Leave the code as is. Just because it's convenient doesn't mean it's important and we have to preserve it. 2) Go back and "unfix" SUREFIRE-350. Notably, in order to fix SUREFIRE-350, I added a new plugin parameter called failIfNoTests, which would fail if no tests were run. (It seemed imaginable that somebody would want to turn that on in a variety of cases, not just in the -Dtest case.) If I were to unfix SUREFIRE-350 it'd still be possible to get a build failure, if you wanted one really badly, by saying "mvn test - Dtest=FooTest -DfailIfNoTests=true". It's not pretty, but perhaps it's an acceptable compromise. 3) Try to add code that detects the reactor case and handles it. I think it's not that hard to detect whether a given invocation of SurefirePlugin is running in a reactor (though I don't actually know how off-hand). But how could I detect the case that this is meant to help, where you accidentally had a typo? There's no way to detect whether a test got run *at some point* in the reactor, is there? -Dan
Re: Surefire 2.4 is coming
Thanks Dan! I'm just catching up again. I am planning to scan through all the commits and see if anything catches my eye. I also saw an issue reported on the users list that I'll drop into JIRA about the new code working with 2.0.6 - which versions of Maven have you been testing with? (I've tended to test with 2.0.4 and 2.0.6, but now 2.0.7 and 2.0.8 made sometimes funky changes that are worth running the integration tests with). WDYT about continuing with the 2.3.1 release immediately, and giving 2.4 a short time more to simmer? If for any reason 2.4 does break certain builds, there is still a better-than-2.3 fallback. You can also use the release ann. to re-encourage testing of 2.4. And again, thanks - the quantity of work you did here was just awesome :) - Brett On 11/12/2007, at 10:25 AM, Dan Fabulich wrote: Surefire trunk currently depends on a couple of SNAPSHOT versions that we expect to be released in the next couple of days, but other than that, as far as I know, there's nothing else standing in the way of a Surefire 2.4 release. It includes more than 50 bug fixes. With that said, the more testing we get for Surefire 2.4 the better, because (as I understand it) people are going to be auto-upgraded to Surefire 2.4 when it gets released; when Surefire 2.3 came out, it wasn't uncommon to hear people complaining that their tests used to work earlier in the week, but post-2.3 no longer work. The easiest way you can help us test Surefire is to pull down surefire trunk and do "mvn clean install"; that will run all of the Surefire integration tests. If any of those fail, please file a bug with a stacktrace (and the log.txt from your surefire-integration- tests/target directory, if applicable). https://svn.apache.org/repos/asf/maven/surefire/trunk The second easiest way you can help test Surefire 2.4 is to try the latest SNAPSHOT out in various projects you support. If you find that Surefire 2.4-SNAPSHOT breaks anything, file bugs and we'll try to address them. (Reduced test cases in the form of minimal Maven projects that reproduce the problem would be greatly appreciated.) -Dan
Re: Surefire 2.3.1, 2.4 and Shade
On 11/12/2007, at 3:17 PM, Jason van Zyl wrote: On 10 Dec 07, at 4:04 PM 10 Dec 07, Brett Porter wrote: On 11/12/2007, at 9:08 AM, Mauro Talevi wrote: Dan Fabulich wrote: Well - it was recently agreed that no vote was required for alphas. That's surprising to me... I'd at least post to dev to make sure you don't get a -1. I distinctly remember that alphas should be released with more ease and not the same formality. But I decided to call for a vote unifying it to the move out of sandbox. I'll then double check the alpha release process and update the webpage for reference. Someone suggested that, but I happen to disagree with it (if you need less formality, make nightlies accessible) - a releases is a releases is a release :) This seems consistent with the Apache foundation documentation also. A release in terms of legal requirements and packaging sure. Official releases should get more attention, while alphas should be expelled as fast as humanly possible for feedback. How can you argue against that? We probably make 10x releases then anyone else at the ASF. Getting something working out to users to try in the least cumbersome way should be the goal. 3 +1's is a requirement, 72 hours is a common courtesy to people who don't have all day to watch the maven lists. I have no objections on a previously-unreleased, still-in-the-sandbox plugin being released faster if it makes some sense. I was just clarifying that it isn't any less formal in terms of voting, though I guess I did a poor job on the clarity aspect. FWIW, my mindset comes from wanting to eradicate the endless alpha mentality from this project. I'd rather see us never do one again and make snapshots easier to consume, and final releases happen more regularly. But I really don't need to go into it here because it's irrelevant to the release-quality work that Dan, Mauro and others have been doing on surefire. - Brett
Re: Surefire 2.3.1, 2.4 and Shade
On 10 Dec 07, at 4:04 PM 10 Dec 07, Brett Porter wrote: On 11/12/2007, at 9:08 AM, Mauro Talevi wrote: Dan Fabulich wrote: Well - it was recently agreed that no vote was required for alphas. That's surprising to me... I'd at least post to dev to make sure you don't get a -1. I distinctly remember that alphas should be released with more ease and not the same formality. But I decided to call for a vote unifying it to the move out of sandbox. I'll then double check the alpha release process and update the webpage for reference. Someone suggested that, but I happen to disagree with it (if you need less formality, make nightlies accessible) - a releases is a releases is a release :) This seems consistent with the Apache foundation documentation also. A release in terms of legal requirements and packaging sure. Official releases should get more attention, while alphas should be expelled as fast as humanly possible for feedback. How can you argue against that? We probably make 10x releases then anyone else at the ASF. Getting something working out to users to try in the least cumbersome way should be the goal. Because we don't have a standard default snapshot repository it's generally a pain for users to try, so they don't, or adding a repository to their POM is enough to stop them. Cheers, Brett Thanks, Jason -- Jason van Zyl Founder, Apache Maven jason at sonatype dot com -- Simplex sigillum veri. (Simplicity is the seal of truth.)
Re: Surefire 2.3.1, 2.4 and Shade
On 10 Dec 07, at 11:10 AM 10 Dec 07, Dan Fabulich wrote: Mauro Talevi wrote: When you guys give the thumbs up, I'll copy stage to repo. +1 Thumbs up! I think it's also time to move the code out of sandbox and into trunk, and call the version beta-1-SNAPSHOT. You would need to call a vote on [EMAIL PROTECTED] and have the vote pass. Well - it was recently agreed that no vote was required for alphas. That's surprising to me... I'd at least post to dev to make sure you don't get a -1. No, I asked that it required 3 +1s and no 72 hours vote. That alphas time to community be reduced, and official releases time to community be elongated. Not that we not vote. -Dan Thanks, Jason -- Jason van Zyl Founder, Apache Maven jason at sonatype dot com -- People develop abstractions by generalizing from concrete examples. Every attempt to determine the correct abstraction on paper without actually developing a running system is doomed to failure. No one is that smart. A framework is a resuable design, so you develop it by looking at the things it is supposed to be a design of. The more examples you look at, the more general your framework will be. -- Ralph Johnson & Don Roberts, Patterns for Evolving Frameworks
Re: Surefire 2.3.1, 2.4 and Shade
On 11/12/2007, at 9:08 AM, Mauro Talevi wrote: Dan Fabulich wrote: Well - it was recently agreed that no vote was required for alphas. That's surprising to me... I'd at least post to dev to make sure you don't get a -1. I distinctly remember that alphas should be released with more ease and not the same formality. But I decided to call for a vote unifying it to the move out of sandbox. I'll then double check the alpha release process and update the webpage for reference. Someone suggested that, but I happen to disagree with it (if you need less formality, make nightlies accessible) - a releases is a releases is a release :) This seems consistent with the Apache foundation documentation also. Cheers, Brett
Running a single test in a reactor
SUREFIRE-350 suggests that "if test parameter is provided, and no match is found, an error should occur, not a successful build with 0 tests." That made sense to me, so I checked in a fix in revision 597952. However, I discovered that this broke a certain maybe-standard usage: if you've got a parent aggregator POM and two children "foo" and "bar" running in a reactor, you used to be able to say -Dtest=BarTest; we'd compile foo and run no tests in foo, then run BarTest in bar. This usage very convenient, because you don't have to know that BarTest is in the bar module; you can just kick off the tests from the root module and run them in whatever module happens to contain them. You could also run multiple individual tests separated by commas in multiple projects. Now, if you try to run that line from the parent, you'll get a build failure in foo, because BarTest isn't defined in foo. I can think of a number of different possible options here. 1) Leave the code as is. Just because it's convenient doesn't mean it's important and we have to preserve it. 2) Go back and "unfix" SUREFIRE-350. Notably, in order to fix SUREFIRE-350, I added a new plugin parameter called failIfNoTests, which would fail if no tests were run. (It seemed imaginable that somebody would want to turn that on in a variety of cases, not just in the -Dtest case.) If I were to unfix SUREFIRE-350 it'd still be possible to get a build failure, if you wanted one really badly, by saying "mvn test -Dtest=FooTest -DfailIfNoTests=true". It's not pretty, but perhaps it's an acceptable compromise. 3) Try to add code that detects the reactor case and handles it. I think it's not that hard to detect whether a given invocation of SurefirePlugin is running in a reactor (though I don't actually know how off-hand). But how could I detect the case that this is meant to help, where you accidentally had a typo? There's no way to detect whether a test got run *at some point* in the reactor, is there? -Dan
Re: Surefire 2.3.1, 2.4 and Shade
Mauro Talevi wrote: When you guys give the thumbs up, I'll copy stage to repo. +1 Thumbs up! I think it's also time to move the code out of sandbox and into trunk, and call the version beta-1-SNAPSHOT. You would need to call a vote on [EMAIL PROTECTED] and have the vote pass. Well - it was recently agreed that no vote was required for alphas. That's surprising to me... I'd at least post to dev to make sure you don't get a -1. -Dan
Re: Surefire 2.3.1, 2.4 and Shade
On Monday 10 December 2007, Mauro Talevi wrote: > On Sat, 8 Dec 2007 22:09:45 -0800 (Pacific Standard Time), Dan Fabulich <[EMAIL PROTECTED]> wrote: > > I think that means that if we release maven-shade-plugin > > alpha-14, even without fixing MSHADE-9, we can release Surefire 2.4, > > which would make me very happy. :-) > > Staged alpha-14 at http://people.apache.org/~mauro/staging-repo > > When you guys give the thumbs up, I'll copy stage to repo. > > Cheers You would need to call a vote on [EMAIL PROTECTED] and have the vote pass. That said, it looks good to me. -- J. Daniel Kulp Principal Engineer IONA P: 781-902-8727C: 508-380-7194 [EMAIL PROTECTED] http://www.dankulp.com/blog
Re: Surefire 2.3.1, 2.4 and Shade
On Mon, 10 Dec 2007 13:47:27 -0500, Daniel Kulp <[EMAIL PROTECTED]> wrote: > On Monday 10 December 2007, Mauro Talevi wrote: >> On Sat, 8 Dec 2007 22:09:45 -0800 (Pacific Standard Time), Dan Fabulich > <[EMAIL PROTECTED]> wrote: >> > I think that means that if we release maven-shade-plugin >> > alpha-14, even without fixing MSHADE-9, we can release Surefire 2.4, >> > which would make me very happy. :-) >> >> Staged alpha-14 at http://people.apache.org/~mauro/staging-repo >> >> When you guys give the thumbs up, I'll copy stage to repo. >> >> Cheers > > You would need to call a vote on [EMAIL PROTECTED] and have the vote pass. > Well - it was recently agreed that no vote was required for alphas. > That said, it looks good to me. > Cool!
Re: Surefire 2.3.1, 2.4 and Shade
On Sat, 8 Dec 2007 22:09:45 -0800 (Pacific Standard Time), Dan Fabulich <[EMAIL PROTECTED]> wrote: > I think that means that if we release maven-shade-plugin > alpha-14, even without fixing MSHADE-9, we can release Surefire 2.4, which > would make me very happy. :-) Staged alpha-14 at http://people.apache.org/~mauro/staging-repo When you guys give the thumbs up, I'll copy stage to repo. Cheers
Re: Measuring integration test code coverage for a Maven plugin (fwd)
Should have cc:ed surefire-dev also. Here's the coverage report. http://docs.codehaus.org/download/attachments/35422245/surefire-clover-2007-12-10.zip -Dan -- Forwarded message -- Date: Mon, 10 Dec 2007 00:02:50 -0800 (Pacific Standard Time) From: Dan Fabulich <[EMAIL PROTECTED]> To: Maven Developers List <[EMAIL PROTECTED]> Subject: Re: Measuring integration test code coverage for a Maven plugin Raphaël Piéroni wrote: My guess is that you should deploy your instrumented plugin in a test repository. That's basically what I wound up doing. For the record, I started by specifying a special local repository in my setting directory. Then I tried just using maven-clover-plugin 3.6, calling instrumentInternal directly (which you're not supposed to do because you risk getting cloverized stuff in your production output, though that's exactly what I wanted to do in my case). That didn't quite work because instrumentInternal includes a function redirectArtifact() that changes the output jar to be blah-clover.jar; once I commented that out and added clover as a dependency to all of my code, I was able to run my integration tests and get a total view of my integration test coverage. For the record, Surefire's unit tests covered 45% of Surefire; with the added integration tests we've covered 79.4% (87 classes, 3,323 / 4,187 elements). It also highlighted some key areas where we could add more tests, which was exactly what I wanted. -Dan 2007/12/7, Dan Fabulich <[EMAIL PROTECTED]>: I recently added a bunch of integration tests for Surefire. These integration tests automatically fork a separate Maven process to run real Maven builds, like the Maven core integration tests do. This naturally led me to wonder: Does Surefire (now) have reasonable code coverage? Specifically, which lines in Surefire were covered by unit tests, which by integration tests, and which weren't covered at all? I know there's a variety of handy code coverage tools that work with Maven, allowing you to instrument classes for code coverage and run your unit tests against the instrumented classes. The catch in this case is that I need to somehow convince Maven to use the instrumented version of my plugin, and not the regular "real" version of the plugin, when I go to run my integration tests. The clover plugin, for example, doesn't seem to want to let me do that. [On the other hand, maybe I should just use an instrumenting JVM instead...? Java 1.5's new java.lang.instrument would probably do the trick, but I'm not aware of any code coverage tool that works with j.l.instrument, and anyway I'd have to fix SUREFIRE-179 just to get it to work... :-)] Has anybody ever done this before? More generally, I don't think I've ever seen an example of anyone using Maven to run multi-process integration tests (e.g. cargo tests) and also measuring code coverage on those integration tests. Has anyone seen a good example of this that I could reuse? -Dan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
