Sorry I didn't get back to folks earlier. I do genuinely believe that branching the integration tests is a good idea. (Though I can't say anything bad about TestNG; we should do that, too!)
My remarks about branching being a good idea are based on about 4 years of experience doing automated testing here at work. Basically, the gist of the problem is: 1) *Most* tests are version specific; they are the norm, not the exception. (It doesn't look like that right now because Maven doesn't have a lot of integration tests that pass in 2.1 but fail in 2.0, mostly because we seem not to have any tests that we know will fail, but I think we can/should expect a whole lot of those in the future. And then think about 2.5, 3.0...) 2) Smart tests are bad tests. It's essential, for the tests to have value, that they be as simple as possible. Running the same tests against multiple versions encourages people to add code paths to the test, e.g. "if this is 2.0, do it this way, but if this is 2.1..." 3) The maintenance burden isn't anywhere near as great as you'd think (because most tests are version specific). There's no point in back-porting that new embedder test or that new bugfix test to Maven 2.0, because the test will never pass in 2.0. More generally, I think the goal of trying to have a single body of tests that gets run against all Maven versions isn't really sound. It'll work right now because 2.1 isn't so different from 2.0, but as soon as the tests get very interesting, it's going to be more difficult to add in (and maintain) those "if this is 2.1" annotations than it will be to just branch the tests. To answer Brett's earlier question as to why we moved the integration tests out in the first place, I think it is useful to keep integration tests out of the "core" build because they aren't just testing core... They're testing "core" and a bunch of plugins *and* a bunch of shared tools, etc. etc. etc. They're a much higher level test than core, so bundling them with "core" doesn't make a lot of sense. -Dan --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
