Hi Arnaud, 2008/8/7 Arnaud HERITIER <[EMAIL PROTECTED]>: > As I said in another thread, we just ended 3. > We named the profile integration-tests but to have the same name as in the > core we'll rename it run-its > > It seems that there is 2 point of view about ITs activation : > -1) ITs are part of the build and always launched > -2) ITs are launched on demand (because they are consumming a lot of time > and ressources) > What do you prefer ? > For me it's (2). > > Another question : What is an integration test ? > For me unit tests don't handle resources like files. In unit tests we are > using mocks to be able to use only in-memory tests. > Everything else which is reading poms, settings or other resources are > integration tests (whatever the framework used : maven-embedder, > maven-verifier, ...). > The problem actually is that our APIs aren't often developped to have unit > tests. > For example, we often have in parameters InputStream or something like than > instead of a Project object which can be mocked. > We should have to review them to be able to have many more UTs. > Like you can see in [1] we don't have an important coverage of UTs. > > ITs are needed for example in plugins when we wrap another tool, but > otherwise, and particularly in the core we should have a good unit tests > suite.
In plugins world, we have a lot of UTs with our test harness and ITs. I think ITs is sometimes misunderstanding by dev or users wich provide patches. IMHO ITs should be used only if the test needs the Maven stack or a part like reactor or aggregate properties for a mojo. In the real world, users open issues with test cases like MJAVADOC-110 or MJAR-60 and dev just adds these test cases as ITs. So, what to do if a user propose a test case? Add it as an IT or transform it to the plugin harness? Cheers, Vincent > [1] http://nemo.sonar.codehaus.org/project/dashboard/19084?sid=669108 > > > Arnaud > > On Wed, Aug 6, 2008 at 5:24 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > >> Hi, >> >> I think we've gotten to the point where we need to decide how we are going >> to test plugins. We need to pick one of the frameworks, settle on a pattern, >> and use that in the plugins otherwise there will be no sane way to validate >> a set of plugins works against a given version of Maven. What I'm thinking >> about here concretely is testing all the plugins that we have here against >> Maven 2.1 to know that we have not screwed something up so terribly that >> things like the deploy plugin doesn't work, or whatever. >> >> I think how this starts is that we: >> >> 1) Pick one of the tools >> 2) Create a touchstone project that can be expanded where necessary for any >> given plugin so that we have a baseline project against which to test >> 3) Pick a standard profile name for invoking this test >> >> This way we create a standard hook point for a larger harness to get hold >> off. We can check out sources and create an aggregator POM with the given >> profile activated to test a set of plugins. I don't know yet what the best >> way would be to share a touchstone project (and that is not to say we won't >> need different projects but we have to start with a baseline), but once we >> start this we can also start plugging in other things like integration >> testing that includes things like coverage or whatever else. >> >> I think the key in moving forward is getting 1-3 sorted out so we're not >> using 5 frameworks and testing plugins with N different patterns where it's >> impossible to hook into for larger scale testing. I think this is the only >> way forward to validate that a set of plugins work against a given version >> of Maven which is vital information to know before releasing 2.1. >> >> For integration testing I have found the SHITTY plugin (we would simply >> have to change that name, sorry Jason Dillon) to be the most useful and >> feature rich. Should be relatively simple to create a test project, and a >> profile name (run-its like the core ITs). Then we figure out how to share >> and version the test project to create a stable baseline. I chatted about >> this briefly in IRC with Benjamin and wanted to get the information out. I >> think it's vital to get this rolling if we want to roll out a 2.1-alpha-1 >> with some degree of confidence we have toasted a bunch of plugins due to >> incompatibilities in the core. >> >> Thanks, >> >> Jason >> >> ---------------------------------------------------------- >> Jason van Zyl >> Founder, Apache Maven >> jason at sonatype dot com >> ---------------------------------------------------------- >> >> What matters is not ideas, but the people who have them. Good people can >> fix bad ideas, but good ideas can't save bad people. >> >> -- Paul Graham >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > > -- > .......................................................... > Arnaud HERITIER > .......................................................... > OCTO Technology - aheritier AT octo DOT com > www.octo.com | blog.octo.com > .......................................................... > ASF - aheritier AT apache DOT org > www.apache.org | maven.apache.org > ........................................................... > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
