+1 to all that! Exactly my sentiment. Thanks -Vincent
> -----Original Message----- > From: Brett Porter [mailto:[EMAIL PROTECTED] > Sent: mardi 7 mars 2006 22:37 > To: Maven Developers List > Subject: Re: plugin testing > > Summarising this thread... > > Terminology I mean when I mention things: > - unit testing, intended to test this module only, independent of its > interaction with Maven and dependencies > - integration testing, in this context, is intended to test that the > plugin works in Maven, that it's metadata is bound correctly, lifecycle > interactions, etc. Done via the embedder. > > I saw this as being the unit testing side, and the reason we needed > stubs is because we weren't using real objects. > > It sounds like what we have now is just great as it is, and I'd like to > start with that and see how far we get. My only issue remains the > confusing POM-that-isn't-a-POM that you were going to fix anyway. > > I was concerned with the perceived direction towards using more real > Maven stuff. I think there are several levels to this: > > * artifact code. I think it's quite reasonable to use this via > maven-artifact as it is fairly pojo like anyway. No new stubs needed. > > * project builder. If you need a whole pom with interpolation and > inheritance, you build up a local repo and use the project builder. I > think this indicates a smell in the mojo (see more details below), but > we might have to live with it in the current setup. The framework > shouldn't care - it just takes in a MavenProject, and someone has > already either instantiated a stub, or built with the project builder > component. Still, we should start to consider the embedder here. > > * artifact code that sets up the whole environment through > maven-artifact-manager. I dislike using this for mojos, though I set up > maven-artifact-test for it. Primarily for use by other libraries > (modello's tests, maybe artifact-ant). I think by this point we should > be thinking about the embedder anyway. > > * the embedder should be used for anything that really relies on the > Maven environment, lifecycle handling, or any testing of how expressions > are set. > > While the embedder certainly has its purposes, it should be used when > unit tests can't do something on their own. > > I don't know if anyone has built the eclipse plugin lately, but here are > some figures: > * time to build skipping tests: 5 seconds > * time to build running tests: 25 seconds > * number of tests: 11 > * number of (non-svn) files in src/test: 241 > * code covered: ~65% > > This really isn't scalable. More than 20 files per test and 2-3 seconds > each. > > We now have 100 integration tests for Maven itself which take maybe 5-6 > seconds each, much that could instead be done through unit tests. It's > just too slow to write and to run, and starting to defeat the purpose. > Bootstrapping is something you have to walk away from and grab a coffee > again. > > So that's my summary of the situation. I'll review the framework and > tests in place and see where we go forward from here. > > Specific responses.... > > Jesse McConnell wrote: > > the repo will need to be real at some point, some plugins needs to > interact > > with the real thing, for getting jars for packaging or for just looking > into > > for resources or something..so the artifact repo seems to be a required > part > > of actual unit tests.. > > Probably, though it should never be a remote repo. There are already > some tools in maven-artifact-test for this. > > I'm still very cautious about this though. It's really pushing into the > realms of integration testing. > > > simple, I'll take care of this today... <plugin>-config.xml maybe? > > I think it can be called whatever you like and passed in as a File, > right? If you look at some of the model tests, there is > pom-that-includes-something.xml, pom-that-doesnt.xml, etc. > > > I think we can get away with all this without actually needing to add > > getters and setters, personally I think adding setters somewhat clouds > the > > interaction with mojo's, have a combo of setters and private field > injection > > seems prone to some kind of abuse > > Well, plexus is using the setters if they are there anyway, so there's > no abuse. I'm fine with leaving that to the discretion of the Mojo > author. As long as its possible. > > > I was getting visions of people just making one test pom and either > reusing > > or copying and modifying a bit. > > So say there are two fields with six possible values each and I need to > test them all with each other for some reason. Do I create 36 poms, or > do I create one and then call the Java setters to modify those values? > > An example you can refer to here are the maven-project tests that test > scopes work from the POM. I felt like that was much harder to work with > than the equivalent unit tests later put into maven-artifact to do the > same thing. > > > I agree with you here, it is disjointed to have the configuration and > the > > asserts in another place...but if we are going to route of a new file > for > > configuring these things, how about just building out an assert setup in > the > > configuration xml? > > > > <test> > > <configuration> > > ... > > </configuration> > > <asserts> > > <assertTrue>expression</ > > .. > > .. > > Well, now you are really starting a separate testing framework and the > Java won't be required at all. That's a lot of work. Something I'd love > to investigate, but a lot of work. We probably should be looking at > using a scripting language to write the tests, perhaps. > > Anyway... more responses. > > Jesse McConnell wrote: > > Mojo's need some way of being tested and having coverage determined. > > This actually works if the embedder is used via surefire. Not sure about > the IT plugin. The purpose is more about being tested easily, and quickly. > > Jason van Zyl wrote: > > Brett Porter wrote: > >> A stub of the maven project builder, or the real one? The real one uses > >> the repo, and that's not really conducive to unit testing. > > > > The harness somewhat blurs the lines but once you execute the Mojo I > > don't really consider it a unit test anymore. > > I don't see why it can't be. What happened to the OJO in MOJO? > > > Even with the simple clean > > plugin example that you started with executed the Mojo. I think that's > > what's useful for testing and many Mojos need a local repository so I > > was suggesting using the real project builder passing in a local > > repository created for testing purposes. > > The clean example is quirky because it basically does nothing outside of > what should be in external libraries, and has very little configuration. > If all of our plugins were like this, it would be great, and none of > this would be necessary, but there is plenty of Maven specific > functionality that needs to go into these if you look at plugins like > jar, eclipse, etc. > > My problem with using a real project is that it is a really complicated > piece of code, that requires a local repository. Setting up that repo is > fine, but its really quite tedious. All of that complicated code runs > before the mojo is instantiated - so the mojo tests have no business > caring what it does. All we need is a prepopulated project object. Now, > we can certainly use the project builder for this, if it is simpler to > do so. I have no objections to it. But I like the idea of having a stub > that is a simple bean I can just call .setFoo() and .getFoo() on as well > on too. > > Another reason I dislike the project builder is that we really should > not be using ${project} on its own in the mojos. That binds you to a > non-trivial library in Maven core that limits the ability to change the > version. More use of ${project.field} instead would be welcomed by me, > and I feel is more in the original spirit of Mojos. > > As before, I think we need to start doing the tests and see where it > leads us. And definitely need to criticially review the tests and try > and find ways to make them simpler in code, and faster to write. > > >> At one point, we were doing away with private field injection and > >> recommending setters, so it is how they'd be used. > > > > Hasn't happened yet. Not sure if that will happen as I'm not sure how > > many Ant tasks are actually used outside of Ant. Given what we do in > > promoting the creation of components that get wrapped by a Mojo to > > expose them in Maven I'm not sure if there's much point to properties. > > If that's what we decide then we can do that. I'm not fussed one way or > > the other. > > Ok, likewise. I think this is tied into the dependence on Maven objects > instead of making them simple POJOs. Maybe it was putting the M in POJO > that was the problem, rather than a lack of OJO in MOJO :) > > > But there are plugins that require a MavenProject and other parts of the > > POM not in the configuration for the plugin. > > The IDEA plugin is one example that needs a POM so having the pom.xml > > file actually be the source of a MavenProject I think would work. There > > is also the resources plugin which requires elements in the <build/> > > element. I don't think we'll get away with just the <configuration/> > > element for the plugin. > > If it needs to be a POM, it should be a POM, loaded with the project > loader. If it is not a POM, it should not look like a POM, and just be > the XML configuration fragment inserted into the instantiated Mojo. Or > it could all be done in java code with a stub project and setters on the > mojo. I'd like the alternative for both. > > > We've had this discussion before :-) They are not properties. We happen > > to abuse the field name being the same as the configuration element. But > > with a property the configuration element can be different then the > > field name as the setter would intervene. > > Ok, I get confused on the terminology, sorry. But isn't it the case > right now that if I add a setter to a mojo, that gets called instead? Or > is that only if you add the property element to the configuration? > > I think this needs to be written down in the documentation, as I suspect > nobody other than yourself knows how the setters truly work right now > and it's probably clouding our perception of how to deal with them here. > property="..." is not listed on > http://maven.apache.org/developers/mojo-api-specification.html. > > > In how many cases in a Mojo can you only set one field and actually make > > it do anything? > > surefirePlugin.setForkMode( "once" ); > surefirePlugin.setForkMode( "pertest" ); > surefirePlugin.setForkMode( "none" ); > > I think there are plenty of examples. > > > I'm all for a Java way, but I think the pom.xml will be > > the most thorough and actually correspond most accurately to real use. > > Whatever kind of testing that actually is. > > That's fine. Let's allow both, and find out through experience. > > > In SuiteRunner the input and assertions were in the same file. Sure, it > > can't hurt to have both methods and I think we need a method in Java > > simply because we will probably find holes in the declarative structure > > and that shouldn't stop you from testing. > > Ok, so we are generally in agreement. Summary at the top. > > Cheers, > Brett > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] ___________________________________________________________________________ Nouveau : téléphonez moins cher avec Yahoo! Messenger ! Découvez les tarifs exceptionnels pour appeler la France et l'international. Téléchargez sur http://fr.messenger.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
