RE: unit testing vs integration testing plugins
On Tue, 2006-01-31 at 05:26 -0600, Allison, Bob wrote: > I also see a need for being able to inject mock objects of Plexus > components (the RPM plugin uses a Plexus Archiver). The unit test will > need to construct these mock objects and set their expectations before > they are injected into the plugin being tested. This is already possible, all you got to do is to make a components.xml file called the same as your test case, e.g. FooTest.java => FooTest.xml. This test file will be merged in after the normal bootstrap so you can override all the components you'd like in there. Take a look at [1] and [2] for an example. [1]: http://svn.plexus.codehaus.org/trunk/plexus-components/plexus-formica-web/src/test/resources/org/codehaus/plexus/formica/action [2]: http://svn.plexus.codehaus.org/trunk/plexus-components/plexus-formica-web/src/test/java/org/codehaus/plexus/formica/action/ -- Trygve - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: unit testing vs integration testing plugins
Brett Porter wrote: Hi, I wanted to get some feedback from other devs on this topic. Historically, I think we've done very little of both and would like to change that. But most of the tests cropping up seem to be integration tests, because they are probably an easier mechanism than to setup the preconditions of the plugin. However, they are much larger in terms of creation, checkout space and time to run. We have turned to the integration in the recent past because they more closely represent what the user will experience. Done more out of a sense of expediency and because it is somewhat difficult right now to quickly because of some of things people have mentioned in that we don't have an easy harness for many of the components. I'd like to see that we "unit test" to the greatest extent possible. IMO, the only reason to go to integration tests is to test lifecycle interactions and interactions with other plugins. We should have plenty of both but to me the integration is the final word on what a user should experience. The symptom here is the decline of unit testing but in a lot of places for me the cause is that the components I'm trying to test are so course that in a short time span it is hard to test a component for which the source is ~1200, ~1400, ~1600 lines long. Namely the plugin manager, project builder and life cycle executor. These are some of the most important components and they are not easy to follow and therefore not easy to test. I feel that's one of the root causes of why more unit testing is not happening. I ran out of time during the last release because I've not been in the core for a while and the project builder is just so big so I punted and made an integration test. What we probably require to do this is convenience methods to construct a decent project and settings object and an expression populator based on the project and settings. The convenience methods and harnesses will certainly help and I think we've got quite a bit of code for that. What would also be cool is to align the components we have in MNG with some archetypes for unit testing components of that particular flavour. Constructed in such a way that they can be isolated and easily attached to a JIRA issue while at the same time be easy to integrate in the existing suite of tests. - Brett -- jvz. Jason van Zyl jason at maven.org http://maven.apache.org First, the taking in of scattered particulars under one Idea, so that everyone understands what is being talked about ... Second, the separation of the Idea into parts, by dividing it at the joints, as nature directs, not breaking any limb in half as a bad carver might. -- Plato, Phaedrus (Notes on the Synthesis of Form by C. Alexander) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: unit testing vs integration testing plugins
I've been writing unit tests for the Cargo m2 plugin and it works quite nice. I didn't feel any need for a framework. In some cases I've used jmock to mock things. You can check it out here: http://svn.cargo.codehaus.org/cargo/trunk/extensions/maven2/src/test/ I agree with Brett about having a healthy balance of unit vs integration/functional tests (in favor of unit tests in number), but there must be functional tests as they are the only type of tests that prove that it works (and it usually doesn't work). Thanks -Vincent > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Carlos > Sanchez > Sent: mardi 31 janvier 2006 01:57 > To: Maven Developers List > Subject: Re: unit testing vs integration testing plugins > > We probably need to provide some plugin test infrastructure like mock > objects and utilities to do the common test tasks. > > On 1/30/06, Brett Porter <[EMAIL PROTECTED]> wrote: > > Hi, > > > > I wanted to get some feedback from other devs on this topic. > > Historically, I think we've done very little of both and would like to > > change that. But most of the tests cropping up seem to be integration > > tests, because they are probably an easier mechanism than to setup the > > preconditions of the plugin. However, they are much larger in terms of > > creation, checkout space and time to run. > > > > I'd like to see that we "unit test" to the greatest extent possible. > > IMO, the only reason to go to integration tests is to test lifecycle > > interactions and interactions with other plugins. > > > > What we probably require to do this is convenience methods to construct > > a decent project and settings object and an expression populator based > > on the project and settings. > > > > Thoughts? > > > > - Brett > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: unit testing vs integration testing plugins
I also see a need for being able to inject mock objects of Plexus components (the RPM plugin uses a Plexus Archiver). The unit test will need to construct these mock objects and set their expectations before they are injected into the plugin being tested. Another item used by the RPM plugin which may need a little refactoring is the Plexus CommandLine object. It is not found by an expression in a parameter like the Archiver, but just built using "new CommandLine". I know that if the Archiver is mocked so that it doesn't actually try to assemble the RPM contents the command to build the RPM will fail. For the test to work, we need to be able to inject a mock object of the CommandLine which only returns the proper status. I would be happy to start testing out samples of a test harness with the RPM plugin. I have been dreading trying to build tests for it, since I have been having a hard time trying to figure out how to verify that it assembled the RPM correctly. If the Archiver and CommandLine can be simulated, the unit tests would be fairly easy to build. -Original Message- From: Brett Porter [mailto:[EMAIL PROTECTED] Sent: Monday, January 30, 2006 18:50 To: Maven Developers List Subject: unit testing vs integration testing plugins Hi, I wanted to get some feedback from other devs on this topic. Historically, I think we've done very little of both and would like to change that. But most of the tests cropping up seem to be integration tests, because they are probably an easier mechanism than to setup the preconditions of the plugin. However, they are much larger in terms of creation, checkout space and time to run. I'd like to see that we "unit test" to the greatest extent possible. IMO, the only reason to go to integration tests is to test lifecycle interactions and interactions with other plugins. What we probably require to do this is convenience methods to construct a decent project and settings object and an expression populator based on the project and settings. Thoughts? - Brett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: unit testing vs integration testing plugins
It might be nice to have a harness which would allow unit testing plugins. The only problem is you have to test the test harness...this is a problem we've run into time and again with all of the test harnesses we have for Maven. I've personally fixed failures based on assumptions in the artifact test harness, the project testing harness, and the PlexusTestCase test harness. My big worry here is that we'd be building a test harness that didn't accurately reflect reality, or which would have to be manually resynchronized with the reality of Maven as it changes. This may introduce subtle bugs... Just my 2c. -john Jesse McConnell wrote: I have always kinda figured the rule of thumb regarding tests and artifacts is unit tests would test anything inside of the artifact and integration tests would encompass extra-artifact scope testing and I totally agree, I think unit testing within mojo projects would be a lot more prelavent with a simple mechanism of bootstraping the plugin requirements.. jesse On 1/30/06, Brett Porter <[EMAIL PROTECTED]> wrote: Hi, I wanted to get some feedback from other devs on this topic. Historically, I think we've done very little of both and would like to change that. But most of the tests cropping up seem to be integration tests, because they are probably an easier mechanism than to setup the preconditions of the plugin. However, they are much larger in terms of creation, checkout space and time to run. I'd like to see that we "unit test" to the greatest extent possible. IMO, the only reason to go to integration tests is to test lifecycle interactions and interactions with other plugins. What we probably require to do this is convenience methods to construct a decent project and settings object and an expression populator based on the project and settings. Thoughts? - Brett - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- jesse mcconnell jesseDOTmcconnellATgmailDOTcom - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: unit testing vs integration testing plugins
We probably need to provide some plugin test infrastructure like mock objects and utilities to do the common test tasks. On 1/30/06, Brett Porter <[EMAIL PROTECTED]> wrote: > Hi, > > I wanted to get some feedback from other devs on this topic. > Historically, I think we've done very little of both and would like to > change that. But most of the tests cropping up seem to be integration > tests, because they are probably an easier mechanism than to setup the > preconditions of the plugin. However, they are much larger in terms of > creation, checkout space and time to run. > > I'd like to see that we "unit test" to the greatest extent possible. > IMO, the only reason to go to integration tests is to test lifecycle > interactions and interactions with other plugins. > > What we probably require to do this is convenience methods to construct > a decent project and settings object and an expression populator based > on the project and settings. > > Thoughts? > > - Brett > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: unit testing vs integration testing plugins
I have always kinda figured the rule of thumb regarding tests and artifacts is unit tests would test anything inside of the artifact and integration tests would encompass extra-artifact scope testing and I totally agree, I think unit testing within mojo projects would be a lot more prelavent with a simple mechanism of bootstraping the plugin requirements.. jesse On 1/30/06, Brett Porter <[EMAIL PROTECTED]> wrote: > > Hi, > > I wanted to get some feedback from other devs on this topic. > Historically, I think we've done very little of both and would like to > change that. But most of the tests cropping up seem to be integration > tests, because they are probably an easier mechanism than to setup the > preconditions of the plugin. However, they are much larger in terms of > creation, checkout space and time to run. > > I'd like to see that we "unit test" to the greatest extent possible. > IMO, the only reason to go to integration tests is to test lifecycle > interactions and interactions with other plugins. > > What we probably require to do this is convenience methods to construct > a decent project and settings object and an expression populator based > on the project and settings. > > Thoughts? > > - Brett > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- jesse mcconnell jesseDOTmcconnellATgmailDOTcom
