Re: Modernizing IT's in core (and elsewhere ?)
I think trying anything new that's easier is good. The only thing we need to be cognizant of is making sure the body of ITs work to test all the versions of Maven. But I think project self-containment is a good goal. I just liked the setup where there was a project and a file that described the output so you could just run it with doing anything else other than setting up the project and declaratively stating what the results should be. On Dec 2, 2012, at 9:47 AM, Kristian Rosenvold wrote: > 2012/12/2 Jason van Zyl : >> I think the way the ITs used to work was better. Where a single project was >> entirely self-contained and there was a declarative file that described the >> desired outputs. > > Well I think this is one of the design goals I *almost* managed to > achieve when I refactored the surefire IT's; the supplied test project > can run with "mvn clean install" straight out of the box to > demonstrate the problem (as described here > http://maven.apache.org/surefire/maven-surefire-plugin/developing.html). > (This is where I think invoker fails miserably). In surefire I did > this by using the system properties for the plugin parameters as well > as defining standard "properties" in the poms for all the props > without predefined -D values. Writing the actual IT to do the > assertion is a walk in the park, and there's a quite high percentage > of surefire patches that actually contain well-functioning IT's. (This > may be related to personal beliefs of anyone > submitting patches to a test framework ;) > > The core IT's are probably the most complex ITs in all of maven. They > are still not *that* much more complex than the > surefire IT's used to be, where the simplification seems to have paid > off well. The effort of rewriting may not be worth it (although it was > in surefire), but establishing a clear norm for "nice" tests that can > be documented is always a good thing. > > I'm no big fan of BDD frameworks like jbehave or cucumber; I think > simple "java" based DSL's do the job better; especially with what is > essentially a very limited domain like this. I like runnable junit > tests that apply to "real" runnable projects that can be made by end > users. I'm a programmer dammit ;) > > Kristian > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > Thanks, Jason -- Jason van Zyl Founder & CTO, Sonatype Founder, Apache Maven http://twitter.com/jvanzyl - First, the taking in of scattered particulars under one Idea, so that everyone understands what is being talked about ... Second, the separation of the Idea into parts, by dividing it at the joints, as nature directs, not breaking any limb in half as a bad carver might. -- Plato, Phaedrus (Notes on the Synthesis of Form by C. Alexander)
Re: Modernizing IT's in core (and elsewhere ?)
2012/12/2 Jason van Zyl : > I think the way the ITs used to work was better. Where a single project was > entirely self-contained and there was a declarative file that described the > desired outputs. Well I think this is one of the design goals I *almost* managed to achieve when I refactored the surefire IT's; the supplied test project can run with "mvn clean install" straight out of the box to demonstrate the problem (as described here http://maven.apache.org/surefire/maven-surefire-plugin/developing.html). (This is where I think invoker fails miserably). In surefire I did this by using the system properties for the plugin parameters as well as defining standard "properties" in the poms for all the props without predefined -D values. Writing the actual IT to do the assertion is a walk in the park, and there's a quite high percentage of surefire patches that actually contain well-functioning IT's. (This may be related to personal beliefs of anyone submitting patches to a test framework ;) The core IT's are probably the most complex ITs in all of maven. They are still not *that* much more complex than the surefire IT's used to be, where the simplification seems to have paid off well. The effort of rewriting may not be worth it (although it was in surefire), but establishing a clear norm for "nice" tests that can be documented is always a good thing. I'm no big fan of BDD frameworks like jbehave or cucumber; I think simple "java" based DSL's do the job better; especially with what is essentially a very limited domain like this. I like runnable junit tests that apply to "real" runnable projects that can be made by end users. I'm a programmer dammit ;) Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Modernizing IT's in core (and elsewhere ?)
On Dec 2, 2012, at 5:48 AM, Kristian Rosenvold wrote: > Some time ago, I modernized the surefire IT's using a builder on top > of Verifier. I think they are much simpler and more concise than they > used to be. > > Typical samples can be seen here; > > https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire907PerThreadWithoutThreadCountIT.java > https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire628ConsoleOutputBeforeAndAfterClassIT.java > https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire809GroupExpressionsIT.java > > Now I just worked on an IT for MNG-5208 and I must admit I felt the > step "back" to old-school verifier tests feels painful. > > The test can be seen at > > https://git-wip-us.apache.org/repos/asf?p=maven-integration-testing.git;a=blob;f=core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5208EventSpyParallelTest.java > > I would like to do "something" about the core it's, maybe define a > simpler standard for newer tests. I'm really just seeking input on > this ;) > > I think there are a number of issues even with my revised tests. As > can be seen the signal to noise ratio becomes worse than ever; > while the test can be precisely expressed as a one-liner, we > consistently add 35 lines of java boiler-plate including license > header. > > The "builder" I made for surefire could be generalized and moved into > verifier; I'm really interested in both feedback and other > pointers to "nice" ways this has been solved.. Maybe I'm missing > something big ;) I think the way the ITs used to work was better. Where a single project was entirely self-contained and there was a declarative file that described the desired outputs. They couldn't not be run in an IDE but I think that could be fixed. The problem with the current setup is that they are impenetrable for anyone who might want to contribute. I can count on one hand the number of ITs contributed by outsiders. Imagine if a single unit test was a completely self contained project and we used something like JBehave so that someone could describe the desired output given the POM for the test project? I still think that we need to be able to debug and run from the IDE but we can probably make a custom test running. I would not change any of the existing tests, but if you wanted to use your builder and it makes the test more succinct go for it. > > Kristian > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > Thanks, Jason -- Jason van Zyl Founder & CTO, Sonatype Founder, Apache Maven http://twitter.com/jvanzyl - We all have problems. How we deal with them is a measure of our worth. -- Unknown
Modernizing IT's in core (and elsewhere ?)
Some time ago, I modernized the surefire IT's using a builder on top of Verifier. I think they are much simpler and more concise than they used to be. Typical samples can be seen here; https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire907PerThreadWithoutThreadCountIT.java https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire628ConsoleOutputBeforeAndAfterClassIT.java https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=blob;f=surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefire809GroupExpressionsIT.java Now I just worked on an IT for MNG-5208 and I must admit I felt the step "back" to old-school verifier tests feels painful. The test can be seen at https://git-wip-us.apache.org/repos/asf?p=maven-integration-testing.git;a=blob;f=core-it-suite/src/test/java/org/apache/maven/it/MavenITmng5208EventSpyParallelTest.java I would like to do "something" about the core it's, maybe define a simpler standard for newer tests. I'm really just seeking input on this ;) I think there are a number of issues even with my revised tests. As can be seen the signal to noise ratio becomes worse than ever; while the test can be precisely expressed as a one-liner, we consistently add 35 lines of java boiler-plate including license header. The "builder" I made for surefire could be generalized and moved into verifier; I'm really interested in both feedback and other pointers to "nice" ways this has been solved.. Maybe I'm missing something big ;) Kristian - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org