Re: RE: Re: Functional Test strategies with Maven
Hi Steven, Oh absolutely, sorry when I said "we" I meant "my organisation" :) Cheers, Martijn On Mar 2, 2009 5:32pm, "Cummings,Steven" wrote: Trying to achieve consistency between organizations ahead of time could prove difficult. I would instead suggest trying for consistency within your organization and sharing notes about what worked and what didn't either on this mailing list or perhaps the Maven wiki. Perhaps that could help provide feedback for if/when integration testing is officially supported. /S -Original Message- From: martijnverb...@gmail.com [mailto:martijnverb...@gmail.com] Sent: Monday, March 02, 2009 11:26 AM To: users@maven.apache.org Subject: Re: Re: Functional Test strategies with Maven Hi David, > Martijn - I've seen this done a couple of different ways. I can't say > that there is any particular set of advantages/disadvantages that makes > one more attractive than another. I'm sure > you've already thought about these, but here they are, just the same: > #1 - The functional tests are named with a specific suffix (ex. _FT) so > that they can be included/excluded by profile. They still live in the > src/test/java tree alongside the unit tests. In > this case, you must explicitly include and/or exclude tests based on a > pattern matching their class names. > #2 - Place all of your functional tests in a different source tree (ex. > src/functionalTest/java). As with #1, you have to do some include/exclude > type work, but in this case, you only > have to do so on your profile that runs the functional tests. However, > you also have to make sure to include this tree on your compilation. > So, IMHO, there's no clear winner. I've been dealing with similar > problems lately for Integration Tests that actually start up my web app > and test it using Selenium. I have the same > set of problems. I think I read that there are plans to enhance Maven > with the knowledge of functional/integration tests, but I don't know what > the timing for any such change is. Thanks David, this helps (along with Ian's comments) confirm that there is currently no 'official' Maven life-cycle support for these types of tests (yet), hopefully as long as we're consistent in our approach any of the suggested solutions will be more than adequate :). Cheers, Martijn PS: I will also post separately about the 'true' integration tests scenario (where we have tests that can be JUnit run but involve many modules and/or projects and can also have 3rd party requirements such as a J2EE server up and running). > The specific purpose in this case is to ensure that we've wired up > messaging flow(s) within our module. This is done by sending a mocked > event > through the various routers/transformers etc that we have. They're not > unit > tests because they're not testing a discrete bit of functionality within a > Java class. > However, it is vital that these tests pass before we package that module, > so I we want to run them as part of a the test life-cycle (as opposed to > integration-test which is post package) or at least a life-cycle phase > that > is before package. > I agree that we could use profiles to run this group of tests separately, > but I'd like to understand where they should live in the source structure. > All of the Maven literature that I have read talks of > name>-->test-->src-->java being used for unit tests, but tests such as > mine > fall into a grey area? > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org -- CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, USA at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
RE: Re: Functional Test strategies with Maven
Trying to achieve consistency between organizations ahead of time could prove difficult. I would instead suggest trying for consistency within your organization and sharing notes about what worked and what didn't either on this mailing list or perhaps the Maven wiki. Perhaps that could help provide feedback for if/when integration testing is officially supported. /S -Original Message- From: martijnverb...@gmail.com [mailto:martijnverb...@gmail.com] Sent: Monday, March 02, 2009 11:26 AM To: users@maven.apache.org Subject: Re: Re: Functional Test strategies with Maven Hi David, > Martijn - I've seen this done a couple of different ways. I can't say > that there is any particular set of advantages/disadvantages that makes > one more attractive than another. I'm sure > you've already thought about these, but here they are, just the same: > #1 - The functional tests are named with a specific suffix (ex. _FT) so > that they can be included/excluded by profile. They still live in the > src/test/java tree alongside the unit tests. In > this case, you must explicitly include and/or exclude tests based on a > pattern matching their class names. > #2 - Place all of your functional tests in a different source tree (ex. > src/functionalTest/java). As with #1, you have to do some include/exclude > type work, but in this case, you only > have to do so on your profile that runs the functional tests. However, > you also have to make sure to include this tree on your compilation. > So, IMHO, there's no clear winner. I've been dealing with similar > problems lately for Integration Tests that actually start up my web app > and test it using Selenium. I have the same > set of problems. I think I read that there are plans to enhance Maven > with the knowledge of functional/integration tests, but I don't know what > the timing for any such change is. Thanks David, this helps (along with Ian's comments) confirm that there is currently no 'official' Maven life-cycle support for these types of tests (yet), hopefully as long as we're consistent in our approach any of the suggested solutions will be more than adequate :). Cheers, Martijn PS: I will also post separately about the 'true' integration tests scenario (where we have tests that can be JUnit run but involve many modules and/or projects and can also have 3rd party requirements such as a J2EE server up and running). > The specific purpose in this case is to ensure that we've wired up > messaging flow(s) within our module. This is done by sending a mocked > event > through the various routers/transformers etc that we have. They're not > unit > tests because they're not testing a discrete bit of functionality within a > Java class. > However, it is vital that these tests pass before we package that module, > so I we want to run them as part of a the test life-cycle (as opposed to > integration-test which is post package) or at least a life-cycle phase > that > is before package. > I agree that we could use profiles to run this group of tests separately, > but I'd like to understand where they should live in the source structure. > All of the Maven literature that I have read talks of > name>-->test-->src-->java being used for unit tests, but tests such as > mine > fall into a grey area? > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org -- CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Re: Functional Test strategies with Maven
Hi David, Martijn - I've seen this done a couple of different ways. I can't say that there is any particular set of advantages/disadvantages that makes one more attractive than another. I'm sure you've already thought about these, but here they are, just the same: #1 - The functional tests are named with a specific suffix (ex. _FT) so that they can be included/excluded by profile. They still live in the src/test/java tree alongside the unit tests. In this case, you must explicitly include and/or exclude tests based on a pattern matching their class names. #2 - Place all of your functional tests in a different source tree (ex. src/functionalTest/java). As with #1, you have to do some include/exclude type work, but in this case, you only have to do so on your profile that runs the functional tests. However, you also have to make sure to include this tree on your compilation. So, IMHO, there's no clear winner. I've been dealing with similar problems lately for Integration Tests that actually start up my web app and test it using Selenium. I have the same set of problems. I think I read that there are plans to enhance Maven with the knowledge of functional/integration tests, but I don't know what the timing for any such change is. Thanks David, this helps (along with Ian's comments) confirm that there is currently no 'official' Maven life-cycle support for these types of tests (yet), hopefully as long as we're consistent in our approach any of the suggested solutions will be more than adequate :). Cheers, Martijn PS: I will also post separately about the 'true' integration tests scenario (where we have tests that can be JUnit run but involve many modules and/or projects and can also have 3rd party requirements such as a J2EE server up and running). The specific purpose in this case is to ensure that we've wired up messaging flow(s) within our module. This is done by sending a mocked event through the various routers/transformers etc that we have. They're not unit tests because they're not testing a discrete bit of functionality within a Java class. However, it is vital that these tests pass before we package that module, so I we want to run them as part of a the test life-cycle (as opposed to integration-test which is post package) or at least a life-cycle phase that is before package. I agree that we could use profiles to run this group of tests separately, but I'd like to understand where they should live in the source structure. All of the Maven literature that I have read talks of name>-->test-->src-->java being used for unit tests, but tests such as mine fall into a grey area? - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Re: Re: Functional Test strategies with Maven
Hi all, Are these functional tests invoked as jUnit / TestNG tests? I have written similar, integration type functional tests with mocks, but they were still based on jUnit / TestNG. I simply stored them in the same name>-->test-->src-->java are being used for unit tests. I differentiated between the two sets through the package names and used profiles to run these optionally. Yes they are JUnit tests and the idea of differentiating by package names is definitely one we've discussed. It's interesting to know that someone is using that technique! My reasoning was that its easiest to stick with the vanilla Maven directory structure as it is a standard. This approach certainly worked for me at the time. I think it would definitely work for us as well, but it's an interesting 'gap' in the standard maven project and so we hesitated on which way to go :) ps: I think you replied directly to me, rather reply to the group email address. Apologies for that, I'm using the iGoogle gmail interface, and the reply picks up the user and not the mailing list address first, add my stupidity on top of that and you get "mailing list fail" :) Original post back to Ian Hi Ian, What is the reasoning beyond the separation of the functional tests? Is it because you would like to run them separately from your other unit tests? If so, have you considered using profiles to control what tests are run by Maven? The specific purpose in this case is to ensure that we've wired up messaging flow(s) within our module. This is done by sending a mocked event through the various routers/transformers etc that we have. They're not unit tests because they're not testing a discrete bit of functionality within a Java class. However, it is vital that these tests pass before we package that module, so I we want to run them as part of a the test life-cycle (as opposed to integration-test which is post package) or at least a life-cycle phase that is before package. I agree that we could use profiles to run this group of tests separately, but I'd like to understand where they should live in the source structure. All of the Maven literature that I have read talks of name>-->test-->src-->java being used for unit tests, but tests such as mine fall into a grey area? --- Thanks again for your insights Ian. Does anyone else follow a different/similar method? Cheers, Martijn On Mon, Mar 2, 2009 at 4:08 PM, martijnverb...@gmail.com> wrote: > Hi Ian, > > > What is the reasoning beyond the separation of the functional tests? Is > it > > because you would like to run them separately from your other unit tests? > If > > so, have you considered using profiles to control what tests are run by > > Maven? > > The specific purpose in this case is to ensure that we've wired up > messaging flow(s) within our module. This is done by sending a mocked event > through the various routers/transformers etc that we have. They're not unit > tests because they're not testing a discrete bit of functionality within a > Java class. > > However, it is vital that these tests pass before we package that module, > so I we want to run them as part of a the test life-cycle (as opposed to > integration-test which is post package) or at least a life-cycle phase that > is before package. > > I agree that we could use profiles to run this group of tests separately, > but I'd like to understand where they should live in the source structure. > All of the Maven literature that I have read talks of > name>-->test-->src-->java being used for unit tests, but tests such as mine > fall into a grey area? > > Cheers, > Martijn > > > > On Mon, Mar 2, 2009 at 3:33 PM, martijnverb...@gmail.com> wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'd like to get opinions from others on how they structure/deal with > what > > > > > we deem 'functional tests' > > > > > > > > > > To us these are tests that are broader in scope than a unit test but > whose > > > > > scope is still limited to that same module and require no 3rd party > > > > > resources to run (we're able to Mock everything we need). Traditionally > a > > > > > Maven structure looks something like: > > > > > > > > > > foobar > > > > > jar > > > > > src > > > > > main > > > > > java > > > > > test > > > > > java > > > > > war > > > > > ... > > > > > sar > > > > > ... > > > > > rar > > > > > ... > > > > > ear > > > > > > > > > > With the 'test' structure being used for Unit Tests. > > > > > > > > > > So we're wondering if it was wise to put the functional te
Re: Re: Functional Test strategies with Maven
Are these functional tests invoked as jUnit / TestNG tests? I have written similar, integration type functional tests with mocks, but they were still based on jUnit / TestNG. I simply stored them in the same -->test-->src-->java are being used for unit tests. I differentated between the two sets through the package names and used profiles to run these optionally. My reasoning was that its easiest to stick with the vanilla Maven directory structure as it is a standard. This approach certainly worked for me at the time. I guess that would be my suggestion. ps: I think you replied directly to me, rather reply to the group email address. On Mon, Mar 2, 2009 at 4:08 PM, wrote: > Hi Ian, > > > What is the reasoning beyond the separation of the functional tests? Is > it > > because you would like to run them separately from your other unit tests? > If > > so, have you considered using profiles to control what tests are run by > > Maven? > > The specific purpose in this case is to ensure that we've wired up > messaging flow(s) within our module. This is done by sending a mocked event > through the various routers/transformers etc that we have. They're not unit > tests because they're not testing a discrete bit of functionality within a > Java class. > > However, it is vital that these tests pass before we package that module, > so I we want to run them as part of a the test life-cycle (as opposed to > integration-test which is post package) or at least a life-cycle phase that > is before package. > > I agree that we could use profiles to run this group of tests separately, > but I'd like to understand where they should live in the source structure. > All of the Maven literature that I have read talks of name>-->test-->src-->java being used for unit tests, but tests such as mine > fall into a grey area? > > Cheers, > Martijn > > > > On Mon, Mar 2, 2009 at 3:33 PM, martijnverb...@gmail.com> wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'd like to get opinions from others on how they structure/deal with > what > > > > > we deem 'functional tests' > > > > > > > > > > To us these are tests that are broader in scope than a unit test but > whose > > > > > scope is still limited to that same module and require no 3rd party > > > > > resources to run (we're able to Mock everything we need). Traditionally > a > > > > > Maven structure looks something like: > > > > > > > > > > foobar > > > > > jar > > > > > src > > > > > main > > > > > java > > > > > test > > > > > java > > > > > war > > > > > ... > > > > > sar > > > > > ... > > > > > rar > > > > > ... > > > > > ear > > > > > > > > > > With the 'test' structure being used for Unit Tests. > > > > > > > > > > So we're wondering if it was wise to put the functional tests under > > > > > jar-->test-->src-->java (perhaps differentiating them by package > structure) > > > > > or introduce a new module (say functional-test) eg > > > > > > > > > > foobar > > > > > jar > > > > > src > > > > > main > > > > > java > > > > > test > > > > > java > > > > > functional-test > > > > > src > > > > > test > > > > > java > > > > > war > > > > > ... > > > > > sar > > > > > ... > > > > > rar > > > > > ... > > > > > ear > > > > > ... > > > > > > > > > > Is there a preferred technique/other options that people have used? > > > > > > > >
