Re: Sharing Test Dependencies
Bernd, If you have two modules 1. Provider API 2. Provider Impl XYZ, ABC and if you write the tests against the API, you can call them integration tests, deploy the JAR to the customer and feel free. It would be just like ordinal module with src/main/java. T On Sun, Jul 4, 2021 at 10:27 PM Bernd Eckenfels wrote: > We have one case in commons, there rhe -test JAR of VFS can be used by > Providers to test their implementation. I did that for my custom provider, > but it is a bit ugly. I think that’s mostly due to relying on some src > files and also the JUnit setup when I remember correctly. But it did work, > even when it’s not what maven project normally do. The test suite could be > its own module, but it would probably not make it much nicer to run in that > case, don’t know. > > I would not expect Test jars to play nicely with application servers, OSGi > bundles or JPMS for that matter. Only with shared classpath junit > “deployment” > > Gruss > Bernd > -- > http://bernd.eckenfels.net > > Von: Tibor Digana > Gesendet: Sunday, July 4, 2021 10:19:45 PM > An: Maven Users List > Betreff: Re: Sharing Test Dependencies > > I did not have time to read it all but I have to say that even the first > point is bad. > Many people want to share test JAR as they initially think it is a good > idea. And then the problems would come. > > sharing stubs? This domain/project may not fit to other domain/project, and > it creates dangerous cohesion. > sharing testing utility classes? Maybe, it depends. It must be universal > and independent of the project's domain. Do it in a separate Git project. > sharing JUnit superclasses? The inheritance must be domain/business > independent. It must be only a technical class. Do it in a separate Git > project. > > T > > > > On Thu, Jul 1, 2021 at 7:15 PM Brandon Mintern > wrote: > > > Hello all, > > > > I'm running up against an issue that I'm sure has come up countless > times. > > How can we share test dependencies in a principled way? I would like to > > configure our projects such that: > > > >1. For a project *P*, its tests are associated with the project, so > that > >`mvn install` from the project directory *p/* fails when P's tests > fail. > >2. Some of P's testing functionality (e.g., stubs, testing utility > >classes, JUnit superclasses) are packaged for use in the tests of > other > >projects* D* that depend on P. > >3. P's shared testing functionality has dependencies on P. For > example, > >P defines a repository interface R, and its tests define and use an > > RStub > >that implements R. We want to package RStub for use by D's tests. > >4. When D declares a scope:test dependency on P's testing > functionality, > >automatically pull in the transitive dependencies of that testing > >functionality. > >5. Avoid cycles in the Maven dependency graph. > > > > Some things I've tried so far: > > > >- Create test-jars for P. This fails on #2 and #4: > >- The test-jars include all of P's test classes instead of just the > > testing functionality that needs to be shared for use by D's tests. > > - All transitive dependencies of P's test-jar must be manually > > specified with scope:test in the D's POM. > >- Extract the stubs to an independent "P - Stubs" project, which P's > >tests depend on. > > - p-stubs depends on P. > > - Stubs, testing utilities, etc. go in p-stubs/src/main/. > > - P depends on p-stubs with scope:test. > > - D depends on p-stubs with scope:test. > > - This fails on #5: it causes a cycle in the dependency graph > since P > > depends on p-stubs (scope:test) and p-stubs depends on P. See > > https://stackoverflow.com/questions/10174542 > >- Create an independent "P - Tests" project, with P's stubs and tests. > > - p-tests depends on P. > > - Stubs, testing utilities, etc. go in p-tests/src/main/. > > - P's tests go in p-tests/src/test/. > > - D depends on p-tests with scope:test. > > - This almost works but fails on #1: P's build succeeds when its > > tests fail. > > - This approach also seems to be fighting Maven and IDE tooling. > For > > example, NetBeans will automatically create p/src/test even > > though we never > > intend to put anything there. > > > > My searches so far have not turned up any complete solutions to this > > problem. The maven-jar-plugin documentation has a
Re: Sharing Test Dependencies
You can easily solve this. Just create (N+1) module which contains test classes. The N+1 module should inherit from the module N having normal sources. The trick is to build module N, and then N+1. On Fri, Jul 9, 2021 at 1:13 AM Brandon Mintern wrote: > Tibor, > > Thanks for your thoughts. Would it be worthwhile for me to construct and > share a minimal concrete example to motivate this discussion? It's not > clear to me that you're open to the possibility that I'm describing a > reasonable use case here. > > On Thu, Jul 8, 2021 at 4:06 PM Tibor Digana > wrote: > > > The tests are dedicated to the module sources and not to the other > > module/s. > > They were not designed to be inherited and it is logical because unit > tests > > have to test a small unit code where the unit is a method, class or a > > module. > > Integration tests are used to test the whole application which is a bunch > > of compiled and packaged modules but these tests also do not need to be > > shared. It's enough if a separate module is called IT and it is built at > > last in the CI process. > > > > On Fri, Jul 9, 2021 at 12:41 AM Andy Feldman > > wrote: > > > > > On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern > > > wrote: > > > > > > > Maybe one of these—or a better alternative—is already possible? I > feel > > > like > > > > I must be missing something. Is something wrong with the way I'm > > > > structuring my projects? Does Maven already provide a way to achieve > > this > > > > out-of-the-box? Is there a plugin that provides something like the > > > "stubs" > > > > functionality? > > > > > > > > > > There was some discussion of this issue on this list a year ago as > well: > > > > > > > > > https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E > > > > > > The conclusion was also that there's no great way to accomplish this. I > > > think one good way to fix this issue would be to have Maven resolve > > > test-scoped dependencies transitively when you depend on test-jars, but > > > perhaps there's a good reason why that's not practical or not a good > > idea. > > > > > > -- > > > Andy Feldman > > > > > >
Re: Sharing Test Dependencies
No, it cannot be based on one use case. It must be based on a theory and generic and representative principles. On Fri, Jul 9, 2021 at 1:13 AM Brandon Mintern wrote: > Tibor, > > Thanks for your thoughts. Would it be worthwhile for me to construct and > share a minimal concrete example to motivate this discussion? It's not > clear to me that you're open to the possibility that I'm describing a > reasonable use case here. > > On Thu, Jul 8, 2021 at 4:06 PM Tibor Digana > wrote: > > > The tests are dedicated to the module sources and not to the other > > module/s. > > They were not designed to be inherited and it is logical because unit > tests > > have to test a small unit code where the unit is a method, class or a > > module. > > Integration tests are used to test the whole application which is a bunch > > of compiled and packaged modules but these tests also do not need to be > > shared. It's enough if a separate module is called IT and it is built at > > last in the CI process. > > > > On Fri, Jul 9, 2021 at 12:41 AM Andy Feldman > > wrote: > > > > > On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern > > > wrote: > > > > > > > Maybe one of these—or a better alternative—is already possible? I > feel > > > like > > > > I must be missing something. Is something wrong with the way I'm > > > > structuring my projects? Does Maven already provide a way to achieve > > this > > > > out-of-the-box? Is there a plugin that provides something like the > > > "stubs" > > > > functionality? > > > > > > > > > > There was some discussion of this issue on this list a year ago as > well: > > > > > > > > > https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E > > > > > > The conclusion was also that there's no great way to accomplish this. I > > > think one good way to fix this issue would be to have Maven resolve > > > test-scoped dependencies transitively when you depend on test-jars, but > > > perhaps there's a good reason why that's not practical or not a good > > idea. > > > > > > -- > > > Andy Feldman > > > > > >
Re: Sharing Test Dependencies
Tibor, Thanks for your thoughts. Would it be worthwhile for me to construct and share a minimal concrete example to motivate this discussion? It's not clear to me that you're open to the possibility that I'm describing a reasonable use case here. On Thu, Jul 8, 2021 at 4:06 PM Tibor Digana wrote: > The tests are dedicated to the module sources and not to the other > module/s. > They were not designed to be inherited and it is logical because unit tests > have to test a small unit code where the unit is a method, class or a > module. > Integration tests are used to test the whole application which is a bunch > of compiled and packaged modules but these tests also do not need to be > shared. It's enough if a separate module is called IT and it is built at > last in the CI process. > > On Fri, Jul 9, 2021 at 12:41 AM Andy Feldman > wrote: > > > On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern > > wrote: > > > > > Maybe one of these—or a better alternative—is already possible? I feel > > like > > > I must be missing something. Is something wrong with the way I'm > > > structuring my projects? Does Maven already provide a way to achieve > this > > > out-of-the-box? Is there a plugin that provides something like the > > "stubs" > > > functionality? > > > > > > > There was some discussion of this issue on this list a year ago as well: > > > > > https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E > > > > The conclusion was also that there's no great way to accomplish this. I > > think one good way to fix this issue would be to have Maven resolve > > test-scoped dependencies transitively when you depend on test-jars, but > > perhaps there's a good reason why that's not practical or not a good > idea. > > > > -- > > Andy Feldman > > >
Re: Sharing Test Dependencies
The tests are dedicated to the module sources and not to the other module/s. They were not designed to be inherited and it is logical because unit tests have to test a small unit code where the unit is a method, class or a module. Integration tests are used to test the whole application which is a bunch of compiled and packaged modules but these tests also do not need to be shared. It's enough if a separate module is called IT and it is built at last in the CI process. On Fri, Jul 9, 2021 at 12:41 AM Andy Feldman wrote: > On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern > wrote: > > > Maybe one of these—or a better alternative—is already possible? I feel > like > > I must be missing something. Is something wrong with the way I'm > > structuring my projects? Does Maven already provide a way to achieve this > > out-of-the-box? Is there a plugin that provides something like the > "stubs" > > functionality? > > > > There was some discussion of this issue on this list a year ago as well: > > https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E > > The conclusion was also that there's no great way to accomplish this. I > think one good way to fix this issue would be to have Maven resolve > test-scoped dependencies transitively when you depend on test-jars, but > perhaps there's a good reason why that's not practical or not a good idea. > > -- > Andy Feldman >
Re: Sharing Test Dependencies
Thanks for the pointer to that conversation! Andreas described exactly my issue in a much clearer and more concise way. For now, test-jars seem to be the best path forward in spite of the drawbacks. I'm considering writing a plugin to prototype the "stubs" approach that I described. On Thu, Jul 8, 2021 at 3:41 PM Andy Feldman wrote: > On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern > wrote: > > > Maybe one of these—or a better alternative—is already possible? I feel > like > > I must be missing something. Is something wrong with the way I'm > > structuring my projects? Does Maven already provide a way to achieve this > > out-of-the-box? Is there a plugin that provides something like the > "stubs" > > functionality? > > > > There was some discussion of this issue on this list a year ago as well: > > https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E > > The conclusion was also that there's no great way to accomplish this. I > think one good way to fix this issue would be to have Maven resolve > test-scoped dependencies transitively when you depend on test-jars, but > perhaps there's a good reason why that's not practical or not a good idea. > > -- > Andy Feldman >
Re: Sharing Test Dependencies
On Thu, Jul 1, 2021 at 12:15 PM Brandon Mintern wrote: > Maybe one of these—or a better alternative—is already possible? I feel like > I must be missing something. Is something wrong with the way I'm > structuring my projects? Does Maven already provide a way to achieve this > out-of-the-box? Is there a plugin that provides something like the "stubs" > functionality? > There was some discussion of this issue on this list a year ago as well: https://lists.apache.org/thread.html/r6bfcf85aa7fd2b9a02a3f2513b9e10f1141b9102fda2bfc533d02379%40%3Cusers.maven.apache.org%3E The conclusion was also that there's no great way to accomplish this. I think one good way to fix this issue would be to have Maven resolve test-scoped dependencies transitively when you depend on test-jars, but perhaps there's a good reason why that's not practical or not a good idea. -- Andy Feldman
Re: Sharing Test Dependencies
We have one case in commons, there rhe -test JAR of VFS can be used by Providers to test their implementation. I did that for my custom provider, but it is a bit ugly. I think that’s mostly due to relying on some src files and also the JUnit setup when I remember correctly. But it did work, even when it’s not what maven project normally do. The test suite could be its own module, but it would probably not make it much nicer to run in that case, don’t know. I would not expect Test jars to play nicely with application servers, OSGi bundles or JPMS for that matter. Only with shared classpath junit “deployment” Gruss Bernd -- http://bernd.eckenfels.net Von: Tibor Digana Gesendet: Sunday, July 4, 2021 10:19:45 PM An: Maven Users List Betreff: Re: Sharing Test Dependencies I did not have time to read it all but I have to say that even the first point is bad. Many people want to share test JAR as they initially think it is a good idea. And then the problems would come. sharing stubs? This domain/project may not fit to other domain/project, and it creates dangerous cohesion. sharing testing utility classes? Maybe, it depends. It must be universal and independent of the project's domain. Do it in a separate Git project. sharing JUnit superclasses? The inheritance must be domain/business independent. It must be only a technical class. Do it in a separate Git project. T On Thu, Jul 1, 2021 at 7:15 PM Brandon Mintern wrote: > Hello all, > > I'm running up against an issue that I'm sure has come up countless times. > How can we share test dependencies in a principled way? I would like to > configure our projects such that: > >1. For a project *P*, its tests are associated with the project, so that >`mvn install` from the project directory *p/* fails when P's tests fail. >2. Some of P's testing functionality (e.g., stubs, testing utility >classes, JUnit superclasses) are packaged for use in the tests of other >projects* D* that depend on P. >3. P's shared testing functionality has dependencies on P. For example, >P defines a repository interface R, and its tests define and use an > RStub >that implements R. We want to package RStub for use by D's tests. >4. When D declares a scope:test dependency on P's testing functionality, >automatically pull in the transitive dependencies of that testing >functionality. >5. Avoid cycles in the Maven dependency graph. > > Some things I've tried so far: > >- Create test-jars for P. This fails on #2 and #4: >- The test-jars include all of P's test classes instead of just the > testing functionality that needs to be shared for use by D's tests. > - All transitive dependencies of P's test-jar must be manually > specified with scope:test in the D's POM. >- Extract the stubs to an independent "P - Stubs" project, which P's >tests depend on. > - p-stubs depends on P. > - Stubs, testing utilities, etc. go in p-stubs/src/main/. > - P depends on p-stubs with scope:test. > - D depends on p-stubs with scope:test. > - This fails on #5: it causes a cycle in the dependency graph since P > depends on p-stubs (scope:test) and p-stubs depends on P. See > https://stackoverflow.com/questions/10174542 >- Create an independent "P - Tests" project, with P's stubs and tests. > - p-tests depends on P. > - Stubs, testing utilities, etc. go in p-tests/src/main/. > - P's tests go in p-tests/src/test/. > - D depends on p-tests with scope:test. > - This almost works but fails on #1: P's build succeeds when its > tests fail. > - This approach also seems to be fighting Maven and IDE tooling. For > example, NetBeans will automatically create p/src/test even > though we never > intend to put anything there. > > My searches so far have not turned up any complete solutions to this > problem. The maven-jar-plugin documentation has a section > < > https://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html#The_preferred_way > > > that touches on this issue: > > The preferred way > > In order to let Maven resolve all test-scoped transitive dependencies you > should create a separate project. > > >1. >2.groupId >3. artifactId-tests >4. version >5. ... >6. > > >- Move the sources files from src/test/java you want to share from the >original project to the src/main/java of this project. The same type of >movement counts for the resources as well of course. >- Move the required test-scoped dependencies and from the original >project to this project and remove the scope (i.
Re: Sharing Test Dependencies
I did not have time to read it all but I have to say that even the first point is bad. Many people want to share test JAR as they initially think it is a good idea. And then the problems would come. sharing stubs? This domain/project may not fit to other domain/project, and it creates dangerous cohesion. sharing testing utility classes? Maybe, it depends. It must be universal and independent of the project's domain. Do it in a separate Git project. sharing JUnit superclasses? The inheritance must be domain/business independent. It must be only a technical class. Do it in a separate Git project. T On Thu, Jul 1, 2021 at 7:15 PM Brandon Mintern wrote: > Hello all, > > I'm running up against an issue that I'm sure has come up countless times. > How can we share test dependencies in a principled way? I would like to > configure our projects such that: > >1. For a project *P*, its tests are associated with the project, so that >`mvn install` from the project directory *p/* fails when P's tests fail. >2. Some of P's testing functionality (e.g., stubs, testing utility >classes, JUnit superclasses) are packaged for use in the tests of other >projects* D* that depend on P. >3. P's shared testing functionality has dependencies on P. For example, >P defines a repository interface R, and its tests define and use an > RStub >that implements R. We want to package RStub for use by D's tests. >4. When D declares a scope:test dependency on P's testing functionality, >automatically pull in the transitive dependencies of that testing >functionality. >5. Avoid cycles in the Maven dependency graph. > > Some things I've tried so far: > >- Create test-jars for P. This fails on #2 and #4: >- The test-jars include all of P's test classes instead of just the > testing functionality that needs to be shared for use by D's tests. > - All transitive dependencies of P's test-jar must be manually > specified with scope:test in the D's POM. >- Extract the stubs to an independent "P - Stubs" project, which P's >tests depend on. > - p-stubs depends on P. > - Stubs, testing utilities, etc. go in p-stubs/src/main/. > - P depends on p-stubs with scope:test. > - D depends on p-stubs with scope:test. > - This fails on #5: it causes a cycle in the dependency graph since P > depends on p-stubs (scope:test) and p-stubs depends on P. See > https://stackoverflow.com/questions/10174542 >- Create an independent "P - Tests" project, with P's stubs and tests. > - p-tests depends on P. > - Stubs, testing utilities, etc. go in p-tests/src/main/. > - P's tests go in p-tests/src/test/. > - D depends on p-tests with scope:test. > - This almost works but fails on #1: P's build succeeds when its > tests fail. > - This approach also seems to be fighting Maven and IDE tooling. For > example, NetBeans will automatically create p/src/test even > though we never > intend to put anything there. > > My searches so far have not turned up any complete solutions to this > problem. The maven-jar-plugin documentation has a section > < > https://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html#The_preferred_way > > > that touches on this issue: > > The preferred way > > In order to let Maven resolve all test-scoped transitive dependencies you > should create a separate project. > > >1. >2.groupId >3. artifactId-tests >4. version >5. ... >6. > > >- Move the sources files from src/test/java you want to share from the >original project to the src/main/java of this project. The same type of >movement counts for the resources as well of course. >- Move the required test-scoped dependencies and from the original >project to this project and remove the scope (i.e. changing it to the >compile-scope). And yes, that means that the junit dependency (or any >other testing framework dependency) gets the default scope too. You'll >probably need to add some project specific dependencies as well to let > it >all compile again. > > Now you have your reusable *test-classes* and you can refer to it as you're > used to... > > This is along the lines of the "P - Stubs" approach I suggested above, but > it unfortunately cannot work since the stubs themselves depend on P (fails > on #3 above). > > Is there a satisfying way to solve this problem? It seems to me like any > one of the following changes would resolve the issue. > >- Allow P to depend on a separate project p-stubs with scope:test, even >though p-stubs depends on P (instead of calling it a dependency cycle). >This means that the build order would be a bit awkward: P (compile) → >p-stubs (compile) → P (test). >- Allow P to indicate that its tests are in another project p-tests. >That way, `mvn install` in p/ would continue to