On 26/08/2011, at 6:45 AM, Adam Murdoch wrote: > Hi, > > I'm not sure I like moving (some of) core/src/test out into a separate > project. As a general pattern, I think I would much rather that the test > fixtures exported by a project live in that project, rather than in a > separate project, and are modelled as a separate publication. > > That is, right now, we have the following dependencies: > > core test -> internalTesting main -> core main > > I'd much rather: > > core test -> core testFixtures -> core main.
core-test is going to have a compile dependency on core-main as well though, so it's not quite that linear (unsure if that's relevant here). > where 'core testFixtures' are the core-specific test fixtures, such as > classes like HelperUtil, AbstractTaskTest, AbstractTaskSpec. This doesn't > include int testing fixtures such as GradleExecutor, and probably doesn't > include general things such as TemporaryFolder, TestFile, and so on. Makes sense. If you are compiling against core, then you'll likely want to compile against coreTestFixtures. However, at some point we'll probably want to publish a core-fixtures jar of public utilities for this and we may want to actually have tests for these test fixtures. This could be the same for any plugin, e.g. we might want to publish a maven-fixtures jar for unit test fixtures for developers who integrate with/use types from the maven plugin(s). The same could be said for integration tests. There may be a maven-integ-fixtures that provides an in memory http server for setting up repos for integration tests. I think this _potentially_ leaves us with the following groups of source for a project main test testFixtures integTest integTestFixtures If we do want to have public test fixtures that might mean another four (names aside): publicTestFixtures publicTestFixturesTest publicIntegTestFixtures publicIntegTestFixturesTest This is starting to look like too much, but all of these (apart from main) would be optional. I wouldn't expect many plugins to expose public utilities for integration testing but it could definitely happen (and should in some cases). Also, I'd rather have this source set explosion and very carefully control what's public than try and jam them together (but I guess having so many projects may be burdensome in the IDE). > One problem - apart from the conceptual awkwardness - with having a project's > test fixtures in a separate project is that it blows our 'must be able to > import into eclipse' story right out of the water, as the eclipse plugin will > map these to: > > core -> internalTesting -> core > > which, of course, eclipse can't handle. If we move the integration testing harness out into its own project (as I think you said we should), won't we have the same problem? Actually, I guess not if we craft it to not depend on core which might be possible. > Another issue (for me) is that idea treats the test fixtures as production > code, not test code. This breaks things such as search for usages. So IDEA knows that the test source sets etc. are not “production”? Does this also mean that for the “public” test fixtures we'd need to have them as separate projects so IDEA knows they are production code? > I think there are some better options: > > 1. We push the separate projects option even further, and bust up every > project into 4 separate projects: main, testFixtures, tests, and integTests. > > 2. Test fixtures live in the owning project, and we map each source set to a > separate eclipse project, so we end up with -main, -test, and -integTests > eclipse projects. Possibly combine this with adding a testFixtures source set > as well. I don't understand the last sentence. Does that just mean a -testFixtures project (just like -integTests) or something else? This seems like the winner to me. > 3. Test fixtures live in the owning project, and we map the integTest source > set of each project to a separate eclipse project. Possibly combine this with > adding a testFixtures source set as well. > > What is currently 'internalTesting' would split into 3 pieces: the general > purpose test fixtures, the core project's test fixtures, and the integration > testing fixtures. These would be treated as per whichever of the above > options we choose. > > My vote is that the text fixtures exported by a project live in that project, > which means either option 2 or 3. Option 2 might be the better option. And it > might be the better option for the Eclipse plugin as a default behaviour, too. > > Short term fix is to move core's (and plugin's) test fixtures back into core > and plugin. I'd really rather not unless directed. It shouldn't be too difficult to do the right thing. At the very least, I'd like to separate core tests from core test fixtures. In pulling out the internalTesting project I had to detangle a few things and I'd rather not open the possibility up to do that again. -- Luke Daley Principal Engineer, Gradleware http://gradleware.com
