Hi Just a thought, but unless it is an already established convention, you might reconsider the use of the # as the class/method separator. The # often means "everything after this is a comment", in particular in unix shells. This might then require quoting of arguments and odd errors where the command line looks "right" but does something very different because of the shell preprocessing.
An alternative may be to stick with "." as the separator, but check both the last word and the second to last word as the class. For example with a pattern like A.B.C you would first look for a test class named C in package A.B, but if that failed you would look for a test class named B in package A with a method named C. I think that more closely mirrors usage in java, and so might be more natural for users. thanks Philip On Mon, Nov 18, 2013 at 6:13 AM, Szczepan Faber < szczepan.fa...@gradleware.com> wrote: > Hey, > > I'll come up with an initial spec for this, please give feedback. The spec > lives here: > https://github.com/gradle/gradle/blob/master/design-docs/test-selection.mdI'm > including the content here because some formatting is not good. > > Use case: legacy integration tests are slow, need a way to execute a > single test method, from command line. > > Options: > > 1. Change the semantics of test.include so that it supports syntax like > {ant-file-pattern}#{testMethodName}, e.g. **/FooTest#someMethod > Test implements PatternFilterable, which declares the exact behavior of > include / exclude. In order to implement this option, we would need to > introduce a mildly breaking change and make Test no longer extend > PatternFilterable. It is kind of a mildly breaking because we would keep > all the existing public methods in the Test type. If users extends Test > task, they would probably need to recompile. For command line, we introduce > --include option for the Test task. Pros: api remains small, Cons: breaking > change. > > 2. Introduce new dsl (various ideas lumped together): > > test { > selection { > //using 'include' wording here could make it confusing with > test.include > select "**/Foo*#someOtherMethod", "**/Foo*#someMethod" > selections = [] > //in/out > in "**/Foo*#someOtherMethod", "**/Foo*#someMethod" > out "**/Foo*#someOtherMethod", "**/Foo*#someMethod" > > //if we deprecate test.include, we could do > include "**/Foo*#someOtherMethod", "**/Foo*#someMethod" > includes = [] > > //keep method selection separate > includeMethod > includeMethods = [] > > include { > method > methods = [] > } > > //some other elements we could add in future > exclude > excludes = [] > > unselect > unselections = [] > > include { > descendantsOf 'com.foo.SomeBaseClass' > annotatedWith 'com.foo.Slow' > matching { descriptor, testClass -> > //... > } > } > } > } > > Then add consistent commandline support, e.g. > > gradle test --select **/Foo.java#someTest > > 3. It would be good to consider the command line interface when designing > the dsl because they need to be consistent. The convenient command line > should support: > - selecting class + method in one option > - allow wildcards for classes (wildcards for methods are not practical to > implement) > - support both, file separators '/' or fqn separators '.' > > --select com/bar/**/Foo.java#someTest > --select Foo#someTest > --select com.foo.bar.Foo#otherTest > > 4. Here's api I was thinking about as a first story, please give feedback: > > test { > //good old include > include '**/*SomeIntegrationTest' > > //new stuff > selection { > include { > method 'method1', 'method2' > methods = [] > } > } > } > > Plus, a convenience method directly in the test task, so that all above > could be inlined into: > > test { > select '**/*SomeIntegrationTest#method1,method2' > //or: > select '**/*SomeIntegrationTest#method1', > '**/*SomeIntegrationTest#method2' > } > > Then, command line support could be: > > gradle test --select **/*SomeIntegrationTest#method1,method2 > > I would also deprecate test.single property > > Cheers! > > > On Fri, Nov 15, 2013 at 4:23 AM, Adam Murdoch <adam.murd...@gradleware.com > > wrote: > >> >> On 13 Nov 2013, at 7:36 am, Szczepan Faber <szczepan.fa...@gradleware.com> >> wrote: >> >> Heya, >> >> I'm keen on solving this use case: developer in a large app has tons of >> long running tests. He wants to run only a *single* test from the command >> line (e.g. we currently support *all* tests from a *single* test class via >> test.single). >> >> There's also a pull request: https://github.com/gradle/gradle/pull/193that >> shows that the community is also keen on getting it sorted out. >> >> How do we want approach this? >> >> 1. Make the test.includes / test.excludes more robust and support some >> kind of notation like "SomeTest#someMethod". This is kind of awkward >> because Test extends PatternFilterable that clearly defines what are >> includes and excludes (they are not test methods, they are ant file >> patterns). On the plus side, this would work with our existing means of >> specifying includes/excludes (existing DSL, existing test.single property) >> >> 2. Add some new API for this, for example: >> >> test.selection.include ... >> >> It might be good to put the test selection api behind some new DSL >> layer/object (e.g. 'selection') because it can potentially grow: >> include/exclude tests by some custom criteria, e.g. test class hierarchy, >> etc. BTW. for the latter we have use cases in our own codebase (cross >> version tests now are picked up by naming convention but it would be nicer >> if they were picked up by parent class, etc.). >> >> If we go down this path, we need to add command line support for it and >> perhaps consider deprecation of the existing include / exclude. >> >> Do we have some other options? Thoughts? >> >> >> I’d go with option #2. The current includes and excludes are really >> intended to specify where to look for tests. Option 2 adds another filter >> over this to specify some criteria to use for selecting tests (beyond where >> they physically live). >> >> I’d also add command-line support and deprecate the system property. >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> >> >> >> > > > -- > Szczepan Faber > Principal engineer@gradle; Founder@mockito >
