On 30/12/2012, at 12:46 AM, Adam Murdoch wrote: > > On 19/12/2012, at 6:18 AM, Matt Khan wrote: > >> Classification: Public >> >> Hello >> >> We have a gradle based build/release/deployment system and we use >> cucumber-jvm (https://github.com/cucumber/cucumber-jvm) for certain types of >> testing. >> >> ***Some preamble about cucumber first which could be skipped if you’re >> familiar with cucumber*** >> >> cucumber (or rather gherkin) is set of keywords organised into a tree like >> structure like (taking the example from their page) >> >> Feature: Addition >> Scenario: Add two numbers >> Given I have entered 50 into the calculator >> And I have entered 70 into the calculator >> When I press add >> Then the result should be 120 on the screen >> >> cucumber-jvm gives you 2 ways to run tests; via cli interface or via a junit >> runner. The junit approach exposes each level in this hierarchy as a junit >> Description where each description is simply it’s displayed name, in this >> example we’d have >> >> a Description with name “Feature: Addition” with 1 child, which is >> a Description with name “Scenario: Add two numbers” with 4 children, which >> are >> a Description for each step with names “Given I have entered 50 into the >> calculator” etc >> >> There are other variations (scenario outlines, data tables etc) but they all >> fall into this sort of hierarchical structure. >> >> *** The meat of the problem *** >> >> As of today (gradle 1.3), the only option to run cucumber-jvm from gradle is >> to create a custom javaexec task that runs the cucumber cli Main with build >> specific kludges to make it agree with how gradle runs testng/junit (like >> checking that tests have run). This is because cucumber’s use of junit does >> appear to agree with gradle’s interpretation. There are, at least, 2 >> problems; >> >> 1) a blocking bug (an NPE in gradle code triggered by the fact that >> cucumber generated Descriptions produce a null “methodName”) that prevents >> it from working at all (described in this thread >> -http://forums.gradle.org/gradle/topics/beforetest_descriptor_is_null_and_the_test_is_not_actually_run_trying_to_run_groovy_and_java_cucumber_step_definition)
I promoted this to issue GRADLE-2615 >> 2) (if one avoids the NPE by just adding a null check) gradle renders >> the results as a list of scenarios only >> >> The 2nd point means that key information is missing (on which step did the >> test fail?). >> >> My Q is pretty simple... what’s the “right” way to get this working in >> gradle such that the full hierarchy of information available is preserved? >> Alternatively should we not expect to see the gradle test reporting >> encompass this sort of content and instead focus on getting the execution >> side working (without the need for custom javaexec tasks) but using cucumber >> to generate test reports? > > I'd like to see Gradle be able to deal with this. To fix this, we'd need to > change the test reporting so that it does not assume that a test == a method > in a class. > > It might also be worthwhile putting together a plugin that connects up > Gradle's TestClassProcessor directly to cucumber, assuming it has some kind > of native execution API. This way, we can get proper descriptions for these > things in the reports (e.g. 'Feature', 'Scenario', 'Step' rather than generic > 'Test'). > > Are you interested in helping with some of this stuff? > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > -- Luke Daley Principal Engineer, Gradleware http://gradleware.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
