Classification: Public
Hello
We have a gradle based build/release/deployment system and we use cucumber-jvm
(https://github.com/cucumber/cucumber-jvm) for certain types of testing.
***Some preamble about cucumber first which could be skipped if you're familiar
with cucumber***
cucumber (or rather gherkin) is set of keywords organised into a tree like
structure like (taking the example from their page)
Feature: Addition
Scenario: Add two numbers
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
Then the result should be 120 on the screen
cucumber-jvm gives you 2 ways to run tests; via cli interface or via a junit
runner. The junit approach exposes each level in this hierarchy as a junit
Description where each description is simply it's displayed name, in this
example we'd have
a Description with name "Feature: Addition" with 1 child, which is
a Description with name "Scenario: Add two numbers" with 4 children, which are
a Description for each step with names "Given I have entered 50 into the
calculator" etc
There are other variations (scenario outlines, data tables etc) but they all
fall into this sort of hierarchical structure.
*** The meat of the problem ***
As of today (gradle 1.3), the only option to run cucumber-jvm from gradle is to
create a custom javaexec task that runs the cucumber cli Main with build
specific kludges to make it agree with how gradle runs testng/junit (like
checking that tests have run). This is because cucumber's use of junit does
appear to agree with gradle's interpretation. There are, at least, 2 problems;
1) a blocking bug (an NPE in gradle code triggered by the fact that
cucumber generated Descriptions produce a null "methodName") that prevents it
from working at all (described in this thread -
http://forums.gradle.org/gradle/topics/beforetest_descriptor_is_null_and_the_test_is_not_actually_run_trying_to_run_groovy_and_java_cucumber_step_definition)
2) (if one avoids the NPE by just adding a null check) gradle renders the
results as a list of scenarios only
The 2nd point means that key information is missing (on which step did the test
fail?).
My Q is pretty simple... what's the "right" way to get this working in gradle
such that the full hierarchy of information available is preserved?
Alternatively should we not expect to see the gradle test reporting encompass
this sort of content and instead focus on getting the execution side working
(without the need for custom javaexec tasks) but using cucumber to generate
test reports?
Depending on the size of the change required, I may get some time in early Jan
to work on this.
Cheers
Matt
---
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and delete this e-mail. Any unauthorized copying,
disclosure or distribution of the material in this e-mail is strictly forbidden.
Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional
EU corporate and regulatory disclosures.
