I think it would be possible to write a step similar to a composite step
to run the necessary steps repeatedly and evaluate the result, however
this has the disadvantage that the steps are hidden in the
implementation of the step and not in the story file (and I think the
steps are not reported individually), though maybe given stories may be
a possibility.
When running individual steps with a statistical outcome, it would be
necessary to keep the state somewhere to be able to evaluate the
probability in the end, possibly in the page object, but this is
difficult to do if there are different tests to be accounted when
multi-threading.
One last thing I'd like to point out is that the evaluation of such a
test is not deterministic unless you have a mockup service that actually
returns a failure every n counts. When running the test 20 times, it
would be valid to have 0 or 2+ failures sometimes, where it would be
misleading to have the test fail. Maybe it would be feasible to run the
20 tests more than once in that case and calculate the average, but this
is still a bit flaky to have a test failing when the expected value is
not reached (either you need an estimator function and a confidence
interval or you can just report the test result as warning with the
calculated value)
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
