On 27-Jan-08, at 12:31 PM, Benjamin Bentmann wrote:
Jason van Zyl-2 wrote:
Correct me if I'm wrong here Benjamin but I think he's talking about
stating what tests you want to run, and how they are grouped
declaratively. As a best practice don't use suites and come up with a
way in Surefire to group things and avoid grouping of things in code.
Not quite sure if I understand you correctly. A TestNG suite XML
file is
quite declarative so I already have the possibility to declaratively
select
and group tests. In fear that I can't get Surefire where I wished
(one-class-per-suite), I opened http://jira.codehaus.org/browse/SUREFIRE-438
which would allow to easily feed a bunch of suite XML files into the
plugin.
But what I really want is to avoid bothering with suites at all.
Sorry if I didn't explain it well but we are in 100% agreement. Make
tests simple, no suites, and group declaratively.
Sure,
TestNG has a powerful suite concept, nice, maybe we may want to use it
someday in the future but for now, we have just the simple use-case
with
non-related test classes, i.e. we are happy by grouping at the class
level.
Explicitly maintaining suites, either in code or via XML, would just
be lost
worktime here (although I already thought about a plugin that
automatically
creates single-class suites from the existing test classes and
prepares the
dozens of suite files before the actual Surefire run...)
We have no suite(s), only a directory with some unrelated test
classes. Now
Surefire goes ahead and creates a suite, automatically, without my
control,
assuming the entire directory is a single suite. This assumption
does not
fit our needs.
dfabulich wrote:
That option sounds really confusing. What would you call it? How
would
you document it? You'd have to know a lot about the guts of
Surefire and
TestNG for an option like that to make much sense.
Could think of something like "suiteMode=class|package|all":
This option controls how Surefire will assemble your test classes
into a
suite. "class" means to run each class individually in its own suite.
"package" groups all classes from the same package into a separate
suite.
"all" will aggregate all found test classes in a single suite. Each
created
suite will then be passed to the testing framework for execution and
will be
run independently of the other suites. Note that test classes that
have
dependencies on each other, must be executed in the same suite.
Therefore,
the values "class" and "package" should be used with care because
Surefire
itself cannot detect such inter-class dependencies.
dfabulich wrote:
"TestNG is designed to cover all categories of tests: unit, [...]"
[...]
If all you're doing is unit tests, you may prefer to use JUnit 4. :-)
Well, no comments ;-)
Say one starts of with JUnit, where Surefire executes classes
individually.
Then one goes and converts his tests to use TestNG (they have a tool
for
this). Conceptually, you have the same test design, but now Surefire
insists
on running all classes in a single job/black-box. I am sorry, but I
still
find this odd since I am lacking the required knowledge about the
guts of
Surefire and TestNG ;-)
Regards,
Benjamin Bentmann
--
View this message in context:
http://www.nabble.com/Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-tp15076378p15123679.html
Sent from the Surefire - Developer mailing list archive at Nabble.com.
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
Simplex sigillum veri. (Simplicity is the seal of truth.)
