On 08/07/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote:
>
> So while I like the annotations, and expect we can use them
effectively,
> I have an instinctive skepticism of annotations right now because in
> general (in general in Java), I'm not convinced we've used them enough
> to grok good design patterns.
There's really no reason to get hung up on the annotations. TestNG
works just as well with JavaDoc source comments; annotations are only
another means to that end. (They're probably a better one for the
future, but it's just an implementation detail.)
> Now since I still haven't read the thread fully, I'm jumping to
> conclusions, taking it to the extreme, etc etc, but my thinking in
> writing the above is that if we bury everything about our test
> 'parameter space' in annotations, some of the visible organization we
> have now w/ on-disk layout becomes invisible, and the readable
> "summaries" of aspects of testing that we'd have in an XML metadata
> document (or whatever) also are hard because you need to scan the
> sources to find all instances of annotation "X".
I'm hoping that this would be just as applicable to using JavaDoc
variants, and that the problem's not with annotations per se.
In either case, both are grokkable with tools -- either
annotation-savy readers or a JavaDoc tag processor, and it wouldn't be
hard to configure one of those to periodically scan the codebase to
generate reports. Furthermore, as long as the annotation X is well
defined, *you* don't have to scan it -- you leave it up to TestNG to
figure it out.
Actually, there's a very valid benefit for using TestNG markers (=
annotations/JavaDoc) for grouping tests; the directory structure is a
tree, whereas the markers can form any slice of tests, and the sets
don't need to be strict subsets (with a tree, everything has to be a
strict subset of its parents). That means that it's possible to define
a marker IO to run all the IO tests, or a marker Win32 to run all the
Win32 tests, and both of those will contain IO-specific Win32 tests.
You can't do that in a tree structure without duplicating content
somewhere along the line (e.g. /win/io or /io/win). Neither of these
scale well, and every time you add a new dimension, you're doubling
the structure of the directory, but merely adding a new marker with
TestNG. So if you wanted to have (say) boot classpath tests vs api
tests, then you'd ahve to have /api/win/io and /boot/win/io (or
various permutations as applicable).
Most of the directory-based arguments seem to be along the lines of
"/api/win/io is better! No, /win/io/api is better!". Just have an
'api', 'win', 'io' TestNG marker, and then let TestNG figure out which
ones to run. You can then even get specific, and only run the Windows
IO API tests, if you really want -- but if you don't, you get the
benefit of being able to run all IO tests (both API and boot).
There doesn't seem to be any benefit to having a strict tree-like
structure to the tests when it's possible to have a multi-dimensional
matrix of all possible combinations that's managed by the tool.
Alex.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
