George Harley wrote: > Alexei Zakharov wrote: >>> Actually, there's a very valid benefit for using TestNG markers (= >>> annotations/JavaDoc) for grouping tests; the directory structure is a >>> tree, whereas the markers can form any slice of tests, and the sets >> >> Concerning TestNG vs JUnit. I just like to pay your attention on the >> fact what it is possible to achieve the same level of test >> grouping/slicing with JUnit TestSuites. You may define any number of >> intersecting suites - XXXAPIFailingSuite, XXXHYSpecificSuite, >> XXXWinSpecificSuite or whatever. Without necessity of migrating to new >> (probably unstable) test harness. >> Just my two cents. >> >> > > Hi Alexei, > > You are quite correct that JUnit test suites are another alternative > here. If I recall correctly, their use was discussed in the very early > days of this project but it came to nothing and we instead went down the > route of using exclusion filters in the Ant JUnit task. That approach > does not offer much in the way of fine grain control and relies on us > pushing stuff around the repository. Hence the kicking off of this thread. > > For the purposes of this discussion it would be fascinating to find out > why you refer to TestNG as being an "unstable" test harness. What is > that statement based on ?
Yeah! What he said! :) geir > > Best regards, > George > > >> 2006/7/8, Alex Blewitt <[EMAIL PROTECTED]>: >>> On 08/07/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote: >>> > >>> > So while I like the annotations, and expect we can use them >>> effectively, >>> > I have an instinctive skepticism of annotations right now because in >>> > general (in general in Java), I'm not convinced we've used them enough >>> > to grok good design patterns. >>> >>> There's really no reason to get hung up on the annotations. TestNG >>> works just as well with JavaDoc source comments; annotations are only >>> another means to that end. (They're probably a better one for the >>> future, but it's just an implementation detail.) >>> >>> > Now since I still haven't read the thread fully, I'm jumping to >>> > conclusions, taking it to the extreme, etc etc, but my thinking in >>> > writing the above is that if we bury everything about our test >>> > 'parameter space' in annotations, some of the visible organization we >>> > have now w/ on-disk layout becomes invisible, and the readable >>> > "summaries" of aspects of testing that we'd have in an XML metadata >>> > document (or whatever) also are hard because you need to scan the >>> > sources to find all instances of annotation "X". >>> >>> I'm hoping that this would be just as applicable to using JavaDoc >>> variants, and that the problem's not with annotations per se. >>> >>> In either case, both are grokkable with tools -- either >>> annotation-savy readers or a JavaDoc tag processor, and it wouldn't be >>> hard to configure one of those to periodically scan the codebase to >>> generate reports. Furthermore, as long as the annotation X is well >>> defined, *you* don't have to scan it -- you leave it up to TestNG to >>> figure it out. >>> >>> Actually, there's a very valid benefit for using TestNG markers (= >>> annotations/JavaDoc) for grouping tests; the directory structure is a >>> tree, whereas the markers can form any slice of tests, and the sets >>> don't need to be strict subsets (with a tree, everything has to be a >>> strict subset of its parents). That means that it's possible to define >>> a marker IO to run all the IO tests, or a marker Win32 to run all the >>> Win32 tests, and both of those will contain IO-specific Win32 tests. >>> You can't do that in a tree structure without duplicating content >>> somewhere along the line (e.g. /win/io or /io/win). Neither of these >>> scale well, and every time you add a new dimension, you're doubling >>> the structure of the directory, but merely adding a new marker with >>> TestNG. So if you wanted to have (say) boot classpath tests vs api >>> tests, then you'd ahve to have /api/win/io and /boot/win/io (or >>> various permutations as applicable). >>> >>> Most of the directory-based arguments seem to be along the lines of >>> "/api/win/io is better! No, /win/io/api is better!". Just have an >>> 'api', 'win', 'io' TestNG marker, and then let TestNG figure out which >>> ones to run. You can then even get specific, and only run the Windows >>> IO API tests, if you really want -- but if you don't, you get the >>> benefit of being able to run all IO tests (both API and boot). >>> >>> There doesn't seem to be any benefit to having a strict tree-like >>> structure to the tests when it's possible to have a multi-dimensional >>> matrix of all possible combinations that's managed by the tool. >>> >>> Alex. >>> >>> --------------------------------------------------------------------- >>> Terms of use : http://incubator.apache.org/harmony/mailing.html >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >> >> > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
