Geir Magnusson Jr wrote: > Stepan Mishura wrote: >> Did anybody think of creating a 'global' (i.e. shared by all modules) >> exclude list or every module will have its own exclude list? Or Harmony >> tests will always pass and we don't need it at all :-) > > That would be the goal :) > >> I see at least the following benefits of creating 'global' exclude >> list: all >> know issues are kept in one well known place (they don't spread between >> several private lists) > > That's true, but.... I always imagined that people would be working in > the modules anyway, so there isn't much gain.
Modules should be as self-sufficient as we can make them, so devolving tests and their exclude lists into a module makes sense to me too. They can be linked into a global view quite easily (read on...) > also it is easier to create an exclude list for a >> target platform, for example, linux.exclude or win.exclude. > > Yes, that could be. Interesting idea. The mechanism that George contributed in HARMONY-57 uses a decorator to implement an exclusion list on regular JUnit tests. (I happen to know that George is off enjoying himself for the next few days, so I hope he doesn't mind me describing it here.) The exclusion list is implemented as a (declarative) XML file read when the tests run -- in HARMONY-57 you can see one in Harmony_Tests/src/test/resources/config/jcltest-excludes.xml Picking an entry at random from there: ... <hy:type id="tests.api.java.io.FileTest"> <hy:exclude id="test_Constructor_String_String_112270" shouldfix="true"> <hy:reason>Undiagnosed failure</hy:reason> </hy:exclude> <hy:exclude id="test_Constructor_File_String_112270" shouldfix="true"> <hy:reason>Undiagnosed failure</hy:reason> </hy:exclude> <hy:exclude id="test_deleteOnExit" shouldfix="true"> <hy:reason>Needs investigation...</hy:reason> </hy:exclude> </hy:type> ... You can see how it works -- some tests are excluded because they fail and should be fixed, others (not shown here) can be excluded because they don't make sense on a particular platform, VM, etc. Applying a style sheet makes it easy to read the exclusions list in glorious technicolor, either in an individual module or as a combined global view. Take a look at the incoming contribution and see if it fits your needs. Regards, Tim > However, I did imagine that we'd give the modules a bit of freedom and > independence for testing - a global exclude list might impact that? > > geir > >> >> Thoughts? >> >> Thanks, >> Stepan Mishura >> Intel Middleware Products Division >> > -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.