On Tuesday, Aug 12, 2003, at 07:37 Europe/London, Bruce Snyder wrote:
2) Another issue is how to label the spec compliance tests in order to parse them later for reporting, etc.
For this issue, not only should each spec compliance test be fully
documented via Javadoc, but we need to devise some meta tags for labeling
each test case for doclet parsing. I suggest that we use the following:
@spec name="J2EE" version="1.4" chapter-name="Transaction Management" chapter-number="4" chapter-section-name="Transactional JDBC Technology Support" chapter-section-number="4.2.7"
Can you tie spec tests down to a specific page/chapter, though? I'm sure that the relationship isn't 1-1. For example, a spec requirement may come out of several places in the document; at the very least, on different pages. Further, there may be some spec tests which cover many different types of element, or that go into more detail than the spec document allows (such as CMP mapping, which isn't covered by the spec at all).
I'm also pretty sure that the -name and -number are storing duplicate information; you'd really only need the -number ones, and provide a lookup file elsewhere. In fact, given that the section stores chapter-and-section number, you'd really only need that :-)
Feedback? Better ideas?
Another approach would be to plan the tests on the Wiki first, and then write the tests themselves, rather than the spec tests being self documenting. The former approach will let you build a complete list of tests first; however, if you rely on the individual tests saying that 'section 4.2.7' is tested, then that can lull you into a false sense of security; yes, /one/ of the tests in 4.2.7 may be covered, but that doesn't mean that /all/ of them will be.
Alex.
