Just a reminder that there is more to developing certificaton tests than just porting code under a particular harness.
The methodology for LSB test development, is as follows (quote paraphrased from the docs I put together for the LSB-FHS test development, http://www.opengroup.org/testing/lsb-fhs/process.html and followed by other addons such as the USERGROUPS and LSB-OS) The process for developing the test suite involves a multi-step process where objectivity, and relationship to a written specification are the key. The first step is development of a test specification which should undergo formal review. Secondly, a test suite based on that specification. There are criteria for both the specification and the test suite that should be met . At regular points there should be feedback from the customer audience / underlying specification owners to verify the correct assumptions. In the case of the LSB-FHS test suite, the test specification is based on the FHS 2.0 specification, see www.pathname.com, as modified by the requirements placed by the LSB specification (for example LSB mandates presence of the X Window system which is optional in the FHS). The test specification follows the POSIX 1003.3 guidelines and consists of a set of assertions , that is a set of statements which evaluate as true or false based on the specification under test. This can be thought of the translation between the natural language specification and the testable specification. For example, the FHS has a statement in section 3.1: Required files for /bin The following commands have been included because they are essential. A few are present because of their traditional placement in /bin. {A list of 34 commands} This is written as 34 separate test assertions , for example Reference 3.1-3 (A) The implementation provides an exec-able version of the cat utility in the /bin directory. Result: PASS/FAIL Reference 3.1-4 (A) The implementation provides an exec-able version of the chgrp utility in the /bin directory. Result: PASS/FAIL etc. The above example is a simple case, more complex examples are for conditional tests etc. The key is to produce logical, testable statements that can be reasoned about, these are called test assertions. By producing test assertions, we can then produce tests. This allows test development to proceed based on the specifications. At this point an iterative process occurs as test assertions are validated or shown to be false. This may be due to one an incorrect assumption by the author of the test assertion, or in some cases what is now considered an error in the underlying specification. The latter should only be decided by the authors of the specifications, and not the test suite author, and these issues should be fed back to the specification owners so that corrective action can be taken. The test suite will note the issue in its output , but the final correct strategy cannot be undertaken until the specification owners have issued corrections or possibly a version of the specification. regards Andrew -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with subject of "unsubscribe". Trouble? Email [EMAIL PROTECTED]
