Hello. > I introduced a new texinfo document in `doc/' named `gnupdf-tsd.texi' > (for GNU PDF Library Test Specification Document). It is the place > where we will document all the unit tests, subsystem tests and system > tests of the library. Is this really worth while? It seems quite imprecise - example:
pdf_realloc Test: pdf_realloc_001 Reallocate some bytes. Success condition The call should not produce an error. It would be better to express some initial conditions, a set of interesting variations (realloc smaller, realloc same size, realloc larger), and then check it actually worked (not just that the error handling is turned off :-)) We can do it more precise by introducing more fields into the test descriptions. For example: Test: pdf_realloc_001 Reallocate some bytes. Initial Conditions: foo bar Success condition: ... As for the variations, can be implemented with more tests. That was the reason to identify functions with testcases: Test: pdf_realloc_002 Realloc smaller ... Test: pdf_realloc_003 Realloc same size ... Test: pdf_realloc_004 Realloc larger ... etc At this level, expressing the test in C would seem better. I agree in that it would be quite convenient to document the tests in the source code. The reason I introduced a separated test specification document is that we need a place to define the tests before to implement these: there are separate tasks in the project plan involving testing: - Design the unit tests for foo - Implement the unit tests for foo In that way we can start the design of the unit tests for the functions of a given module as early as we have the module API designed. The testmaster (I would like to appoint someone to play that role; would you like to dedicate some time for this? :)) will need to continuously maintain some statistics about the state of the library validation system: - How much functions have unit tests designed? - How much functions have unit tests implemented? - ... A script could be written to inspect the TSD, the Reference Manual and the torture/unit/ source tree and automatically generate such statistics. An alternative is to generate the TSD from the torture/unit/ sources. What do you think about that? I am not saying that there shouldn't be a test plan / spec, just that it should be at a higher level - expressing goals rather than detail: * Goal: all functions should have unit tests that verify correct operation. * Goal: all functions should have unit tests that check bad parameter combinations result in meaningful errors. * Goal: subsystems shall... * Goal: systems shall... Something like a test plan is already depicted in the http://gnupdf.org/Lib:Torture_Chamber wiki page. Maybe would be good to put that information in a texinfo document instead.
