I very often review RTIs that have little to no testing details in the
"Test Results" section.  Often, the section will have some terse text
basically stating that "some testing was done and it passed" with no
effective details regarding what was done.  This doesn't imply that the
RE didn't do adequate testing, but it is not enough information for the
CRT advocate to determine if the testing was in fact adequate (one of
the primary purposes of RTIs to begin with).

In addition to this message being a reminder to developers to place that
information in the RTI (you went through the trouble of doing testing,
why not show off what you did by providing the proof) ;-), I'd like to
add a section to the rti-nits page
(http://www.opensolaris.org/os/community/on/crt/rti-nits/) as a
reminder.  Here's some suggested text for a new bullet under
"Submitters":

* Test Results
  * This section should contain enough details for the CRT advocate to
    evaluate if testing was adequate.  Logs of automated tests, or
    terminal output of manual tests are welcome.  A statement of the
    form, "I tested the code and it passed" is inadequate.
  * If tests could be added to an existing automated test suite to
    facilitate in testing the affected code, then either integrate
    such a test, or file an RFE against the test suite detailing what
    is missing.  Without that, code-coverage of existing test suites
    will never improve, and bugs will perpetually be re-introduced in
    the same areas due to lack of test coverage.

Thoughts?  

-Seb



Reply via email to