-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Benji York wrote: > On Mon, Apr 19, 2010 at 7:21 AM, Charlie Clark > <charlie.cl...@clark-consulting.eu> wrote: >> Narrative documentation forces you to explain yourself to someone else. >> Neither, however, will necessarily notice if you forget something: tests >> aren't documentation and docu [sic] > > Quite true. It is also true that tests should be well documented and > documentation should be well tested. Some prefer doctests for both.
I would disagree that "tests should be well documented", at least in a thread whose main purpose is to point up the very grave shortcomings in the narrative documentation for the ZTK package set as a whole. Writing clean, understandable code, with good class / method / function / variable names, comments where appropriate, etc., is not the same thing as documenting that code. Overloading the word "well-documented" to describe code which conforms to such (completely uncontroversial) standards muddies the point here. I am not arguing that writing good doctests is impossible, any more than writing good unit tests. What I am arguing is that we have lots of mediocre-to-poor tests (of both kinds), and that we have basically no documentation of the sort that most people mean when they use the term. I want us to take off the "I'm testing the code" hat when writing documentation, and focus on producing a much more useful documentation set, one which actually encourages folks who haven't already drunk our Kool-Aid to use the code. For instance, the ratio of lines of narrative text to lines of examples in the documentation ought to be *much* higher than the near-parity which is the rule in most ZTK doctests. I don't think we can get there by continuing to pretend that our existing doctests are working as documenation, overall, which was my rationale for moving the ones which might form the basis for real docs *out* of the code, and manage them separately as Sphinx documentation. As I have said elasewhere in the thread, I also believe that the snipptes in that documentation should be testable, where possible: I think that the Sphinx addons which allow running the snippets as part of building the tests are a really good fit for ensuring that the snippets don't bit-rot. Unfortunately, no tool is going to help ensure that the narrative itself hasn't bit-rotted: that takes a dedicated investment over time by the team maintaining the packages. I also believe that the goal of testability of document snippets remains subordinate to the goals of comprehensibility and of completness *as documentation*. Doumentation writers must work to remove any artifacts of testability which shows up in the rendered docs, where such artifacts don't actively aid comprehension. When we wear the testing hat, as opposed to the documenation hat, we should come to some kind of agreement about how to achieve the goal of getting our code thoroughly tested. I am not adamantly opposed to using doctests to aid in this goal, particularly for covering the "main line" of usage of an API. However, because they frequently test non-essential artifacts (hash ordering, exception rendering, etc.), they tend to be the most fragile of our tests across things like major Python version changes, platforms, etc. Also, because they take a narrative form, doctests alos encourage test writere to use shared state between logically-discrete tests, which leads to really ugly coupling and reduces the ability of the test writer to reason about semantic completeness of the tests. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvMjPAACgkQ+gerLs4ltQ5X0QCcDdeRhjUKLQtjcNZVXW7fGulv WLMAoLbNRu9SMximbOrqnsV2mqkWrbxz =SExz -----END PGP SIGNATURE----- _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )