bartc wrote: > Testing everything comprehensively just wouldn't be useful for me who > works on whole applications, whole concepts, not just a handful of > functions with well-defined inputs and outputs.
I had this experience with Pyrex (the precursor to Cython). The various parts are so interdependent that it doesn't really make sense to test individual classes or methods on their own. It would require a huge amount of scaffolding and wouldn't really prove much. Instead, what I did was write tests for units of *functionality*. (Can it compile an assignment statement? An expression that adds two things? Subtracts them? etc.) I wrote a very large number of them, plus a test framework to run them and compare the output with known good output. I can't claim that I always wrote the tests *first* (sometimes I did, sometimes I didn't). But I never had any trouble motivating myself to write the tests. Partly this is because the way my test framework worked, there was very little difference between a manual test and an automated one. I had to write a test file to test it at all, and having done that, it was just as easy to save it away together with its output so that it became part of the automated test suite. It was also partly because the benefits of having a comprehensive automatic test suite were so blindingly obvious in that project. Its complexity meant that it was particularly easy to break something accidentally when adding a new feature. If that happened, the tests would almost certainly pick it up immediately. Without those tests, I would never have been able to develop it to the point I did. >> So there was something else wrong with the chip. I'm not sure what >> your point is. > > The extensive testing was like unit testing, but needed to be even more > thorough because of the commitment involved. It failed to spot a problem. Testing obviously isn't guaranteed to find all problems in a system with complex behaviour, but that doesn't mean testing is useless. It would have been hugely more risky to go ahead and build the chip without testing the design at all! BTW, hardware is a bit different from software. It's possible for a design to be *logically* correct and pass all the simulations, but for the hardware to fail for messy physics- related reasons. (Charles Moore tells a story like that about one of his Forth chips. A transistor driving a particular signal was too small, and if you executed a particular instruction too many times in too short a time interval, it would overheat and produce incorrect results.) -- Greg -- https://mail.python.org/mailman/listinfo/python-list