Nate Lowrie wrote: > On 2/26/07, Dabo <[EMAIL PROTECTED]> wrote: >> #4: Improve unit testing coverage >> ----------------------+----------------------------------------------------- >> Reporter: paul | Owner: paul >> Type: task | Status: new >> Priority: critical | Milestone: 1.0 >> Component: app | Version: >> Keywords: | >> ----------------------+----------------------------------------------------- >> We are on the right track with unit testing, but we desperately need to >> improve coverage. Most (all?) properties of dCursorMixin and dBizobj are >> currently being tested in some way, but hardly any methods are. Start by >> improving the coverage here, and then move to the ui layer.
Thanks for replying. I had no idea that the dabo-dev list was getting these notifications, although I was indeed trying to make that happen. > Notes on unit testing...... > > 1) I think that unit tests are absolutely vital to the continuing > success of Dabo and that I am surprised that the code has a high level > of cohesion with a relatively low number of defects without it. We > need to bring all tiers under the unit test. Yep. > 2) I like the simplicity that the no API of py.test brings, but I > think that in the case of Dabo this is not good. For such a large > structure as Dabo I see the need for at least some form of API in the > test suite. I like that Paul went with PyUnit for the test library. > It has a nice structured API and the formatting is easy. On another > note, I have an absolute disdain for the doctest suite and would like > to avoid its use if at all possible. I don't really like doctest that much, either. > 3) I do think that we need to completely separate the unit test > structure from the Dabo directory (and move it into the trunk root > under the folder test). This way the test suite will be separate from > the framework and things will be clearer. Also, since the tests > already import Dabo there will be no changes that need to be made to > the code. Ok, that's fine by me. I've seen it done both ways and don't really have an opinion on the matter. > 4) I don't know if you have played with CruiseControl or AntHill, but > they are wonderful programs for use in test-driven development. > Basically, we use them at work for Java and C# projects. They will > check out the latest code from an SVN repository and run the unit test > suite at pre-determined time intervals specified by the user. If any > of the tests fail, it will automatically email the entire development > team with the test log. I would like to see something similar for > dabo. I don't think that there is a program like this but it should > be a simple script to write and it could have a user interface in dabo > even. Regardless, this Continuous Integration is necessary for a > test-driven environment which is what we are striving for. I don't know that we are striving for test-driven development, but I'm attempting to get at least some of it going. The unit tests and continuous integration are just so valuable that we can't not do them. > 5) For the unit test on the UI layer, I suggest we use Python Mock or > Pmock to create the mock object we need for solid unit tests. I don't > know of any other way to decouple some of the classes. Yes, for testing our code, but we also need to know how wx processes some of the interactions, so I don't think we can get away from running the tests against wx, at least where it is possible (no modal user input). > 6) A suggestion that I would like to make is that we set up unit tests > for the various libraries that we subclass. I know that unit tests > for dabo will be enough work, but look at it this way. Suppose we had > a set of unit tests for wxPython when it was in 2.6 and then 2.8 came > out. We would have had a much easier time finding the API > discrepancies and fixing the code since we could have run the unit > tests and figured out which ones had failed. By doing this we > basically create a safety net to ensure that new releases don't > introduce new bugs in the system through code changes in the > subclassed libraries. Oh boy, let's do it one step at a time. That will have to wait, unless you have a big itch to scratch on that. > Sorry I am chiming in so late on this. I have been busy with > finishing a thesis and planning a wedding. I am a big advocate of > test-driven development cycle, especially with a library that is big > like Dabo. After I free up a lot of time starting the end of March, > this is one of the top priorities I will have. Paul, I would like to > work closely with you on this. I have a feeling that we might have to > refactor some code to define the unit tests properly. But, I do > believe that once this is implemented, bugs will plummet, code is > easier to debug, and we can refactor code and add new code with > certainty that the old code is most likely not broken. Nate, please feel free to do whatever you want with reorganizing and adding to the unit testing. I'll do my part and add tests when bugs surface and adding new features, but I'm not sure I have time to get tests written for all the prior work. Also, I'm pretty green at writing unit tests, so don't worry about stepping on my feet if you think some tests can be written better. > Note that this does not eliminate the need for acceptance testing, > user testing, and integration testing. We will still have to do these > but there should be less defects found as a result of them. Absolutely agree. I think Ed and I are just naturally good at "testing" our changes, which means lots of manual testing of common and edge cases using the UI, which is why we haven't gotten in way over our heads yet. But we can't catch everything, and bugs have slipped through on plenty of occasions where, had the proper unit tests been in place, we would have caught 'em early on. > My $.02, sorry for the long email. Don't be sorry, but put some of that energy into writing tests! :) -- pkm ~ http://paulmcnett.com _______________________________________________ Post Messages to: [email protected] Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
