Re: [pygame] unit and doc testing questions
On Thu, Jan 15, 2009 at 07:44:00AM +1100, René Dudfield wrote: > it can be good to mark your slow tests somehow, so you can run them > separately. > > With pygame we extend the python unittest module to allow specifying > tags(categories) for tests. This allows us to also use interactive > tests, and other tests you may not want to run every time. > > However you could just move them into a different directory, and > change your test runner to look in a different directory given a cmd > line option... you don't need any fancy testing framework. I prefer phrasing it as "Python lets you trivially create your own fancy testing framework in 15 minutes". ;-) > doctests can be a quick way to do testing. You can create your code > in the interactive interpreter, then paste the output into the doc > strings. You can use doc tests, and unittests together too... so the > doc tests can be run in a unittest test runner. This is, in fact, my preferred mode of testing. Example: http://mg.pov.lt/pyspacewar/trac/browser/trunk/src/pyspacewar/tests/test_world.py (I dislike long doctests in the docstrings of real functions, they obscure the code too much and make it hard to read. Short doctests there are fine. Long doctests belong in a separate test module.) > On Thu, Jan 15, 2009 at 5:57 AM, Jake b wrote: > > I say 'compiling', because I found a method in the python cookbook that says > > "you want to ensure your modules tests are run each time your module is > > compiled." > > > > It pretty much does "if module is imported, but not main script, and needs > > recompile(because was changed): then run tests" This sounds very interesting. Do you have a URL? I tried searching on http://code.activestate.com/recipes/langs/python/, but didn't find anything. Marius Gedminas -- Writing setattr hooks properly is a black art. Writing persistent setattr hooks is more like hearding bees blindfolded... -- Casey Duncan signature.asc Description: Digital signature
Re: [pygame] unit and doc testing questions
On Wed, Jan 14, 2009 at 12:58 PM, Jake b wrote: >> are pretty much inherently checking for compiler errors, so there is no >> need to test for the compiler >> explicitly. > > Don't know what you mean? I'm not testing for compile errors? You mentioned having compile time tests, there is no reason to, but stuffing some tests into a module to have it run whenever the module is imported can be a quick way to go about it if you don't need the flexibility of test runners. It sounds like for what you are doing you might need as much flexibility as you can get. > I can convert most of the render test like you said. But I'm more curious > when a case comes up that it just takes too long. ( Not specifically > rendering , but just a unittest in general that requires time.) What is the test you are talking about that is too slow? My tests are collectively slow, but I don't have any major problem cases that sound like what you describe. I guess sometimes though you can't get away from it. Structure it however necessary that the other tests tell you everything else, and set things up to still let you run everything when you need to (always running everything when you are think you are "finished" with something.) Another option for slow tests, is if the part that is slow can be reused in more than one test, you can save time by caching that operation. This is the purpose of test fixtures - build/set up some stuff, then run these tests using it. But fixtures can cause issues if you aren't careful, because running the tests in different orders or at different times may have different results, depending on what's going on inside the fixture between runs. Don't forget to do integration testing in addition to unit testing! I have an experiments file where I just play with the system and classes. I have found bugs there that my unit tests missed. Sometimes, I was able to take the experiment and turn it into another unit test, other times it wasn't worth it to do so. This was my first serious attempt at testing, so I was a bit surprised when I had many unit tests and thought things were peachy, but found many problems when I started experimenting.
Re: [pygame] unit and doc testing questions
> explicitly. That cookbook example sounds like it is less about > compiling, and more > of a way to implicitly run tests when the application runs. > Right. it auto-runs your tests when the code is compiled. are pretty much inherently checking for compiler errors, so there is no need > to test for the compiler > explicitly. Don't know what you mean? I'm not testing for compile errors? I can convert most of the render test like you said. But I'm more curious when a case comes up that it just takes too long. ( Not specifically rendering , but just a unittest in general that requires time.) The tagging system looks interesting, I'm going to look more into it. -- Jake
Re: [pygame] unit and doc testing questions
hi, it can be good to mark your slow tests somehow, so you can run them separately. With pygame we extend the python unittest module to allow specifying tags(categories) for tests. This allows us to also use interactive tests, and other tests you may not want to run every time. However you could just move them into a different directory, and change your test runner to look in a different directory given a cmd line option... you don't need any fancy testing framework. doctests can be a quick way to do testing. You can create your code in the interactive interpreter, then paste the output into the doc strings. You can use doc tests, and unittests together too... so the doc tests can be run in a unittest test runner. cheers, On Thu, Jan 15, 2009 at 5:57 AM, Jake b wrote: > >> > 2) Do you have two seperate unittests ? (A) One is ran every time source >> > files are compiled, and a second, (B) slower one is ran at a longer >> > interval >> > for slow tests ? >> >> Such separation is rarely necessary. > > Something I started on, requires rendering. It's not a huge delay, but I > don't want it happening every time I hit f5, while the other tests are fast > that they can every time. > > I'm not totally sure how I'm going to handle this. For now I'm manually > running the slow one, but. > >> >> "Compiling" is a very nebulous, automatic and invisible step in Python. >> I run the tests whenever I want to get some feedback on my code, and >> also before committing my changes to the source control system (you have >> one, right?) > > > So for yours, you manually hit a button / click shortcut / whatever to run > tests? None are automatic? ( Or maybe you do on central code check-in > computer. Where it runs fast ones right then, and the slow tests on an > interval. ) > > I say 'compiling', because I found a method in the python cookbook that says > "you want to ensure your modules tests are run each time your module is > compiled." > > It pretty much does "if module is imported, but not main script, and needs > recompile(because was changed): then run tests" > -- > Jake >
Re: [pygame] unit and doc testing questions
On Wed, Jan 14, 2009 at 10:57 AM, Jake b wrote: > >> > 2) Do you have two seperate unittests ? (A) One is ran every time source >> > files are compiled, and a second, (B) slower one is ran at a longer >> > interval >> > for slow tests ? >> >> Such separation is rarely necessary. > > Something I started on, requires rendering. It's not a huge delay, but I > don't want it happening every time I hit f5, while the other tests are fast > that they can every time. > > I'm not totally sure how I'm going to handle this. For now I'm manually > running the slow one, but. In my system, all of my tests are categorized, and I have a flag in the run tests script that determines which categorys I want. While debugging, I only run "new" tests (any tests I am working on or recently added, I tag as new). Before I check into version control, I make sure to run all of the tests first, but setting the flag in the run tests script to all categories. Running all of the tests does take a while (more than a minute), but the slow tests don't delay my debugging/coding. Another option, which might have merit, is to completely separate rendering tests and other unit tests. You can have one runner that runs the rendering tests, and another runner for everything else. I think rendering is a bit fuzzy as a unit test. You should prefer tests that don't have to render to those that do, and structure code in such a way that you can test as many things without rendering as possible. For instance, animation code doesn't need to render to check and make sure self.frame is incrementing properly. But, there are certainly cases where rendering needs to be tested. Just make sure to only test it when necessary. Example (semi-real): I have 20ish tests for positioning an object. If I were to draw the object in each test, it would be very slow. Instead, the 20 tests check the result of a get_screen_pos() rather than render the object outright. Most of the logic that used to be in the object.draw() method is refactored to the get_screen_pos method. That way I can get by with just a few tests of object.draw() itself, and know that since it uses get_screen_pos (which is fully covered by the other tests), that draw() is well covered as well. Any of your tests or test files that import the module are pretty much inherently checking for compiler errors, so there is no need to test for the compiler explicitly. That cookbook example sounds like it is less about compiling, and more of a way to implicitly run tests when the application runs.
Re: [pygame] unit and doc testing questions
> > 2) Do you have two seperate unittests ? (A) One is ran every time source > > files are compiled, and a second, (B) slower one is ran at a longer > interval > > for slow tests ? > > Such separation is rarely necessary. Something I started on, requires rendering. It's not a huge delay, but I don't want it happening every time I hit f5, while the other tests are fast that they can every time. I'm not totally sure how I'm going to handle this. For now I'm manually running the slow one, but. > "Compiling" is a very nebulous, automatic and invisible step in Python. > I run the tests whenever I want to get some feedback on my code, and > also before committing my changes to the source control system (you have > one, right?) So for yours, you manually hit a button / click shortcut / whatever to run tests? None are automatic? ( Or maybe you do on central code check-in computer. Where it runs fast ones right then, and the slow tests on an interval. ) I say 'compiling', because I found a method in the python cookbook that says "you want to ensure your modules tests are run each time your module is compiled." It pretty much does "if module is imported, but not main script, and needs recompile(because was changed): then run tests" -- Jake
Re: [pygame] unit and doc testing questions
On Tue, Jan 13, 2009 at 06:44:10AM -0600, Jake b wrote: > I'm learning about doctest, and unittests. Now if I create a file, like > pygame/test/rect_test.py , is there and easy way to import the project in > its parent directory? Make sure it's in sys.path, then import using the absolute package name. (Hint: 'pygame' is a bad name for a project, it will clash with the pygame library.) > I thought "import ../mymodule" might work, ( but at least not in 2.5 ). > Then I got it to work appending sys.path. > 1) What is your advice on imports from test files? ( Do you append your > sys.path, or something else? ) The test runner makes sure your project is importable by adding it to sys.path, if necessary. > 2) Do you have two seperate unittests ? (A) One is ran every time source > files are compiled, and a second, (B) slower one is ran at a longer interval > for slow tests ? Such separation is rarely necessary. I generally have a separate test module for every code module (with separate tests for each function/method), and then a top-level script that imports all test module, collects all the tests into a single unittest.TestSuite, and runs them. http://pypi.python.org/pypi/nose can be used instead of a custom script, but I got used to writing those before nose was available and so far hadn't gotten to try it out. "Compiling" is a very nebulous, automatic and invisible step in Python. I run the tests whenever I want to get some feedback on my code, and also before committing my changes to the source control system (you have one, right?) > 3) Other advice relating to testing ? Keep them short and fast. Marius Gedminas -- Microsoft has performed an illegal operation and will be shut down. -- Judge Jackson signature.asc Description: Digital signature
