On 27 March 2015 at 20:27, Thomas De Schampheleire <patrickdeping...@gmail.com> wrote: > That sounds great, thanks! > In practice, how will this work?
Well, I'm kind of making it up as I go, but I envisage something like · initial familiarisation with project, discussion of aims (which we have done a bit here and now on the wiki page) · basics: pytest can run all tests, tests run without errors/failures, CI server set up (I haven't seen one for Kallithea yet - is there one?), probably with coverage - pytest helpers submit PRs, project authors review and discuss Then you have a choice of focusing on existing tests or exploring needs for new tests. Existing tests: Progressively rewrite to be more pytesthonic. You mentioned improving the speed and that is another definite area to look at! New tests: Add unit tests for low coverage areas. Figure out what fixtures might be needed, especially for functional tests. I would suggest the latter could be especially useful for Kallithea as you already mentioned fixtures for different vcs repos - sounds perfect. As I put 3 helpers with Kallithea they are people with a variety of pytest experience, so something like converting or writing new unit tests is a good task for someone with less experience, and analysing the speed and writing fixtures is a good task for someone with more. Like that is actually heaps for a months' effort, right? I get really excited and I'm really bad at keeping the scope down. :) But that's pretty much what I think - then if you/Kallithea would like a different focus, or have other ideas, point the way! Below are some notes that I wrote on the web page and in an email to the helpers. They are written with a bit of assumption that the project contributors are not very familiar with pytest, which seems like is not quite the case with Kallithea. HTH, Brianna What does it mean to “adopt pytest”? There can be many different definitions of “success”. Pytest can run many nose and unittest tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right? Progressive success might look like: · tests can be run (by pytest) without errors (there may be failures) · tests can be run (by pytest) without failures · test runner is integrated into CI server · existing tests are rewritten to take advantage of pytest features - this can happen in several iterations, for example: ·· changing to native assert statements (pycmd has a script to help with that, pyconvert_unittest.py) ·· changing setUp/tearDown methods to fixtures ·· adding markers ·· other changes to reduce boilerplate · assess needs for future tests to be written, e.g. new fixtures, distributed testing tweaks “Success” should also include that the development team feels comfortable with their knowledge of how to use pytest. In fact this is probably more important than anything else. So spending a lot of time on communication, giving examples, etc will probably be important - both in running the tests, and in writing them. It may be after the month is up, the partner project decides that pytest is not right for it. That’s okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies. -------------------- Notes sent to pytest helpers: If you are happy with the project I have put you with, please make contact with the project person and start having a look at the project itself. Install it, build it, play with it. Join the mailing list and IRC channel if they exist. Read the docs and any existing tests. Run the tests and see if they all pass! (Also in the existing test runner.) Take note of any test files which are noticably slower than the others. Try running them distributed and see if any extra failures pop up, this is a notoriously good way of bringing out test interdependencies or reliance on globals. Before you start writing any tests, be sure to discuss with the project person about what the project is, what their goals are, what their knowledge of testing and pytest is, what they see as areas that need attention, any specific things they want to achieve. Try to make a rough plan which has lots of steps, each step being an achievement in its own right. A month is not very long, so for a large project don't expect to get everything 100% perfect. Some projects have not yet made a pypi release yet - see if they want help with that. Some projects don't have CI set up yet - I would suggest that is a good early goal to set, along with getting into the practice of having all existing tests passing. Also record the test coverage, although keep in mind it can be misleading. Projects that are closer to being libraries may be happy with just unit tests. Projects closer to applications, will probably benefit from some functional/system tests. Find out which areas of the code are bug-prone, or are already planned to be rewritten/refactored - these are excellent candidates for functional tests, NOT unit tests. If you find yourself rewriting tests several times, this is not a bad thing at all. Tests should evolve as everyone's knowledge evolves and also as the code evolves. When you submit PRs, be extra verbose at the start to explain what you are doing and why. Encourage them to ask you about anything they don't understand. Suggest useful command line options for them to use when running the tests. Also, keep in mind that this is an opportunity to see pytest from the eyes of a newcomer. Take notes of what seems confusing to them, or where you notice some detail missing in the pytest docs - or some functionality in pytest itself. I will ask you all about this at the end and it's much easier to record these details as you go rather than try to remember everything at the end of the month! -- They've just been waiting in a mountain for the right moment: http://modernthings.org/ _______________________________________________ kallithea-general mailing list kallithea-general@sfconservancy.org http://lists.sfconservancy.org/mailman/listinfo/kallithea-general