Hi Elena, I like fuzz testing and how Hypothesis does it. I have used it on a couple of projects and it always found bugs. In fact it was my first Pycon that I sat with David and sprinted on adding it to a project :)
That said I've always found it quite a leap from straightforward tests, being slower and non-deterministic. There is a hard balance to make writing fuzz tests versus writing individual test cases to ensure edge cases keep being covered. And also we generally want tests to be less complex than the code they are testing - short functions with several parameters can easily need many lines of hypothesis code to set up accurate testing. Personally I tend to settle on using a little bit of fuzzing with random data from factory boy, but adding control to the random seed used in testing using pytest-randomly. I gave this approach a bit of a write up when I started using it in 2014: https://adamj.eu/tech/2014/09/03/factory-boy-fun/ . As for Django's test sutie: I don't believe there is any fuzz testing there. But it is being tested separately instead with Google's OSS-Fuzz project: https://github.com/google/oss-fuzz/tree/master/projects/django . This was set up by Guido Vranken who announced it in this mailing list thread: https://groups.google.com/forum/#!topic/django-developers/-WweB07YiVQ . Apparently it has caught a few bugs since implementation, and there's scope to expand it quite a lot: https://groups.google.com/d/msg/django-developers/PMtvmMlsjyw/_rOj_LzvCQAJ . I *think* if we were to increase our fuzz testing, we'd probably want to concentrate efforts there. Google also offer $ rewards to critical projects like Django for reaching certain levels of coverage, which is nice of them. Thanks, Adam -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM2%2Bbz%2BwVgiZiVG7YijwO0zzxZcfVE%3Dpxmcper7D3iUJQg%40mail.gmail.com.
