Hi all, I fuzzed parsemail with python-afl/afl-fuzz.
Predictably, it did not hold up well. Nothing major, just a bunch of edge cases. Date parsing held up particularly poorly. Currently, if we hit any of the errors this patch set fixes, the entire mail will be rejected. With this patch set, the emails have a much better chance of surviving. This is helpful for us - we have previously had bug reports where a stray mail has been rejected due to a corrupt header, which is suboptimal. Test cases are included. They may not email particularly well so I have also put them up on my github as a signed tag: https://github.com/daxtens/patchwork/releases/tag/fuzz-testing For the interested, the bulk of the fuzzing was done on an AWS cloud instance, using something quite similar to the standard docker-compose setup. Several hundered thousand executions were done - nowhere near what I'd like but I can't seem to get execution speed above about 1 execution per second. If I was able to confidently mock out the db or extract the parser out of the django framework it would go faster, but both of those were a bit more than I was easily able to do. I also picked up some bugs when interfacing with the db (patch 7) so I feel like this approach was vindicated. I also included a patch to allow people to replicate my setup. It's not really ready to merge and I'm not convinced it's really necessary as it's not a lot of work to replicate. Regards, Daniel Daniel Axtens (10): parser: fix charset 'guessing' algorithm parser: don't assume headers are strings parser: codec lookup fails when a NUL (\x00) is in the name parser: catch failures in decoding headers parser: deal with headers entirely failing to parse parser: better date parsing parser: Don't pass a message-id longer than 255 chars to the db parse(mail|archive): handle early fail within email module Add fuzzer-generated tests [RFC] Fuzzing harness patchwork/management/commands/fuzz.py | 88 +++++++++++++++ patchwork/management/commands/parsearchive.py | 9 ++ patchwork/management/commands/parsemail.py | 31 +++--- patchwork/parser.py | 126 +++++++++++++++++----- patchwork/tests/__init__.py | 1 + patchwork/tests/fuzztests/base64err.mbox | 46 ++++++++ patchwork/tests/fuzztests/charset.mbox | 131 +++++++++++++++++++++++ patchwork/tests/fuzztests/codec-null.mbox | Bin 0 -> 8192 bytes patchwork/tests/fuzztests/date-oserror.mbox | Bin 0 -> 8209 bytes patchwork/tests/fuzztests/date-too-long.mbox | Bin 0 -> 1828 bytes patchwork/tests/fuzztests/date.mbox | 44 ++++++++ patchwork/tests/fuzztests/dateheader.mbox | Bin 0 -> 580 bytes patchwork/tests/fuzztests/earlyfail.mbox | Bin 0 -> 1712 bytes patchwork/tests/fuzztests/msgid-len.mbox | Bin 0 -> 1809 bytes patchwork/tests/fuzztests/msgid-len2.mbox | 37 +++++++ patchwork/tests/fuzztests/msgidheader.mbox | 131 +++++++++++++++++++++++ patchwork/tests/fuzztests/refshdr.mbox | Bin 0 -> 816 bytes patchwork/tests/fuzztests/unknown-encoding.mbox | Bin 0 -> 1751 bytes patchwork/tests/fuzztests/value2.mbox | Bin 0 -> 1598 bytes patchwork/tests/fuzztests/year-out-of-range.mbox | Bin 0 -> 1660 bytes patchwork/tests/test_parser.py | 60 ++++++++++- tools/docker/Dockerfile | 2 + tools/fuzzer_dict | 52 +++++++++ 23 files changed, 714 insertions(+), 44 deletions(-) create mode 100644 patchwork/management/commands/fuzz.py create mode 100644 patchwork/tests/fuzztests/base64err.mbox create mode 100644 patchwork/tests/fuzztests/charset.mbox create mode 100644 patchwork/tests/fuzztests/codec-null.mbox create mode 100644 patchwork/tests/fuzztests/date-oserror.mbox create mode 100644 patchwork/tests/fuzztests/date-too-long.mbox create mode 100644 patchwork/tests/fuzztests/date.mbox create mode 100644 patchwork/tests/fuzztests/dateheader.mbox create mode 100644 patchwork/tests/fuzztests/earlyfail.mbox create mode 100644 patchwork/tests/fuzztests/msgid-len.mbox create mode 100644 patchwork/tests/fuzztests/msgid-len2.mbox create mode 100644 patchwork/tests/fuzztests/msgidheader.mbox create mode 100644 patchwork/tests/fuzztests/refshdr.mbox create mode 100644 patchwork/tests/fuzztests/unknown-encoding.mbox create mode 100644 patchwork/tests/fuzztests/value2.mbox create mode 100644 patchwork/tests/fuzztests/year-out-of-range.mbox create mode 100644 tools/fuzzer_dict -- 2.11.0 _______________________________________________ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork