Thomas Petazzoni reported that Patchwork would occasionally lose Buildroot email. Andrew - having talked to jk and sfr - suggested that this may be race-condition related.
I investigated and found some bugs. I first had to develop some tools. Along the way I found other unrelated bugs too. Patches 1-4 are tooling - ways to do parallel parsing of messages and get and compare the output. (Patch 1 fixes an issue I found when running the tool from patch 2) Patch 5 is an unrelated fix that came up along the way and demonstrates that humans remain the best fuzzers, and that Python's email module is still adorably* quirky. Patch 6 is a bug that came up very quickly in testing but is unlikely to be the actual bug Buildroot is hitting, as it can only occur the first time an email address is seen. Patch 7 is a related tidy-up/optimisation. Patch 8 fixes up a MySQL-only bug, but also adds some robustness. I think patch 9 closes the most likely issue for Buildroot patches. Pending review, patches 5, 6, 8 and 9 should go to stable. Regards, Daniel Cc: Andrew Donnellan <andrew.donnel...@au1.ibm.com> Cc: Thomas Petazzoni <thomas.petazz...@free-electrons.com> Daniel Axtens (9): tools/docker: assume terminal supports utf-8 debugging: add command to dump patches and series tools/scripts: split a mbox N ways tools/scripts: parallel_parsearchive - load archives in parallel parser: Handle even more exotically broken headers parser: close a TOCTTOU bug on Person creation parser: avoid an unnecessary UPDATE of Person parser: use Patch.objects.create instead of save() parser: don't fail on multiple SeriesReferences patchwork/management/commands/debug_dump.py | 46 +++++++++++++++ patchwork/parser.py | 92 +++++++++++++++++++---------- patchwork/tests/fuzztests/x-face.mbox | 58 ++++++++++++++++++ patchwork/tests/test_parser.py | 44 +++++++------- tools/docker/Dockerfile | 1 + tools/scripts/parallel_parsearchive.sh | 55 +++++++++++++++++ tools/scripts/split_mail.py | 76 ++++++++++++++++++++++++ 7 files changed, 317 insertions(+), 55 deletions(-) create mode 100644 patchwork/management/commands/debug_dump.py create mode 100644 patchwork/tests/fuzztests/x-face.mbox create mode 100755 tools/scripts/parallel_parsearchive.sh create mode 100755 tools/scripts/split_mail.py -- 2.14.1 _______________________________________________ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
