On Thu, 2017-06-29 at 00:06 +1000, Daniel Axtens wrote:
> Andrew Donnellan <andrew.donnel...@au1.ibm.com> writes:
>
> > On 28/06/17 17:48, Daniel Axtens wrote:
> > > Certain really messed up email messages can cause a failure within
> > > the email module (at least on py3). Catch this.
> > >
> > > Signed-off-by: Daniel Axtens <d...@axtens.net>
> > > ---
> > > patchwork/management/commands/parsearchive.py | 9 ++++++++
> > > patchwork/management/commands/parsemail.py | 31 ++++++++++++++++-----
> > > ------
> > > 2 files changed, 27 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/patchwork/management/commands/parsearchive.py
> > > b/patchwork/management/commands/parsearchive.py
> > > index a3c8360186c8..3aab58a45bcd 100644
> > > --- a/patchwork/management/commands/parsearchive.py
> > > +++ b/patchwork/management/commands/parsearchive.py
> > > @@ -77,6 +77,15 @@ class Command(BaseCommand):
> > >
> > > count = len(mbox)
> > >
> > > + # detect broken mails in the mbox
> > > + # see earlyfail fuzz test on py3
> > > + try:
> > > + for m in mbox:
> > > + pass
> > > + except AttributeError:
> > > + logger.warning('Broken mbox/Maildir, aborting')
> > > + return
> > > +
> >
> > This seems a bit non-obvious and could do with a little bit of explanation?
>
> The message, or the code structure? I structured the code this way
> rather than the more obvious
> try:
> mbox = [m for m in mbox]
> ...
> because the more obvious way requires loading the entire mbox/maildir
> into memory and I was a bit worried about the memory consumption of that
> when parsing a large mbox.
>
> I agree a more helpful comment would have been in order. Stephen, do you
> want a v2 of this patch by itself? I can resend the series but it seems
> a bit excessive... Or I could do a follow-up.
Yeah, let's just do a follow-up. I'm going to go ahead and merge the rest of
these.
Stephen
_______________________________________________
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
