OT: culprit in MIME recoding and breaking signatures
The winner appears to be Perl, namely the Mail::Audit module (and whatever other modules it relies on). I had a couple of scripts that did gentle transformations of incoming mails. The transformations were supposed to only ever touch the headers, but I used a Mail::Audit object to write back the entire message including the body. I had complete trust that the body would be bitwise identical to the original, but not so. This is a very unpleasant surprise, I have thought of Perl and the more popular CPAN modules as the good old workhorse which would always do the job efficiently and reliably - if not always legibly :) I intentionally preferred Perl for this task over some combination of procmail, formail, sed and pcregrep which would have worked. My world is crumbling :( -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. To reply privately _only_ on Usenet and on broken lists which rewrite From, fetch the TXT record for no-use.mooo.com.
Re: OT: culprit in MIME recoding and breaking signatures
On 11/23/18 1:40 AM, Ian Zimmerman wrote: The winner appears to be Perl, namely the Mail::Audit module (and whatever other modules it relies on). I had a couple of scripts that did gentle transformations of incoming mails. The transformations were supposed to only ever touch the headers, but I used a Mail::Audit object to write back the entire message including the body. I had complete trust that the body would be bitwise identical to the original, but not so. What change did it make? I suppose it might use a different MIME transfer encoding, changing the representation but not the meaning. Also, the most common variants of mbox are known to break cryptographic signatures, with the notorious ">From " escaping.
Re: OT: culprit in MIME recoding and breaking signatures
On 2018-11-23 09:47, Kurt Hackenberg wrote: > > The winner appears to be Perl, namely the Mail::Audit module (and > > whatever other modules it relies on). I had a couple of scripts that > > did gentle transformations of incoming mails. The transformations were > > supposed to only ever touch the headers, but I used a Mail::Audit object > > to write back the entire message including the body. I had complete > > trust that the body would be bitwise identical to the original, but not > > so. > > What change did it make? I suppose it might use a different MIME > transfer encoding, changing the representation but not the meaning. It keeps the original QP encoding, but it recodes some (but not all) sequences of spaces as =20, and vice versa. Please see a recent thread I started here for exact details, and also my blog: https://very.loosely.org/itz-blog/ > Also, the most common variants of mbox are known to break > cryptographic signatures, with the notorious ">From " escaping. I stopped using mbox some years ago, and this could have been one of the reasons (though the main reason was fragility when taking apart folders and reassembling them). -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. To reply privately _only_ on Usenet and on broken lists which rewrite From, fetch the TXT record for no-use.mooo.com.
Re: OT: culprit in MIME recoding and breaking signatures
On Thu, Nov 22, 2018 at 10:40:22PM -0800, Ian Zimmerman wrote: The winner appears to be Perl, namely the Mail::Audit module (and whatever other modules it relies on). I had a couple of scripts that did gentle transformations of incoming mails. Congratulations on tracking this down. This is a very unpleasant surprise, I have thought of Perl and the more popular CPAN modules as the good old workhorse which would always do the job efficiently and reliably - if not always legibly :) I intentionally preferred Perl for this task over some combination of procmail, formail, sed and pcregrep which would have worked. My world is crumbling :( I'm a bit surprised too. Although it has fallen out of favor, I have used and relied on Perl for a long time too, and it never let me down. -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
