On May 1, 2009, at 3:01 AM, Petr Hroudný wrote:
due to Python defaults, mailman exhibits strange behaviour when processing UTF-8 emails. When no header/footer is configured, mailman passes UTF-8 emails in original form, i.e. 8bit. However, when either header or footer is configured in mailman, it uses Python's libraries to add them and as a side effect it converts 8bit emails into 7bit base64 encoded ones.
Just to be clear, we're talking about headers and footers added by appending the text to the main body, right? When headers and footers are attached via MIME, Mailman always does the right thing?
This is highly undesirable in some cases. For instance, mailinglist might be used to distribute trouble tickets or other content which is expected to be easily parsable by automated text-based utilities. With base64, emails grow in size by 33 % and such emails are getting much higher spam scores since base64 is typically used by spammers to obfuscate the payload. There are of course much more reasons for not using base64 as the primary encoding method for UTF-8 email.
One option of course would be to simply not add headers and footers for these types of mailing lists. Another option, which I proposed in bug 373083, is to add an option to force a mailing list to always attach headers and footers as MIME. You risk some users not seeing them because their MUA doesn't display them inline and they don't know how to click on the attachments, but I think that's an acceptable risk for the minority of mailing lists that care about this. At least for MM2.1.
The fix is quite simple and is already widely used by other projects.
All that needs to be done is
to redefine Python's UTF-8 charset properties, i.e. in every place
where you have
from email.Charset import Charset
you need to add:
email.Charset.add_charset('utf-8',email.Charset.SHORTEST, None, None)
With such setting, mailman will keep the 8bit encoding also when it's
adding header/footer and won't downconvert to 7bit+base64. So I'd like
to propose the above addition, at least as a configurable option
if there's any fear that enabling it by default could cause some
problems.
I'll note that you don't need to change Mailman at all do to this. You simply need to add this to your mm_cfg.py file and I'll bet it will just work for you.
-Barry
PGP.sig
Description: This is a digitally signed message part
_______________________________________________ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
