[issue20747] Charset.header_encode in email.charset doesn't take a maxlinelen argument and has inconsistent behavior with different encodings
Rik added the comment: Ok, do you think there's any risk in making `base64mime.header_encode` handle `maxlinelen=None`? I think it would be more consistent if `base64mime.header_encode` and `quoprimime.header_encode` interpret their arguments similarly. -- ___ Python tracker <http://bugs.python.org/issue20747> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20747] Charset.header_encode in email.charset doesn't take a maxlinelen argument and has inconsistent behavior with different encodings
Rik added the comment: Ok, so you suggest to use `maxlinelen=None` for the `base64mime.header_encode` which will act the same as giving `maxlinelen=None` to `email.quoprimime`, so that we don't need to change the API? And this change would then also be reflected in the Python 3.5 legacy interface? -- ___ Python tracker <http://bugs.python.org/issue20747> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20747] Charset.header_encode in email.charset doesn't take a maxlinelen argument and has inconsistent behavior with different encodings
Rik added the comment: Line wrapping is indeed done by `Header`, but why do `base64mime` and `quoprimime` then have their own line wrapping? I assume so that you can also use them independently. So that's why I would think `Charset.header_encode` should also accept a `maxlinelen` so that you can use `Charset` independently too. -- ___ Python tracker <http://bugs.python.org/issue20747> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20747] Charset.header_encode in email.charset doesn't take a maxlinelen argument and has inconsistent behavior with different encodings
New submission from Rik:
If you look at the `header_encode` method in the `Charset` class in
`email.charset`, you'll see that depending on the `header_encoding` that is set
on the `Charset` instance, it will either encode it using base64 or
quoted-printable (QP):
http://hg.python.org/cpython/file/3a1db0d2747e/Lib/email/charset.py#l351
However, QP always uses `maxlinelen=None` and base64 doesn't. This results in
the following behaviour:
- If you use base64 encoding and your header size is longer than the default
`maxlinelen`, it will be split over multiple lines.
- If you use QP encoding with the same header it doesn't get split over
multiple lines.
You can easily test it with this snippet:
from email.charset import Charset, BASE64, QP
header = (
'tejkstj tlkjes takldjf aseio neaoiflk asnfoieas nflkdan foeias '
'naskln ioeasn kldan flkansoie naslk dnaslk fndaslk fneoisaf '
'neklasn dfklasnf oiasenf lkadsn lkfanldk fas dfknaioe nas'
)
charset = Charset('utf-8')
charset.header_encoding = BASE64
print 'BASE64:'
print charset.header_encode(header)
charset.header_encoding = QP
print 'QP:'
print charset.header_encode(header)
Which will output:
BASE64:
=?utf-8?b?dGVqa3N0aiB0bGtqZXMgdGFrbGRqZiBhc2VpbyBuZWFvaWZsayBhc25mb2llYXMg?=
=?utf-8?b?bmZsa2RhbiBmb2VpYXMgbmFza2xuIGlvZWFzbiBrbGRhbiBmbGthbnNvaWUgbmFz?=
=?utf-8?b?bGsgZG5hc2xrIGZuZGFzbGsgZm5lb2lzYWYgbmVrbGFzbiBkZmtsYXNuZiBvaWFz?=
=?utf-8?b?ZW5mIGxrYWRzbiBsa2ZhbmxkayBmYXMgZGZrbmFpb2UgbmFz?=
QP:
=?utf-8?q?tejkstj_tlkjes_takldjf_aseio_neaoiflk_asnfoieas_nflkdan_foeias_naskln_ioeasn_kldan_flkansoie_naslk_dnaslk_fndaslk_fneoisaf_neklasn_dfklasnf_oiasenf_lkadsn_lkfanldk_fas_dfknaioe_nas?=
This is inconsistent behavior.
Aside from that, I think the `header_encode` method should accept an argument
`maxlinelen` that defaults to an appropriate value (probably 76), but which you
can overwrite on free will.
This is (I think) also necessary because the `Header` class in `email.header`
has a `maxlinelen` attribute that is used for the same purpose. Normally this
works fine, but when you specified a charset for your header, it uses the
`Charset` class and the `maxlinelen` is lost. This is happening here:
http://hg.python.org/cpython/file/3a1db0d2747e/Lib/email/header.py#l368
You see, the `_encode_chunks` takes the `maxlinelen` argument but doesn't pass
it on to the `header_encode` method of `charset` (which is a `Charset`
instance).
As such, you can see this issue in action with the following snippet:
from email.header import Header
maxlinelen = 999
print 'No charset:'
print Header(
u'asdfjk lasjdf sajdfl ajsdfaj sdlkfjas kfladjs flkajsdflk jsadklf
jadslkfj adslkfj asdlkjf lksadjfkldas jfkldasj fkadsj fladsjf kladsjfk
asdjfkldasasd kfaj kfladsj fkadsjf asdf ',
maxlinelen=maxlinelen
).encode()
print 'Charset with special characters:'
print Header(
u'attachment; filename="ajdsklfj klasdjfkl asdjfkl jadsfja sdflkads fad
fads adsf dasjfkl jadslkfj dlasf asd \u6211\u6211\u6211 jo \u6211\u6211 jo
\u6211\u6211"',
charset='utf-8',
maxlinelen=999
).encode()
Which will output:
No charset:
asdfjk lasjdf sajdfl ajsdfaj sdlkfjas kfladjs flkajsdflk jsadklf jadslkfj
adslkfj asdlkjf lksadjfkldas jfkldasj fkadsj fladsjf kladsjfk asdjfkldasasd
kfaj kfladsj fkadsjf asdf
Charset with special characters:
=?utf-8?b?YXR0YWNobWVudDsgZmlsZW5hbWU9ImFqZHNrbGZqIGtsYXNkamZrbCBhc2RqZmts?=
=?utf-8?b?IGphZHNmamEgc2RmbGthZHMgZmFkIGZhZHMgYWRzZiBkYXNqZmtsIGphZHNsa2Zq?=
=?utf-8?b?IGRsYXNmIGFzZCDmiJHmiJHmiJEgam8g5oiR5oiRIGpvIOaIkeaIkSI=?=
This is currently an issue we're experiencing in Django, see our issue in the
issue tracker:
https://code.djangoproject.com/ticket/20889#comment:4
--
components: Library (Lib), email
messages: 212011
nosy: barry, r.david.murray, rednaw
priority: normal
severity: normal
status: open
title: Charset.header_encode in email.charset doesn't take a maxlinelen
argument and has inconsistent behavior with different encodings
type: behavior
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue20747>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11218] pattern=None when following documentation for load_tests and unittest.main()
Rik Poggi added the comment: I wasn't trying to make any argument, just thinking that such particular signature was intentional. Also notice that there might be code that doesn't pass the pattern argument, and fall back on the default value. So a signature change will break their code. I think that the best solution would be to provide a better documentation. I don't know what's the rational behind all that. -- ___ Python tracker <http://bugs.python.org/issue11218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11218] pattern=None when following documentation for load_tests and unittest.main()
Rik Poggi added the comment: I think the doc should be improved (http://docs.python.org/library/unittest.html#load-tests-protocol), it's not clear how pattern in the example (last one) could not be None. Changing the discover signature doesn't seem to be an option since the TestLoader.discover doc http://docs.python.org/library/unittest.html#unittest.TestLoader.discover says: The pattern is deliberately not stored as a loader attribute so that packages can continue discovery themselves. -- nosy: +rik.poggi ___ Python tracker <http://bugs.python.org/issue11218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11060] distutils2 sdist does not complain about version that is not PEP 386 compliant
Rik Poggi added the comment: Moving on, I've understood what the bug is about. I've made a couple of tests for this issue. I'm waiting for a review before adding others (if necessary). The fix is not going to be easy, because I'm not sure about the Metadata design. I think that what should be done is to make available the old version scheme check for metadata-version 1.0 and 1.1. -- Added file: http://bugs.python.org/file25162/issue_tests.diff ___ Python tracker <http://bugs.python.org/issue11060> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11060] distutils2 sdist does not complain about version that is not PEP 386 compliant
Rik Poggi added the comment: My assumption that the "random" log message was related to this bug seems to be completely wrong. It seems, instead, related to a starstar call of create_dist in the support module that will loose the order (of an OrderedDict obviously). The behaviour is still strange because in order to reproduce it I had to re-run the test different. (test_star_star is the "random" failing one) Attaching the diff. -- Added file: http://bugs.python.org/file25147/test_support.diff ___ Python tracker <http://bugs.python.org/issue11060> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11060] distutils2 sdist does not complain about version that is not PEP 386 compliant
Rik Poggi added the comment: Thanks, I'll wait! In the meanwhile I couldn't help to dig a little deeper and found that the Metadata class is currently logging a warning. Should the commands raise something when there's a warning in strict mode? I was playing with the check command about this when I stumbled in an odd looking piece of code in metadata.py with a FIXME note: "# FIXME this rejects UNKNOWN, is that right?" (see attached file). I'm not sure how, but it seems related to a "random" (I couldn't find any pattern) log message that sometimes give: "'UNKNOWN': '0.4.5dev' is not a valid version (field 'Version')" and others: "'Name': '0.4.5dev' is not a valid version (field 'Version')" (I had this with consecutive execution from a simple test that I wrote in test_command_check, with metadata['name'] == 'Name'). I hoped to not have wasted your time, but I thought that it could may be related to this bug since it seems that the version gets rightfully (but strangefully) warned as not valid from this "middle-layer" check point. -- Added file: http://bugs.python.org/file25141/metadata_set_extract.py ___ Python tracker <http://bugs.python.org/issue11060> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11060] distutils2 sdist does not complain about version that is not PEP 386 compliant
Rik Poggi added the comment: Strictly related or not to this bug, a bit more test coverage shouldn't hurt. So while waiting for a reply I started writing a couple of tests for pypi/dist.py, hope they look good. -- Added file: http://bugs.python.org/file25134/test_pypi_dist.diff ___ Python tracker <http://bugs.python.org/issue11060> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11060] distutils2 sdist does not complain about version that is not PEP 386 compliant
Rik Poggi added the comment: Hi, I'd like to contribute to this bug if it's possible. I would've directly started, but it seems the situation has moved/changed a bit, so I'm not sure where the tests are needed. There's a try/except, like the one mentioned above, in pypi/dist.py. Was that a consequence of this bug? Are tests needed for that conversion? The sdist command seems to be not complaining about any kind of irrational/invalid version. Should this be fixed? Should sdist check for a valid version number and do something like what Éric Araujo mentioned in the last comment? -- nosy: +rik.poggi ___ Python tracker <http://bugs.python.org/issue11060> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
