[issue11492] email.header.Header doesn't fold headers correctly

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 10378 to show that I don't think that this bug must be fixed in 
Python 2: it would break any application relying on the current folding 
algorithm.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +9680

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[Srikanth S]

Changes by Srikanth S sriks...@gmail.com:


--
nosy: +srikanths

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[Roundup Robot]

Roundup Robot devnull@devnull added the comment:

New changeset 51a551acb60d by R David Murray in branch '3.2':
#11492: rewrite header folding algorithm.  Less code, more passing tests.
http://hg.python.org/cpython/rev/51a551acb60d

New changeset fcd20a565b95 by R David Murray in branch 'default':
Merge: #11492: rewrite header folding algorithm.  Less code, more passing tests.
http://hg.python.org/cpython/rev/fcd20a565b95

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


--
resolution:  - fixed
stage: patch review - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

This was quite the adventure.  The more I worked on fixing the tests, the more 
if/else cases the existing splitting algorithm grew.  When I reached the point 
where fixing one test broke two others, I thought maybe it was time to try a 
different approach.

Based on the knowledge gathered by banging my head on the old algorithm, I 
developed a new one.  This one is more RFC2822/RFC5322 compliant, I believe.  
It breaks only at FWS, but still gives preference to breaking after commas or 
semicolons by default.

I had to adjust several tests that tested broken behavior: the folded lines 
were longer than maxlen even though there were suitable fold points.

I'm very happy with this patch because there are 70 fewer lines of code but the 
module passes more tests.

Even though the code changes are extensive, I plan to apply this to 3.2.  It 
fixes bugs, and the new code is at least somewhat easier to understand than the 
old code (if only because there is less of it!)  I don't plan to apply it to 
3.1 because one older test fails if the patch is applied and I don't understand 
why (it appears to have nothing to do with line wrapping, and the same test 
works fine in 3.2).

--
stage: needs patch - patch review
versions:  -Python 3.1
Added file: http://bugs.python.org/file21607/better_header_spliter.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Note that this fix solves issue 11772, so I've closed that one as a duplicate.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers at spaces if value contains '; 's

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

OK, it looks like the wrapping problem arises when the line contains runs of 
blank delimited tokens longer than maxlinelen *and* the line also contains 
';'s.  The line is then split at the ';' and the remaining overlong pieces are 
not split.

I'll work on a fix.

--
stage: committed/rejected - needs patch
title: email.header.Header doesn't fold headers at spaces - 
email.header.Header doesn't fold headers at spaces if value contains ';'s

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Here is a patch containing three test cases that demonstrate three different 
failings of the header folding algorithm.  I'm working on the fix, but it is 
non-trivial.

--
components: +Library (Lib) -None
keywords: +patch
title: email.header.Header doesn't fold headers at spaces if value contains 
';'s - email.header.Header doesn't fold headers correctly
Added file: http://bugs.python.org/file21563/header_folding_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


Removed file: http://bugs.python.org/file21563/header_folding_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Note that 2.7 fails two of these tests as well, but for different reasons.  I'm 
not currently planning to fix 2.7, as its behavior at least (a) doesn't lose 
non-whitespace information and (b) doesn't exceed the maxheaderlen.

--
Added file: http://bugs.python.org/file21564/header_folding_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

Changes by R. David Murray rdmur...@bitdance.com:


Removed file: http://bugs.python.org/file21564/header_folding_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Here is an updated test patch that brings the test coverage of the relevant 
code much closer to 100%.  There are still three lines and one branch 
uncovered, but it appears as though one of the bugs is preventing the test case 
that would produce full coverage from getting to the relevant code path.  This 
gives me enough coverage to feel safer mucking about with the algorithm.

--
Added file: http://bugs.python.org/file21576/header_folding_tests.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers correctly

[Roundup Robot]

Roundup Robot devnull@devnull added the comment:

New changeset 10725fc76e11 by R David Murray in branch '3.1':
#11492: fix header truncation on folding when there are runs of split chars.
http://hg.python.org/cpython/rev/10725fc76e11

New changeset 74ec64dc3538 by R David Murray in branch '3.2':
Merge #11492: fix header truncation on folding when there are runs of split 
chars.
http://hg.python.org/cpython/rev/74ec64dc3538

New changeset 5ec2695c9c15 by R David Murray in branch 'default':
Merge #11492: fix header truncation on folding when there are runs of split 
chars.
http://hg.python.org/cpython/rev/5ec2695c9c15

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers

[Scott Kitterman]

Scott Kitterman skl...@kitterman.com added the comment:

Not so fast ...  I may have done this wrong, but I get:

print(Header(hdrin,maxlinelen=78))
Received: from mailout00.controlledmail.com (mailout00.controlledmail.com 
[72.81.252.19]) by mailwash7.pair.com (Postfix) with ESMTP id 16BB5BAD5 for 
b...@kitterman.com; Sun, 13 Mar 2011 23:46:05 -0400 (EDT)

all in one line with python3.2, so maxlinelen doesn't appear to do anything.  
With python2.7 it seems to when invoked that way:

Python 2.7.1+ (r271:86832, Mar 24 2011, 00:39:14) 
[GCC 4.5.2] on linux2
Type help, copyright, credits or license for more information.
 from email.header import Header
 hdrin = 'Received: from mailout00.controlledmail.com 
 (mailout00.controlledmail.com [72.81.252.19]) by mailwash7.pair.com 
 (Postfix) with ESMTP id 16BB5BAD5 for b...@kitterman.com; Sun, 13 Mar 
 2011 23:46:05 -0400 (EDT)'
 print(Header(hdrin))
Received: from mailout00.controlledmail.com (mailout00.controlledmail.com
 [72.81.252.19]) by mailwash7.pair.com (Postfix) with ESMTP id 16BB5BAD5 for
 b...@kitterman.com; Sun, 13 Mar 2011 23:46:05 -0400 (EDT)
 print(Header(hdrin, maxlinelen=30))
Received: from
 mailout00.controlledmail.com
 (mailout00.controlledmail.com
 [72.81.252.19]) by
 mailwash7.pair.com (Postfix)
 with ESMTP id 16BB5BAD5 for
 b...@kitterman.com;
 Sun, 13 Mar 2011 23:46:05
 -0400 (EDT)

--
resolution: invalid - 
status: closed - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers at spaces

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

You have to do an 'encode' to get the wrapped header. __str__ uses 
maxlinelen=None.

However, there does seem to be a problem with the line wrapping algorithm 
revealed by your example: it is only doing a line break at the ';', not at any 
spaces.  I will look in to this.

--
title: email.header.Header doesn't fold headers - email.header.Header doesn't 
fold headers at spaces

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Ah, it isn't broken, it's just that the default changed.  In 2.x, the default 
was maxlinelen=78, in 3.x, the default is maxlinelen=None (unlimited), but 
generator passes in an override of 78 when formatting output.  So you can 
specify an explicit maxlinelen=78 and that will wrap the headers in both 2.x 
and 3.x.  (There are differences in the wrapping algorithms, though!)

--
resolution:  - invalid
stage: needs patch - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers

[Scott Kitterman]

New submission from Scott Kitterman skl...@kitterman.com:

Header folding is very different (non-existent as far as I've found so far) in 
Python3.  Here's a short example:

#!/usr/bin/python
# -*- coding: ISO-8859-1

from email.header import Header

hdrin = 'Received: from mailout00.controlledmail.com 
(mailout00.controlledmail.com [72.81.252.19]) by mailwash7.pair.com (Postfix) 
with ESMTP id 16BB5BAD5 for b...@kitterman.com; Sun, 13 Mar 2011 23:46:05 
-0400 (EDT)'

print(Header(hdrin))

With python2.6 the output is:


Received: from mailout00.controlledmail.com (mailout00.controlledmail.com
 [72.81.252.19]) by mailwash7.pair.com (Postfix) with ESMTP id 16BB5BAD5 for
 b...@kitterman.com; Sun, 13 Mar 2011 23:46:05 -0400 (EDT)

With python3.1 or 3.2 the output is one line:

Received: from mailout00.controlledmail.com (mailout00.controlledmail.com 
[72.81.252.19]) by mailwash7.pair.com (Postfix) with ESMTP id 16BB5BAD5 for 
b...@kitterman.com; Sun, 13 Mar 2011 23:46:05 -0400 (EDT)

This makes it very difficult to write header processing code that works for 
both Python2 and Python3 even if one can fold headers at all in Python3.

--
components: None
messages: 130793
nosy: kitterma
priority: normal
severity: normal
status: open
title: email.header.Header doesn't fold headers

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers

[Barry A. Warsaw]

Changes by Barry A. Warsaw ba...@python.org:


--
nosy: +barry, r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11492] email.header.Header doesn't fold headers

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

It exists, but clearly it is broken.  I'll look in to it.

--
assignee:  - r.david.murray
stage:  - needs patch
type:  - behavior
versions: +Python 3.1, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11492
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com