[issue12455] urllib2 forces title() on header names, breaking some requests

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy:  -demian.brecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

Mark,

I'm happy to followup. 
I will be in favor of removing any capitalization and not to change headers 
whatever they are. Because it doesn't matter per spec. Browsers do not care 
about the capitalization. And I haven't identified Web Compatibility issues 
regarding the capitalization.

That said, it seems that Cal msg139512 had an issue, I would love to know which 
server/API had this behavior to fill a but at http://webcompat.com/

So…

Where do we stand? Feature or removing anything which modifies the 
capitalization of headers?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Mark Lawrence]

Mark Lawrence added the comment:

@Karl do you intend following up on this issue?

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

terry.reedy:


You said: and that has_header and get_header *require* the .capitalize() form 
and reject the .title() form.

I made a patch for these two. 
See http://bugs.python.org/issue5550

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

Are there issues related to removing the capitalize() and title() appears?

# title()

* http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l1239

# capitalize()

* http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l359
* http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l363
* http://hg.python.org/cpython/file/886df716cd09/Lib/urllib/request.py#l1206

Because the behavior is inconsistent, I would live to propose a patch removing 
them and be sure to be completely neutral with regards to them.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

tests in 
http://hg.python.org/cpython/file/886df716cd09/Lib/test/test_wsgiref.py#l370 
also checking that everything is case insensitive. 

And the method to get the headers in wsgiref, make sure they are lower-case
http://hg.python.org/cpython/file/886df716cd09/Lib/wsgiref/headers.py#l82

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

Note that HTTP header fields are case-insensitive.
See http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging#section-3.2

   Each HTTP header field consists of a case-insensitive field name
   followed by a colon (:), optional whitespace, and the field value.

Basically the author of a request can set them to whatever he/she wants. But we 
should, IMHO, respect the author intent. It might happen that someone will 
choose a specific combination of casing to deal with broken servers and/or 
proxies. So a cycle of set/get/send should not modify at all the headers.

--
nosy: +karlcow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[karl]

karl added the comment:

So looking at the casing of headers, I discovered other issues. I opened 
another bug. http://bugs.python.org/issue17322

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy: +dbrecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Terry J. Reedy]

Terry J. Reedy added the comment:

The comment about urllib.request forcing .title() is consistent with 
'Content-Length' and 'Content-Type' in the docs but puzzling and inconsistent 
given that in 3.3, header names are printed .capitalize()'ed and not 
.title()'ed and that has_header and get_header *require* the .capitalize() form 
and reject the .title() form.

import urllib.request
opener = urllib.request.build_opener()
request = urllib.request.Request(http://example.com/;, headers =
{Content-Type: application/x-www-form-urlencoded})
opener.open(request, 1.encode(us-ascii))
print(request.header_items(),
  request.has_header(Content-Type),
  request.has_header(Content-type),
  request.get_header(Content-Type),
  request.get_header(Content-type), sep='\n')
 
[('Content-type', 'application/x-www-form-urlencoded'), ('Content-length', 
'1'), ('User-agent', 'Python-urllib/3.3'), ('Host', 'example.com')]
False
True
None
application/x-www-form-urlencoded

Did .title in 2.7 urllib2 request get changed to .capitalize in 3.x 
urllib.request (without the examples in the doc being changed) or is request 
inconsistent within itself?

Cal did not the 2.7 code exhibiting the problme, but when I add this code in 
3.3, the output start as shown.

request.add_header('Content-MD5', 'xxx')
print(request.header_items())
#
[('Content-md5', 'xxx'), ...

So is 3.3 sending 'Content-Md5' or 'Content-md5'

My guess is the former, as urllib.request has the same single use of .title in 
.do_open as Cal quoted. The two files also have the same three uses of 
.capitalize in .add_header, .add_unredirected_header, and .do_request. So it 
seems that header names are normalized to .capitalize on entry and .title on 
sending, or something like that. Ugh. Is there any good justification for this?

I do not see anything in the doc about headers names being normalized either 
way or about the requirements of has_/get_header. If the behavior were 
consistent and the same since forever, then I would say the current docs should 
be improved and a change would be an enhancement request. Since the behavior 
seems inconsistent, I am more inclined to think there is a bug.

I realize that this message expands the scope of the issue, but it is all about 
the handing of header names in requests.

--
nosy: +terry.reedy
versions: +Python 3.4 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


Removed file: http://bugs.python.org/file22537/unnamed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Senthil Kumaran]

Senthil Kumaran sent...@uthcode.com added the comment:

AFAIR, that capitalize part is somewhere a requirement in RFC, if the server 
did not behave in proper manner, it may not be a good idea for the client to 
change (or be permissive the flag).

--
nosy: +orsenthil

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Senthil Kumaran]

Senthil Kumaran sent...@uthcode.com added the comment:

Sorry, not Capitalize, but the Title part. One can some bugs which lead to 
this change in the urllib2.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Quoting http://tools.ietf.org/html/rfc2068#section-4.2:

  Field names are case-insensitive.

Which is only logical, since they are modeled on email headers, and email 
header names are case insensitive.  So, the server in question is broken, yes, 
but that doesn't mean we can't provide a facility to allow Python to 
inter-operate with it.  Email, for example, preserves the case of the field 
names it parses or receives from the application program, but otherwise treats 
them case-insensitively.  However, since the current code coerces to title 
case, we have to provide this feature as a switchable facility defaulting to 
the current behavior, for backward compatibility reasons.

And someone needs to write a patch

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Cal Leeming]

Cal Leeming cal.leem...@simplicitymedialtd.co.uk added the comment:

Thats full understandable that the default won't change. I'll put this in my
todo list to write a patch in a week or two.
On 1 Jul 2011 08:45, R. David Murray rep...@bugs.python.org wrote:

 R. David Murray rdmur...@bitdance.com added the comment:

 Quoting http://tools.ietf.org/html/rfc2068#section-4.2:

 Field names are case-insensitive.

 Which is only logical, since they are modeled on email headers, and email
header names are case insensitive. So, the server in question is broken,
yes, but that doesn't mean we can't provide a facility to allow Python to
inter-operate with it. Email, for example, preserves the case of the field
names it parses or receives from the application program, but otherwise
treats them case-insensitively. However, since the current code coerces to
title case, we have to provide this feature as a switchable facility
defaulting to the current behavior, for backward compatibility reasons.

 And someone needs to write a patch

 --

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue12455
 ___

--
nosy: +sleepycal
Added file: http://bugs.python.org/file22537/unnamed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___pThats full understandable that the default won#39;t change. I#39;ll put 
this in my todo list to write a patch in a week or two./p
div class=gmail_quoteOn 1 Jul 2011 08:45, quot;R. David Murrayquot; 
lt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt; 
wrote:br type=attributiongt; brgt; R. David Murray lt;a 
href=mailto:rdmur...@bitdance.com;rdmur...@bitdance.com/agt; added the 
comment:br
gt; brgt; Quoting a 
href=http://tools.ietf.org/html/rfc2068#section-4.2;http://tools.ietf.org/html/rfc2068#section-4.2/a:brgt;
 brgt;   Field names are case-insensitive.brgt; brgt; Which is only 
logical, since they are modeled on email headers, and email header names are 
case insensitive.  So, the server in question is broken, yes, but that 
doesn#39;t mean we can#39;t provide a facility to allow Python to 
inter-operate with it.  Email, for example, preserves the case of the field 
names it parses or receives from the application program, but otherwise treats 
them case-insensitively.  However, since the current code coerces to title 
case, we have to provide this feature as a switchable facility defaulting to 
the current behavior, for backward compatibility reasons.br
gt; brgt; And someone needs to write a patchbrgt; brgt; 
--brgt; brgt; ___brgt; 
Python tracker lt;a 
href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;br
gt; lt;a 
href=http://bugs.python.org/issue12455;http://bugs.python.org/issue12455/agt;brgt;
 ___br/div
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

A feature request for a way to control this is reasonable.  However, new 
features can only go into 3.3.

--
stage:  - needs patch
title: urllib2 Request() forces capitalize() on header names, breaking some 
requests - urllib2 forces title() on header names, breaking some requests
type: behavior - feature request
versions: +Python 3.3 -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Cal Leeming]

Cal Leeming cal.leem...@simplicitymedialtd.co.uk added the comment:

Damn 3.3 huh? Ah well, at least it's in the pipeline ^_^

Thanks for your help on this @r.david.murray!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12455] urllib2 forces title() on header names, breaking some requests

[Santoso Wijaya]

Changes by Santoso Wijaya santoso.wij...@gmail.com:


--
nosy: +santa4nt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12455
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com