[issue1401] urllib2 302 POST

[RuixiaZhang]

RuixiaZhang added the comment:

I am using  python 2.7,the same ,come to 302 lost cookies,
I try add the code :
req.headers.pop('content-length')
to urllib2.py:536
but, have another error:
KeyError: 'content-length'
So, I want to ask is there any better method to solver this error?

--
components: +Windows -XML
nosy: +paul.moore, steve.dower, tim.golden, zach.ware, zhangruixia0108
versions: +Python 2.7 -Python 2.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[John J Lee]

John J Lee <[EMAIL PROTECTED]> added the comment:

I think this was actually not a bug, and the fix should not have been
applied.  I guess this comment is just "for the record", as the fix is
probably cruft that can't be removed now, since people will have started
relying on it.

The only way that the Content-Length header could be in req.headers in
the first place is if the user had explicitly requested that it be added
(urllib2 adds that header to .unredirected_hdrs, but not to .headers). 
The user can no doubt provoke all kinds of other errors by adding random
HTTP headers -- what makes this particular one special?  .add_header()
has always been a "you need to know what you're doing, or you'll break
stuff" interface, and it's not really possible to "fix" that (at least
not without changing the meaning of .add_header()).

--
nosy: +jjlee

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Facundo Batista]

Facundo Batista added the comment:

Fixed in r60648, in the trunk.

Now the content-length and content-type headers are removed from from
the headers in the redirection.

Thank you all!

--
resolution:  -> fixed
status: open -> closed

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Antoine Pitrou]

Antoine Pitrou added the comment:

Content-Type is probably meaningless but harmless as well. I'd say the
priority is in getting rid of headers whose misinterpretation can be
annoying, such as Content-Length.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Facundo Batista]

Facundo Batista added the comment:

So, the general agreement is that:

- After receiven the 302, it's ok to generate a GET request.

- There's a problem with the generated GET request (there should not be
a Content-Length field in the headers)

Right?

If this is ok, I'll fix it. But, what other fields should disappear?
What about Content-Type, for example?

Thank you!

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Johann Tonsing]

Changes by Johann Tonsing:


--
nosy: +jtonsing

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Antoine Pitrou]

Antoine Pitrou added the comment:

Hi,

Sending a 302 in response to a POST is a very common practice so that
the browser is redirected to a "normal", non state-changing page after
the POST request has been processed. It is useful in that it allows the
user to reload the resulting page (fetched with GET) without it popping
up a warning dialog, and without re-submitting the request if the user
validates the dialog.

So, a 302 response after a POST should generate a GET to the new URL.
And, of course, without a Content-Length (there's no content in a GET so
it can't have a length, does it?).

--
nosy: +pitrou

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Jim Jewett]

Jim Jewett added the comment:

> But you said that #2 solution was more RFC compliant... 
> Could you please quote the RFC part that describes this behaviour?

RFD2616
http://www.faqs.org/rfcs/rfc2616.html

section 4.3 Message Body ...

   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests.

[I couldn't actually find a quote saying that GET has no body, but ... it 
doesn't.]

Section 10.3 Redirection 3xx says 

   The action
   required MAY be carried out by the user agent without interaction
   with the user if and only if the method used in the second request is
   GET or HEAD.

In other words, changing it to GET may not be quite pure, but leaving it as 
POST would technically mean that the user MUST confirm that the redirect is 
OK.  This MUST NOT becomes more explicit later, such as in 10.3.2 (301 Moved 
Permanently).  Section 10.3.3 (302 Found) says that 307 was added 
specifically to insist on keeping it a POST, and even 307 says it MUST NOT 
automatically redirect unless it can be confirmed by the user.

Which is why user agents change redirects to a GET and try that...

--
components: +XML -None
nosy: +jimjjewett

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Andres Riancho]

Andres Riancho added the comment:

As I said in my original bug report, if you don't remove the
content-length header or add the data, you are sending an invalid request:

START Request=
GET http://f00/1.php HTTP/1.1
Content-length: 63
Accept-encoding: identity
Accept: */*
User-agent: w3af.sourceforge.net
Host: f00
Content-type: application/x-www-form-urlencoded


 END REQUEST ===

I opened this bug report because this is a bug, not because i'm a RFC
purist. I I have time, I'll test the urllib2 patch I coded yesterday and
submit it for revision.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Senthil]

Senthil added the comment:

Hello Andres,
I think we are mixing up 2 or 3 things. Here are my comments.

1) urllib2 POST issue.
- Discussion broke on this. No conclusion.

2) GET Request.

>> If we create a GET request (handling 302 as 303) we should
>> remove the content length header!

I fail to find a reference for this statement in the RFC.

Currently urllib2, gets the headers from the original request and passes
the same headers to the redirected url. RFC mentions that we "should not
change the headers". I quoted the section previously.

3) Handling 30* headers the same way.
Yes, urllib2 handles them in the way stating most of the clients
apparently handle it the same way. Most of us are okay with it.
("Practicality beats Purity"). But when we find that something can be
improved, we just to add/extend the same. I am working on urilib
(sandbox/trunk/urilib) which has some extensions like cached redirection
for temporary redirections etc.

And for this issue, we just have to find the supporting evidence for:

>> If we create a GET request (handling 302 as 303) we should
>> remove the content length header!

If we don't find it,then it is not a bug.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Andres Riancho]

Andres Riancho added the comment:

According to the RFC:

If urllib2 gets a 302 in response to a request, it MUST send the *same*
request to the URI specified in the Location header, without modifying
the method, headers, or any data (urllib2 is not RFC compliant here)

In urllib2, a 301 and a 307 message should be handled just like a 302.

If urllib2 gets a 303 in response to a request, it MUST send a GET
request to the URI specified in the Location header (urllib2 is "half"
RFC compliant here, this only works if the original request WASN'T a
POST request)

I will code a complete patch to make urllib2 work as RFC Compliant as
possible. Whenever it's ready i'll send it.

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Facundo Batista]

Facundo Batista added the comment:

So, for 302 error we should resend the request as POST (header with
lenght and data), and for the others we need to keep current behaviour.

Actually, we just need to code a new handling function for 302, and
leave the existing one as is.

What do you think?

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Andres Riancho]

Andres Riancho added the comment:

As mentioned in the RFC, and quoted by orsenthil, "however, most
existing user agent implementations treat 302 as if it were a 303
response", which is true for urllib2.py too ( see line 585 ):

http_error_301 = http_error_303 = http_error_307 = http_error_302

Which means: "all redirections are treated the same way". So, if we want
to strictly respect the RFC, we should implement 4 different methods:

   - http_error_301
   - http_error_303
   - http_error_307
   - http_error_302

But urllib2 is implementing the RFC "the easy way", this is: "threat all
redirections the same, threat them as 303". 303 redirections perform a
GET on the URI, which urllib2 does here:

return Request(newurl,
   headers=req.headers,
   origin_req_host=req.get_origin_req_host(),
   unverifiable=True)

These line does not clone the old request completely, it creates a GET
request. If we create a GET request (handling 302 as 303) we should
remove the content length header!

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Senthil]

Senthil added the comment:

I agree with facundobatista here.
1) I am not sure, if what you mention that doing POST in the 302
redirect url does not carry forward the POST data. Some examples would
help to verify that and if this is case, IMO its a bug in urllib2.
2) You mention solution 1 as RFC compliant? Does RFC mention about
changes to request headers for a redirection? Please quote the relevant
section
also I find in RFC2616 under 302 Found that:
  Note: RFC 1945 and RFC 2068 specify that the client is not
allowedto change the method on the redirected request. However, most   
   existing user agent implementations treat 302 as if it were a 303   
   response, performing a GET on the Location field-value regardless   
   of the original request method.
But could not find any references for behaviour with respect to POST on
redirection.

--
nosy: +orsenthil

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Facundo Batista]

Facundo Batista added the comment:

I don't understand why after receiving a redirection, and going to a new
URL, you say to NOT send the POST data. Shouldn't the HTTP request be
exactly like the original one, but to another URL destination?

But you said that #2 solution was more RFC compliant... Could you please
quote the RFC part that describes this behaviour?

--
nosy: +facundobatista

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1401] urllib2 302 POST

[Andres Riancho]

Changes by Andres Riancho:


--
title: urllib 302 POST -> urllib2 302 POST

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com