[issue5038] urrlib2/httplib doesn't reset file position between requests

[Irit Katriel]

Change by Irit Katriel :


--
components: +Documentation
stage: patch review -> needs patch
versions: +Python 3.10, Python 3.9 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[nr]

nr  added the comment:

the pull request now passed the build checks, please review the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[nr]

nr  added the comment:

I will fix the build errors first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[nr]

nr  added the comment:

I added a new pull request.
Martin, you are right I realized when looking through the code that just 
setting the file pointer to zero inside http lib might interfere with requests 
that don't have authentication enabled.

The new pull requests does number 2.) of your suggestion for both Basic and 
Digest authentication.

Can you please review the code? Thank you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[nr]

Change by nr :


--
pull_requests: +11930

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Martin Panter]

Martin Panter  added the comment:

For 3.7+ (where iterable objects are supported), I suggest:

1. Document the problem as a limitation of handlers like 
AbstractBasicAuthHandler, and consider raising an exception instead of trying 
to upload a file or iterable a second time.

2. Clarify the behaviour for different types of the “urllib.request” data 
parameter. I understand “file-like objects” means objects with a “read” 
attribute, and the “read” method is called in preference to iteration or 
treating the parameter as a “bytes” object.

Despite the bug title, I don’t think the library should mess with the file 
position. Certainly not when making a single request. But it should already be 
possible for the caller to supply a custom iterable object that resets the file 
position:

class FileReiterator:
def __iter__(self):
self.file.seek(0)
while True:
chunk = self.file.read(self.chunksize)
yield chunk
if len(chunk) < self.chunksize:
break

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[nr]

nr  added the comment:

PR 11843 should fix the issue in master, I didn't check python 2.6 or prior 
versions. The problem is that in the first request sent to HTTP service the 
POST data is sent correctly. After that the HTTP server responds with 401 and 
the request is resent but the mmap file pointer is pointing now to the end of 
the file because it has been fully read in the requests before. The PR just 
seeks to the beginning of the file after the file has been read and sends the 
request with auth credentials including POST body.

--
nosy: +nr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +11873
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[David Heiberg]

Change by David Heiberg :


--
nosy: +dheiberg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Martin Panter]

Martin Panter  added the comment:

Here is a demonstration script in case it helps. I haven’t tested it with 
versions before Python 2.6.

Older versions send “Content-Length: 11”, but leave the server hanging trying 
to read the data. Newer versions (I presume since Issue 12319, 3.6+) send a 
valid HTTP 1.1 chunked request, but with empty data.

--
Added file: https://bugs.python.org/file47780/auth-mmap.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Lorenz Mende]

Lorenz Mende  added the comment:

Issue shall be closed, as no reproduction code is provided.
No patch provided and no comments since 2015.

--
nosy: +LorenzMende

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Martin Panter]

Martin Panter added the comment:

Actually, I do not think any “Expect: 100-continue” headers are explicitly sent 
by the Python standard library. The Python client does not support waiting for 
a “100 Continue” response; see Issue 1346874.

There is Issue 23740 opened about fixing or clarifying the various data types 
accepted by “http.client”.

On the other hand, the documentation for urlopen() says only bytes and 
iterables are supported. If mmap objects are being treated as file objects by 
urlopen() that is unexpected, and the documentation or implementation needs 
fixing there. Also, iterating a mmap() object is different from iterating 
either the equivalent bytearray() or file object, so there is something weird 
going on there.

--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Anthony Kong]

Changes by Anthony Kong anthony.hw.k...@gmail.com:


--
nosy: +Anthony.Kong

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Senthil Kumaran]

Senthil Kumaran added the comment:

I think, this requires triaging in terms of is the feature request still 
applicable. Except 100 is sent by httplib and the support for this was added 
few years ago, much later then this bug was originally raised.

--
assignee:  - orsenthil

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Daniel Diniz]

Changes by Daniel Diniz aja...@gmail.com:


--
keywords: +easy
priority:  - normal

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Daniel Diniz]

Changes by Daniel Diniz aja...@gmail.com:


--
nosy: +ajaksu2, orsenthil
stage:  - test needed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Daniel Diniz]

Changes by Daniel Diniz aja...@gmail.com:


--
nosy: +jjlee

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[jan matejek]

New submission from jan matejek jmate...@suse.cz:

since 2.6 httplib supports reading from file-like objects.

Now consider the following situation:
There are two handlers in urrlib2, first is plain http, second is basic
auth.
I want to POST a file to a service, and pass the open file object as
data parameter to urllib2.urlopen.
First handler is invoked, it sends the file data, but gets 401
Unauthorized return code and fails with that.
Second handler in chain is invoked (at least that's how i understand
urrlib2, please correct me if i'm talking rubbish). At that point the
open file is at EOF, so empty data is sent.

furthermore, the obvious solution you can't do this through urllib so
go read the file yourself doesn't apply that well - the file object in
question is actually a mmap.mmap instance.
This code is in production since python 2.4. Until file object support
in httplib was introduced, it worked fine, handling the mmap'ed file as
a string. Now it is picked up as read()-able and this problem occurs.
Only workaround to restore pre-2.6 behavior that comes to mind is
building a wrapper class for the mmap object that hides its read() method.

--
components: Library (Lib)
messages: 80419
nosy: matejcik
severity: normal
status: open
title: urrlib2/httplib doesn't reset file position between requests
type: behavior
versions: Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5038] urrlib2/httplib doesn't reset file position between requests

[Gabriel Genellina]

Gabriel Genellina gagsl-...@yahoo.com.ar added the comment:

This happens in other implementations too, not just urllib2.

If the server supports it, the best way is to send an 'Expect: 100-
Continue' header field before attempting to send the actual file.

--
nosy: +gagenellina

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5038
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com