[issue41345] Remote end closed connection without response

[Martin Panter]

Martin Panter  added the comment:

Previous report about Requests to the Python bug tracker: Issue 33620.

I suspect this is an unavoidable race condition with trying a POST (or other 
non-idempotent) request on an idle HTTP connection. I think it has to be up to 
the higher-level application or user to decide if it is safe to retry a POST 
request. Otherwise you risk e.g. accidentally ordering two pizzas because the 
server received two POST requests but something interfered with the response of 
the first response. On the other hand, I noticed some browsers seem to 
automatically retry a POST once if it is interrupted, which makes me uneasy.

A concrete example of the problem is a firmware upload that triggers a reboot. 
If the reboot is too quick and prevents the POST response being sent, I found 
that a web browser will repeat the firmware upload once more after my firmware 
boots up again.

If it is not safe to retry the POST request, other options would be to avoid 
the server thinking the connection is stale:

* always do the POST request on a fresh HTTP connection
* "ping" the connection with a dummy request (e.g. OPTIONS, HEAD, or GET) 
immediately before the POST request

Another option that comes to mind is to try using the 100 Continue mechanism, 
but this is not a general solution and depends on the application.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Michal Arbet]

Michal Arbet  added the comment:

Hello,

Thank you very much for help, I created pull request for urllib3 ->
https://github.com/urllib3/urllib3/pull/1911
Can u just confirm that this could be fixed in urllib3 as it is in pull
request ? Have I fixed it by good way ?

Thanks.
Michal Arbet
( kevko )

pá 24. 7. 2020 v 20:29 odesílatel Guido van Rossum 
napsal:

>
> Guido van Rossum  added the comment:
>
> This surely is some application corner case. Closing.
>
> --
> nosy: +gvanrossum
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Guido van Rossum]

Guido van Rossum  added the comment:

This surely is some application corner case. Closing.

--
nosy: +gvanrossum
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Ilia Androshchuk]

Ilia Androshchuk  added the comment:

Hi Michal,
I found some information in the package documentation
Please check this page
https://requests.readthedocs.io/en/master/user/advanced/#keep-alive

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Ilia Androshchuk]

Ilia Androshchuk  added the comment:

Hi, Michal
It took a lot of time, but I achieved exactly the same behavior.

My Traceback :

DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 3138
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in 
urlopen
httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in 
_make_request
six.raise_from(e, None)
  File "", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in 
_make_request
httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse
response.begin()
  File "/usr/lib/python3.8/http/client.py", line 303, in begin
version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 272, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in 
urlopen
retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in 
increment
raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise
raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in 
urlopen
httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in 
_make_request
six.raise_from(e, None)
  File "", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in 
_make_request
httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse
response.begin()
  File "/usr/lib/python3.8/http/client.py", line 303, in begin
version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 272, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', 
RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test1.py", line 14, in 
s.post('http://localhost', data={counter:txt})
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in 
request
resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', 
RemoteDisconnected('Remote end closed connection without response'))

I'm using same dist and settings:

lerfi@UbuntuVM:~/work/psf/41345$ uname -a
Linux UbuntuVM 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 
x86_64 x86_64 x86_64 GNU/Linux
erfi@UbuntuVM:~/work/psf/41345$ dpkg -l | grep apache
ii  apache22.4.41-4ubuntu3  
 amd64Apache HTTP Server
ii  apache2-bin2.4.41-4ubuntu3  
 amd64Apache HTTP Server (modules and other binary files)
ii  apache2-data   2.4.41-4ubuntu3  
 all  Apache HTTP Server (common files)
ii  apache2-utils  2.4.41-4ubuntu3  
 amd64Apache HTTP Server (utility programs for web servers)
pip3 requests==2.22.0

it looks like there's a problem with this urllib3 package

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Michal Arbet]

Michal Arbet  added the comment:

I'm using  Ubuntu 20.04 with dist apache2 and default settings :

michalarbet@pixla:~/work$ dpkg -l | grep apache
ii  apache22.4.41-4ubuntu3
  amd64Apache HTTP Server
ii  apache2-bin2.4.41-4ubuntu3
  amd64Apache HTTP Server (modules and other binary files)
ii  apache2-data   2.4.41-4ubuntu3
  all  Apache HTTP Server (common files)
ii  apache2-utils  2.4.41-4ubuntu3
  amd64Apache HTTP Server (utility programs for web servers)
ii  libapache2-mod-php 2:7.4+75
 all  server-side, HTML-embedded scripting language (Apache
2 module) (default)
ii  libapache2-mod-php7.4  7.4.3-4ubuntu2.2
 amd64server-side, HTML-embedded scripting language (Apache
2 module)

michalarbet@pixla:~/work$ uname -a
Linux pixla 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux

When reproducing issue, be patient , you have to wait several iterations
before it will fail.
Some bug reports and discussion related to :

https://bugs.python.org/issue41345
https://github.com/psf/requests/issues/4664
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=899406

po 20. 7. 2020 v 14:54 odesílatel Ilia Androshchuk 
napsal:

>
> Ilia Androshchuk  added the comment:
>
> Hi Michal,
> Which version of operating system and apache are you running?
> I'll try to reproduce
>
> --
> nosy: +an7e
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Ilia Androshchuk]

Ilia Androshchuk  added the comment:

Hi Michal,
Which version of operating system and apache are you running?
I'll try to reproduce

--
nosy: +an7e

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41345] Remote end closed connection without response

[Michal Arbet]

New submission from Michal Arbet :

Hi,

I'm not sure if this is really python bug, but I hope that you can check the 
issue.

Issue is that from time to time i'm getting exception from python when sending 
request to server which has http keepalive option turned on.

Requests send a request but in few miliseconds apache2 server is closing 
persistent connection by sending FIN packet which generate traceback.

I can reproduce it by following simple script.

#!/usr/bin/python3

import requests
from time import sleep

import logging
logging.basicConfig(level=logging.DEBUG)

s = requests.Session()
s.verify = False  # self-signed cert

counter = 0
txt = "test"
while True:
counter = counter + 1
s.post('http://localhost', data={counter:txt})
sleep(5)

Everything is working fine, but from time to time I get following traceback. 
When FIN is received right after request was sent.

michalarbet@pixla:~/work$ ./request_test.py 
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:80
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost
DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost