New submission from Benno Leslie:

he http.client HTTPConnection._send_output method has an optimization for 
avoiding bad interactions between delayed-ack and the Nagle algorithm:

http://hg.python.org/cpython/file/f32f67d26035/Lib/http/client.py#l884

Unfortunately this interacts rather poorly if the case where the message_body 
is a bytes instance and is rather large.

If the message_body is bytes it is appended to the headers, which causes a copy 
of the data. When message_body is large this duplication of data can cause a 
significant spike in memory usage.

(In my particular case I was uploading a 200MB file to 30 hosts at the same 
leading to memory spikes over 6GB.

[There is a short thread discussing this issue on python-dev; Subject: 
"http.client Nagle/delayed-ack optimization"; Date: Dec 15, 2012]

----------
components: Library (Lib)
files: http_opt.diff
keywords: patch
messages: 178728
nosy: bennoleslie
priority: normal
severity: normal
status: open
title: httpc.lient delayed ack / Nagle algorithm optimisation performs badly 
for large messages
type: performance
versions: Python 3.3
Added file: http://bugs.python.org/file28518/http_opt.diff

_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue16833>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to