New submission from Benno Leslie: he http.client HTTPConnection._send_output method has an optimization for avoiding bad interactions between delayed-ack and the Nagle algorithm:
http://hg.python.org/cpython/file/f32f67d26035/Lib/http/client.py#l884 Unfortunately this interacts rather poorly if the case where the message_body is a bytes instance and is rather large. If the message_body is bytes it is appended to the headers, which causes a copy of the data. When message_body is large this duplication of data can cause a significant spike in memory usage. (In my particular case I was uploading a 200MB file to 30 hosts at the same leading to memory spikes over 6GB. [There is a short thread discussing this issue on python-dev; Subject: "http.client Nagle/delayed-ack optimization"; Date: Dec 15, 2012] ---------- components: Library (Lib) files: http_opt.diff keywords: patch messages: 178728 nosy: bennoleslie priority: normal severity: normal status: open title: httpc.lient delayed ack / Nagle algorithm optimisation performs badly for large messages type: performance versions: Python 3.3 Added file: http://bugs.python.org/file28518/http_opt.diff _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue16833> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com