[issue43332] Make http.client._tunnel send one byte string over the network
Zveinn added the comment: No problem, Hopefully this will improve the performance on some network devices and proxy services. -- ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] Make http.client._tunnel send one byte string over the network
Zveinn added the comment: Hey! First of all, thank you for not shitting all over me <3 I have never really used python and the only reason I found this is because I was developing a tool that accepted a LOT of CONNECT requests for python and I just happened to stumble upon this little nugget. Regarding a PR or a local path, it would have been too much work for me since I'm already swamped and I don't know anything about python or it's ecosystem :S Anyways, I leave this in your hands now. Regards, Zveinn -- ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] http/client.py: - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end
Zveinn added the comment: also found this: https://dynatrace.github.io/OneAgent-SDK-for-Python/docs/encoding.html It might be relevant ? -- ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] http/client.py: - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end
Zveinn added the comment:
def _tunnel(self):
connect_str = "CONNECT %s:%d HTTP/1.0\r\n" % (self._tunnel_host,
self._tunnel_port)
connect_bytes = connect_str.encode("ascii") <
<https://bugs.python.org/issue43332>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] http/client.py: - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end
Zveinn added the comment: P.s. Sorry for the formatting of the previous message, I´m new :S -- ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] http/client.py: - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end
Change by Zveinn : -- title: def _tunnel(self): - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end -> http/client.py: - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] def _tunnel(self): - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end
Change by Zveinn : -- title: def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end -> def _tunnel(self): - uses multiple network writes, possibly causing excessive network usage and increased implementation complexity on the other end ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43332] def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end
New submission from Zveinn : Hey, some time ago I ran into some code in the cpython code I thought might be possible to improve it a little bit. https://github.com/python/cpython/blob/master/Lib/http/client.py#L903 This code specifically. Notice how the self.send() method is used multiple time to construct the CONNECT request. When the network load is high, these different parts actually get split into separate network frames. This causes additional meta data to be sent, effectively costing more bandwidth and causing the request to be split into multiple network frames. This has some interesting behavior on the receiving end as well. If you send everything as a single network frame, then the receiver can read the entire thing in a single read call. If you send multiple frames, the main reader pipe now needs a temporary buffer to encapsulate the multiple calls. Because of this, sending requests as many network frames actually causes a rise in processing complexity on the receiving end. Here is a github issue I made about this problem some time ago: https://github.com/psf/requests/issues/5384 In this issue you will find detailed information and screenshots. My recommendation would be to construct the query as a whole before using a single self.send() to send the whole payload in one network frame. Even if we ignore the added complexity on the receivers end, the gain in network performance is worth it. -- components: C API messages: 387742 nosy: zveinn priority: normal severity: normal status: open title: def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end type: performance versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43332> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
