Yury Selivanov <yseliva...@gmail.com> added the comment: > 1. What happens if size of read data is greater than pre-allocated buffer?
Let's say we have 2Kb of data in the socket's network buffer, and we only preallocated 0.5Kb in our buffer. We will the receive 0.5Kb in our buffer, 'Protocol.buffer_updated()' will be called with nbytes=512, and 1.5Kb of data will be left in the network buffer. So the loop will call get_buffer()/buffer_updated() again, and the cycle will continue until there's no data left. > 2. Is flow control logic changed or not? If I understand correctly > pause_reading() / resume_reading() continue to work as earlier. Flow control will continue working absolutely the same for BufferedProtocols. > I have another question: what happens if there is a partial read? > For example, let's says I return a 1024-bytes buffer in get_buffer(), but > recv_into() receives data in 512 chunks. Is it: It will be as follows: 1. Protocol.get_buffer() is called, returns 1024 bytes buffer 2. recv_into() receives 512 bytes, writes them in buf[0:512] 3. Protocol.buffer_updated() is called with nbytes=512 Now it's the responsibility of the Protocol to return a correct view over buffer the next time `get_buffer()` is called. The general idea is to: 1. allocate a big buffer 2. keep track of how much data we have in that buffer, let's say we have a 'length' integer for that. 3. when get_buffer() is called, return 'memoryview(big_buffer)[length:]' 4. when buffer_updated(nbytes) is called, do 'length += nbytes; parse_buffer_if_possible()' I've implemented precisely this approach here: https://gist.github.com/1st1/1c606e5b83ef0e9c41faf21564d75ad7#file-get_buffer_bench-py-L27-L43 ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32251> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
