On Mon, Nov 18, 2013 at 4:30 PM, Nick Coghlan <ncogh...@gmail.com> wrote:
> On 19 November 2013 09:57, Antoine Pitrou <solip...@pitrou.net> wrote: > > On Mon, 18 Nov 2013 16:44:59 -0600 > > Tim Peters <tim.pet...@gmail.com> wrote: > >> [Tim] > >> >> But it has a different kind of advantage: PREFETCH was optional. As > >> >> Guido said, it's annoying to bloat the size of small pickles (which > >> >> may, although individually small, occur in great numbers) by 8 bytes > >> >> each. There's really no point to framing small chunks of data, > right? > >> > >> [Antoine] > >> > You can't know how much space the pickle will take until the pickling > >> > ends, though, which makes it difficult to decide whether you want to > >> > emit a PREFETCH opcode or not. > >> > >> Ah, of course. Presumably the outgoing pickle stream is first stored > >> in some memory buffer, right? If pickling completes before the buffer > >> is first flushed, then you know exactly how large the entire pickle > >> is. If "it's small" (say, < 100 bytes), don't write out the PREFETCH > >> part. Else do. > > > > Yet another possibility: keep framing but use a variable-length > > encoding for the frame size: > > > > - first byte: bits 7-5: N (= frame size bytes length - 1) > > - first byte: bits 4-0: first 5 bits of frame size > > - remaning N bytes: remaining bits of frame size > > > > With this scheme, very small pickles have a one byte overhead; small > > ones a two byte overhead; and the max frame size is 2**61 rather than > > 2**64, which should still be sufficient. > > > > And the frame size is read using either one or two read() calls, which > > is efficient. > > And it's only a minimal change from the current patch. Sounds good to me. > Food for thought: maybe we should have variable-encoding lengths for all opcodes, rather than the current cumbersome scheme? -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com