Hi Tobias,

On Wed, Jan 18, 2017 at 5:27 PM, Tobias Oberstein <
tobias.oberst...@gmail.com> wrote:

> Are you aware of
>
> https://github.com/alex/zero_buffer
>
> ?
>
> This emulates Python strings using zero-copy read-only buffer views.
>

​yes, I saw it in the past and I considered to use it for capnpy. IIRC, I
measured that at the end of the days, the overhead of using it was larger
than simply doing string slicing, especially for short strings.
It might be useful for very large strings, however.

Depending of what you need to do, for zero copy you could also consider
returning a memoryview slice of the original underlying buffer.



> Holy grail for me (use case being IPC) would be:
>
> Python process 1 mmap's a file shared with Python process 2.
>
> Python process 1 puts a string into mmap'ed file, pointer to that is
> "somehow transferred" to process 2 (eg pushing the index into the mmap'ed
> file over Unix domain socket .. a single uint64), and Python code in
> process 2 can do stuff with this string _without_ copying - probably via
> zero_buffer.
>
>
​Have you actually measured that copying the data between processes is the
bottleneck? Using shared memory is something I tried also for a client of
mine but at the end we switched back to pass messages using network because
the extra complexity was not worth the gain. But again, I suppose it
depends on the size of the message.

​ciao,
Anto​
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Reply via email to