so quick question - what's the win compared to multiple processes?
On Mon, Jun 20, 2016 at 8:51 AM, Omer Katz <omer.d...@gmail.com> wrote: > Hi all, > There was an experiment based on CPython's code called PyParallel that > allows running threads in parallel without STM and modifying source code of > both Python and C extensions. The only limitation is that they disallow > mutation of global state in parallel context. > I briefly mentioned it before on PyPy's freenode channel. > I'd like to discuss why the approach is useful, how it can benefit PyPy > users and how can it be implemented. > Allowing to run in parallel without mutating global state can help servers > use each thread to handle a request. It can also allow to log in parallel or > send an HTTP request (or an AMQP message) without sharing the response with > the main thread. This is useful in some cases and since PyParallel managed > to keep the same semantics it (shouldn't) break CPyExt. > If we keep to the following rules: > > No global state mutation is allowed > No new keywords or code modifications required > No CPyExt code is allowed (for now) > > I believe that users can somewhat benefit from this implementation if done > correctly. > As for implementation, if we can trace the code running in the thread and > ensure it's not mutating global state and that CPyExt is never used during > the thread's course we can simply release the GIL when such a thread is run. > That requires less knowledge than using STM and less code modifications. > However I think that attempting to do so will introduce the same issue with > caching traces (Armin am I correct here?). > > As for CPyExt, we could copy the same code modifications that PyParallels > did but I suspect that it will be so slow that the benefit of running in > parallel will be completely lost for all cases but very long threads. > > Is what I'm suggesting even possible? How challenging will it be? > > Thanks, > Omer Katz. > > _______________________________________________ > pypy-dev mailing list > pypy-dev@python.org > https://mail.python.org/mailman/listinfo/pypy-dev > _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
