On Mon, Apr 4, 2011 at 1:20 PM, John Ladasky <lada...@my-deja.com> wrote:
> When should one pickle and copy? When to implement an object in > shared memory? Why is pickling apparently such a non-trivial process > anyway? And, given that multi-core CPU's are apparently here to stay, > should it be so difficult to make use of > them?<http://mail.python.org/mailman/listinfo/python-list> Pickle and copy when your distinct processes will benefit from having multiple distinct copies of the data - that is, copies that can diverge from each other if written to in one or more of the processes. Use shared memory if your distinct processes need to be able to see a single copy of the same data - EG, because one needs to see the result of the changes made in another. Pickling's not a big deal - it's just turning structured data (from an int to a nested series of objects) into a stream of bytes. Not all extension types support it without extra work though. Yes, multicore is increasingly important, and Python needs to support it well. multiprocessing is one good way of doing so. Also, Python 3.2 has a new module facilitating this, and an improved GIL situation. Then there are things like greenlets and stackless python. Oh, and if you move to jython or ironpython, you get better multithreading. Multithreading is generally a bit lighter-weight than multiprocessing, but it also gives a little tighter coupling between different parts of the program and a greater probability of race conditions.
-- http://mail.python.org/mailman/listinfo/python-list
