On Wed, Oct 29, 2008 at 8:05 PM, Glenn Linderman <[EMAIL PROTECTED]> wrote: > On approximately 10/29/2008 3:45 PM, came the following characters from the > keyboard of Patrick Stinson: >> >> If you are dealing with "lots" of data like in video or sound editing, >> you would just keep the data in shared memory and send the reference >> over IPC to the worker process. Otherwise, if you marshal and send you >> are looking at a temporary doubling of the memory footprint of your >> app because the data will be copied, and marshaling overhead. > > Right. Sounds, and is, easy, if the data is all directly allocated by the > application. But when pieces are allocated by 3rd party libraries, that use > the C-runtime allocator directly, then it becomes more difficult to keep > everything in shared memory. > > One _could_ replace the C-runtime allocator, I suppose, but that could have > some adverse effects on other code, that doesn't need its data to be in > shared memory. So it is somewhat between a rock and a hard place. > > By avoiding shared memory, such problems are sidestepped... until you run > smack into the GIL.
If you do not have shared memory: You don't need threads, ergo: You don't get penalized by the GIL. Threads are only useful when you need to have that requirement of large in-memory data structures shared and modified by a pool of workers. -jesse -- http://mail.python.org/mailman/listinfo/python-list
