Re: Python Multiprocessing in Softimage
Even without considering all the good points brought up insofar, the GIL in Python (CPy at least) makes multithreading live, in general, somewhere between downright impossible and not worth trying. Depending on the process your only option might be to manage the threading yourself inside a cpp op, or you might have no options at all if you too frequently have to halt and listen to something (going between Soft's own callbacks, event loop and so on). ICE, on the other hand, is a bit blackboxed but ultimately very comfortable and efficient at threading in some scenarios. So if you want to parallelize simply because you work on a very large set of points or something like that, then writing it in ICE form (as in an ICE node) would most likely be both the easiest and the safest way. -- Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are!
Re: Python Multiprocessing in Softimage
Hey Vincent, multithreading is different of multiprocessing... short long story, multithreading is about running many threads in the same memory space (so shared memory and so use of patterns such as mutex, semaphore,...) while multiprocessing is about running different external processes (all having their own memory space)... It becomes tedious when you want to share data then across those different processes as they don't share a common memory space, Python designers achieve this by simply pickling the datas you want to share (and so you have to be sure they are pickable) and saving them somewhere in memory or on disk (where processes known where to look for). multiprocessing is a powerful concept equivalent in theory to pipes or even co-routines. As Matt said, the sdk is not thread-safe, so forgetting about multithreading is a good rule to follow, while I wouldnt even think about multiprocessing to access scene graph components. To boost your operator, review and be sure your code is optimized (thats just all what it takes 90% of the cases) ! if this is still too slow, move to Cpp... --jon 2013/5/30 Matt Lind > Outside of ICE, the SDK is not thread safe. > > Matt > > > -Original Message- > From: softimage-boun...@listproc.autodesk.com [mailto: > softimage-boun...@listproc.autodesk.com] On Behalf Of Vincent Ullmann > Sent: Thursday, May 30, 2013 3:43 PM > To: softimage@listproc.autodesk.com > Subject: Re: Python Multiprocessing in Softimage > > Hmm... ok, thanks > > Is it possible to write a multiThreaded CustomOperator in C++ ? > I already build a simpler version of my ScriptedOp in C, so i might "just" > make it multiThreaded ;-) But first it would be nice to know if > Softimage could handle this > > > Am 31.05.2013 00:36, schrieb Matt Lind: > > Outside of ICE, the SDK is not thread safe. > > > > Matt > > >
RE: Python Multiprocessing in Softimage
Outside of ICE, the SDK is not thread safe. Matt -Original Message- From: softimage-boun...@listproc.autodesk.com [mailto:softimage-boun...@listproc.autodesk.com] On Behalf Of Vincent Ullmann Sent: Thursday, May 30, 2013 3:43 PM To: softimage@listproc.autodesk.com Subject: Re: Python Multiprocessing in Softimage Hmm... ok, thanks Is it possible to write a multiThreaded CustomOperator in C++ ? I already build a simpler version of my ScriptedOp in C, so i might "just" make it multiThreaded ;-) But first it would be nice to know if Softimage could handle this Am 31.05.2013 00:36, schrieb Matt Lind: > Outside of ICE, the SDK is not thread safe. > > Matt
Re: Python Multiprocessing in Softimage
Hmm... ok, thanks Is it possible to write a multiThreaded CustomOperator in C++ ? I already build a simpler version of my ScriptedOp in C, so i might "just" make it multiThreaded ;-) But first it would be nice to know if Softimage could handle this Am 31.05.2013 00:36, schrieb Matt Lind: Outside of ICE, the SDK is not thread safe. Matt
RE: Python Multiprocessing in Softimage
Outside of ICE, the SDK is not thread safe. Matt -Original Message- From: softimage-boun...@listproc.autodesk.com [mailto:softimage-boun...@listproc.autodesk.com] On Behalf Of Vincent Ullmann Sent: Thursday, May 30, 2013 3:34 PM To: softimage@listproc.autodesk.com Subject: Python Multiprocessing in Softimage Hi List, i tried to optimize performance of a scripted operator i wrote, by adding multithread-support. So i got a Test-Script working fine in Sublime-Text, but as soon as i try to run is from inside Softimage's ScriptEditor or in a ScriptedOperator it breaks. It gives me this error: # PicklingError: Can't pickle : it's not found as __ax_main__.Worker Cant find any results in Google, but maybe someone here know how to do it PS: No, in this case i cant go to ICE and do my stuff there. ;-)
