I can't comment at all on how the OIIO python bindings specifically behave in a threaded usage, but from a general standpoint AFAIK the library would have to be dropping the GIL when it can, in order for calls to be able to yield to other threads. That would mean that under the condition that it wasn't dropping the GIL, your multi-threaded calls into oiio would end up blocking on each other and you would end up with a semi-serial result.
The reason Python threading works better with IO-bound calls is that it can release the GIL and let other threads run for a time slice. I added similar GIL-dropping to one of my Cython bindings to a C++ library so that multiple threaded calls into the library could execute concurrently. Do you have 24 physical cores in your box? Or are those hyper-threaded logical cores (12 physical cores)? I would think, if using the python multiprocessing approach, you would want to do something like a 12 process pool, and let each task have 2 thread? Again, this was totally a generic python response :-) On Mon, Sep 1, 2014 at 12:35 PM, Dave Lajoie < [email protected]> wrote: > Hello Guys! > > I have been searching the archive and I could not find anything on this. > Basically I am using open image IO as slate generation tool (among other > things of course). It works great! I am looking at speeding up the process, > so I have attempted to call my python function using various threading > modules, and I am not getting a lot of speed up. I am using nthread=0 to > ensure the call uses all available threads/cores. > > I do know that Python GIL is well known to be great at certain type of > multi-threading calls, especially anything IO bound offers best > performance. However I was wondering if there are any know issues with > python multithreading, and/or if there was any list of do's and don'ts. > > Currently the only way I could get all the cores to be utilized to 100% > was to spawn a separate python process, which will process a single frame, > so on my box I would have 24 concurrent python subprocesses invoking OIIO. > The machine grinds almost to halt ( which is good in this case), but I was > wondering what are the other things I don't know regarding this subject. :) > > Any help would be appreciated. > Best > Dave Lajoie > R&D Director Digital District Montreal > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
