Robin skrev: > Ah, I hadn't realised it was an OS constraint - I thought it was > possible to unload dlls - and that was why matlab provides the clear > function. mex automatically clears a function when you rebuild it - I > thought that was how you can rebuild and reload mex functions without > restarting matlab. It is OS dependent if DLLs can be unloaded or not. In Windows you can call CloseHandle on the DLL handle and it's gone. If you can unload a MEX function you can also unload Python or NumPy.
But when you unload the MEX function, the DLLs loaded by the MEX are not automatically cleared. There is no garbage collector. In Windows, a load and an unload will result in calls to a function called "DllMain" in the DLL. And if the DLL has other DLLs to load or clear, that is where you need to place it. You can put a custom DllMain function in a MEX file. But be careful: a DLL is only loaded once, i.e. DLLs are imported as singletons in the process. If two MEX functions load Python26.dll, they get handles to the same instance. If you unload Python26.dll in one MEX, it is unloaded globally from the process, so the other MEX will crash. There is no reference counting by the OS kernel. The solution to this type of DLL Hell in Windows is COM. A COM object is a DLL but not a singleton. You can have multiple instances of the save COM object in one process. On Mac I guess your options are to either statically link everything to the MEX file, or find a way for multiple MEX files to share Python interpreter, e.g. implement some sort of reference counting scheme, which by the way is what COM does on Windows. > I really want a cross platform solution so that rules out COM. CORBA or XMLRPC seem to be the standards. I'm not sure I would use either. > Do you think I'm likely to run into more problems? I got the feeling > from asking questions on IRC that embedding Python is kind of > discouraged but I thought in this case it seemed the neatest way. > > It is discouraged because you get a singleton. You can create subinterpreters (cf. Apache's mod_python), but that will bring problems of it's own. For example you cannot use extensions that require the simplified GIL API (e.g. ctypes). I think this is a major design flaw of CPython. For example with Java's JNI, you get a context pointer to the VM, so you don't have any of these problems. But with CPython, both the interpreter and extensions are basically implemented to be loaded as singletons. > I was aware of this - I thought I would be OK on the mac - at least > Python and Numpy and my mex function are built with apple gcc although > I'm not sure about Matlab. I guess Windows will be more difficult... > But in any case I don't plan to pass any file handles around. > > It applies to any CRT resource, not just files. Compiler is not important, but which CRT is loaded. And if you statically link the same CRT twice, that becomes two CRTs that cannot share resources. In Windows, Microsoft has made sure there are multiple versions of their CRT (msvcrt.dll, msvcr71.dll, msvcr80.dll, msvcr90.dll, ...) that cannot share resources. And anyone not careful with this can experice crashes at random locations. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion