Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Pauli Virtanen
pe, 2009-11-13 kello 22:36 +0100, Sturla Molden kirjoitti:
[clip]
> > I was aware of this - I thought I would be OK on the mac - at least
> > Python and Numpy and my mex function are built with apple gcc although
> > I'm not sure about Matlab. I guess Windows will be more difficult...
> > But in any case I don't plan to pass any file handles around.
>
> It applies to any CRT resource, not just files. Compiler is not 
> important, but which CRT is loaded. And if you statically link the same 
> CRT twice, that becomes two CRTs that cannot share resources. In 
> Windows, Microsoft has made sure there are multiple versions of their 
> CRT (msvcrt.dll, msvcr71.dll, msvcr80.dll, msvcr90.dll, ...) that cannot 
> share resources. And anyone not careful with this can experice crashes 
> at random locations.

Well, for Matlab/Python integration meant for numerical computations I
would be surprised if CRT is really a problem. You essentially can pass
data to Matlab only via Matlab's own interface -- CRT stuff like
ownership of pointers to allocated memory, file handles etc. typically
do not cross this boundary.

-- 
Pauli Virtanen



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Sturla Molden
Robin skrev:
> Ah, I hadn't realised it was an OS constraint - I thought it was
> possible to unload dlls - and that was why matlab provides the clear
> function. mex automatically clears a function when you rebuild it - I
> thought that was how you can rebuild and reload mex functions without
> restarting matlab.
It is OS dependent if DLLs can be unloaded or not. In Windows you can 
call CloseHandle on the DLL handle and it's gone. If you can unload a 
MEX function you can also unload Python or NumPy.

But when you unload the MEX function, the DLLs loaded by the MEX are not 
automatically cleared. There is no garbage collector.

In Windows, a load and an unload will result in calls to a function 
called "DllMain" in the DLL. And if the DLL has other DLLs to load or 
clear, that is where you need to place it. You can put a custom DllMain 
function in a MEX file.

But be careful: a DLL is only loaded once, i.e. DLLs are imported as 
singletons in the process. If two MEX functions load Python26.dll, they 
get handles to the same instance. If you unload Python26.dll in one 
MEX,  it is unloaded globally from the process, so the other MEX will 
crash. There is no reference counting by the OS kernel.

The solution to this type of DLL Hell in Windows is COM. A COM object is 
a DLL but not a singleton. You can have multiple instances of the save 
COM object in one process.

On Mac I guess your options are to either statically link everything to 
the MEX file, or find a way for multiple MEX files to share Python 
interpreter, e.g. implement some sort of reference counting scheme, 
which by the way is what COM does on Windows.

> I really want a cross platform solution so that rules out COM. 
CORBA or XMLRPC seem to be the standards. I'm not sure I would use either.


> Do you think I'm likely to run into more problems? I got the feeling
> from asking questions on IRC that embedding Python is kind of
> discouraged but I thought in this case it seemed the neatest way.
>
>   
It is discouraged because you get a singleton. You can create 
subinterpreters (cf. Apache's mod_python), but that will bring problems 
of it's own. For example you cannot use extensions that require the 
simplified GIL API (e.g. ctypes). I think this is a major design flaw of 
CPython. For example with Java's JNI, you get a context pointer to the 
VM, so you don't have any of these problems. But with CPython, both the 
interpreter and extensions are basically implemented to be loaded as 
singletons.



> I was aware of this - I thought I would be OK on the mac - at least
> Python and Numpy and my mex function are built with apple gcc although
> I'm not sure about Matlab. I guess Windows will be more difficult...
> But in any case I don't plan to pass any file handles around.
>
>   
It applies to any CRT resource, not just files. Compiler is not 
important, but which CRT is loaded. And if you statically link the same 
CRT twice, that becomes two CRTs that cannot share resources. In 
Windows, Microsoft has made sure there are multiple versions of their 
CRT (msvcrt.dll, msvcr71.dll, msvcr80.dll, msvcr90.dll, ...) that cannot 
share resources. And anyone not careful with this can experice crashes 
at random locations.








___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robin
On Fri, Nov 13, 2009 at 7:48 PM, Sturla Molden  wrote:
> Robin skrev:
>> I had assumed when matlab unloads the mex function it would also
>> unload python - but it looks like other dynamic libs pulled in from
>> the mex function (in this case python and in turn numpy) aren't
>> unloaded...
>>
> Matlab MEX functions are DLLs, Python interpreter is a DLL, NumPy .pyd
> files are DLLs... If you cannot unload Python or NumPy, you cannot
> unload MEX functions either. The same OS constraints apply.

Ah, I hadn't realised it was an OS constraint - I thought it was
possible to unload dlls - and that was why matlab provides the clear
function. mex automatically clears a function when you rebuild it - I
thought that was how you can rebuild and reload mex functions without
restarting matlab. I thought it was just a quirk of matlab that there
was no way to unload other libraries pulled in through the mex
function. I must admit to being a bit out of my depth with this stuff
though so I stand corrected.

> If you are using Windows, I would recommend that you expose your NumPy
> code as a COM object using pywin32, and make a Matlab wrapper (ordinary
> .m file) that calls the COM object. If you are not using Windows, you
> could create an XMLRPC server in Python and call that using Matlab's
> built-in Java VM.
>
> Or you can spawn a separate Python process from Matlab, and communicate
> using pipes or a tcp/ip socket (Matlab's Java or MEX).
>
> There are many solutions that are easier than embedding Python in Matlab.

I really want a cross platform solution so that rules out COM. I
thought about using web services or something which I guess would be
the easiest way to do communication through a tcp socket (least work
since I could use existing web services libraries on both sides). But
actually I have found it pretty easy to embed Python so far... with
about 5 lines of code I'm able run simple strings and with a small
cython module I get stdout to the console. I haven't tried passing
data back and forth yet but from Pauli's pythoncall it doesn't look
like that is too difficult. I was sort of hoping that eventually I
could create a numpy array from a matlab data pointer - at least for
read only input - so that I could use python code to work on large
amounts of data without so much memory overhead (returning smaller
results by copying).

Do you think I'm likely to run into more problems? I got the feeling
from asking questions on IRC that embedding Python is kind of
discouraged but I thought in this case it seemed the neatest way.

> Be careful when you are using file handles. You have to be sure that
> Matlab, Python and NumPy are all linked against the same CRT.

I was aware of this - I thought I would be OK on the mac - at least
Python and Numpy and my mex function are built with apple gcc although
I'm not sure about Matlab. I guess Windows will be more difficult...
But in any case I don't plan to pass any file handles around.

Cheers

Robin
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Sturla Molden
Robin skrev:
> So far the only remotely tricky thing I did was redirect sys.stdout
> and sys.stderr to a wrapper that uses mexPrintf so output goes to the
> matlab console.
>   
Be careful when you are using file handles. You have to be sure that 
Matlab, Python and NumPy are all linked against the same CRT.

Sturla
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Sturla Molden
Robin skrev:
> I had assumed when matlab unloads the mex function it would also
> unload python - but it looks like other dynamic libs pulled in from
> the mex function (in this case python and in turn numpy) aren't
> unloaded...
>   
Matlab MEX functions are DLLs, Python interpreter is a DLL, NumPy .pyd 
files are DLLs... If you cannot unload Python or NumPy, you cannot 
unload MEX functions either. The same OS constraints apply.

If you are using Windows, I would recommend that you expose your NumPy 
code as a COM object using pywin32, and make a Matlab wrapper (ordinary 
.m file) that calls the COM object. If you are not using Windows, you 
could create an XMLRPC server in Python and call that using Matlab's 
built-in Java VM.

Or you can spawn a separate Python process from Matlab, and communicate 
using pipes or a tcp/ip socket (Matlab's Java or MEX).

There are many solutions that are easier than embedding Python in Matlab.

Sturla
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Pauli Virtanen
Fri, 13 Nov 2009 19:12:26 +, Robin wrote:
[clip]
> How does the RTLD_GLOBAL thing manifest itself? So far I have only a
> very basic start which basically consists of:
> cmd = mxArrayToString(prhs[0]);
> PyRun_SimpleString(cmd);
> but I haven't noticed anything not working - I can run numpy testsuite,
> and do simple commands as expected (initiliase arrays in the
> interpreter, run numpy functions on them). Perhaps recent versions of
> Matlab behave differently (I am using R2009a on a mac).

The RTLD_GLOBAL issue prevented Numpy from being imported. I think 
everything was well, until you tried to run "import numpy" in the 
embedded process -- loading multiarray.so would fail because of missing 
symbols.

But if it worked for you without the hack, then it must have been changed 
in the Matlab versions since then (and Pythoncall needs updating...).

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robin
On Fri, Nov 13, 2009 at 6:50 PM, Pauli Virtanen  wrote:
> Fri, 13 Nov 2009 17:23:19 +, Robin wrote:
>> I'm trying to embed Python in a MATLAB mex file. I've been coming under
>> some pressure to make my Python code available to my MATLAB colleagues
>> so I am trying to come up with a relatively general way of calling
>> numerical python code from Matlab.
>
> Out of curiosity, how are you handling the Matlab RTLD_GLOBAL issue. Last
> time [1] I did something similar, I had to hack around it in an ugly way.
>
> .. [1] http://www.iki.fi/pav/software/pythoncall/index.html

Actually I was completely unaware of it (the RTLD_GLOBAL thing). I had
googled for a while trying to find a similar project (I had assumed
someone must have done it) but somehow didn't find your pythoncall
project. It's great - the interface is very close to what I had in
mind, but it's much more comprehensive then anything I thought of (I
was hoping to handle just contiguous numpy arrays).

How does the RTLD_GLOBAL thing manifest itself? So far I have only a
very basic start which basically consists of:
cmd = mxArrayToString(prhs[0]);
PyRun_SimpleString(cmd);
but I haven't noticed anything not working - I can run numpy
testsuite, and do simple commands as expected (initiliase arrays in
the interpreter, run numpy functions on them). Perhaps recent versions
of Matlab behave differently (I am using R2009a on a mac).

So far the only remotely tricky thing I did was redirect sys.stdout
and sys.stderr to a wrapper that uses mexPrintf so output goes to the
matlab console.

Cheers

Robin
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Pauli Virtanen
Fri, 13 Nov 2009 17:23:19 +, Robin wrote:
> I'm trying to embed Python in a MATLAB mex file. I've been coming under
> some pressure to make my Python code available to my MATLAB colleagues
> so I am trying to come up with a relatively general way of calling
> numerical python code from Matlab.

Out of curiosity, how are you handling the Matlab RTLD_GLOBAL issue. Last 
time [1] I did something similar, I had to hack around it in an ugly way.

.. [1] http://www.iki.fi/pav/software/pythoncall/index.html

-- 
Pauli Virtanen

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robin
On Fri, Nov 13, 2009 at 6:13 PM, Robert Kern  wrote:
> On Fri, Nov 13, 2009 at 12:05, Travis Oliphant  wrote:
>> I wonder if this is related to the fact that you can't "unload" a
>> dynamically linked module (like NumPy has).   So, when you call
>> Py_Finalize you are not really "finalizing" your usage of Python
>> extension modules.
>>
>> I'm not sure though.
>
> Right. We do some global things when numpy is imported. Since there is
> no unload step for extension modules, we can't undo them. The second
> time the interpreter starts up, it doesn't know that numpy has already
> been loaded and that numpy shouldn't try to do those global things
> again.

Thanks for the quick responses...

I'm sure you're right - in fact it looks like there was a similar issue here:
http://www.mathworks.co.uk/matlabcentral/newsreader/view_thread/262089

I had assumed when matlab unloads the mex function it would also
unload python - but it looks like other dynamic libs pulled in from
the mex function (in this case python and in turn numpy) aren't
unloaded...

I wonder if it would be possible to link python statically to my mex
function, so it really is unloaded when the mex function is... but I'm
getting a bit out of my depth with linker options, and I guess numpy
is always loaded dynamically anyway and will stick around.

Easy enough to work around it anyway - but just wanted to check what
was going on.

Cheers

Robin
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robert Kern
On Fri, Nov 13, 2009 at 12:05, Travis Oliphant  wrote:
>
> On Nov 13, 2009, at 11:23 AM, Robin wrote:
>
>> Hi,
>>
>> I'm trying to embed Python in a MATLAB mex file. I've been coming
>> under some pressure to make my Python code available to my MATLAB
>> colleagues so I am trying to come up with a relatively general way of
>> calling numerical python code from Matlab.
>>
>> I am making some progress - but get a reliable crash from numpy. This
>> only occurs the second time I am loading it. So I Py_Initialize,
>> import numpy, Py_Finalize (all works fine), but then if I clear the
>> mex file (clear funcname in matlab - which calls Py_Finalize through a
>> mexAtExit handler)  and try to run the function again (which will
>> reinitialize interpreter and import numpy again) I get the followin
>> stack trace from multisarray.so. Wondered if anyone could through any
>> light. I have already run into this bug
>> http://bugs.python.org/issue6869 which prevents me using ctypes... I
>> wondered if this was related.
>>
>> For now its not such a big problem - I will just avoid unloading the
>> mex function (with clear function). But I thought it might be
>> indicative of a memory leak or some other problem since I think in
>> theory it should work (It does if numpy isn't imported).
>
> I wonder if this is related to the fact that you can't "unload" a
> dynamically linked module (like NumPy has).   So, when you call
> Py_Finalize you are not really "finalizing" your usage of Python
> extension modules.
>
> I'm not sure though.

Right. We do some global things when numpy is imported. Since there is
no unload step for extension modules, we can't undo them. The second
time the interpreter starts up, it doesn't know that numpy has already
been loaded and that numpy shouldn't try to do those global things
again.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Travis Oliphant

On Nov 13, 2009, at 11:23 AM, Robin wrote:

> Hi,
>
> I'm trying to embed Python in a MATLAB mex file. I've been coming
> under some pressure to make my Python code available to my MATLAB
> colleagues so I am trying to come up with a relatively general way of
> calling numerical python code from Matlab.
>
> I am making some progress - but get a reliable crash from numpy. This
> only occurs the second time I am loading it. So I Py_Initialize,
> import numpy, Py_Finalize (all works fine), but then if I clear the
> mex file (clear funcname in matlab - which calls Py_Finalize through a
> mexAtExit handler)  and try to run the function again (which will
> reinitialize interpreter and import numpy again) I get the followin
> stack trace from multisarray.so. Wondered if anyone could through any
> light. I have already run into this bug
> http://bugs.python.org/issue6869 which prevents me using ctypes... I
> wondered if this was related.
>
> For now its not such a big problem - I will just avoid unloading the
> mex function (with clear function). But I thought it might be
> indicative of a memory leak or some other problem since I think in
> theory it should work (It does if numpy isn't imported).

I wonder if this is related to the fact that you can't "unload" a  
dynamically linked module (like NumPy has).   So, when you call  
Py_Finalize you are not really "finalizing" your usage of Python  
extension modules.

I'm not sure though.

-Travis

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robin
>> I'm trying to embed Python in a MATLAB mex file. I've been coming
>> under some pressure to make my Python code available to my MATLAB
>> colleagues so I am trying to come up with a relatively general way of
>> calling numerical python code from Matlab.

I get a similar but different error trying to do the same with scipy -
(importing scipy after a Py_Finalize and matlab function clear) - this
time in umath.so:


 Bus error detected at Fri Nov 13 17:53:43 2009


Configuration:
  MATLAB Version:   7.8.0.347 (R2009a)
  MATLAB License:   161051
  Operating System: Darwin 10.0.0 Darwin Kernel Version 10.0.0: Fri
Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386
  Window System:The X.Org Foundation (10402000), display
/tmp/launch-2p1ZWg/:0
  Current Visual:   0x24 (class 4, depth 24)
  Processor ID: x86 Family 6 Model 15 Stepping 10, GenuineIntel
  Virtual Machine:  Java 1.6.0_15-b03-219 with Apple Inc. Java
HotSpot(TM) Client VM mixed mode
  Default Encoding:  ISO-8859-1

Fault Count: 1

Register State:
  eax = 33028046  ebx = 327464cb
  ecx = 3382b110  edx = 330180b0
  esi = 304d6ac0  edi = 336e6660
  ebp = b08fa8f8  esp = b08fa8cc
  eip = 23810006  flg = 00010a92

Stack Trace:
  [0] 0x23810006(0x336e6660, 0x3352e415 "remainder", 0x32824d74, 0)
  [1] umath.so:initumath~(0xb08faebb "numpy.core.umath", 0xb08faec6
"umath", 0xb08faa07 "/Library/Frameworks/Python.frame..", 0xa043aab0)
+ 2152 bytes
  [2] Python:_PyImport_LoadDynamicModule~(0xb08faebb
"numpy.core.umath", 0xb08faa07 "/Library/Frameworks/Python.frame..",
0xa043aab0, 0x331ae7d3) + 153 bytes
  [3] Python:load_module~(3, 0, 0xb08fae08 "∞™C†", 0xb08fae0c) + 203 bytes
  [4] Python:import_submodule~(0xb08faec6 "umath", 0x326eecd4 "umath",
5, 0x33148fc9) + 293 bytes
  [5] Python:load_next~(0xb08faebb "numpy.core.umath", 0xb08fb2bc,
0xb08faf68, 0x331a75e6) + 195 bytes
  [6] Python:import_module_level~(0x33200aa0, 0x, 0xee9e70b3,
0x33189f4d "«E") + 142 bytes
  [7] Python:PyImport_ImportModuleLevel~(0x326eecd4 "umath",
0x336340c0, 0x336340c0, 0x33200aa0) + 45 bytes
  [8] Python:builtin___import__~(0, 0x33845ea0, 0, 0x3318fcfe) + 156 bytes
  [9] Python:PyObject_Call~(0x326ba7b0, 0x33845ea0, 0, 0x3313f3cd) + 45 bytes
  [10] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845ea0,
0, 0x336340c0) + 112 bytes
  [11] Python:PyEval_EvalFrameEx~(0x304d5fd0, 0, 0x336340c0,
0x336340c0) + 8138 bytes
  [12] Python:PyEval_EvalCodeEx~(0x30386e30, 0x336340c0, 0x336340c0,
0) + 1819 bytes
  [13] Python:PyEval_EvalCode~(0x30386e30, 0x336340c0, 0x336340c0,
0x9325378f) + 87 bytes
  [14] Python:PyImport_ExecCodeModuleEx~(0xb08fc4cb "numpy.core",
0x30386e30, 0xb08fb76f "/Library/Frameworks/Python.frame..",
0x4adc73cc) + 193 bytes
  [15] Python:load_source_module~(1, 0, 0xb08fbfbc "X™C†0D80†370", 0)
+ 726 bytes
  [16] Python:load_package~(5, 0, 0xb08fc418, 0xb08fc41c) + 427 bytes
  [17] Python:import_submodule~(0xb08fc4d1 "core", 0x3384512a
"core.numeric", 4, 0x33148fc9) + 293 bytes
  [18] Python:load_next~(0xb08fc4cb "numpy.core", 0xb08fc8cc, 9,
0x331a75e6) + 195 bytes
  [19] Python:import_module_level~(0x33200aa0, 0x, 0x746e6920,
0x33189f4d "«E") + 213 bytes
  [20] Python:PyImport_ImportModuleLevel~(0x33845124
"numpy.core.numeric", 0x33634ae0, 0x33634ae0, 0x33200aa0) + 45 bytes
  [21] Python:builtin___import__~(0, 0x33845f30, 0, 0x3318fcfe) + 156 bytes
  [22] Python:PyObject_Call~(0x326ba7b0, 0x33845f30, 0, 0x3313f3cd) + 45 bytes
  [23] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845f30,
0, 0x33634ae0) + 112 bytes
  [24] Python:PyEval_EvalFrameEx~(0x304caf70, 0, 0x33634ae0,
0x33634ae0) + 8138 bytes
  [25] Python:PyEval_EvalCodeEx~(0x30386e78, 0x33634ae0, 0x33634ae0,
0) + 1819 bytes
  [26] Python:PyEval_EvalCode~(0x30386e78, 0x33634ae0, 0x33634ae0,
0x9325378f) + 87 bytes
  [27] Python:PyImport_ExecCodeModuleEx~(0xb08fd68b
"numpy.lib.type_check", 0x30386e78, 0xb08fcd7f
"/Library/Frameworks/Python.frame..", 6771) + 193 bytes
  [28] Python:load_source_module~(1, 0, 0xb08fd5d8, 0xb08fd5dc) + 726 bytes
  [29] Python:import_submodule~(0xb08fd695 "type_check", 0x3038a494
"type_check", 10, 0x33148fc9) + 293 bytes
  [30] Python:load_next~(0xb08fd68b "numpy.lib.type_check",
0xb08fda8c, 0xb08fd738, 0x331a75e6) + 195 bytes
  [31] Python:import_module_level~(0x32703e30, 0x, 0xee9e70b3,
0x33189f4d "«E") + 142 bytes
  [32] Python:PyImport_ImportModuleLevel~(0x3038a494 "type_check",
0x33634150, 0x33634150, 0x32703e30) + 45 bytes
  [33] Python:builtin___import__~(0, 0x33845540, 0, 0x3318fcfe) + 156 bytes
  [34] Python:PyObject_Call~(0x326ba7b0, 0x33845540, 0, 0x3313f3cd) + 45 bytes
  [35] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845540,
0, 0x33634150) + 112 bytes
  [36] Python:PyEval_EvalFrameEx~(0x304b4fd0, 0, 0x33634150,
0x33634150) + 8138 bytes
  [37] Python:PyEva

Re: [Numpy-discussion] bus error in embedded numpy

2009-11-13 Thread Robin
I forgot to add it doesn't happen if Py_Finalize isn't called - if I
take that out and just let the OS/matlab unload the mex function then
I can run it many times without the error, but it does leak a bit of
memory each time.

Cheers

Robin

On Fri, Nov 13, 2009 at 5:23 PM, Robin  wrote:
> Hi,
>
> I'm trying to embed Python in a MATLAB mex file. I've been coming
> under some pressure to make my Python code available to my MATLAB
> colleagues so I am trying to come up with a relatively general way of
> calling numerical python code from Matlab.
>
> I am making some progress - but get a reliable crash from numpy. This
> only occurs the second time I am loading it. So I Py_Initialize,
> import numpy, Py_Finalize (all works fine), but then if I clear the
> mex file (clear funcname in matlab - which calls Py_Finalize through a
> mexAtExit handler)  and try to run the function again (which will
> reinitialize interpreter and import numpy again) I get the followin
> stack trace from multisarray.so. Wondered if anyone could through any
> light. I have already run into this bug
> http://bugs.python.org/issue6869 which prevents me using ctypes... I
> wondered if this was related.
>
> For now its not such a big problem - I will just avoid unloading the
> mex function (with clear function). But I thought it might be
> indicative of a memory leak or some other problem since I think in
> theory it should work (It does if numpy isn't imported).
>
> Cheers
>
> Robin
>
> 
>             Bus error detected at Fri Nov 13 17:11:57 2009
> 
>
> Configuration:
>  MATLAB Version:   7.8.0.347 (R2009a)
>  MATLAB License:   161051
>  Operating System: Darwin 10.0.0 Darwin Kernel Version 10.0.0: Fri
> Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386
>  Window System:    The X.Org Foundation (10402000), display
> /tmp/launch-2p1ZWg/:0
>  Current Visual:   0x24 (class 4, depth 24)
>  Processor ID:     x86 Family 6 Model 15 Stepping 10, GenuineIntel
>  Virtual Machine:  Java 1.6.0_15-b03-219 with Apple Inc. Java
> HotSpot(TM) Client VM mixed mode
>  Default Encoding:  ISO-8859-1
>
> Fault Count: 1
>
> Register State:
>  eax = 0001  ebx = 307b12ab
>  ecx =   edx = 305ef148
>  esi = 305ef140  edi = 32a79d60
>  ebp = b097c6c8  esp = b097c670
>  eip = 307b14be  flg = 00010202
>
> Stack Trace:
>  [0] multiarray.so:PyArray_FromScalar~(0x305ef140, 0, 2, 0x32e23287)
> + 542 bytes
>  [1] multiarray.so:gentype_nonzero_number~(0x305ef140, 0x2eaa6db0, 0,
> 0x32e73cfe) + 36 bytes
>  [2] Python:PyObject_IsTrue~(0x305ef140, 0x2ea95de0, 2, 0) + 63 bytes
>  [3] Python:PyEval_EvalFrameEx~(0x33c95160, 0, 0x332941e0, 0) + 12598 bytes
>  [4] Python:PyEval_EvalFrameEx~(0x32a70f80, 0, 0x332941e0, 0) + 24217 bytes
>  [5] Python:PyEval_EvalCodeEx~(0x33f05068, 0x332941e0, 0, 0x32a96794)
> + 1819 bytes
>  [6] Python:PyEval_EvalFrameEx~(0x32a96640, 0, 0x332941e0, 0) + 16561 bytes
>  [7] Python:PyEval_EvalCodeEx~(0x33f051d0, 0x332941e0, 0, 0x32a6fcc0)
> + 1819 bytes
>  [8] Python:PyEval_EvalFrameEx~(0x32a6fb60, 0, 0x33bc99c0, 0) + 16561 bytes
>  [9] Python:PyEval_EvalCodeEx~(0x334abda0, 0x33bc99c0, 0, 0x32ad3b24)
> + 1819 bytes
>  [10] Python:PyEval_EvalFrameEx~(0x32ad39d0, 0, 0x33bc94b0, 0) + 16561 bytes
>  [11] Python:PyEval_EvalCodeEx~(0x332919b0, 0x33bc94b0, 0,
> 0x33ce5294) + 1819 bytes
>  [12] Python:PyEval_EvalFrameEx~(0x33ce5150, 0, 0x33bc94b0, 0) + 16561 bytes
>  [13] Python:PyEval_EvalCodeEx~(0x332918d8, 0x33bc94b0, 0,
> 0x34d5156c) + 1819 bytes
>  [14] Python:PyEval_EvalFrameEx~(0x34d51430, 0, 0x33bc9300,
> 0x33bc9300) + 16561 bytes
>  [15] Python:PyEval_EvalCodeEx~(0x33291968, 0x33bc9300, 0x33bc9300,
> 0) + 1819 bytes
>  [16] Python:PyEval_EvalCode~(0x33291968, 0x33bc9300, 0x33bc9300,
> 0x9325378f) + 87 bytes
>  [17] Python:PyImport_ExecCodeModuleEx~(0xb097e4cb
> "numpy.core._internal", 0x33291968, 0xb097dbbf
> "/Library/Frameworks/Python.frame..", 0x332a2000) + 193 bytes
>  [18] Python:load_source_module~(1, 0, 0xb097e418 "X™C†", 0xb097e41c)
> + 726 bytes
>  [19] Python:import_submodule~(0xb097e4d6 "_internal", 0x33d1ce1c
> "_internal", 9, 0x32e2cfc9) + 293 bytes
>  [20] Python:load_next~(0xb097e4cb "numpy.core._internal",
> 0xb097e8cc, 0xb097e578, 0x32e8b5e6) + 195 bytes
>  [21] Python:import_module_level~(0x32ee4aa0 "TD", 0x,
> 0xee9e70b3, 0x32e6df4d "«E") + 142 bytes
>  [22] Python:PyImport_ImportModuleLevel~(0x33d1ce1c "_internal",
> 0x33b25150, 0x33b25150, 0x32ee4aa0 "TD") + 45 bytes
>  [23] Python:builtin___import__~(0, 0x348854e0, 0, 0x32e73cfe) + 156 bytes
>  [24] Python:PyObject_Call~(0x33da9a08, 0x348854e0, 0, 0x32e233cd) + 45 bytes
>  [25] Python:PyEval_CallObjectWithKeywords~(0x33da9a08, 0x348854e0,
> 0, 0x33b25150) + 112 bytes
>  [26] Python:PyEval_EvalFrameEx~(0x33cd2da0, 0, 0x33b25150,
> 0x33b25150) + 8138 bytes
>  [27] Python:PyEval_EvalCodeEx~(0x2068, 0x3