Hi,
Thanks to everyone for the suggestions. I can't get the __CYGWIN__ +
PLUGIN_CPP_FILE(OBFormat) trick to work - it still segfaults, and I'm already
using g++ to
link the module. Using Noel's idea to set RTLD_GLOBAL does the trick, but that
completely breaks Numpy - I trade a segfault in OBFormat for one in
PyArray_SimpleNewFromData. This appears to be a known problem:
http://projects.scipy.org/numpy/ticket/1148.
Additionally, I think there's something about the dlopen flags that I don't
quite understand. If I set dlopen flags as the very first thing I do in my
script,
then I get the behavior described above (OpenBabel works fine, Numpy breaks).
The Python code that uses the C library is actually embedded in a class, so I
tried toggling the dlopen flags at method entry and exit (rather than at the
script global level):
@staticmethod
def frommolfile(filename):
flags = sys.getdlopenflags()
sys.setdlopenflags(flags | dl.RTLD_GLOBAL)
argtuple = _input.loadMolset(filename)
sys.setdlopenflags(flags)
return molset.fromtuple(argtuple)
This doesn't work, though. It doesn't crash, but it appears that
OBConversion::Read no longer works properly. Specifically, I get zero molecules
read
(OBConversion::Read returns false the first time I call it). The SDF file I'm
using to test has 30 molecules in it, and loads fine if RTLD_GLOBAL is set at
script init. I also tried moving the import of _input (the C++-OpenBabel
library) into the method, after setting the dlopenflags, but that doesn't
change
anything. Here's the C++ code that is doing the reading:
extern "C" int molFromFile(string filename,[...])
{
OBConversion conv;
ifstream ifs(filename.c_str(),ifstream::in|ifstream::binary);
OBFormat* inFormat = conv.FormatFromExt(filename.c_str());
conv.SetInFormat(inFormat);
OBMol mol;
int nread = 0;
while (conv.Read(&mol,&ifs)) {
nread++;
[ do some stuff with mol and stuff a transformed copy into a list]
mol.Clear();
}
if (!nread) {
cout << "FATAL ERROR: could not read file "<<filename<<endl;
exit(1);
}
return nread;
}
That "FATAL ERROR" is triggering when I try to do the method-level
setdlopenflags trickery. Does anyone know how to either get this to work
without RTLD_GLOBAL,
or how I can get OpenBabel and Numpy to coexist?
Thanks,
Imran
On 11/18/10 9:55 AM, Craig A. James wrote:
> On 11/17/10 5:35 PM, Imran Haque wrote:
>> Hi,
>>
>> I have a large set of C++ molecule-manipulation libraries that
>> internally use OpenBabel to handle molecule input and structure, and I'm
>> trying to wrap them into a Python library using standard Python C API
>> techniques. These routines work fine when called from a C program, but
>> when called from the .so used for the Python library,
>> OBConversion::FindFormat segfaults. I don't know how to follow the code
>> path in the source code, as it appears to be generated by macros and
>> eventually calls down into an independent set of .so's for each file
>> format. Does anyone know how to get the molecule loader routines to work
>> from dynamic libraries?
>
> We do something similar: we link OpenBabel (C++) into the Postgres database
> (written in C). We discovered that it is *critical* that you use the C++
> linker,
> NOT the C linker, to link the final shared library.
>
> There's a lot of linker magic that goes on behind the curtains regarding
> different versions of system libraries. C++ has a lot of constructors and
> initialization that is triggered by the loader/linker, but if you link the
> shared object with the ordinary C linker, that stuff doesn't get initialized.
> It
> causes all sorts of weird and unexplainable segfaults. Simply replacing gcc
> with g++ to create the .so solved our problem.
>
> Craig
>
>>
>> I've put up a minimal test case at
>> http://cs.stanford.edu/people/ihaque/ob_bug_repro.tar. You may have to
>> adjust the paths in the Makefile to correspond to your Python and OB
>> installations. Here's sample output (note that I added a tracing printf
>> to OBConversion::FormatFromExt that isn't in the mainline OB 2.3.0 source):
>>
>> $ python numatoms.py test.sdf
>> Molecule hasFilename test.sdf
>> FormatFromExt calling FindFormat with arg sdf
>> Segmentation fault
>>
>> $ ./numatoms test.sdf
>> Filename test.sdf
>> FormatFromExt calling FindFormat with arg sdf
>> Detected format: MDL MOL format
>> Reads and writes V2000 and V3000 versions
>> Read Options, e.g. -as
>> s determine chirality from atom parity flags
>> This is valid only for 0D information. Atom
>> parity is always ignored on reading for MOL files
>> containing 2D or 3D information.
>> T read title only
>> P read title and properties only
>> When filtering an sdf file on title or properties
>> only, avoid lengthy chemical interpretation by
>> using the T or P option together with copy format.
>>
>> Write Options, e.g. -x3
>> 3 output V3000 not V2000 (used for>999 atoms/bonds)
>> m write no properties
>> w recalculate wedge and hash bonds(2D structures only)
>> A output in Alias form, e.g. Ph, if present
>>
>>
>> Molecule has 27 atoms.
>>
>>
>> Thanks,
>>
>> Imran Haque
>> http://cs.stanford.edu/people/ihaque
>> [email protected]
>>
>>
>> PS - Using Pybel is not an option here because I don't want to
>> reimplement all the C++ functionality in Python - I just need to be able
>> to call into it.
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9 supports
>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2& L3.
>> Spend less time writing and rewriting code and more time creating great
>> experiences on the web. Be a part of the beta today
>> http://p.sf.net/sfu/msIE9-sfdev2dev
>> _______________________________________________
>> OpenBabel-discuss mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/openbabel-discuss
>>
>
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
OpenBabel-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openbabel-discuss
