[C++-sig] dynamic linking in Linux

[Jim Bosch]

I've just run into the "wontfix" problem detailed here:

https://svn.boost.org/trac/boost/ticket/3210

Essentially, shared information held in one module (such as RTTI stuff 
needed for dynamic cast) isn't available to another module, leading to 
segfaults.


I got the impression from reading the ticket comments and the linked 
email threads that the best solution is to do something like this, where 
module "B" depends on module "A":


liba.so  C++ objects for A and Boost.Python wrappers.

a.so --- module, linked against liba.so, just invokes wrapper 
generator functions defined in liba.so within a BOOST_PYTHON_MODULE block.


libb.so  C++ objects for B and Boost.Python wrappers; linked against 
liba.so


b.so --- module, linked against libb.so and liba.so, just invokes 
wrapper generator functions defined in libb.so within a 
BOOST_PYTHON_MODULE block



However, that's not working, and the only solution I've found is to put 
my modules in packages with something like the following in __init__.py:


import ctypes
ctypes.PyDLL("%s/_a" % tuple(__path__), ctypes.RTLD_GLOBAL)
from . import _a

Another similar option, using sys.setdlopenflags, seems to be less 
favored because the dl module needed to get the flags has been 
deprecated in Python itself.


Is this just the current state of things?  Are there any efforts 
underway to improve matters?  Or was I on track in the first case, and I 
was just doing it wrong?


Thanks!

Jim Bosch
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] dynamic linking in Linux

[Niall Douglas]

On 23 Jul 2010 at 0:08, Jim Bosch wrote:

> I've just run into the "wontfix" problem detailed here:
> 
> https://svn.boost.org/trac/boost/ticket/3210
> 
> Essentially, shared information held in one module (such as RTTI stuff 
> needed for dynamic cast) isn't available to another module, leading to 
> segfaults.
> [snip]
> Another similar option, using sys.setdlopenflags, seems to be less 
> favored because the dl module needed to get the flags has been 
> deprecated in Python itself.
> 
> Is this just the current state of things?  Are there any efforts 
> underway to improve matters?  Or was I on track in the first case, and I 
> was just doing it wrong?

I am sorry to hear that this is STILL a problem so many years later 
(it was a change I submitted to GCC back in 2005 which created this 
problem). My advice then as it is now is to change the default 
dlopenflags to RTLD_GLOBAL across the Python language - it is the 
only sane choice. I would even go so far as to say that RTLD_LOCAL 
needs deprecating in GNU libc.

If you search Google for "dlopenflags python" you will see tales of 
woe and misoperation going back eight years now. My view is that 
Guido should simply impose RTLD_GLOBAL on Python and if people's 
extension modules break then they should be excluded from the core 
build until they are fixed.

Laziness is bad enough anywhere in the world, but laziness which 
wastes productivity for hundreds, perhaps thousands of busy and time 
valuable people is inexcusable. This problem has been allowed to 
fester for far too long, and it particularly annoys me because it is 
very easy to fix.

Sorry for your particular woes, and I do apologise for my part in 
causing them.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.



___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] dynamic linking in Linux

[Stefan Seefeld]

 On 07/23/2010 09:35 AM, Niall Douglas wrote:

I would even go so far as to say that RTLD_LOCAL
needs deprecating in GNU libc.


That would open the door for all sorts of ABI issues (symbol collisions 
resulting in the wrong objects being looked up, etc.).

I don't think making symbols visible globally will solve anything.

Stefan

--

  ...ich hab' noch einen Koffer in Berlin...

___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] dynamic linking in Linux

[Niall Douglas]

On 23 Jul 2010 at 9:49, Stefan Seefeld wrote:

>   On 07/23/2010 09:35 AM, Niall Douglas wrote:
> > I would even go so far as to say that RTLD_LOCAL
> > needs deprecating in GNU libc.
> 
> That would open the door for all sorts of ABI issues (symbol collisions 
> resulting in the wrong objects being looked up, etc.).
> I don't think making symbols visible globally will solve anything.

No, all that's happening with RTLD_LOCAL is that symbol collisions 
are being HIDDEN from view when a properly maintained ABI ought to 
explicitly hide all symbols not absolutely necessary from an external 
use perspective. Failure to correctly do this is programmer laziness 
as I said.

It took Microsoft well over a decade of MSVCRT instance conflicts 
before someone forced the default to be the DLL rather than static 
link in MSVC7.1 - countless millions of dollars of wasted 
productivity happened in the intervening time while people made 
statements like the one you just made. I am not criticising you 
personally Stefen, this response is extremely common in the 
Unix/POSIX community. Unfortunately it's just plain wrong.

As I said back in 2005, and many, many times since, programmers need 
to be forced to get off their lazy arses and fix the plain out broken 
and lazy ABI management endemic in the GCC-based ecosystem. The tools 
(-fvisibility and __attribute__(visibility())) have been there since 
GCC 4.0. Unless major projects such as Python simply force the issue 
nothing is going to happen.

I for one was extremely disappointed to see Python 3.x baulk yet 
again on the default for dlopenflags. But I'm tired of ranting about 
it. Maybe we might have more success in getting GCC to make the 
default -fvisibility=hidden, then we might get some traction at last.

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.



___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] dynamic linking in Linux

[Jim Bosch]

On 07/23/2010 12:08 AM, Jim Bosch wrote:

I've just run into the "wontfix" problem detailed here:

https://svn.boost.org/trac/boost/ticket/3210

Essentially, shared information held in one module (such as RTTI stuff
needed for dynamic cast) isn't available to another module, leading to
segfaults.




An update: apparently, I'm not experiencing the garden-variety version 
of this problem.  Using setdlopenflags *doesn't* actually fix it (and 
neither does the similar ctypes method); it was just intermittent, and 
happened to work twice after I added the ctypes trick.  And it appears 
to only happens when pickling a Python builtin (dict, in my case) that 
contains a wrapped C++ object.  It doesn't matter whether I use pickle 
or cPickle.


Given that many people (including myself) have pickled Boost.Python 
objects inside dicts many times before without issue, it might be tricky 
to make a contained example that reproduces it this problem.  And I'm 
less motivated now because I can just skip pickling the object in this 
case and move on with my life...


But if anyone has any new ideas on where to look, it does seem like 
mysterious behavior that would be nice to track down.


Jim
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

[C++-sig] How to derive from an abstract base class

[Lutz Maibaum]

Dear all,

I am totally new to Boost.Python, and I am going through the tutorial to get up 
to speed. I apologize if this is a very obvious question.

Imagine I have a base class in C++ that I want to derive from both in C++ and 
in Python. The following code works wonderfully well:

#include 
using namespace boost::python;

class Base {
public:
  virtual int f() {return -1;}
};
class Derived : public Base {
public:
  int f() {return 1;}
};
class BaseWrap : public Base, public wrapper {
public:
  int f() {
if (override f = this->get_override("f"))
  return f();
return Base::f();
  }
  int default_f() {return this->Base::f();}
};

int func(Base& b) {
  return b.f();
}

BOOST_PYTHON_MODULE(foo) {
  class_ ("Base");
  class_ > ("Derived")
.def("f", &Derived::f);
  class_("Base")
.def("f", &Base::f, &BaseWrap::default_f);
  def("func", func, "Calls the method f of its argument");
}


With this code, I can do the following:

>>> import foo
>>> d1=foo.Derived()
>>> class D(foo.Base):
... def f(self): return 2
... 
>>> d2=D()
>>> foo.func(d1)
1
>>> foo.func(d2)
2


So far so good. Now I would like to make Base::f a pure virtual function, and I 
can't get it to work.

1. Attempt: 

class Base {
public:
  virtual int f() = 0;
};
class Derived : public Base {
public:
  int f() {return 1;}
};
class BaseWrap : public Base, public wrapper {
public:
  int f() {return this->get_override("f")();}
};

int func(Base& b) {
  return b.f();
}

BOOST_PYTHON_MODULE(foo) {
  class_ > ("Derived")
.def("f", &Derived::f);
  class_("Base")
.def("f", pure_virtual(&Base::f));
  def("func", func, "Calls the method f of its argument");
}

This compiles, but when I try to import the resulting module I get he following 
error message:

RuntimeError: extension class wrapper for base class Base has not been created 
yet

This makes sense, because the Python side doesn't now about the abstract  C++ 
class Base yet. Therefore…

2. Attempt:

I change the wrapping code to the following:

BOOST_PYTHON_MODULE(foo) {
  class_ ("Derived")
.def("f", &Derived::f);
  class_("Base")
.def("f", pure_virtual(&Base::f));
  def("func", func, "Calls the method f of its argument");
}

Again this compiles, but I cannot call the c++ function func with the C++ class 
Derived anymore:

>>> import foo
>>> d1=foo.Derived()
>>> foo.func(d1)
Traceback (most recent call last):
  File "", line 1, in 
Boost.Python.ArgumentError: Python argument types in
foo.func(Derived)
did not match C++ signature:
func(Base {lvalue})

Apparently the Python side doesn't know anymore that the C++ class Derived is 
derived from Base.

The core of the problem seems to be that I cannot expose the abstract base 
class Base directly to Python, but only the wrapper class BaseWrap. But then 
C++ classes derived from Base and Python classes derived from BaseWrap do not 
have a common ancestor in the class hierarchy.

If anyone could point me in the right direction, it would much appreciated.

Best,

  Lutz



___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] How to derive from an abstract base class

[Jim Bosch]

On 07/23/2010 03:09 PM, Lutz Maibaum wrote:

Dear all,

I am totally new to Boost.Python, and I am going through the tutorial to get up 
to speed. I apologize if this is a very obvious question.





class Base {
public:
   virtual int f() = 0;
};
class Derived : public Base {
public:
   int f() {return 1;}
};
class BaseWrap : public Base, public wrapper  {
public:
   int f() {return this->get_override("f")();}
};

int func(Base&  b) {
   return b.f();
}

BOOST_PYTHON_MODULE(foo) {
   class_  >  ("Derived")
 .def("f",&Derived::f);
   class_("Base")
 .def("f", pure_virtual(&Base::f));
   def("func", func, "Calls the method f of its argument");
}



Try this:

BOOST_PYTHON_MODULE(foo) {
class_("Base")
  .def("f", pure_virtual(&Base::f));
class_  >  ("Derived")
  .def("f",&Derived::f);
def("func", func, "Calls the method f of its argument");
}

(just swap the order of the class_ statements)


Jim Bosch



___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [C++-sig] How to derive from an abstract base class

[Lutz Maibaum]

On Jul 23, 2010, at 3:55 PM, Jim Bosch wrote:
> Try this:
> 
> BOOST_PYTHON_MODULE(foo) {
>class_("Base")
>  .def("f", pure_virtual(&Base::f));
>class_  >  ("Derived")
>  .def("f",&Derived::f);
>def("func", func, "Calls the method f of its argument");
> }
> 
> (just swap the order of the class_ statements)

Fantastic! That works like charm.

Thank you very much,

  Lutz

___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

[C++-sig] Help with Jamroot file

[Filipe M. S. de Campos]

Hello!

I'm writing a wrapper of a c++ program that uses
OpenCVlibrary. I think I have to
add some information to the Jamroot file to make
bjam import opencv headers and compile the wrapper successfully.
But I couldn't find the right command and location in Jamroot file that I
need to add it. Could somebody help me?

Regards

Filipe Morgado Simões de Campos
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig