Re: [C++-sig] Function handling with Boost
Hi,
On Wed, Jun 24, 2009 at 7:52 AM, Christopher Schramm
wrote:
>
> Stefan Seefeld wrote:
> > An alternative is not to use BOOST_PYTHON_MODULE at all, but set up
> > converters in ordinary C++ code. In the following I set up a Python
> > interpreter in my main application, inject a (C++) base class, run a
> > Python script that adds a derived class, then instantiate and run that
> > derived class from C++ code. I this may just be what you want:
>
> Sure. That's what I'm doing (just within a submodule below main). But in
> your example - how do get simple c++ functions into the python scope
> which aren't members of any class? That's my problem.
This seems to work:
#include
#include
namespace bp = boost::python;
void myfunction() {
std::cout << "Hello world!\n";
}
int main(int argc, char* argv[]) {
Py_Initialize();
bp::object main = bp::import("__main__");
bp::object global = main.attr("__dict__");
bp::object function = bp::object(myfunction);
global["function"] = function;
bp::exec("function()", global, global);
return 0;
}
Is this what you wanted?
Boost python's def returns void, which is natural I think, since you
can't assign to the "result" of def in python either.
Hope this helps,
Thomas
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Function handling with Boost
Thomas Berg wrote: > bp::object function = bp::object(myfunction); Great! And it was that simple... Thanks! ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Function handling with Boost
> Thomas Berg wrote: >> bp::object function = bp::object(myfunction); > > Great! And it was that simple... But wait... Giving that a second thought I don't think that's going to exhaust bpy's full potential. At least I don't see a way to use it's docstring handling or call policies. I'll test it tomorrow. ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Function handling with Boost
On Wed, Jun 24, 2009 at 10:26 PM, Christopher Schramm
wrote:
>
> > Thomas Berg wrote:
> >> bp::object function = bp::object(myfunction);
> >
> > Great! And it was that simple...
>
> But wait... Giving that a second thought I don't think that's going to
> exhaust bpy's full potential. At least I don't see a way to use it's
> docstring handling or call policies.
>
> I'll test it tomorrow.
Just discovered a way of doing it properly with "def". By creating a
"scope" object you can control where objects created by "def" are
inserted, like this:
#include
#include
namespace bp = boost::python;
void myfunction() {
std::cout << "Hello world!\n";
}
int main(int argc, char* argv[]) {
Py_Initialize();
bp::object main = bp::import("__main__");
bp::object global = main.attr("__dict__");
{
bp::scope sc(main);
bp::def("function", myfunction, "function helpstring");
}
bp::exec("function()", global, global);
bp::exec("help(function)", global, global);
return 0;
}
The scope object sets a global variable which bp::def uses when
defining the function. In this case I passed the main module to the
scope, so def inserts the definition into it. Getting closer now?
Found this out by reading the source code of
boost::python::detail::init_module, called by the BOOST_PYTHON_MODULE
macro.
Cheers,
Thomas
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
[C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied?
Hi again,
I've run across another issue confusing me as I try to use pybindgen,
in this case it's the treatment of reference parameters in virtual
methods where you're allowing the class to be subclassed from python.
It looks like the helper class that is generated is making copies of
parameters that are being passed as references, which is not what we
want. Take the following example C++ classes:
class A {
public:
A(const int val): mVal(val) {}
private:
int mVal;
A();
A(const A& rhs);
A& operator=(const A& rhs);
};
class B {
public:
B() {}
virtual ~B() {}
virtual void some_virtual_method(const A& a) const { std::cerr <<
&a << std::endl; }
private:
};
I'd like to be able to create subclasses of B from python, so I try
wrapping these classes with the following pybindgen code:
mod = Module("ref_param_example")
mod.add_include('"classes.hh"')
a = mod.add_class("A")
b = mod.add_class("B", allow_subclassing=True)
a.add_constructor([param("int", "val")])
b.add_constructor([])
b.add_method("some_virtual_method", None, [param("A&", "a")],
is_const=True, is_virtual=True)
The resulting generated code contains the following:
void
PyB__PythonHelper::some_virtual_method(A & a) const
{
PyGILState_STATE __py_gil_state;
B *self_obj_before;
PyObject *py_retval;
PyA *py_A;
__py_gil_state = (PyEval_ThreadsInitialized() ?
PyGILState_Ensure() : (PyGILState_STATE) 0);
if (!PyObject_HasAttrString(m_pyself, (char *)
"_some_virtual_method")) {
B::some_virtual_method(a);
if (PyEval_ThreadsInitialized())
PyGILState_Release(__py_gil_state);
return;
}
self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj;
reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const
B*) this);
py_A = PyObject_New(PyA, &PyA_Type);
py_A->obj = new A(a);
As you can see, the last line quoted here is trying to make a copy of
the instance "a" that was passed in. In this example this will not
compile because the copy constructor for A is private. In general we
don't want to make copies of reference parameters regardless of course.
Am I once again missing something here? Is there some way I can
declare that the passed parameters for "some_virtual_method" should
really be treated as references and not copied?
Thanks!
Mike.
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied?
2009/6/24 J. Michael Owen
> Hi again,
>
> I've run across another issue confusing me as I try to use pybindgen, in
> this case it's the treatment of reference parameters in virtual methods
> where you're allowing the class to be subclassed from python. It looks like
> the helper class that is generated is making copies of parameters that are
> being passed as references, which is not what we want. Take the following
> example C++ classes:
>
> class A {
> public:
> A(const int val): mVal(val) {}
> private:
> int mVal;
> A();
> A(const A& rhs);
> A& operator=(const A& rhs);
> };
>
> class B {
> public:
> B() {}
> virtual ~B() {}
> virtual void some_virtual_method(const A& a) const { std::cerr << &a <<
> std::endl; }
> private:
> };
>
> I'd like to be able to create subclasses of B from python, so I try
> wrapping these classes with the following pybindgen code:
>
> mod = Module("ref_param_example")
> mod.add_include('"classes.hh"')
> a = mod.add_class("A")
> b = mod.add_class("B", allow_subclassing=True)
> a.add_constructor([param("int", "val")])
> b.add_constructor([])
> b.add_method("some_virtual_method", None, [param("A&", "a")],
> is_const=True, is_virtual=True)
>
> The resulting generated code contains the following:
>
> void
> PyB__PythonHelper::some_virtual_method(A & a) const
> {
>PyGILState_STATE __py_gil_state;
>B *self_obj_before;
>PyObject *py_retval;
>PyA *py_A;
>
>__py_gil_state = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() :
> (PyGILState_STATE) 0);
>if (!PyObject_HasAttrString(m_pyself, (char *) "_some_virtual_method"))
> {
>B::some_virtual_method(a);
>if (PyEval_ThreadsInitialized())
>PyGILState_Release(__py_gil_state);
>return;
>}
>self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj;
>reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const B*)
> this);
>py_A = PyObject_New(PyA, &PyA_Type);
>py_A->obj = new A(a);
>
>
> As you can see, the last line quoted here is trying to make a copy of the
> instance "a" that was passed in. In this example this will not compile
> because the copy constructor for A is private. In general we don't want to
> make copies of reference parameters regardless of course.
>
> Am I once again missing something here? Is there some way I can declare
> that the passed parameters for "some_virtual_method" should really be
> treated as references and not copied?
>
Sorry, but in this case the answer is no. The virtual method wrapper here
receives a parameter from C++ and has to create a Python wrapper for it.
The wrapper needs to own the object. The reason is that the Python method
that is called may decide to keep a reference to the object. If the Python
wrapper kept a "shared" reference to the same C++ object then it could at
some point in time be keeping a reference to an object when the caller has
already deallocated it.
To summarize, the object copy is done because the alternative would be
memory unsafe.
This is one case where a C++ API blindly designed without considering
language bindings may make it difficult to bind later on. Ideally, in this
case you should pass a pointer to A, and even better A would have reference
counting so that both caller and callee could own a reference to the same
object at the same time...
--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied?
I can see what you're worried about here, but as a practical matter it is very useful to be able to pass stuff by reference, and copying it under the hood will break code. For instance, if the virtual functions modify the state of the parameters passed by reference it will just not function correctly when a copy is made. Or in the example I sent when copy constructors are not available it won't even compile! In Boost.Python this can be handled by wrapping the intermediate arguments with the boost::ref() function, which tells Boost.Python to pass the thing along as reference rather than making a copy and on my own head be it if I do something nefarious with that reference. I guess what I would like in this case is the equivalent of an "on my own head be it" option. This is kind of how I think of the pointer passing policy, like with the "caller_owns_return" option. Wouldn't it be reasonable to allow us to specify optionally that references should not be copied, but rather passed along natively? With the default policy still being the more memory safe option of doing the copy? On Jun 24, 2009, at 3:31 PM, Gustavo Carneiro wrote: Sorry, but in this case the answer is no. The virtual method wrapper here receives a parameter from C++ and has to create a Python wrapper for it. The wrapper needs to own the object. The reason is that the Python method that is called may decide to keep a reference to the object. If the Python wrapper kept a "shared" reference to the same C++ object then it could at some point in time be keeping a reference to an object when the caller has already deallocated it. To summarize, the object copy is done because the alternative would be memory unsafe. This is one case where a C++ API blindly designed without considering language bindings may make it difficult to bind later on. Ideally, in this case you should pass a pointer to A, and even better A would have reference counting so that both caller and callee could own a reference to the same object at the same time... ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied?
2009/6/25 J. Michael Owen
> I can see what you're worried about here, but as a practical matter it is
> very useful to be able to pass stuff by reference, and copying it under the
> hood will break code. For instance, if the virtual functions modify the
> state of the parameters passed by reference it will just not function
> correctly when a copy is made. Or in the example I sent when copy
> constructors are not available it won't even compile!
>
> In Boost.Python this can be handled by wrapping the intermediate arguments
> with the boost::ref() function, which tells Boost.Python to pass the thing
> along as reference rather than making a copy and on my own head be it if I
> do something nefarious with that reference. I guess what I would like in
> this case is the equivalent of an "on my own head be it" option. This is
> kind of how I think of the pointer passing policy, like with the
> "caller_owns_return" option. Wouldn't it be reasonable to allow us to
> specify optionally that references should not be copied, but rather passed
> along natively? With the default policy still being the more memory safe
> option of doing the copy?
I can't really fault your logic.
I just thought of something that makes some sense. If the wrapper did
something like:
_wrap_my_virtual_method (A &a)
{
PyA *py_A;
py_A = PyObject_New(PyA, &PyA_Type);
py_A->obj = &a;
// --- code to call into Python (without consuming a reference to py_A)
---
py_A->obj = NULL;
Py_DECREF(py_A);
}
At the end of the virtual method call, if the called python code decided to
keep a reference to py_A, it will have a py_A with NULL C++ object and so
any method call on it will crash.
And this would not need any options. It's always best to do the smart thing
when possible without asking too many questions ;-)
>
> On Jun 24, 2009, at 3:31 PM, Gustavo Carneiro wrote:
>
> Sorry, but in this case the answer is no. The virtual method wrapper here
>> receives a parameter from C++ and has to create a Python wrapper for it.
>> The wrapper needs to own the object. The reason is that the Python method
>> that is called may decide to keep a reference to the object. If the Python
>> wrapper kept a "shared" reference to the same C++ object then it could at
>> some point in time be keeping a reference to an object when the caller has
>> already deallocated it.
>>
>> To summarize, the object copy is done because the alternative would be
>> memory unsafe.
>>
>> This is one case where a C++ API blindly designed without considering
>> language bindings may make it difficult to bind later on. Ideally, in this
>> case you should pass a pointer to A, and even better A would have reference
>> counting so that both caller and callee could own a reference to the same
>> object at the same time...
>>
>
> ___
> Cplusplus-sig mailing list
> Cplusplus-sig@python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied?
That sounds great to me!
On Jun 24, 2009, at 5:03 PM, Gustavo Carneiro wrote:
I can't really fault your logic.
I just thought of something that makes some sense. If the wrapper
did something like:
_wrap_my_virtual_method (A &a)
{
PyA *py_A;
py_A = PyObject_New(PyA, &PyA_Type);
py_A->obj = &a;
// --- code to call into Python (without consuming a reference to
py_A) ---
py_A->obj = NULL;
Py_DECREF(py_A);
}
At the end of the virtual method call, if the called python code
decided to keep a reference to py_A, it will have a py_A with NULL C+
+ object and so any method call on it will crash.
And this would not need any options. It's always best to do the
smart thing when possible without asking too many questions ;-)
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
[C++-sig] return_internal_reference & overriding methods in a class.
Hello,
I'm having trouble trying to override a method on a class exposed to
Python. I cannot modify the original class, so I need to wrap it to
change the behaviour of some methods to be Python compatible. The
classes look like the following:
struct DocumentWrap : Document, wrapper
{
DocumentWrap() : Document(){ };
DocumentWrap(const Document&) { };
wstring toString()
{
wstring w(_T("test"));
return w;
}
int test(){
return 1;
}
};
class_("Hits", init())
.def("doc", &Hits::doc, return_internal_reference<>());
class_("Document")
.def("toString", &DocumentWrap::toString)
.def("test", &DocumentWrap::test)
;
Hits::doc returns a Document. If I do:
>>> d = Document()
>>> d.test()
1
>>> d.toString()
"test"
I get the right answer. However, If I obtain the class through Hits I
get a reference to a c++ object of type Document (not DocumentWrap) so
trying to execute the methods yield:
Traceback (most recent call last):
File "t.py", line 10, in
d.test()
Boost.Python.ArgumentError: Python argument types in
Document.test(Document)
did not match C++ signature:
test(DocumentWrap {lvalue})
Traceback (most recent call last):
File "t.py", line 10, in
d.toString()
Boost.Python.ArgumentError: Python argument types in
Document.toString(Document)
did not match C++ signature:
toString(DocumentWrap {lvalue})
Is there any way to get python to understand the internal reference to
Document can be treated as a DocumentWrap?
Thanks in advance!
Best Regards,
--
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] return_internal_reference & overriding methods in a class.
I tried adding:
implicitly_convertible();
implicitly_convertible();
but it doesn't appear to change anything. I would apreciate any idea
on thes regard.
Thanks again!
On Thu, Jun 25, 2009 at 3:31 AM, Nicolas Lara wrote:
> Hello,
>
> I'm having trouble trying to override a method on a class exposed to
> Python. I cannot modify the original class, so I need to wrap it to
> change the behaviour of some methods to be Python compatible. The
> classes look like the following:
>
> struct DocumentWrap : Document, wrapper
> {
> DocumentWrap() : Document(){ };
> DocumentWrap(const Document&) { };
> wstring toString()
> {
> wstring w(_T("test"));
> return w;
> }
> int test(){
> return 1;
> }
> };
>
> class_("Hits", init())
> .def("doc", &Hits::doc, return_internal_reference<>());
> class_("Document")
> .def("toString", &DocumentWrap::toString)
> .def("test", &DocumentWrap::test)
> ;
>
> Hits::doc returns a Document. If I do:
>
d = Document()
d.test()
> 1
d.toString()
> "test"
>
> I get the right answer. However, If I obtain the class through Hits I
> get a reference to a c++ object of type Document (not DocumentWrap) so
> trying to execute the methods yield:
>
> Traceback (most recent call last):
> File "t.py", line 10, in
> d.test()
> Boost.Python.ArgumentError: Python argument types in
> Document.test(Document)
> did not match C++ signature:
> test(DocumentWrap {lvalue})
>
> Traceback (most recent call last):
> File "t.py", line 10, in
> d.toString()
> Boost.Python.ArgumentError: Python argument types in
> Document.toString(Document)
> did not match C++ signature:
> toString(DocumentWrap {lvalue})
>
> Is there any way to get python to understand the internal reference to
> Document can be treated as a DocumentWrap?
>
> Thanks in advance!
>
> Best Regards,
>
> --
> Nicolas Lara
> Linux user #380134
> http://nicolas-lara.blogspot.com/
> Public key id: 0x152e7713 at http://subkeys.pgp.net/
>
--
Nicolas Lara
Linux user #380134
http://nicolas-lara.blogspot.com/
Public key id: 0x152e7713 at http://subkeys.pgp.net/
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] return_internal_reference & overriding methods in a class.
On Thu, Jun 25, 2009 at 6:31 AM, Nicolas Lara wrote:
> Hello,
>
> I'm having trouble trying to override a method on a class exposed to
> Python. I cannot modify the original class, so I need to wrap it to
> change the behaviour of some methods to be Python compatible. The
> classes look like the following:
>
> struct DocumentWrap : Document, wrapper
> {
> DocumentWrap() : Document(){ };
> DocumentWrap(const Document&) { };
> wstring toString()
> {
> wstring w(_T("test"));
> return w;
> }
> int test(){
> return 1;
> }
> };
>
> class_("Hits", init())
> .def("doc", &Hits::doc, return_internal_reference<>());
> class_("Document")
> .def("toString", &DocumentWrap::toString)
> .def("test", &DocumentWrap::test)
> ;
>
> Hits::doc returns a Document. If I do:
>
d = Document()
d.test()
> 1
d.toString()
> "test"
>
> I get the right answer. However, If I obtain the class through Hits I
> get a reference to a c++ object of type Document (not DocumentWrap) so
> trying to execute the methods yield:
>
> Traceback (most recent call last):
> File "t.py", line 10, in
> d.test()
> Boost.Python.ArgumentError: Python argument types in
> Document.test(Document)
> did not match C++ signature:
> test(DocumentWrap {lvalue})
>
> Traceback (most recent call last):
> File "t.py", line 10, in
> d.toString()
> Boost.Python.ArgumentError: Python argument types in
> Document.toString(Document)
> did not match C++ signature:
> toString(DocumentWrap {lvalue})
>
> Is there any way to get python to understand the internal reference to
> Document can be treated as a DocumentWrap?
I don't think so. What you ask doesn't make sense.
The better solution would be to replace problematic function in another way:
struct Doc{
xyz** do_smth();
};
add new global function:
boost::python::list do_smth4py( Doc& d){
///call your function
}
and register it instead of the original function
class_< Doc>(...)
.def( "do_smth", &::do_smth4py )
The Python user will not note the difference.
HTH
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Function handling with Boost
Thomas Berg wrote:
> bp::scope sc(main);
> bp::def("function", myfunction, "function helpstring");
Now that looks interesting. I'll implement that and let you know if it
leads to any further problems. But it sounds like the perfect solution.
> Found this out by reading the source code of
> boost::python::detail::init_module, called by the BOOST_PYTHON_MODULE
> macro.
Good idea.
I'm really wondering how someone could develop such a mighty tool like
boost python and not even think about writing a useful documentation for it.
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
