[C++-sig] problem with boost::python and pickling
I've run into some problem i do not understand. Consider following example,
which is a slightly modified pickle2.cpp from boost's libs/python/test/:
#include
#include
#include
#include
#include
namespace boost_python_test {
struct dummy // my addition
{
};
// A friendly class.
class world
{
public:
world(const std::string& country) : secret_number(0) {
this->country = country;
}
world(dummy) {}; // my addition
std::string greet() const { return "Hello from " + country + "!"; }
std::string get_country() const { return country; }
void set_secret_number(int number) { secret_number = number; }
int get_secret_number() const { return secret_number; }
private:
std::string country;
int secret_number;
};
struct world_pickle_suite : boost::python::pickle_suite
{
static
boost::python::tuple
getinitargs(const world& w)
{
using namespace boost::python;
return make_tuple(dummy()); // change to use dummy instead of string
}
static
boost::python::tuple
getstate(const world& w)
{
using namespace boost::python;
return make_tuple(w.get_secret_number());
}
static
void
setstate(world& w, boost::python::tuple state)
{
using namespace boost::python;
boost::python::object lenRes = state.attr("__len__")(); // old
boost
int const stateLen = boost::python::extract(lenRes);
if (stateLen != 1)
{
PyErr_SetObject(PyExc_ValueError,
("expected 1-item tuple in call to __setstate__; got
%s"
% state).ptr()
);
throw_error_already_set();
}
long number = extract(state[0]);
if (number != 42)
w.set_secret_number(number);
}
};
}
BOOST_PYTHON_MODULE(pickle2_ext)
{
boost::python::class_< boost_python_test::dummy>( "dummy",
boost::python::no_init );
boost::python::class_(
"world", boost::python::init())
.def( boost::python::init< std::string const & >())
.def("greet", &boost_python_test::world::greet)
.def("get_secret_number", &boost_python_test::world::get_secret_number)
.def("set_secret_number", &boost_python_test::world::set_secret_number)
.def_pickle(boost_python_test::world_pickle_suite())
;
}
Code is compiled with gcc41, boost 1.33.1; compile options that might be
relevant:
-pthread -fno-strict-aliasing -O0 -fno-inline -g0 -m64 -fPIC
then i use follwing test:
import pickle2_ext
import pickle
a = pickle2_ext.world("asd")
a.set_secret_number(54)
print a.get_secret_number()
pickle.dump(a, open('a.p', 'w'), pickle.HIGHEST_PROTOCOL)
r = pickle.load(open('a.p'))
print r.get_secret_number()
This test yields (python 2.4):
54
Traceback (most recent call last):
File "test.py", line 7, in ?
r = pickle.load(open('a.p'))
File "/usr/lib64/python2.4/pickle.py", line 1390, in load
return Unpickler(file).load()
File "/usr/lib64/python2.4/pickle.py", line 872, in load
dispatch[key](self)
File "/usr/lib64/python2.4/pickle.py", line 1153, in load_reduce
value = func(*args)
Boost.Python.ArgumentError: Python argument types in
world.__init__(world, dummy)
did not match C++ signature:
__init__(_object*, std::string)
__init__(_object*, boost_python_test::dummy)
The original example, with std::string as a constructor parameter, works fine.
What do i miss?
regards,
Jakub Żytka
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] problem with boost::python and pickling
On 07/02/10 15:42, Jakub Zytka wrote:
Ok, on newer boost (1.43) error message pointed the actual problem:
even for such simple dummy class some pickling support must be provided, eg.
struct dummy_pickle_suite : boost::python::pickle_suite
{
static
boost::python::tuple
getinitargs(const dummy& w)
{
return boost::python::tuple();
}
static
boost::python::tuple
getstate(const dummy& w)
{
using namespace boost::python;
return boost::python::tuple();
}
static
void
setstate(dummy&, boost::python::tuple)
{
}
};
I was mislead by this ArgumentError...
regards,
Jakub Żytka
I've run into some problem i do not understand. Consider following example,
which is a slightly modified pickle2.cpp from boost's libs/python/test/:
#include
#include
#include
#include
#include
namespace boost_python_test {
struct dummy // my addition
{
};
// A friendly class.
class world
{
public:
world(const std::string& country) : secret_number(0) {
this->country = country;
}
world(dummy) {}; // my addition
std::string greet() const { return "Hello from " + country + "!"; }
std::string get_country() const { return country; }
void set_secret_number(int number) { secret_number = number; }
int get_secret_number() const { return secret_number; }
private:
std::string country;
int secret_number;
};
struct world_pickle_suite : boost::python::pickle_suite
{
static
boost::python::tuple
getinitargs(const world& w)
{
using namespace boost::python;
return make_tuple(dummy()); // change to use dummy instead of string
}
static
boost::python::tuple
getstate(const world& w)
{
using namespace boost::python;
return make_tuple(w.get_secret_number());
}
static
void
setstate(world& w, boost::python::tuple state)
{
using namespace boost::python;
boost::python::object lenRes = state.attr("__len__")(); // old boost
int const stateLen = boost::python::extract(lenRes);
if (stateLen != 1)
{
PyErr_SetObject(PyExc_ValueError,
("expected 1-item tuple in call to __setstate__; got %s"
% state).ptr()
);
throw_error_already_set();
}
long number = extract(state[0]);
if (number != 42)
w.set_secret_number(number);
}
};
}
BOOST_PYTHON_MODULE(pickle2_ext)
{
boost::python::class_< boost_python_test::dummy>( "dummy",
boost::python::no_init );
boost::python::class_(
"world", boost::python::init())
.def( boost::python::init< std::string const & >())
.def("greet", &boost_python_test::world::greet)
.def("get_secret_number", &boost_python_test::world::get_secret_number)
.def("set_secret_number", &boost_python_test::world::set_secret_number)
.def_pickle(boost_python_test::world_pickle_suite())
;
}
Code is compiled with gcc41, boost 1.33.1; compile options that might be
relevant:
-pthread -fno-strict-aliasing -O0 -fno-inline -g0 -m64 -fPIC
then i use follwing test:
import pickle2_ext
import pickle
a = pickle2_ext.world("asd")
a.set_secret_number(54)
print a.get_secret_number()
pickle.dump(a, open('a.p', 'w'), pickle.HIGHEST_PROTOCOL)
r = pickle.load(open('a.p'))
print r.get_secret_number()
This test yields (python 2.4):
54
Traceback (most recent call last):
File "test.py", line 7, in ?
r = pickle.load(open('a.p'))
File "/usr/lib64/python2.4/pickle.py", line 1390, in load
return Unpickler(file).load()
File "/usr/lib64/python2.4/pickle.py", line 872, in load
dispatch[key](self)
File "/usr/lib64/python2.4/pickle.py", line 1153, in load_reduce
value = func(*args)
Boost.Python.ArgumentError: Python argument types in
world.__init__(world, dummy)
did not match C++ signature:
__init__(_object*, std::string)
__init__(_object*, boost_python_test::dummy)
The original example, with std::string as a constructor parameter, works fine.
What do i miss?
regards,
Jakub Żytka
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
[C++-sig] pyplusplus bug related to boost::equality_comparable (and possibly other operators as well)
Actually, I haven't had time to investigate it deeper yet, but it seems to be a
pyplusplus's bug. I also haven't found anything relevant on the net, so here we go:
Test case:
class to be exposed (file a.h)
#include
class A: boost::equality_comparable
{
public:
A(int a): a(a) {};
bool operator==(A const & other) const
{
return this->a == other.a;
}
private:
int a;
};
generator used (gen.py):
from pyplusplus import module_builder
mb = module_builder.module_builder_t(files = ['a.h'])
mb.decl('alloca').exclude()
mb.build_code_creator( module_name = 'testA')
mb.write_module('testA.cpp')
file generated:
// This file has been generated by Py++.
#include "boost/python.hpp"
#include "a.h"
namespace bp = boost::python;
BOOST_PYTHON_MODULE(testA){
{ //::A
typedef bp::class_< A > A_exposer_t;
A_exposer_t A_exposer = A_exposer_t( "A", bp::init< int >((
bp::arg("a") )) );
bp::scope A_scope( A_exposer );
bp::implicitly_convertible< int, A >();
A_exposer.def( bp::self == bp::self );
// OOPS! something is missing here
}
}
C++ class test:
#include
#include "a.h"
using namespace std;
int main()
{
A a1(3);
A a2(3);
cout << "a1 == a2: " << (a1 == a2) << endl;
cout << "a1 != a2: " << (a1 != a2) << endl;
return 0;
}
C++ class test result:
a1 == a2: 1
a1 != a2: 0
python test:
>>> import testA
>>> a1 = testA.A(3)
>>> a2 = testA.A(3)
>>> a1 == a2
True
>>> a1 != a2
True # drat!
The reason: missing A_exposer.def( bp::self != bp::self ); in the generated
code;
If any of you think it is easy to fix in pyplusplus (Roman, perhaps you're the
one to know best?) I may try. I'm also considering a workaround of checking
bases for all the classes and add missing "self != self" for all those which
derive from equality_comparable;
Perhaps there is the same problem with less_than_comparable and others.
regards,
Kuba
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] pyplusplus bug related to boost::equality_comparable (and possibly other operators as well)
On 09/09/10 20:08, Roman Yakovenko wrote: I don't think fixing py++ is the right approach. I'm also considering a workaround of checking bases for all the classes and add missing "self != self" for all those which derive from equality_comparable; Perhaps there is the same problem with less_than_comparable and others. I would implement the following solution: 1. Introduce new "def_visitor" ( http://www.boost.org/doc/libs/1_44_0/libs/python/doc/v2/def_visitor.html ) - so it will register all member function from the appropriate class In our case, equality_comparable, the visitor will register == and != ( this code could be contributed later to the Boost.Python ) 2. Apply that visitors on all relevant classes using py++ Does it help? Partially ;) I don't understand the concept with a new visitor. Creating a visitor for eg. equality_comparable doesn't solve the problem in general (and btw, equality_comparable doesn't provide member functions) I think all the needed visitors are already there (eg. boost::python::self != boost::python::self). The problem is that pyplusplus doesn't use them (or - to be specific - it *sometimes* doesn't use them). Why I consider it as a py++ bug? The problem is *sometimes* py++ adds some visitors to a class, and sometimes not. I suspect (not 100% verified) it doesn't work if an operator is declared as a non member function. I haven't found such warning in the docs, thus my surprise. If it is there, well, my fault. It escaped me. The uncertainty about operator== and operator!= is very unfortunate, because in python you have == or != by default, and in python code you don't know whether you're using pythonic comparison or the one you've defined in C++. So perhaps it is at least worth pointing out in docs (if it is not there, and didn't escape ;). I'd also add a tip that for each class_ one may (or should) define __cmp__ to raise some fancy exception in order to avoid problems as below: b1 = BoundClass(...) b2 = BoundClass(...) b1 == b2 # are we using pythonic pointer comparison or C++ defined one? If __cmp__ is defined to raise exception: 1. if operator== is defined in C++ and correct visitor is applied then python's __eq__ will be used, and comparison works as expected (that is: as in C++) 2. if operator== is not defined or no visitor has been applied (__eq__ is not defined) __cmp__ will be called and exception will happen. There will be no need to trace bugs that occurred in seemingly impossible branches. In my case I applied point 2 as a workaround: I look for all the operators (member or not) and apply related visitors on the classes that are parameters of these operators. This is far for ideal, but works in most cases. Having exception raising __cmp__ I'm able to spot problems immediately. regards, Kuba ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] gtkglext & mesa = __cxa_allocate_exception
On 09/14/10 21:18, Leonard Ritter wrote: On Tue, Sep 14, 2010 at 9:12 PM, Leonard Ritter wrote: it seems that when gtk.glext is imported in python, and a boost python module hits a StopIteration exception (as it is usual for generators), the exception is not caught and the app segfaults. that is, unless OpenGL.GL is imported before gtk.glext - in this case, no segfaults; it's the translation of a sites suggestion to use "env LD_PRELOAD=libGL.so python script.py", which is essentially the same thing. i wonder what kind of voodoo is at work here :) Last time I run into this kind of problem it was because python was using different memory allocator than the library I was importing (that is: the library was linked against our slightly modified tcmalloc) ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] TypeError: No to_python (by-value) converter found for C++ type: class std::basic_ostream >
> I get this error:
> *TypeError: No to_python (by-value) converter found for C++ type: class
> std::basic_ostream >*
>
> This is the python function
> *def serialise(self, gameobj, file):
> if(gameobj in self.components):
> file.write("HEY", 3)*
>
> What's the problem here? Is is because the ostream argument is an reference?
The problem is you are trying to copy a noncopyable object:
return class_,
boost::noncopyable>("ostream", no_init)
ostream is noncopyable
.def("write", write ,
return_value_policy())
but you define write to return copy (of ostream)
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] TypeError: No to_python (by-value) converter found for C++ type: class std::basic_ostream >
On 09/29/10 14:11, Simon W wrote:
Thank you for the answer. Is there any way I can prevent it from copying the
stream object even though std::ostream::write() returns a std::ostream& ?
You can use other return policy, or define one on your own.
http://www.boost.org/doc/libs/1_44_0/libs/python/doc/tutorial/doc/html/python/functions.html#python.call_policies
But honestly, I do not think it is the correct way to go. Read below.
> I'm trying to serialise some data. I have a c++ class:
> class ostream
> {
> ...
> could to the c++ way and add a integer before the byte stream that reveal how
> big it is. But How can i serialise an int from python that takes up constant
> "byte-size" for an integer between lets say 0-1!?
Everything can be done, but I have a different question:
I haven't followed all your previous emails, but do you *really* need to
implement serialization code in python?
Somehow I feel it is very weird that on one hand you want to have serialization
code in python, and yet use ostream.
What would be wrong with coding serialization purely in C++, and then providing
appropriate pickle suite to python:
struct MyPickleSuite
{
// ...
boost::python::tuple MyPickleSuite::getstate(MyClass & object)
{
std::ostringstream oStream(ios_base::binary);
serialize(oStream, object);
return boost::python::make_tuple(oStream.str());
}
void MyPickleSuite::setstate(MyClass & object, boost::python::tuple state)
{
// verify tuple correctness etc.
std::string serializedData = extract(state[0]);
std::istringstream iStream(serializedData, ios_base::binary);
serialize(iStream, object); // NOTE: you can use the same code for serialization
and deserialization. stream type should determine actual behavior
}
}
That way you do not have to expose ostream to python at all. In fact if you
wanted to switch to another type of stream (imagine that you have a custom
stream implementation, which provides eg. type checking) it is cheap. You only
change a few functions in your pickle suite, nothing more.
Even if you do really need to define serialization in python I see no reason to
play with streams. I'd rather had a wrapper which hides underlying
implementation (eg. std::streams) and operates on strings.
Please reconsider your design.
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] help on abstract class export
On 04/29/11 09:14, zeb wrote:
mb.free_fun('createWorker').call_policies =
call_policies.return_value_policy(call_policies.return_pointee_value)
you do not want to have *value* of an abstract class.
I found that if the IWorker class is not abstract, for example:
class IWorker {
public:
virtual int getData() { return 0; }
};
Python doesn't report that error, but the result of the following code is
not expected(I think it should be 100).
You expectancy is wrong. If you keep IWorker by value there will be no dynamic
dispatch.
This is not related to py++, python or boost::python. Its just the same as in
c++. If you wrote a c++ function that takes IWorker parameter by value you'd get
the same behavior.
Basic rule of thumb: if you're dealing with inheritance, do not pass object by
value.
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
