[C++-sig] compiling python extensions to work for multiple versions of python
I am working on compiling a python extension (specifically libtorrent-rasterbar), and would like to wind up with a single executable that I can use on multiple versions of MacOS X (specifically 10.5 and 10.6) without having to force the version of python used on each version (10.5 has 2.5, 10.6 has 2.5 and 2.6, but defaults to 2.6). When I compile using bjam: bjam debug-symbols=on dht-support=on boost=source architecture=combined toolset=darwin address-model=32_64 (with BOOST_BUILD and BOOST_ROOT set approrpiatly) I get a working binary (libtorrent.so) that works great on the platform I am working on, and is a "Universal binary" (ppc/ppc64/i386/x86_64), but when I try to "import" it on the other platform without specifying the version of python I get: Fatal Python error: Interpreter not initialized (version mismatch?) Abort trap Forcing the version of python (so forcing 10.6 to use 2.5) allows the module to be imported and used without problem. After an evening of searching around on mailing lists I almost came to the conclusion that it is not possible to compile python modules that work with multiple versions of python... except that I have built a version of the MySQL module that does exactly that. It runs without complaint on 10.5 (ppc and intel), and 10.6 (intel only), on both python 2.5 and 2.6. So my questions: 1) Is this a limitation of Boost.Python? A trick from MySQL that I can emulate? Am I missing something obvious? I have tried a large number of variations in build commands using combinations of the following: link=shared --layout=versioned --enable-framework --build-type=complete link=shared,static macosx-version=10.5 macosx-version-min=10.5 2) If this is not a possibility with Boost.Python-built modules, what would be the best work-around. I can probably build separate modules with the target python version in their names, and play games with importing them. But that is obviously a less-desired version of things, since it would require: a) multiple 60Meg binaries, and b) having 2 at the moment, and presumably a third when 10.7 ships. Note: I am using Boost 1.46.1 and libtorrent-rasterbar 0.15.6, the latest releases of both as of a couple of days ago. -- Karl Kuehn [email protected] ___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] compiling python extensions to work for multiple versions of python
On 5/25/11 17:21 , Karl Kuehn wrote: 1) Is this a limitation of Boost.Python? A trick from MySQL that I can emulate? Am I missing something obvious? I have tried a large number of variations in build commands using combinations of the following: This is a limitation of python, and I would be very afraid of using a MySQL module if it was compiled for another python version, even if it does load. Python versions do not offer a stable ABI: it will change depending on python version, OS, python configure options, compile flags, etc. You should always try to compile Python modules against the specific Python version you want to use. Wichert. ___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] compiling python extensions to work for multiple versions of python
On May 25, 2011, at 8:25 AM, Wichert Akkerman wrote: >> 1) Is this a limitation of Boost.Python? A trick from MySQL that I can >> emulate? Am I missing something obvious? I have tried a large number of >> variations in build commands using combinations of the following: > > This is a limitation of python, and I would be very afraid of using a MySQL > module if it was compiled for another python version, even if it does load. > Python versions do not offer a stable ABI: it will change depending on python > version, OS, python configure options, compile flags, etc. You should always > try to compile Python modules against the specific Python version you want to > use. Thanks for the pointer. Obviously not what I wanted to hear, but valuable none-the-less. I will just have to go about compiling the modules with separate names, and just swallow the extra bandwidth required to have multiple binaries. From what you say, it is odd that the MySQL is working as well as it is then. A single compiled binary has been working very reliably for more than 2 years running on 10.4 (since dropped), 10.5, and 10.6… all of which default to different versions of python. Since this is a testing system we would have glaring holes in our data if it was not working reliably. Ah, well… off to start up a few compiles… Thanks much! -- Karl Kuehn [email protected]___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
[C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions
Hi all,
Hopefully this is the correct way to post a question, If not I apologize as I
haven't used mailing lists before.
Anyways, here is my problem:
I have some class functions that takes in parameters by pointer. However, when
I expose and override this class/these functions in boost::python, any changes
I make to the parameters are not reflected back to the original, meaning it
isn't being passed by pointer as I expect. I previously had them as
pass-by-reference, but the same problem still occurred. For an example of what
I mean, here's some of my code with unimportant functions and details omitted.
class StatusEffect
{
public:
virtual void TickCharacter(Character::BaseCharacter* character,
Battles::BattleField *field, int ticks = 1);
}
//In my Scripting Code header file
struct StatusEffectWrapper : Game::StatusEffects::StatusEffect
{
virtual void TickCharacter(Game::Character::BaseCharacter* character,
Game::Battles::BattleField *field, int ticks = 1);
virtual void TickCharacterDefault(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks = 1);
}
//In the actual scripting code
void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
call_method(self, "TickCharacter", character, field, ticks);
}
void
StatusEffectWrapper::TickCharacterDefault(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
this->StatusEffect::TickCharacter(character, field, ticks);
}
class_ >("StatusEffect")
.def("TickCharacter", &StatusEffect::TickCharacter,
&StatusEffectWrapper::TickCharacterDefault)
and finally in my Python file, I try and do something like this
class StatCreepEffect(StatusEffect):
def __init__(self, name, stat_, rate_, descript, uid, turns, prior,
canrefresh, poseffect, cats, conds, flgs, sep, pers):
StatusEffect.__init__(self, name, descript, uid, turns, prior,
canrefresh, poseffect, cats, conds, flgs, sep, pers)
self.rate = rate_
self.stat = stat_
def TickCharacter(self, Character, Field, Ticks):
if (self.stat == Stat.HP):
Character.SetHP(int(round(Character.GetHP() +
Character.GetInBattleStat(self.stat) * self.rate)))
print("HP Regenerated on character " + Character.GetName() + "
New HP: " + str(Character.GetHP()))
elif (self.stat == Stat.MP):
Character.SetMP(int(round(Character.GetMP() +
Character.GetInBattleStat(self.stat) * self.rate)))
elif (self.stat == Stat.SP):
Character.SetSP(int(round(Character.SetSP() +
Character.GetInBattleStat(self.stat) * self.rate)))
Regen = StatCreepEffect("Regen", Stat.HP, 0.05, "Regenerates a small amount
of HP every turn", PrimaryEngine.GetUID(), 14, 0, True, True,
StatusCategories.SC_Regen, 0, StatusFlags.TickEffect, True,
StatusPersistence.SP_Timer_Cure)
SELibrary.AddPythonEffect(Regen);
If it matters, the AddPythonEffect is defined as follows:
void StatusEffectsLibrary::AddPythonEffect(boost::shared_ptr
effect)
{
if (effect.get() != nullptr)
{
NameToSEMap[effect->GetName()] = effect;
IDToSEMap[effect->GetUID()] = effect;
}
}
I call the effect like so in my C++ code
StatusEffect* regen =
game.GetSELibrary().GetStatusEffect(std::string("Regen")); //This simply does a
lookup to the NameToSEMap inside the StatusEffectLibrary and does a .get() on
the shared_ptr
while(character->GetHP() < 600)
{
regen->TickCharacter(character, field, 1);
std::cout << "Character HP: " << character->GetHP() << std::endl;
}
The script compiles and the effect is added correctly. The print line in the
python script prints the correct value. The character object starts with 400 HP
and the print line outputs 430. However, the std::cout line in my C++ code
always outputs 400, meaning the extra 30 HP added by the script is not
reflected back in the original object. For clarity, here's the console output
from the above C++ code fragment:
HP Regenerated on character Arian New HP: 430
Character HP: 400
I've tried adding boost::non_copyable to the class wrapper but to no effect.
I've replaced my pointers with pass by reference to no effect. I've switched
between shared_ptr, auto_ptr, and a straight reference for the HeldType to no
effect. I can confirm using shared_ptrs over regular pointers in the function
does work, but I'd prefer to avoid using them unless there really isn't another
way as it would be tedious and time consuming to do so.
Can someone tell me how to get the behaviour I expect?
To be clear, I'm not looking for the python scri
Re: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions
On 05/25/2011 02:26 PM, Jay Riley wrote:
I have some class functions that takes in parameters by pointer.
However, when I expose and override this class/these functions in
boost::python, any changes I make to the parameters are not reflected
back to the original, meaning it isn't being passed by pointer as I
expect. I previously had them as pass-by-reference, but the same problem
still occurred. For an example of what I mean, here's some of my code
with unimportant functions and details omitted.
I note two spots that bear further investigation.
The most likely source is here:
//In the actual scripting code
void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
call_method(self, "TickCharacter", character, field, ticks);
}
I think there's a very good chance that call_method is copying your
character argument when it converts it to Python. Boost.Python is
extra-careful about making sure the C++ objects it puts inside Python
objects don't turn into dangling references, and it has no way to
control the lifetime of a raw pointer or reference. I think you could
probably fix this by using shared_ptr here, because
Boost.Python does know how to convert that safely to Python without
copying the pointee.
If that fails, look here:
.def("TickCharacter", &StatusEffect::TickCharacter,
&StatusEffectWrapper::TickCharacterDefault)
I've personally never seen this invocation of .def, with two member
function pointer arguments. That doesn't necessarily mean it's
incorrect, because overloading virtual methods in Python is feature I
almost never use, and it seems like there are a half-dozen ways to do it
and I never know which one is considered best (perhaps someone else on
this list does?). But you might want to make sure this does what you
expect.
HTH
Jim Bosch
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functio
You may want to try having Py++ do an example wrapping of a header file or
two, thats always what I did while I was wrapping parts of my code that I
was not sure about the right conventions in boost python.
That being said, I have also never tried passing a pointer to python for
python to edit and give back. That kind of design kind of stinks to me, I
think a better solution would be to write an interface for a python visitor
that you could apply to your C++ game objects, but that is not important
here.
There are a couple of suggestions I can make, if you switch back to passing
by reference, you need to remember to use boost::ref() for the arguments
you do not want copied.
Eg:
call_method(self, "TickCharacter", boost::ref( character ),
boost:;ref( field ), ticks)
I use a constant ref in some of my functions and I found I always need to
tell boost about it other wise the object does not work right.
Also, why are you using call_method like this?
I thought the right way for calling python functions was
if( bp::override func = bp::get_override( "TickCharacter" ) )
func( boost::ref( character ), boost::ref( field ), ticks );
That is what I do anyway and it was in the docs at some point. Maybe the
difference in calling convention is causing boost python to not accurately
pass your pointers?
Charles
On , Jim Bosch wrote:
On 05/25/2011 02:26 PM, Jay Riley wrote:
I have some class functions that takes in parameters by pointer.
However, when I expose and override this class/these functions in
boost::python, any changes I make to the parameters are not reflected
back to the original, meaning it isn't being passed by pointer as I
expect. I previously had them as pass-by-reference, but the same problem
still occurred. For an example of what I mean, here's some of my code
with unimportant functions and details omitted.
I note two spots that bear further investigation.
The most likely source is here:
//In the actual scripting code
void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
call_method(self, "TickCharacter", character, field, ticks);
}
I think there's a very good chance that call_method is copying your
character argument when it converts it to Python. Boost.Python is
extra-careful about making sure the C++ objects it puts inside Python
objects don't turn into dangling references, and it has no way to control
the lifetime of a raw pointer or reference. I think you could probably
fix this by using shared_ptr here, because Boost.Python does know how to
convert that safely to Python without copying the pointee.
If that fails, look here:
.def("TickCharacter", &StatusEffect::TickCharacter,
&StatusEffectWrapper::TickCharacterDefault)
I've personally never seen this invocation of .def, with two member
function pointer arguments. That doesn't necessarily mean it's incorrect,
because overloading virtual methods in Python is feature I almost never
use, and it seems like there are a half-dozen ways to do it and I never
know which one is considered best (perhaps someone else on this list
does?). But you might want to make sure this does what you expect.
HTH
Jim Bosch
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions
Your reply actually helped alot. I hadn't considered that the call_method might
have been being used incorrectly. Turns out it was. After looking up the
documentation for call_method, by changing
void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
call_method(self, "TickCharacter", character, field, ticks);
}
to this
void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
character, Game::Battles::BattleField *field, int ticks)
{
call_method(self, "TickCharacter", ptr(character), ptr(field),
ticks);
}
I get the behave I expected. Thanks!
> Date: Wed, 25 May 2011 15:35:31 -0700
> From: [email protected]
> To: [email protected]
> Subject: Re: [C++-sig] boost::python - how to pass by reference/pointer to
> python overriden class functions
>
> On 05/25/2011 02:26 PM, Jay Riley wrote:
> > I have some class functions that takes in parameters by pointer.
> > However, when I expose and override this class/these functions in
> > boost::python, any changes I make to the parameters are not reflected
> > back to the original, meaning it isn't being passed by pointer as I
> > expect. I previously had them as pass-by-reference, but the same problem
> > still occurred. For an example of what I mean, here's some of my code
> > with unimportant functions and details omitted.
> >
>
> I note two spots that bear further investigation.
>
> The most likely source is here:
>
> > //In the actual scripting code
> > void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter*
> > character, Game::Battles::BattleField *field, int ticks)
> > {
> > call_method(self, "TickCharacter", character, field, ticks);
> > }
>
> I think there's a very good chance that call_method is copying your
> character argument when it converts it to Python. Boost.Python is
> extra-careful about making sure the C++ objects it puts inside Python
> objects don't turn into dangling references, and it has no way to
> control the lifetime of a raw pointer or reference. I think you could
> probably fix this by using shared_ptr here, because
> Boost.Python does know how to convert that safely to Python without
> copying the pointee.
>
> If that fails, look here:
>
> > .def("TickCharacter", &StatusEffect::TickCharacter,
> > &StatusEffectWrapper::TickCharacterDefault)
>
> I've personally never seen this invocation of .def, with two member
> function pointer arguments. That doesn't necessarily mean it's
> incorrect, because overloading virtual methods in Python is feature I
> almost never use, and it seems like there are a half-dozen ways to do it
> and I never know which one is considered best (perhaps someone else on
> this list does?). But you might want to make sure this does what you
> expect.
>
> HTH
>
> Jim Bosch
> ___
> Cplusplus-sig mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
