[C++-sig] Python Exported Method that Takes a Boost::Function Object
I have a method that takes a boost function object as so
bool createTimer( boost::int64_t interval, boost::function< bool() >
function, bool recurring = false )
that I would like to call in python. I tried exporting the class in Py++
but it did not look like it does anything special with that argument.
However, when I try to use this from python I get
Boost.Python.ArgumentError: Python argument types in
createTimer( int, method, bool )
did not match C++ signature:
createTimer( unsigned long interval, class boost::function function, bool recurring=False)
that is with the export code
.def( "createTimer", &Class::createTimer, ( bp::arg( "interval" ), bp::arg(
"function" ), bp::arg( "recurring" ) = false ) )
Py++ recommends exporting it like
{ //::Class::createTimer
typedef ::boost::int32_t ( ::Class::*createTimer_function_type
)( ::boost::uint32_t,::boost::function< bool ()() >,bool ) ;
Class_exposer.def(
"createTimer"
, createTimer_function_type( &::Class::createTimer )
, ( bp::arg("interval"), bp::arg("function"),
bp::arg("recurring")=(bool)(false) ) );
}
which btw there is an error here in Py++, ::boost::function< bool ()() >
should be ::boost::function< bool () >
that code also fails in python with the same error.
I have seen the page here
http://wiki.python.org/moin/boost.python/HowTo#boost.functionobjects about
boost function objects but I do not think that is exactly what I am looking
for.
If there an easy way to get python to work with this function definition?
Or am I trying to do something stupid?
with this code I can do createTimer( 3, boost::bind( &longFunc, arg1, arg2 )
) which is extremely convenient. I would like to have the same ability in
python.
Thanks
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
[C++-sig] OpenMP and boost-python?
Hello group, I have C++ code which uses OpenMP to parallellize some algorithms. When wrapped with boost-python only one thread seems to run at a time, and there is no speedup (as seen with pure C++) when using multiple core machines. The FAQ says this is pretty much expected, but hints at a possible patch: http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/faq.html#threadsupport How difficult is it to make boost-python play nice with OpenMP? Anyone done it? Is this ever going to be a feature of boost-python, or always going to require special patching and hacking? thanks, Anders ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] OpenMP and boost-python?
On 02/01/2010 12:07 PM, Anders Wallin wrote: Hello group, I have C++ code which uses OpenMP to parallellize some algorithms. When wrapped with boost-python only one thread seems to run at a time, and there is no speedup (as seen with pure C++) when using multiple core machines. The FAQ says this is pretty much expected, but hints at a possible patch: http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/faq.html#threadsupport How difficult is it to make boost-python play nice with OpenMP? Anyone done it? Is this ever going to be a feature of boost-python, or always going to require special patching and hacking? I'm not sure in how much boost.python would actually need to care about this. Presumably, some OpenMP support needs to be provided by the runtime library (such as the thread management), so I would expect the Python binary (or whatever your main application is) to be responsible for initializing the right runtime library / support. What compiler are you using ? Does it provide some documentation on what needs to be done to activate OpenMP at runtime ? Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] OpenMP and boost-python?
Anders Wallin wrote: Hello group, I have C++ code which uses OpenMP to parallellize some algorithms. When wrapped with boost-python only one thread seems to run at a time, and there is no speedup (as seen with pure C++) when using multiple core machines. The FAQ says this is pretty much expected, but hints at a possible patch: http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/faq.html#threadsupport How difficult is it to make boost-python play nice with OpenMP? Anyone done it? Is this ever going to be a feature of boost-python, or always going to require special patching and hacking? Boost.MPI has python bindings, you could look at those for ideas. -t ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] OpenMP and boost-python?
>> How difficult is it to make boost-python play nice with OpenMP? Anyone >> done it? Is this ever going to be a feature of boost-python, or always >> going to require special patching and hacking? > I'm not sure in how much boost.python would actually need to care about > this. Presumably, some OpenMP support needs to be provided by the runtime > library (such as the thread management), so I would expect the Python binary > (or whatever your main application is) to be responsible for initializing > the right runtime library / support. > What compiler are you using ? Does it provide some documentation on what > needs to be done to activate OpenMP at runtime ? I am using GCC 4.4.1 The pure C++ code with OpenMP compiles and runs fine using many threads. There is almost a linear speedup of trivially parallel for-loops. When this same code is wrapped with boost-python and called from Python that speedup disappears, i.e. the Python GIL(?, or something similar), forces the C++ code to run in a single thread, or only one thread to execute at a time. MPI is clearly one option, but requires a re-write of the whole code, and results in code that does not resemble the single-thread code much. I would like to use OpenMP, since it's a "for dummies" way of parallellizing things. Any other suggestions for writing parallel C++ code which is called from Python ?? AW ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] OpenMP and boost-python?
On 02/01/2010 12:58 PM, Anders Wallin wrote: How difficult is it to make boost-python play nice with OpenMP? Anyone done it? Is this ever going to be a feature of boost-python, or always going to require special patching and hacking? I'm not sure in how much boost.python would actually need to care about this. Presumably, some OpenMP support needs to be provided by the runtime library (such as the thread management), so I would expect the Python binary (or whatever your main application is) to be responsible for initializing the right runtime library / support. What compiler are you using ? Does it provide some documentation on what needs to be done to activate OpenMP at runtime ? I am using GCC 4.4.1 The pure C++ code with OpenMP compiles and runs fine using many threads. There is almost a linear speedup of trivially parallel for-loops. When this same code is wrapped with boost-python and called from Python that speedup disappears, i.e. the Python GIL(?, or something similar), forces the C++ code to run in a single thread, or only one thread to execute at a time. I'm not convinced of this. It is true that there may only ever be one thread accessing the Python runtime (thus the GIL to enforce it). But you may certainly have as many threads doing other work as you want. And if your OpenMP-enabled code is pure C++, with no hooks into the Python runtime, then I don't see why that may not work. Again: If your OpenMP-using code and the Python-exposed API are well isolated, I don't think either needs to know about the other. Stefan -- ...ich hab' noch einen Koffer in Berlin... ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] OpenMP and boost-python?
> I'm not convinced of this. It is true that there may only ever be one thread > accessing the Python runtime (thus the GIL to enforce it). But you may > certainly have as many threads doing other work as you want. And if your > OpenMP-enabled code is pure C++, with no hooks into the Python runtime, then > I don't see why that may not work. > Again: If your OpenMP-using code and the Python-exposed API are well > isolated, I don't think either needs to know about the other. Thanks, in fact I now got my trivial example working with a good speedup! For the real code I need to look at isolating C++/Python. AW ___ Cplusplus-sig mailing list Cplusplus-sig@python.org http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Python Exported Method that Takes a Boost::Function Object
On Mon, Feb 1, 2010 at 5:22 PM, Charles Solar wrote:
> I have a method that takes a boost function object as so
>
> bool createTimer( boost::int64_t interval, boost::function< bool() >
> function, bool recurring = false )
>
> that I would like to call in python. I tried exporting the class in Py++
> but it did not look like it does anything special with that argument.
Py++ generates the wrong code in this situation. It definitely could
do better job.
>
> which btw there is an error here in Py++, ::boost::function< bool ()() >
> should be ::boost::function< bool () >
Hmm. This is how gccxml reports the class name. I will take a look on this bug.
> that code also fails in python with the same error.
>
> I have seen the page here
> http://wiki.python.org/moin/boost.python/HowTo#boost.functionobjects about
> boost function objects but I do not think that is exactly what I am looking
> for.
>
> If there an easy way to get python to work with this function definition?
> Or am I trying to do something stupid?
No, I would call this a "challenge".
The following is just an untested idea:
1. create the following class:
class callback_wrapper_t{
callback_wrapper_t( boost::python::object callable ){ store it in the class}
bool operator()( bool ){
call the function
}
}
2. create a wrapper
bool createTimerWrapper( boost::int64_t interval,
boost::python::object function, bool recurring = false ){
createTimer( interval, boost::function( callback_wrapper_t(
function ) ), recurring )
}
3. Register the createTimerWrapper
I think this could work. May be there is a better way. If you have a
lot of functions like this and the idea works, I can help you to
generate it with Py++.
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Python Exported Method that Takes a Boost::Function Object
Nice! A little bit of cleaning up and editing and that worked like a charm,
thanks
Here is what I had to do:
struct timer_func_wrapper_t
{
timer_func_wrapper_t( bp::object callable ) : _callable( callable ) {}
bool operator()()
{
PyGILState_STATE gstate = PyGILState_Ensure();
bool ret = _callable();
PyGILState_Release( gstate );
return ret;
}
bp::object _callable;
};
boost::int32_t createTimerWrapper( Class* class, boost::uint64_t interval,
bp::object function, bool recurring = false )
{
return class->createTimer( interval, boost::function(
timer_func_wrapper_t( function ) ), recurring );
}
and then in the class def:
.def( "createTimer", &createTimerWrapper, ( bp::arg( "interval" ), bp::arg(
"function" ), bp::arg( "recurring" ) = false ) )
I think I kind of cheated with the wrapper function, the this pointer just
fills in the Class* hole.. I was actually surprised that it worked so
easily.
Anyway now I can do magic like
import MyLib
import time
def callMePls():
print( "Hello world" )
return True
class = MyLib.Class()
class.createTimer( 3, callMePls )
time.sleep( 1 )
Works great!
About the Py++ stuff though, I mainly use Py++ to see how I should export
something I am unsure of. Which is why I mentioned that Py++ did not do
anything special, since I was unsure how to export this in the first place.
I type up and maintain my real export code though, so no rush on Py++
support, although I bet its as easy as generating a patch or something..
Anyway, great tool, great advice, thanks again.
On Mon, Feb 1, 2010 at 1:40 PM, Roman Yakovenko
wrote:
> On Mon, Feb 1, 2010 at 5:22 PM, Charles Solar
> wrote:
> > I have a method that takes a boost function object as so
> >
> > bool createTimer( boost::int64_t interval, boost::function< bool() >
> > function, bool recurring = false )
> >
> > that I would like to call in python. I tried exporting the class in Py++
> > but it did not look like it does anything special with that argument.
>
> Py++ generates the wrong code in this situation. It definitely could
> do better job.
>
> >
> > which btw there is an error here in Py++, ::boost::function< bool ()() >
> > should be ::boost::function< bool () >
>
> Hmm. This is how gccxml reports the class name. I will take a look on this
> bug.
>
> > that code also fails in python with the same error.
> >
> > I have seen the page here
> > http://wiki.python.org/moin/boost.python/HowTo#boost.functionobjectsabout
> > boost function objects but I do not think that is exactly what I am
> looking
> > for.
> >
> > If there an easy way to get python to work with this function definition?
> > Or am I trying to do something stupid?
>
> No, I would call this a "challenge".
>
> The following is just an untested idea:
>
> 1. create the following class:
>
> class callback_wrapper_t{
>callback_wrapper_t( boost::python::object callable ){ store it in the
> class}
>
> bool operator()( bool ){
>call the function
> }
> }
>
> 2. create a wrapper
>
> bool createTimerWrapper( boost::int64_t interval,
> boost::python::object function, bool recurring = false ){
>createTimer( interval, boost::function( callback_wrapper_t(
> function ) ), recurring )
> }
>
> 3. Register the createTimerWrapper
>
> I think this could work. May be there is a better way. If you have a
> lot of functions like this and the idea works, I can help you to
> generate it with Py++.
>
>
> --
> Roman Yakovenko
> C++ Python language binding
> http://www.language-binding.net/
> ___
> Cplusplus-sig mailing list
> Cplusplus-sig@python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
___
Cplusplus-sig mailing list
Cplusplus-sig@python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig
