Hi Renato,
Okay, I have installed Python 2.4.2 with gcc 4.1.2 on Linux and it's
crashing in the same place as on Solaris.
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x41074940 (LWP 12004)]
0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
(gdb) where
#0 0x000000307740c850 in sem_post () from /lib64/libpthread.so.0
#1 0x000000308b2b78e9 in PyThread_release_lock ()
from /usr/lib64/libpython2.4.so.1.0
#2 0x00002b46531d9f82 in Ticker::operator() (this=0xbd0f478) at
ticker.cc:28
#3 0x00002b46531d9fb0 in boost::detail::thread_data<Ticker>::run (
this=0xbd0f370)
at /opt/atm/include/boost-1_39/boost/thread/detail/thread.hpp:56
#4 0x00002b46533ee14b in thread_proxy ()
from /opt/atm/lib64/libboost_thread-gcc41-mt-1_39.so.1.39.0
#5 0x0000003077406367 in start_thread () from /lib64/libpthread.so.0
#6 0x00000030768d2f7d in clone () from /lib64/libc.so.6
With the global interpret lock added my code now looks like this...
#include <boost/python.hpp>
#include <boost/thread/thread.hpp>
#include <boost/thread/xtime.hpp>
using namespace boost::python;
class Ticker
: public wrapper<Ticker>
{
private:
bool run_;
volatile bool * running_;
boost::thread * thread_;
boost::xtime xt_;
PyGILState_STATE state_;
public:
Ticker() :running_(&run_) { *running_ = false; }
void operator()()
{
while (*running_)
{
boost::xtime_get(&xt_, boost::TIME_UTC);
++xt_.sec;
boost::thread::sleep(xt_);
state_ = PyGILState_Ensure();
onTick();
PyGILState_Release(state_);
}
}
void run()
{
if (*running_ == false)
{
*running_ = true;
thread_ = new boost::thread(*this);
}
}
void stop()
{
if (*running_ == true)
{
*running_ = false;
thread_->join();
delete thread_;
}
}
virtual void onTick() { get_override("onTick")(); }
void default_onTick() {}
};
BOOST_PYTHON_MODULE(tick)
{
class_<Ticker, boost::noncopyable> ("Ticker")
.def("run", &Ticker::run)
.def("stop", &Ticker::stop)
.def("onTick", &Ticker::default_onTick);
}
Thanks again,
Paul
"Renato Araujo" <[email protected]> wrote in message
news:[email protected]...
I'm using gcc/linux and python >= 2.4 and works fine for me.
On Mon, Jul 6, 2009 at 7:39 AM, Paul Scruby<[email protected]> wrote:
> Hello again,
>
> Sorry, I spoke too soon. The good news is that wrapping my virtual method
> calls into Python between PyGILState_Ensure() and PyGILState_Release()
> fixed
> the crash under Python2.6.2 on Windows, but not when I tested it again
> under
> Python 2.4.4 on Solaris. Under Python 2.4.4 on SolarisSolaris it now calls
> and exits the virtual method in Python sucessfully, but then crashes when
> C++ trys to release the global interpretor lock.
>
> The call-stack with the Sun C++ 5.9 compiler under Solaris is
> t...@2 (l...@2) signal SEGV (no mapping at the fault address) in sem_invalid
> at 0xfec453ed
> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
> Current function is Ticker::operator()
> 28 PyGILState_Release(state_);
> (dbx) where
> current thread: t...@2
> [1] sem_invalid(0x0), at 0xfec453ed
> [2] _sem_post(0x0), at 0xfec454c4
> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56
> in "thread.hpp"
> [9] thread_proxy(0x810a288), at 0xfea78ce4
> [10] _thr_setup(0xfe670200), at 0xfee159b9
> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
> Do you think it's worth repeating this test using gcc/linux, or do you
> think
> that this is just a limitation of using Python with threads?
>
> Thanks again,
>
> Paul
>
>
> t...@2 (l...@2) signal SEGV (no mapping at the fault address) in sem_invalid
> at
> 0xfec453ed
> 0xfec453ed: sem_invalid+0x0013: movzwl 0x00000006(%eax),%eax
> Current function is Ticker::operator()
> 28 PyGILState_Release(state_);
> (dbx) where
> current thread: t...@2
> [1] sem_invalid(0x0), at 0xfec453ed
> [2] _sem_post(0x0), at 0xfec454c4
> [3] PyThread_release_lock(0x0, 0xfe77ef2c, 0xfef43eba, 0x80c55c0,
> 0xfe77ef38, 0xfef441b5), at 0xfef492dc
> [4] PyEval_ReleaseLock(0x80c55c0, 0xfe77ef38, 0xfef441b5, 0xfeb12aec,
> 0xfe77ef70, 0xfeafb2e3), at 0xfef27abe
> [5] PyThreadState_DeleteCurrent(0xfeb12aec, 0xfe77ef70, 0xfeafb2e3, 0x1,
> 0x0, 0x80a3140), at 0xfef43eba
> [6] PyGILState_Release(0x1, 0x0), at 0xfef441b5
> =>[7] Ticker::operator()(this = 0x810a304), line 28 in "ticker.cc"
> [8] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line 56 in
> "thread.hpp"
> [9] thread_proxy(0x810a288), at 0xfea78ce4
> [10] _thr_setup(0xfe670200), at 0xfee159b9
> [11] _lwp_start(0xfe77ef08, 0xfec454c4, 0x0, 0xfe77ef54, 0x80c55c0,
> 0xfe77ef14), at 0xfee15ca0
>
>
>
> "Paul Scruby" <[email protected]> wrote in message
> news:[email protected]...
>> Hiya,
>>
>> That's fantastic, all I needed to do was to put PyGILState_Ensure();
>> before my virtual function calls into python from another thread and my
>> program no longer crashes. Problem solved, isn't boost::python great!
>>
>> Many thanks,
>>
>> Paul
>>
>>
>> "Renato Araujo" <[email protected]> wrote in message
>> news:[email protected]...
>> Hi Paul
>>
>> In my bindings I had a problem like this, to solve I created a simple
>> class like that:
>>
>> class thread_locker
>> {
>> thread_locker()
>> {
>> if (thread_support::enabled())
>> m_gstate = PyGILState_Ensure();
>> }
>>
>> ~thread_locker()
>> {
>> if (thread_support::enabled())
>> PyGILState_Release(m_gstate);
>> }
>> };
>>
>> then in my wrapper virtual implementation I did this:
>>
>> ...
>> void wrapper::virtual_func(..)
>> {
>> thread_locker lock;
>> .. my code ..
>> }
>> ....
>>
>> this solve my problems with call of virtual functions in thread
>> enviroment.
>>
>> BR
>>
>>
>>
>>
>> On Sat, Jul 4, 2009 at 5:03 PM, William Ladwig<[email protected]> wrote:
>>> Whoops, I think this problem is a little uglier than I thought, since
>>> you
>>> overrode the onTick() function in python with a call to print, which
>>> needs access to the interpreter in your new thread. See the link Thomas
>>> posted for dealing with the GIL (along with worrying about any possible
>>> garbage collection issues). Now I remember why I kept my C++ threads
>>> isolated from Python stuff....sorry, it's been a while....
>>>
>>> Bill
>>> ________________________________________
>>> From: [email protected]
>>> [[email protected]] On Behalf Of
>>> William Ladwig [[email protected]]
>>> Sent: Saturday, July 04, 2009 1:34 PM
>>> To: Development of Python/C++ integration
>>> Subject: Re: [C++-sig] boost::python and threads
>>>
>>> It looks to me like you have a garbage collection problem going on. If
>>> you create a wrapped c++ object in python, then python is going to own
>>> the object and will destroy it when its reference count goes to 0. In
>>> your python example script at the bottom, you call the Ticker's run()
>>> function, which from the python point of view, returns quickly and the
>>> script ends. Python has no idea that you spawned off a new thread from
>>> the C++ side, so when the script ends, python destroys the object and
>>> now
>>> you have a problem. One way that you can check to see if this is what is
>>> going on is to add this to the bottom of the test script and see if the
>>> crashes go away:
>>>
>>> # Warning, you'll need to kill this script manually
>>> import time
>>> while True:
>>> time.sleep(1)
>>>
>>> Generally when I want to fire off a new C++ thread, I hold any objects
>>> that the thread needs in an auto_ptr (see held type for class wrappers),
>>> take ownership of them in C++ (see the FAQ in the documentation) and
>>> start the thread from the C++ side. Or, you can also create C++ wrappers
>>> which accept shared_ptr arguments, while holding your classes in
>>> shared_ptrs, and this should handle the reference counting as well.
>>> Unfortunately, this may require some interface changes to what you have
>>> already written (or possibly some clever wrapping).
>>>
>>> Hope this helps,
>>> Bill
>>>
>>>
>>> ________________________________________
>>> From: [email protected]
>>> [[email protected]] On Behalf Of Paul
>>> Scruby [[email protected]]
>>> Sent: Friday, July 03, 2009 6:15 AM
>>> To: [email protected]
>>> Subject: [C++-sig] boost::python and threads
>>>
>>> I am having some problems using boost::python with boost::thread. I'm
>>> using
>>> threads because I want to run some tasks in the background when I'm
>>> using
>>> the Python's interactive shell. However, when I use get_override() to
>>> call
>>> a Python method from another boost::thread it crashes internally. For
>>> example:
>>>
>>> #include <boost/python.hpp>
>>> #include <boost/thread/thread.hpp>
>>> #include <boost/thread/xtime.hpp>
>>>
>>> using namespace boost::python;
>>>
>>> class Ticker
>>> : public wrapper<Ticker>
>>> {
>>> private:
>>> bool run_;
>>> volatile bool * running_;
>>> boost::thread * thread_;
>>> boost::xtime xt_;
>>> public:
>>> Ticker() : running_(&run_) { *running_ = false; }
>>>
>>> void operator()()
>>> {
>>> while (*running_)
>>> {
>>> boost::xtime_get(&xt_, boost::TIME_UTC);
>>> ++xt_.sec;
>>> boost::thread::sleep(xt_);
>>> onTick();
>>> }
>>> }
>>>
>>> void run()
>>> {
>>> if (*running_ == false)
>>> {
>>> *running_ = true;
>>> thread_ = new boost::thread(*this);
>>> }
>>> }
>>>
>>> void stop()
>>> {
>>> if (*running_ == true)
>>> {
>>> *running_ = false;
>>> thread_->join();
>>> delete thread_;
>>> }
>>> }
>>>
>>> virtual void onTick() { get_override("onTick")(); }
>>> void default_onTick() {}
>>> };
>>>
>>> BOOST_PYTHON_MODULE(tick)
>>> {
>>> class_<Ticker, boost::noncopyable> ("Ticker")
>>> .def("run", &Ticker::run)
>>> .def("stop", &Ticker::stop)
>>> .def("onTick", &Ticker::default_onTick);
>>> }
>>>
>>> Here is a test script that which will crash when you import it into
>>> Python's
>>> interactive shell.
>>>
>>> from tick import Ticker
>>>
>>> class MyTicker(Ticker):
>>> def onTick(self):
>>> print "Each second"
>>>
>>> myticker = MyTicker()
>>> myticker.run()
>>>
>>> I ran this test initially on Python 2.4.4 with the Sun C++ 5.9 compiler
>>> on
>>> Solaris and I also tested it using Python 2.6.2 with Visual Studio 2008
>>> on
>>> Windows XP.
>>>
>>> The call-stack in dbx on Solaris:
>>>
>>> >>> t...@2 (l...@2) signal SEGV (no mapping at the fault address) in
>>> PyErr_Restore at 0xfef38fa1
>>> 0xfef38fa1: PyErr_Restore+0x0031: movl 0x00000028(%edi),%ecx
>>> Current function is boost::python::override::operator()
>>> 99 detail::method_result x(
>>>
>>> (dbx) where
>>> current thread: t...@2
>>> [1] PyErr_Restore(0x80652fc, 0x80f1220, 0x0, 0xfe77ee90, 0xfef3951e,
>>> 0x80652fc), at 0xfef38fa1
>>> [2] PyErr_SetObject(0x80652fc, 0x80f1220), at 0xfef3901e
>>> [3] PyErr_Format(0x80652fc, 0xfef5c2d8, 0xfef7902c), at 0xfef3951e
>>> [4] PyObject_Call(0xfef88768, 0x806102c, 0x0), at 0xfeee291a
>>> [5] PyEval_CallObjectWithKeywords(0xfef88768, 0x806102c, 0x0), at
>>> 0xfef2bf02
>>> [6] PyEval_CallFunction(0xfef88768, 0xfeb02004), at 0xfef434c5
>>> =>[7] boost::python::override::operator()(this = 0xfe77ef30), line 99 in
>>> "override.hpp"
>>> [8] Ticker::onTick(this = 0x810a304), line 48 in "ticker.cc"
>>> [9] Ticker::operator()(this = 0x810a304), line 25 in "ticker.cc"
>>> [10] boost::detail::thread_data<Ticker>::run(this = 0x810a288), line
>>> 56 in "thread.hpp"
>>> [11] thread_proxy(0x810a288), at 0xfea78ce4
>>> [12] _thr_setup(0xfe670200), at 0xfee159b9
>>> [13] _lwp_start(0xfe77ef54, 0x80f1220, 0xfe77ee7c, 0xfef3901e,
>>> 0x80652fc, 0x80f1220), at 0xfee15ca0
>>>
>>> The call-stack in Visual Studio 2008:
>>>
>>> python26.dll!1e013595()
>>> [Frames below may be incorrect and/or missing, no symbols loaded for
>>> python26.dll]
>>> python26.dll!1e09ee7d()
>>> > tick.pyd!boost::python::override::operator()() Line 103 + 0x16 bytes
>>> C++
>>> 00f3fd64()
>>> tick.pyd!Ticker::operator()() Line 27 + 0xe bytes C++
>>> tick.pyd!boost::detail::thread_data<Ticker>::run() Line 57 C++
>>> tick.pyd!boost::`anonymous namespace'::thread_start_function(void *
>>> param=0x00245f30) Line 168 C++
>>> msvcr90d.dll!_callthreadstartex() Line 348 + 0xf bytes C
>>> msvcr90d.dll!_threadstartex(void * ptd=0x00d46938) Line 331 C
>>> kernel32.dll!7c80b729()
>>>
>>>
>>> Have a missed a trick using the wrapper, or does boost::python not
>>> support
>>> threading?
>>>
>>> Many thanks,
>>>
>>> Paul
>>>
>>>
>>>
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> [email protected]
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> [email protected]
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> [email protected]
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>>
>>
>>
>>
>> --
>> Renato Araujo Oliveira Filho
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
--
Renato Araujo Oliveira Filho
_______________________________________________
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
