[
https://issues.apache.org/jira/browse/QPID-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Schreiber updated QPID-5644:
---------------------------------
Description:
Sometimes, when running qpidd it crashes on startup with the following
assertion appearing in /var/log/messages
{quote} Mar 24 11:08:03 localhost qpidd[441]: qpidd:
/usr/include/boost/smart_ptr/shared_ptr.hpp:653: typename
boost::detail::sp_member_access<T>::type boost::shared_ptr<T>::operator->()
const [with T = qmf::org::apache::qpid::broker::Broker; typename
boost::detail::sp_member_access<T>::type =
qmf::org::apache::qpid::broker::Broker*]: Assertion `px != 0' failed.
{quote}
Trying to increase log levels to trace+ gives us this:
{quote}
2014-03-24 11:07:37 [Security] info SSL connector not enabled, you must set
QPID_SSL_CERT_DB to enable it.
2014-03-24 11:07:43 [Model] trace Mgmt create memory. id:amqp-broker
2014-03-24 11:07:43 [Management] debug Management object added: amqp-broker
2014-03-24 11:07:43 [Broker] info Management enabled
2014-03-24 11:07:43 [Management] info ManagementAgent restored broker ID:
e349c55c-69ef-47ad-bd53-a4c898f43e0a
2014-03-24 11:07:49 [Management] debug ManagementAgent boot sequence: 4
2014-03-24 11:07:49 [Management] debug ManagementAgent added package
org.apache.qpid.broker
2014-03-24 11:07:49 [Management] debug SEND PackageInd
package=org.apache.qpid.broker to=schema.package
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:system
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:memory
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:broker
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:agent
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:vhost
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:queue
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:exchange
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:binding
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:subscription
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:connection
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:incoming
2014-03-24 11:07:50 [Management] debug ManagementAgent added class
org.apache.qpid.broker:outgoing
2014-03-24 11:07:57 [Management] debug Management agent periodic processing:
management snapshot: 1 packages, 0 objects (0 deleted), 1 new objects (0
deleted), 0 pending deletes
2014-03-24 11:07:57 [Management] trace Management agent periodic processing:
new objects
amqp-broker
{quote}
We managed to find out what the problem was after we got a core dump of qpidd.
The stack trace was:
{quote}
#0 0x00007fc41f1fe9e9 in raise () from /lib64/libc.so.6
#1 0x00007fc41f2000f8 in abort () from /lib64/libc.so.6
#2 0x00007fc41f1f7956 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007fc41f1f7a02 in __assert_fail () from /lib64/libc.so.6
#4 0x00007fc4213dd0e5 in
boost::shared_ptr<qmf::org::apache::qpid::broker::Broker>::operator->() const
[clone .isra.49] [clone .part.50] () from /lib64/libqpidbroker.so.2
#5 0x00007fc4215edd8d in
qpid::management::ManagementAgent::periodicProcessing() () from
/lib64/libqpidbroker.so.2
#6 0x00007fc4215d27d0 in qpid::management::(anonymous
namespace)::Periodic::fire() () from /lib64/libqpidbroker.so.2
#7 0x00007fc42102d70b in
qpid::sys::Timer::fire(boost::intrusive_ptr<qpid::sys::TimerTask>) () from
/lib64/libqpidcommon.so.2
#8 0x00007fc42102ed26 in qpid::sys::Timer::run() () from
/lib64/libqpidcommon.so.2
#9 0x00007fc420fc20da in qpid::sys::(anonymous namespace)::runRunnable(void*)
() from /lib64/libqpidcommon.so.2
#10 0x00007fc41f591c53 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fc41f2bedbd in clone () from /lib64/libc.so.6
{quote}
What happens is that in Broker's constructor it calls
managementAgent->configure() prior to setting mgmtObject with a proper
shared_ptr. ManagementAgent::configure() starts a periodic task which tries to
access mgmObject using Broker::GetManagementObject(). So it's a timing issue:
if the periodic task happened to occur before setting Broker::mgmtObject, the
code will crash on a null shared_ptr.
This also happens on 0.26.
was:
Sometimes, when running qpidd it crashes on startup with the following
assertion appearing in /var/log/messages
{quote} Mar 24 11:08:03 localhost qpidd[441]: qpidd:
/usr/include/boost/smart_ptr/shared_ptr.hpp:653: typename
boost::detail::sp_member_access<T>::type boost::shared_ptr<T>::operator->()
const [with T = qmf::org::apache::qpid::broker::Broker; typename
boost::detail::sp_member_access<T>::type =
qmf::org::apache::qpid::broker::Broker*]: Assertion `px != 0' failed.
{quote}
Trying to increase log levels to trace+ gives us this:
{quote}
2014-03-24 11:07:37 [Security] info SSL connector not enabled, you must set
QPID_SSL_CERT_DB to enable it.
2014-03-24 11:07:43 [Model] trace Mgmt create memory. id:amqp-broker
2014-03-24 11:07:43 [Management] debug Management object added: amqp-broker
2014-03-24 11:07:43 [Broker] info Management enabled
2014-03-24 11:07:43 [Management] info ManagementAgent restored broker ID:
e349c55c-69ef-47ad-bd53-a4c898f43e0a
2014-03-24 11:07:49 [Management] debug ManagementAgent boot sequence: 4
2014-03-24 11:07:49 [Management] debug ManagementAgent added package
org.apache.qpid.broker
2014-03-24 11:07:49 [Management] debug SEND PackageInd
package=org.apache.qpid.broker to=schema.package
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:system
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:memory
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:broker
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:agent
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:vhost
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:queue
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:exchange
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:binding
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:subscription
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:connection
2014-03-24 11:07:49 [Management] debug ManagementAgent added class
org.apache.qpid.broker:incoming
2014-03-24 11:07:50 [Management] debug ManagementAgent added class
org.apache.qpid.broker:outgoing
2014-03-24 11:07:57 [Management] debug Management agent periodic processing:
management snapshot: 1 packages, 0 objects (0 deleted), 1 new objects (0
deleted), 0 pending deletes
2014-03-24 11:07:57 [Management] trace Management agent periodic processing:
new objects
amqp-broker
{quote}
We managed to find out what the problem was after we got a core dump of qpidd.
The stack trace was:
{quote}
#0 0x00007fc41f1fe9e9 in raise () from /lib64/libc.so.6
#1 0x00007fc41f2000f8 in abort () from /lib64/libc.so.6
#2 0x00007fc41f1f7956 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007fc41f1f7a02 in __assert_fail () from /lib64/libc.so.6
#4 0x00007fc4213dd0e5 in
boost::shared_ptr<qmf::org::apache::qpid::broker::Broker>::operator->() const
[clone .isra.49] [clone .part.50] () from /lib64/libqpidbroker.so.2
#5 0x00007fc4215edd8d in
qpid::management::ManagementAgent::periodicProcessing() () from
/lib64/libqpidbroker.so.2
#6 0x00007fc4215d27d0 in qpid::management::(anonymous
namespace)::Periodic::fire() () from /lib64/libqpidbroker.so.2
#7 0x00007fc42102d70b in
qpid::sys::Timer::fire(boost::intrusive_ptr<qpid::sys::TimerTask>) () from
/lib64/libqpidcommon.so.2
#8 0x00007fc42102ed26 in qpid::sys::Timer::run() () from
/lib64/libqpidcommon.so.2
#9 0x00007fc420fc20da in qpid::sys::(anonymous namespace)::runRunnable(void*)
() from /lib64/libqpidcommon.so.2
#10 0x00007fc41f591c53 in start_thread () from /lib64/libpthread.so.0
#11 0x00007fc41f2bedbd in clone () from /lib64/libc.so.6
{quote}
What happens is that in Broker's constructor it calls
managementAgent->configure() prior to setting mgmtObject with a proper
shared_ptr. ManagementAgent::configure() starts a periodic task which tries to
access mgmObject using Broker::GetManagementObject(). So it's a timing issue:
if the periodic task happened to occur before setting Broker::mgmtObject, the
code will crash on a null shared_ptr.
This also happens on 0.26. Updating this bug.
> [C++ Broker] null Broker shared pointer
> ---------------------------------------
>
> Key: QPID-5644
> URL: https://issues.apache.org/jira/browse/QPID-5644
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker
> Affects Versions: 0.24, 0.26
> Environment: Fedora 19
> Reporter: Amit Schreiber
>
> Sometimes, when running qpidd it crashes on startup with the following
> assertion appearing in /var/log/messages
> {quote} Mar 24 11:08:03 localhost qpidd[441]: qpidd:
> /usr/include/boost/smart_ptr/shared_ptr.hpp:653: typename
> boost::detail::sp_member_access<T>::type boost::shared_ptr<T>::operator->()
> const [with T = qmf::org::apache::qpid::broker::Broker; typename
> boost::detail::sp_member_access<T>::type =
> qmf::org::apache::qpid::broker::Broker*]: Assertion `px != 0' failed.
> {quote}
> Trying to increase log levels to trace+ gives us this:
> {quote}
> 2014-03-24 11:07:37 [Security] info SSL connector not enabled, you must set
> QPID_SSL_CERT_DB to enable it.
> 2014-03-24 11:07:43 [Model] trace Mgmt create memory. id:amqp-broker
> 2014-03-24 11:07:43 [Management] debug Management object added: amqp-broker
> 2014-03-24 11:07:43 [Broker] info Management enabled
> 2014-03-24 11:07:43 [Management] info ManagementAgent restored broker ID:
> e349c55c-69ef-47ad-bd53-a4c898f43e0a
> 2014-03-24 11:07:49 [Management] debug ManagementAgent boot sequence: 4
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added package
> org.apache.qpid.broker
> 2014-03-24 11:07:49 [Management] debug SEND PackageInd
> package=org.apache.qpid.broker to=schema.package
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:system
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:memory
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:broker
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:agent
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:vhost
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:queue
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:exchange
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:binding
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:subscription
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:connection
> 2014-03-24 11:07:49 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:incoming
> 2014-03-24 11:07:50 [Management] debug ManagementAgent added class
> org.apache.qpid.broker:outgoing
> 2014-03-24 11:07:57 [Management] debug Management agent periodic processing:
> management snapshot: 1 packages, 0 objects (0 deleted), 1 new objects (0
> deleted), 0 pending deletes
> 2014-03-24 11:07:57 [Management] trace Management agent periodic processing:
> new objects
> amqp-broker
> {quote}
> We managed to find out what the problem was after we got a core dump of
> qpidd. The stack trace was:
> {quote}
> #0 0x00007fc41f1fe9e9 in raise () from /lib64/libc.so.6
> #1 0x00007fc41f2000f8 in abort () from /lib64/libc.so.6
> #2 0x00007fc41f1f7956 in __assert_fail_base () from /lib64/libc.so.6
> #3 0x00007fc41f1f7a02 in __assert_fail () from /lib64/libc.so.6
> #4 0x00007fc4213dd0e5 in
> boost::shared_ptr<qmf::org::apache::qpid::broker::Broker>::operator->() const
> [clone .isra.49] [clone .part.50] () from /lib64/libqpidbroker.so.2
> #5 0x00007fc4215edd8d in
> qpid::management::ManagementAgent::periodicProcessing() () from
> /lib64/libqpidbroker.so.2
> #6 0x00007fc4215d27d0 in qpid::management::(anonymous
> namespace)::Periodic::fire() () from /lib64/libqpidbroker.so.2
> #7 0x00007fc42102d70b in
> qpid::sys::Timer::fire(boost::intrusive_ptr<qpid::sys::TimerTask>) () from
> /lib64/libqpidcommon.so.2
> #8 0x00007fc42102ed26 in qpid::sys::Timer::run() () from
> /lib64/libqpidcommon.so.2
> #9 0x00007fc420fc20da in qpid::sys::(anonymous
> namespace)::runRunnable(void*) () from /lib64/libqpidcommon.so.2
> #10 0x00007fc41f591c53 in start_thread () from /lib64/libpthread.so.0
> #11 0x00007fc41f2bedbd in clone () from /lib64/libc.so.6
> {quote}
> What happens is that in Broker's constructor it calls
> managementAgent->configure() prior to setting mgmtObject with a proper
> shared_ptr. ManagementAgent::configure() starts a periodic task which tries
> to access mgmObject using Broker::GetManagementObject(). So it's a timing
> issue: if the periodic task happened to occur before setting
> Broker::mgmtObject, the code will crash on a null shared_ptr.
> This also happens on 0.26.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
