[jira] Resolved: (QPID-1952) C++: client request memory leak on windows
[ https://issues.apache.org/jira/browse/QPID-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Huston resolved QPID-1952. Resolution: Fixed Fix Version/s: 0.6 Assignee: Steve Huston I tested the patch here with Purify - no more leak. Fix is in r791997. C++: client request memory leak on windows -- Key: QPID-1952 URL: https://issues.apache.org/jira/browse/QPID-1952 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: 0.5 Environment: Windows XP Pro w/SP3 Reporter: David Rennalls Assignee: Steve Huston Fix For: 0.6 Attachments: AsynchIO.cpp.diff, Connector_memory_leak.patch, Main2.cpp I'm experiencing a memory leak when using the client library on Windows (0.5 release) when sending a single request. From what debugging I've done so far it looks like the The AsynchIO object used by the TCPConnector is not being deleted. When the connection is being closed aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up. There used to be some code in AsynchIO::completion() that seems like it would do the necessary delete (see below). It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550 because 'It fixed a memory corruption on the broker side by deleting too early' (from Steve H.) -// Lock released; ok to delete if all is done. -if (opsInProgress == 0 queuedDelete) -delete this; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-1952) C++: client request memory leak on windows
[ https://issues.apache.org/jira/browse/QPID-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Huston updated QPID-1952: --- Attachment: AsynchIO.cpp.diff David, could you please try the attached AsynchIO.cpp.diff with your test case? It goes back to your original diagnosis that the issue is in AsynchIO.cpp - my other fix took care of the broker cases, but left a leak with the client where the operations are done in reverse order, usually. C++: client request memory leak on windows -- Key: QPID-1952 URL: https://issues.apache.org/jira/browse/QPID-1952 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: 0.5 Environment: Windows XP Pro w/SP3 Reporter: David Rennalls Attachments: AsynchIO.cpp.diff, Connector_memory_leak.patch, Main2.cpp I'm experiencing a memory leak when using the client library on Windows (0.5 release) when sending a single request. From what debugging I've done so far it looks like the The AsynchIO object used by the TCPConnector is not being deleted. When the connection is being closed aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up. There used to be some code in AsynchIO::completion() that seems like it would do the necessary delete (see below). It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550 because 'It fixed a memory corruption on the broker side by deleting too early' (from Steve H.) -// Lock released; ok to delete if all is done. -if (opsInProgress == 0 queuedDelete) -delete this; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-1952) C++: client request memory leak on windows
[ https://issues.apache.org/jira/browse/QPID-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Rennalls updated QPID-1952: - Attachment: Connector_memory_leak.patch This fixed the leak for me. I'm only using the client library and don't know if this leak exists in the broker's usage. C++: client request memory leak on windows -- Key: QPID-1952 URL: https://issues.apache.org/jira/browse/QPID-1952 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: 0.5 Environment: Windows XP Pro w/SP3 Reporter: David Rennalls Attachments: Connector_memory_leak.patch, Main2.cpp I'm experiencing a memory leak when using the client library on Windows (0.5 release) when sending a single request. From what debugging I've done so far it looks like the The AsynchIO object used by the TCPConnector is not being deleted. When the connection is being closed aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up. There used to be some code in AsynchIO::completion() that seems like it would do the necessary delete (see below). It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550 because 'It fixed a memory corruption on the broker side by deleting too early' (from Steve H.) -// Lock released; ok to delete if all is done. -if (opsInProgress == 0 queuedDelete) -delete this; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: C++: client request memory leak on windows
David Rennalls wrote:
Hi,
I'm experiencing a memory leak when using the client library on
Windows (0.5 release) when sending a single request. From what debugging
I've done so far it looks like the The AsynchIO object used by the
TCPConnector is not being deleted. When the connection is being closed
aio-queueForDeletion() is called from TCPConnector::closeInternal()..
However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the
'delete this' branch is not done. There are no more queueForDeletion()
calls made on the aio object (after opsInProgress is 0) and so it never
gets deleted. The ConnectionImpl object involved is not cleaned up
either, the shared_ptr to it has a non-zero ref count but I'm guessing
it's related to the AsynchIO object not being cleaned up.
There used to be some code in AsynchIO::completion() that seems like it
would do the necessary delete. It was removed with code changes in
https://issues.apache.org/jira/browse/QPID-1550
The extra handling in completion() was as follows..
-// Lock released; ok to delete if all is done.
-if (opsInProgress == 0 queuedDelete)
-delete this;
Should that code be re-introduced (I don't know the history nor the 'big
picture'), or could this be due to my usage ? My reproduction code is as
follows.. The CRT memory leak report gets printed when the test program
exits, that's what lead me to aio allocation that isn't being freed.
I'm not sure about that code, astitcher might know more
#define _CRTDBG_MAP_ALLOC
#include cstdlib
#include crtdbg.h
#include qpid/client/Connection.h
#include qpid/client/Session.h
#include qpid/client/Message.h
#include qpid/client/SubscriptionManager.h
#include qpid/sys/Time.h
#include iostream
using namespace qpid::client;
using namespace qpid::framing;
using qpid::sys::TIME_SEC;
int main(int argc, char **argv)
{
_CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );
_CrtSetReportMode( _CRT_ERROR, _CRTDBG_MODE_DEBUG | _CRTDBG_MODE_WNDW);
Connection connection;
Message response;
try
{
//qpid::TcpAddress::DEFAULT_PORT
connection.open(10.39.170.224);
Session session = connection.newSession();
// Create a response queue using the client's session id as the
name of
// the response queue
string responseQueue = x.client + session.getId().getName();
// Use the name of the response queue as the routing key as well
session.queueDeclare(arg::queue=responseQueue);
session.exchangeBind(arg::exchange=amq.direct,
arg::queue=responseQueue, arg::bindingKey=responseQueue);
// Setup a message using our response queue so the
// the service knows where to route response messages.
Message req;
DeliveryProperties deliveryProps = req.getDeliveryProperties();
deliveryProps.setRoutingKey(x.request);
MessageProperties messageProps = req.getMessageProperties();
messageProps.setReplyTo(ReplyTo(amq.direct, responseQueue));
session.messageTransfer(arg::content=req,
arg::destination=amq.direct, arg::acceptMode=0);
// Create a listener for the response queue and listen for
response messages.
SubscriptionManager subsManager(session);
if(!subsManager.get(response, responseQueue, TIME_SEC))
std::cout Timed out waiting for a response std::endl;
}
catch(const std::exception error)
{
std::cout exception thrown.. std::endl
error.what() std::endl;
}
return 0;
}
- David
Try adding an explicit connection.close() to your code and see if that solves
the leak. In principle we should clean up properly even if close() is not
called, but in practice I think there are some cases where we don't.
I thought there was already a JIRA for this but I can't find it, can you add one
with your reproducer code?
Cheers,
Alan.
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: C++: client request memory leak on windows
On Thu, 2009-06-25 at 08:32 -0400, Alan Conway wrote: David Rennalls wrote: Hi, I'm experiencing a memory leak when using the client library on ... I'm not sure about that code, astitcher might know more This is windows only code, of which I have little knowledge. Andrew - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Created: (QPID-1952) C++: client request memory leak on windows
C++: client request memory leak on windows -- Key: QPID-1952 URL: https://issues.apache.org/jira/browse/QPID-1952 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: 0.5 Environment: Windows XP Pro w/SP3 Reporter: David Rennalls I'm experiencing a memory leak when using the client library on Windows (0.5 release) when sending a single request. From what debugging I've done so far it looks like the The AsynchIO object used by the TCPConnector is not being deleted. When the connection is being closed aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up. There used to be some code in AsynchIO::completion() that seems like it would do the necessary delete (see below). It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550 because 'It fixed a memory corruption on the broker side by deleting too early' (from Steve H.) -// Lock released; ok to delete if all is done. -if (opsInProgress == 0 queuedDelete) -delete this; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
Re: C++: client request memory leak on windows
Alan Conway wrote: David Rennalls wrote: ..snip.. Try adding an explicit connection.close() to your code and see if that solves the leak. In principle we should clean up properly even if close() is not called, but in practice I think there are some cases where we don't. My 'real' code does have the explicit close(). When I created the smaller reproducible sample I trimmed as much as possible including the explcit close(). I thought there was already a JIRA for this but I can't find it, can you add one with your reproducer code? done, see https://issues.apache.org/jira/browse/QPID-1952 I'll attach the test code in a bit (same as my original post), I want to re-test with the .close() added but I've lost my server (qpidd was on centos 5) momentarily. I'll try and get the windows broker running for an easier reproduction setup as well when I have time. Cheers, Alan. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Updated: (QPID-1952) C++: client request memory leak on windows
[ https://issues.apache.org/jira/browse/QPID-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Rennalls updated QPID-1952: - Attachment: Main2.cpp Test code that exhibits the problem. This was run against a local broker (using .exe from http://www.apache.org/dist/qpid/0.5-windows/qpidc-0.5.msi) with no queues defined etc.. just needs a working broker to accept the request. The real code where the queue is defined and a response is received etc. has the same problem C++: client request memory leak on windows -- Key: QPID-1952 URL: https://issues.apache.org/jira/browse/QPID-1952 Project: Qpid Issue Type: Bug Components: C++ Client Affects Versions: 0.5 Environment: Windows XP Pro w/SP3 Reporter: David Rennalls Attachments: Main2.cpp I'm experiencing a memory leak when using the client library on Windows (0.5 release) when sending a single request. From what debugging I've done so far it looks like the The AsynchIO object used by the TCPConnector is not being deleted. When the connection is being closed aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up. There used to be some code in AsynchIO::completion() that seems like it would do the necessary delete (see below). It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550 because 'It fixed a memory corruption on the broker side by deleting too early' (from Steve H.) -// Lock released; ok to delete if all is done. -if (opsInProgress == 0 queuedDelete) -delete this; -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
C++: client request memory leak on windows
Hi,
I'm experiencing a memory leak when using the client library on Windows (0.5 release) when
sending a single request. From what debugging I've done so far it looks like the The AsynchIO object
used by the TCPConnector is not being deleted. When the connection is being closed
aio-queueForDeletion() is called from TCPConnector::closeInternal().. However in
AsynchIO::queueForDeletion() aio.opsInProgress 0 so the 'delete this' branch is not done. There
are no more queueForDeletion() calls made on the aio object (after opsInProgress is 0) and so it
never gets deleted. The ConnectionImpl object involved is not cleaned up either, the shared_ptr to
it has a non-zero ref count but I'm guessing it's related to the AsynchIO object not being cleaned up.
There used to be some code in AsynchIO::completion() that seems like it would do the necessary
delete. It was removed with code changes in https://issues.apache.org/jira/browse/QPID-1550
The extra handling in completion() was as follows..
-// Lock released; ok to delete if all is done.
-if (opsInProgress == 0 queuedDelete)
-delete this;
Should that code be re-introduced (I don't know the history nor the 'big picture'), or could this be
due to my usage ? My reproduction code is as follows.. The CRT memory leak report gets printed when
the test program exits, that's what lead me to aio allocation that isn't being freed.
#define _CRTDBG_MAP_ALLOC
#include cstdlib
#include crtdbg.h
#include qpid/client/Connection.h
#include qpid/client/Session.h
#include qpid/client/Message.h
#include qpid/client/SubscriptionManager.h
#include qpid/sys/Time.h
#include iostream
using namespace qpid::client;
using namespace qpid::framing;
using qpid::sys::TIME_SEC;
int main(int argc, char **argv)
{
_CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );
_CrtSetReportMode( _CRT_ERROR, _CRTDBG_MODE_DEBUG | _CRTDBG_MODE_WNDW);
Connection connection;
Message response;
try
{
//qpid::TcpAddress::DEFAULT_PORT
connection.open(10.39.170.224);
Session session = connection.newSession();
// Create a response queue using the client's session id as the name of
// the response queue
string responseQueue = x.client + session.getId().getName();
// Use the name of the response queue as the routing key as well
session.queueDeclare(arg::queue=responseQueue);
session.exchangeBind(arg::exchange=amq.direct,
arg::queue=responseQueue, arg::bindingKey=responseQueue);
// Setup a message using our response queue so the
// the service knows where to route response messages.
Message req;
DeliveryProperties deliveryProps = req.getDeliveryProperties();
deliveryProps.setRoutingKey(x.request);
MessageProperties messageProps = req.getMessageProperties();
messageProps.setReplyTo(ReplyTo(amq.direct, responseQueue));
session.messageTransfer(arg::content=req, arg::destination=amq.direct,
arg::acceptMode=0);
// Create a listener for the response queue and listen for response
messages.
SubscriptionManager subsManager(session);
if(!subsManager.get(response, responseQueue, TIME_SEC))
std::cout Timed out waiting for a response std::endl;
}
catch(const std::exception error)
{
std::cout exception thrown.. std::endl
error.what() std::endl;
}
return 0;
}
- David
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
