Rajith Attapattu wrote:
Jodi,
Thanks for the feedback.
Comments inline
Regards,
Rajith
On 9/11/07, Jodi Moran <[EMAIL PROTECTED]> wrote:
Hi all,
I'm having problems using the Qpid M2 Java client (JMS to AMQP) for
publishing messages in a load test. When I publish messages as quickly
as possible, the publishing client runs out of memory (no matter what
limit I set). I am using only the minimum of the JMS interface:
InitialContext jndiContext = new
InitialContext(additionalJNDIProps);
connectionFactory = (ConnectionFactory)
jndiContext.lookup(connectionFactoryJNDIName);
destination = (Destination) jndiContext.lookup(topicName);
connection = connectionFactory.createConnection();
jmsSession = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
jmsMessageProducer = jmsSession.createProducer(destination);
jmsMessageProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
And later, in a loop:
BytesMessage message = jmsSession.createBytesMessage();
message.writeBytes(messageContent);
jmsMessageProducer.send(message);
After making use of profilers and heap dumps, it appears that the OOM is
caused by the fact that the publishing thread by default does not block
during the send but just adds the write request to an (unbounded) queue
internal to Mina. Since in my case (it seems) the I/O is slower than the
publishing thread, the write request queue continues to grow until it
causes the OOM.
I've noticed that there is functionality in the BasicMessageProducer
that allows the user to block on writes (_waitUntilSent), but it seems
that this functionality is not even exposed via the extended interfaces
(i.e. org.apache.qpid.jms.MessageProducer or
org.apache.qpid.jms.Session) and so requires a cast to
BasicMessageProducer or to AMQSession to use. Is the only way to get
flow control in my publishing client to make use of _waitUntilSent or is
there some other way I can achieve the same effect?
Currently this is the only way to set this.
However we could provide a jvm argument to set it, so that u don't have to
cast it to any AMQ specific class.
We might repsin the M2 release again. We can add this feature if it helps.
Doing this block will slow down your application.
Without the block atleast your application can continue publishing at a
higher rate, until the internal MINA queue starts to grow due to IO being
slow.
Is there any way you can throttle the publish rate in your application?
After some experimenting you maybe able to find a sweet spot that is right
for your environment. This might yeild a higher average publish rate than a
block for every publish type of solution.
We should really do the throttling automatically, i.e. we should block
when the MINA queue exceeds a certain limit, but not if it is below that
limit
--Rafael
