Rajith Attapattu wrote:
On 9/11/07, Rafael Schloming <[EMAIL PROTECTED]> wrote:
Rajith Attapattu wrote:
Jodi,
Thanks for the feedback.
Comments inline
Regards,
Rajith
On 9/11/07, Jodi Moran <[EMAIL PROTECTED]> wrote:
Hi all,
I'm having problems using the Qpid M2 Java client (JMS to AMQP) for
publishing messages in a load test. When I publish messages as quickly
as possible, the publishing client runs out of memory (no matter what
limit I set). I am using only the minimum of the JMS interface:
InitialContext jndiContext = new
InitialContext(additionalJNDIProps);
connectionFactory = (ConnectionFactory)
jndiContext.lookup(connectionFactoryJNDIName);
destination = (Destination) jndiContext.lookup(topicName);
connection = connectionFactory.createConnection();
jmsSession = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
jmsMessageProducer = jmsSession.createProducer
(destination);
jmsMessageProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
And later, in a loop:
BytesMessage message = jmsSession.createBytesMessage();
message.writeBytes(messageContent);
jmsMessageProducer.send(message);
After making use of profilers and heap dumps, it appears that the OOM
is
caused by the fact that the publishing thread by default does not block
during the send but just adds the write request to an (unbounded) queue
internal to Mina. Since in my case (it seems) the I/O is slower than
the
publishing thread, the write request queue continues to grow until it
causes the OOM.
I've noticed that there is functionality in the BasicMessageProducer
that allows the user to block on writes (_waitUntilSent), but it seems
that this functionality is not even exposed via the extended interfaces
(i.e. org.apache.qpid.jms.MessageProducer or
org.apache.qpid.jms.Session) and so requires a cast to
BasicMessageProducer or to AMQSession to use. Is the only way to get
flow control in my publishing client to make use of _waitUntilSent or
is
there some other way I can achieve the same effect?
Currently this is the only way to set this.
However we could provide a jvm argument to set it, so that u don't have
to
cast it to any AMQ specific class.
We might repsin the M2 release again. We can add this feature if it
helps.
Doing this block will slow down your application.
Without the block atleast your application can continue publishing at a
higher rate, until the internal MINA queue starts to grow due to IO
being
slow.
Is there any way you can throttle the publish rate in your application?
After some experimenting you maybe able to find a sweet spot that is
right
for your environment. This might yeild a higher average publish rate
than a
block for every publish type of solution.
We should really do the throttling automatically, i.e. we should block
when the MINA queue exceeds a certain limit, but not if it is below that
limit
Rafi, I thought about this initially, but since this queue is internal to
MINA, I wasn't sure if we can know the current queue size etc ?
Well you can modify MINA per Robert's suggestion in another post,
however I don't think you actually care about the queue size. What you
care about is how much memory the queue consumes, and this is strictly
speaking independent of the queue size.
I believe it should be possible to implement a byte limit without
modifying MINA. MINA invokes a callback whenever a message has actually
been sent over the wire. On the client this is
AMQProtocolHandler.messageSent(...). Currently the said method already
tracks how many bytes have been written over the wire. All we would need
to do is track how many bytes we've actually asked MINA to write via the
writeFrame(...) methods on that same class. The difference between the
two values is how much we have queued in memory, and we should be able
to block if that exceeds our threshold.
That said, it does seem like fairly obvious functionality that MINA lacks.
--Rafael
