On 9/11/07, Rafael Schloming <[EMAIL PROTECTED]> wrote: > > Rajith Attapattu wrote: > > Jodi, > > > > Thanks for the feedback. > > Comments inline > > > > Regards, > > > > Rajith > > > > On 9/11/07, Jodi Moran <[EMAIL PROTECTED]> wrote: > >> Hi all, > >> > >> I'm having problems using the Qpid M2 Java client (JMS to AMQP) for > >> publishing messages in a load test. When I publish messages as quickly > >> as possible, the publishing client runs out of memory (no matter what > >> limit I set). I am using only the minimum of the JMS interface: > >> > >> InitialContext jndiContext = new > >> InitialContext(additionalJNDIProps); > >> connectionFactory = (ConnectionFactory) > >> jndiContext.lookup(connectionFactoryJNDIName); > >> destination = (Destination) jndiContext.lookup(topicName); > >> connection = connectionFactory.createConnection(); > >> jmsSession = connection.createSession(false, > >> Session.AUTO_ACKNOWLEDGE); > >> jmsMessageProducer = jmsSession.createProducer > (destination); > >> > >> jmsMessageProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT); > >> > >> And later, in a loop: > >> > >> BytesMessage message = jmsSession.createBytesMessage(); > >> message.writeBytes(messageContent); > >> jmsMessageProducer.send(message); > >> > >> After making use of profilers and heap dumps, it appears that the OOM > is > >> caused by the fact that the publishing thread by default does not block > >> during the send but just adds the write request to an (unbounded) queue > >> internal to Mina. Since in my case (it seems) the I/O is slower than > the > >> publishing thread, the write request queue continues to grow until it > >> causes the OOM. > >> > >> I've noticed that there is functionality in the BasicMessageProducer > >> that allows the user to block on writes (_waitUntilSent), but it seems > >> that this functionality is not even exposed via the extended interfaces > >> (i.e. org.apache.qpid.jms.MessageProducer or > >> org.apache.qpid.jms.Session) and so requires a cast to > >> BasicMessageProducer or to AMQSession to use. Is the only way to get > >> flow control in my publishing client to make use of _waitUntilSent or > is > >> there some other way I can achieve the same effect? > > > > > > Currently this is the only way to set this. > > However we could provide a jvm argument to set it, so that u don't have > to > > cast it to any AMQ specific class. > > We might repsin the M2 release again. We can add this feature if it > helps. > > > > Doing this block will slow down your application. > > Without the block atleast your application can continue publishing at a > > higher rate, until the internal MINA queue starts to grow due to IO > being > > slow. > > Is there any way you can throttle the publish rate in your application? > > After some experimenting you maybe able to find a sweet spot that is > right > > for your environment. This might yeild a higher average publish rate > than a > > block for every publish type of solution. > > We should really do the throttling automatically, i.e. we should block > when the MINA queue exceeds a certain limit, but not if it is below that > limit
Rafi, I thought about this initially, but since this queue is internal to MINA, I wasn't sure if we can know the current queue size etc ? Rajith --Rafael >
