Looks like my memory problems are all related to the kaha HashIndex:
By the time I create 3000 queues, there are about 5000 HashIndex
object allocated. The retained size of those HashIndex objects is
about 187 megs. 163 megs of that are just byte[]s being held directly
by the HashIndex and the DataByteArraOutputStream that the HashIndex
holds onto.
Rob, any chances of reducing the number of HashIndex (or the amount of
memory each one needs) objects held in memory?
Regards,
Hiram
On Fri, May 16, 2008 at 2:44 PM, Hiram Chirino <[EMAIL PROTECTED]> wrote:
> I modified your case so that it's a JUnit test and so that it starts
> an embedded broker.. I did not see the threading problem against the
> trunk, but I did see :
>
> javax.jms.JMSException: Java heap space
> at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
> at
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1201)
> at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1648)
> at
> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
> at
> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:300)
> at org.apache.activemq.App.testSend(App.java:34)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.activemq.util.DataByteArrayOutputStream.<init>(DataByteArrayOutputStream.java:45)
> at
> org.apache.activemq.kaha.impl.index.hash.HashIndex.load(HashIndex.java:246)
> at
> org.apache.activemq.kaha.impl.container.MapContainerImpl.load(MapContainerImpl.java:107)
> at
> org.apache.activemq.store.kahadaptor.KahaPersistenceAdapter.getSubsMapContainer(KahaPersistenceAdapter.java:222)
> at
> org.apache.activemq.store.kahadaptor.KahaReferenceStoreAdapter.createTopicReferenceStore(KahaReferenceStoreAdapter.java:171)
> at
> org.apache.activemq.store.amq.AMQPersistenceAdapter.createTopicMessageStore(AMQPersistenceAdapter.java:456)
> at
> org.apache.activemq.broker.region.DestinationFactoryImpl.createDestination(DestinationFactoryImpl.java:94)
> at
> org.apache.activemq.broker.region.AbstractRegion.createDestination(AbstractRegion.java:425)
> at
> org.apache.activemq.broker.jmx.ManagedTopicRegion.createDestination(ManagedTopicRegion.java:56)
> at
> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:120)
> at
> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:266)
> at
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:141)
> at
> org.apache.activemq.advisory.AdvisoryBroker.addDestination(AdvisoryBroker.java:147)
> at
> org.apache.activemq.broker.BrokerFilter.addDestination(BrokerFilter.java:141)
> at
> org.apache.activemq.broker.MutableBrokerFilter.addDestination(MutableBrokerFilter.java:148)
> at
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:425)
> at
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:125)
> at
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
> at
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:132)
> at
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:437)
> at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:624)
> at
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
> at
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:180)
> at
> org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:104)
> at
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at
> org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:205)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> at java.lang.Thread.run(Thread.java:613)
>
> If you run my modified test case do you still see the threading issue?
>
> Here's my new test case:
>
> package org.apache.activemq;
>
> import javax.jms.Connection;
> import javax.jms.ConnectionFactory;
> import javax.jms.Destination;
> import javax.jms.JMSException;
> import javax.jms.MessageProducer;
> import javax.jms.Session;
> import javax.jms.TextMessage;
>
> import junit.framework.TestCase;
>
> public class App extends TestCase {
>
> Connection conn;
>
> @Override
> protected void setUp() throws Exception {
> ConnectionFactory cf = new ActiveMQConnectionFactory("vm://localhost");
> conn = cf.createConnection();
> }
>
> @Override
> protected void tearDown() throws Exception {
> conn.close();
> }
>
> public void testSend() throws JMSException {
> Session session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
> MessageProducer producer = session.createProducer(null);
> for (int i = 0; i < 5000; i++) {
> Destination destination = session.createTopic("TOPIC" + i);
> TextMessage message = session.createTextMessage("");
> producer.send(destination, message);
> System.out.println("Sent: " + i+": "+Thread.activeCount());
>
> }
> producer.close();
> }
> }
>
> On Fri, May 16, 2008 at 2:29 PM, Danilo Tuler <[EMAIL PROTECTED]> wrote:
>>
>>
>>
>> Hiram Chirino wrote:
>>>
>>> Still it would be nice if you could create a test case that shows the
>>> problem and submit it so we can get to the root of the problem.
>>>
>>
>> The attached code shows the problem.
>> With UseDedicatedTaskRunner=false it works ok.
>>
>> http://www.nabble.com/file/p17281318/App.java App.java
>>
>> --
>> View this message in context:
>> http://www.nabble.com/java.lang.OutOfMemoryError%3A-unable-to-create-new-native-thread-on-AMQ-5.0-tp17005441s2354p17281318.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://open.iona.com
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
