I'm using stomp, java clients and camel to move messages through
a 'pipeline' of activemq queues. The messages can be very big
(>120MB) and unfortunately the OS is 32-bit, so I want to keep
memory usage to a minimum (2GB limit). There isn't a huge number
of messages, but I keep running out of heap space. Each queue has
only one producer and one consumer.
I've attempted to read everything I can about producer flow control,
message cursors, prefetch limits, etc. and I still can't get it
to do what I want. I don't care about speed at all, I have plenty
of disk space, and all messages are persistent.
The JVM has the max 2024MB of memory.
bits of activemq.xml:
...
<policyEntry queue=">" producerFlowControl="true"
memoryLimit="150mb" queuePrefetch="1">
<pendingQueuePolicy>
<fileQueueCursor />
</pendingQueuePolicy>
</policyEntry>
...
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="1024 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="25 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="5 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
I've tried it with and without producerFlowControl. I get stuff like:
2011-03-07 18:11:28,004 | ERROR | Failed to page in more queue messages
| org.apache.activemq.broker.region.Queue | Queue:scif_mets_pre_jhove_in
java.lang.OutOfMemoryError: Java heap space
at
org.apache.activemq.openwire.v6.BaseDataStreamMarshaller.looseUnmarshalByteSequence(BaseDataStreamMarshaller.java:639)
and
2011-03-07 18:17:25,011 | WARN | Execution of JMS message listener
failed, and no ErrorHandler has been set. |
org.springframework.jms.listener.DefaultMessageListenerContainer |
DefaultMessageListenerContainer-1
java.lang.OutOfMemoryError: Java heap space
Anybody help me? How can I keep the memory usage to a minimum, at
the expense of performance, disk space, anything else?
Am I just expecting too much because my messages are too big?
--
Illtud Daniel illtud.dan...@llgc.org.uk
Prif Swyddog Technegol Chief Technical Officer
Llyfrgell Genedlaethol Cymru National Library of Wales
