Problem with producer flow control and message dropping

[aparsuri]

Hello,

I am trying to create an activeMQ configuration (5.4.2.1) for the broker
which would make me achieve the following scenario -
1. If the consumer is slow, the producer SHOULD NOT slow down.
2. After the queue gets full in the broker, the broker should start writing
to disk and keep doing so (for an extremely large number of messages ( > 100
million or more or depending on the disk size available at least a few
gigs). 

I tried a lot of combinations for the config and either I have the producer
slowing down to catch up with the consumer or, I have a LOT of dropped
messages (not even close to the amount mentioned). The configs I have
currently is follows -

<broker xmlns="http://activemq.apache.org/schema/core";
brokerName="localhost" dataDirectory="${activemq.base}/data"
destroyApplicationContextOnStop="true">
             
        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry topic=">" producerFlowControl="false">
                        <messageEvictionStrategy>
                                        <oldestMessageEvictionStrategy />
                        </messageEvictionStrategy>
                        <pendingMessageLimitStrategy>
                                        <constantPendingMessageLimitStrategy 
limit="100000000"/>
                        </pendingMessageLimitStrategy>
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy> 
 
        <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

        <persistenceAdapter>
            <kahaDB directory="${activemq.base}/data/kahadb"/>
        </persistenceAdapter>
        
        
        <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage limit="64 mb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="100 gb"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="10 gb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>
                
                  
        <transportConnectors>
            <transportConnector name="openwire"
uri="tcp://0.0.0.0:61616?wireFormat.maxInactivityDuration=0"/>
        </transportConnectors>

</broker>

I tried to comment to systemUsage tag altogether and that does not seem to
effect anything. I also tried to change the number for
constantPendingMessageLimitStrategy and it did not seem to change a whole
lot (lot of messages are still being dropped). I also tried
<prefetchRatePendingMessageLimitStrategy multiplier="2"/> instead of
constantPendingMessageLimitStrategy  but, that doesn't seem to work at all.
The producers kept slowing down rather than messages being dropped. 

Any help would be appreciated. Thank you.

- Apar Suri.

--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Problem-with-producer-flow-control-and-message-dropping-tp4161641p4161641.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.