Hello.
I'm trying to find a minimal parameter set to test a JDBC adapter
configuration (using postgresql as DB) for ActiveMQ. For now I'm working
with an old ServiceMix that includes ActiveMQ version 5.14.5. It's more or
less the version of my customer platform.
To check if my troubles come from the JDBC adapter, I've also tested with
the base KahaDB persistence, and I've similar issues.
In the activemq.xml file I've added a datasource bean and replaced the
KahaDB adapter with the JDBC adapter. With the default settings for the
broker, a simple Camel context with two routes works as expected (or seems
to do it). My first route reads files and write them to a queue. The second
route just reads the queue.
I've tested this with files of several sizes. The considered maximum size
should be about 80 MB, and the average between 16 and 32 MB. My tests, with
this Camel context as is, still work fine with files of 128 MB.
I have an OutOfMemoryError when I start the broker after some messages have
been written into the queue, but the consuming route has been stopped. I've
tested with 128 messages of 16 MB, and my heap maximum size is set with 8
GB, and the broker is given 75% of the memory. It seems that despite no
consumer being created (the route that consumes is not started), the broker
loads all the remaining messages from the queue, and crashes.
I've tried some parameters I've found googling, but none resolves my OOM
error : I hoped that something like that "<policyEntry queue=">"
maxPageSize="20" queuePrefetch="1" lazyDispatch="true"/>" could help me,
but it doesn't. I've also set some optimization on the JDBC driver about
fetching (postgresql fetches all the rows of a resultset)...
Does someone know some way to make my broker start when messages remain
waiting in the queues. I guess that this situation is well known ?
Thanks for your help.
Regards.
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<beans
xmlns="http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd";>
<broker xmlns="http://activemq.apache.org/schema/core";
brokerName="${broker-name}"
dataDirectory="${data}"
start="false">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
<policyEntry queue=">" maxPageSize="20" queuePrefetch="1" lazyDispatch="true"/>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#postgres-ds" lockKeepAlivePeriod="5000">
<locker>
<lease-database-locker lockAcquireSleepInterval="10000"/>
</locker>
<adapter>
<postgresql-jdbc-adapter/>
</adapter>
</jdbcPersistenceAdapter>
</persistenceAdapter>
<plugins>
<jaasAuthenticationPlugin configuration="karaf" />
</plugins>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="75"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="5 mb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
</broker>
<bean id="postgres-ds" class="org.postgresql.ds.PGPoolingDataSource">
<property name="serverName" value="localhost"/>
<property name="portNumber" value="5432"/>
<property name="databaseName" value="AMQ"/>
<property name="user" value="amq1"/>
<property name="password" value="amq1"/>
<property name="initialConnections" value="1"/>
<property name="maxConnections" value="10"/>
<property name="applicationName" value="SMX ActiveMQ Broker"/>
<property name="defaultRowFetchSize" value="5"/>
</bean>
</beans>
