I just looked at the source.
Could please one explain the following settings from HashIndex
DEFAULT_PAGE_SIZE =
Integer.parseInt(System.getProperty("defaultPageSize", "1024"));
DEFAULT_KEY_SIZE =
Integer.parseInt(System.getProperty("defaultKeySize", "96"));
DEFAULT_BIN_SIZE=
Integer.parseInt(System.getProperty("defaultBinSize", "1024"));
MAXIMUM_CAPACITY =
Integer.parseInt(System.getProperty("maximumCapacity", "16384"));
DEFAULT_LOAD_FACTOR=Integer.parseInt(System.getProperty("defaultLoadFactor","50"));
which are used within AMQPersistanceAdapter:
private int maxFileLength =
AsyncDataManager.DEFAULT_MAX_FILE_LENGTH;
private int indexBinSize = HashIndex.DEFAULT_BIN_SIZE;
private int indexKeySize = HashIndex.DEFAULT_KEY_SIZE;
private int indexPageSize = HashIndex.DEFAULT_PAGE_SIZE;
private int indexMaxBinSize = HashIndex.MAXIMUM_CAPACITY;
private int indexLoadFactor = HashIndex.DEFAULT_LOAD_FACTOR;
If I currently look into my activemq/data/kr-store/data directory it looks
like
-rw-r--r-- 1 activemq java 2097119 2009-01-22 09:06 data-queue-data-994
-rw-r--r-- 1 activemq java 2097108 2009-01-22 09:06 data-queue-data-995
-rw-r--r-- 1 activemq java 2097119 2009-01-22 09:07 data-queue-data-996
-rw-r--r-- 1 activemq java 2097141 2009-01-22 09:08 data-queue-data-997
-rw-r--r-- 1 activemq java 2097134 2009-01-22 09:09 data-queue-data-998
-rw-r--r-- 1 activemq java 2097133 2009-01-22 09:10 data-queue-data-999
-rw-r--r-- 1 activemq java 1036253185 2009-01-22 12:05
hash-index-queue-data_queue#3a#2f#2ftracking
-rw-r--r-- 1 activemq java 153 2009-01-21 16:47 index-container-roots
-rw-r--r-- 1 activemq java 491847009 2009-01-22 12:05 index-queue-data
-rw-r--r-- 1 activemq java 0 2009-01-21 16:46 lock
It looks like each file is about 2MB big. Which attribute controls this file
size?
I don't see it match any of the above attibutes.
What is the content of "hash-index-queue-data_queue#3a#2f#2ftracking"?
This file is quite big.
Thanks for helping!
Jörg
rajdavies wrote:
>
> The index will grow up to a max number of bins - increasing that
> maximum will increase performance - e.g.
>
> <bean id="store"
> class="org.apache.activemq.store.amq.AMQPersistenceAdapter">
> <property name="directory" value="${activemq.base}/
> data"/>
> <property name="maxFileLength" value="10000000"/>
> <property name="checkpointInterval" value="60000"/>
> <property name="cleanupInterval" value="30000"/>
> <property name="persistentIndex" value="true"/>
> <property name="indexMaxBinSize" value="131072"/>
> </bean>
>
>
>
>
> cheers,
>
> Rob
>
> Rob Davies
> http://fusesource.com
> http://rajdavies.blogspot.com/
>
>
> On 22 Jan 2009, at 10:32, Cybexion wrote:
>
>>
>> Hi,
>>
>> I have a queue that should hold up to 30 GB of small persistent
>> messages:
>> <systemUsage sendFailIfNoSpace="true">
>> <memoryUsage>
>> <memoryUsage limit="1gb"/>
>> </memoryUsage>
>> <storeUsage>
>> <storeUsage limit="30gb" store="#store"/>
>> </storeUsage>
>>
>> The queue consists of 10 MB files and are specified like this:
>> <bean id="store"
>> class="org.apache.activemq.store.amq.AMQPersistenceAdapter">
>> <property name="directory" value="${activemq.base}/
>> data"/>
>> <property name="maxFileLength" value="10000000"/>
>> <property name="checkpointInterval" value="60000"/>
>> <property name="cleanupInterval" value="30000"/>
>> <property name="persistentIndex" value="true"/>
>> </bean>
>>
>> This night I made a test with lots of produceds an just 5 consumer.
>> I wanted
>> to fill the queue to its limit.
>> Well, so far I managed to send 18 Million messages into the queue. I
>> have 20
>> % Store usage so far.
>> What I noticed is, that mostly the messages are send within 0-20
>> milliseconds. However sometimes I have a lot of messages which take
>> 10+
>> seconds to be send. This is not good in my scenario. I need a
>> producer that
>> should be very very fast in sending the messages.
>> Question now:
>> Could it be that the number of data files or the size of the
>> datafiles has
>> impact on the performance?
>> Would it be better to have one queue big file with 30GB of size
>> instead of
>> hundreds of files having 1 MB?
>>
>> The key index ist persistent. I tested this and it seems to be the
>> only way
>> of NOT getting an out of memory error.
>> Does it make sense to change the indexkeysize or the indexpagesize
>> in the
>> store? Would this maybe gice additional constant performance?
>>
>> Btw:
>> I'm using AMQ 5.2
>>
>> Thanks and best regards
>>
>> Jörg
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Performance-in-relation-to-queue-size-tp21601585p21601585.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
>
>
>
--
View this message in context:
http://www.nabble.com/Performance-in-relation-to-queue-size-tp21601585p21605634.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
