Hi Darrell,
Derby is the bottleneck for the Journal, in version 4.x by default
the entire message will be moved from the journal into the store (by
default Derby) - which is slow. In reality relational databases
aren't well suited to streaming queued data through. Kaha persistence
on the overhand doesn't use a journal, and is optimized for streaming
queued data. However, the big difference is in reality down to the
fact that Kaha doesn't ensure data is synced to disk between writes
by default.
For version 5 of activemq we've made big changes to the way messages
are passed through the broker. In version 4, references to the
messages in store were usually held in memory, which enabled
reasonable performance, meant the code was easier to maintain but
ultimately meant that activemq could not scale to coping with queue
depths of 100s of millions of messages without requiring a very large
memory footprint to hold those message references!
For version 5, message references are no longer held, but messages
are instead paged from the store when required. Its relatively
straight forward to pull messages from a queue when they are
required, but that is generally slow. So we've catered for a hybrid
push/pull model: We push (as in version 4) as long as the consumers
can keep up, but if they can't - we swap back to pulling out of the
store. This enables persistent messaging to have very good
performance when the consumers aren't slow, but also enables scaling
too.
However, having done this, the requirement on the message store is
more demanding than version 4. So the other big change we've done
for version 5 is to change the default messaging to use the journal
with kaha for the default messaging. We only need to hold references
in kaha - pointing to the actual message in the journal. This
combination is actually faster and more resilient than kaha in
version 4 - as if the reference store (kaha by default) is corrupted,
we can generate it again from the journal.
So long story short - use the default message store (AMQ) in version 5!
cheers,
Rob
http://rajdavies.blogspot.com/
On Sep 28, 2007, at 7:00 PM, DarrellFuller wrote:
Our system is taking 6~7 seconds to pull a message off of a queue
using a
simple selector (id = ?). The queue has about 20,000 messages
which are
fairly small. I ran through a quick series of tests and this is
what I came
up with.
** NO PERSISTENCE **
# of Messages = 20,000
Avg time to Send = 64 ms (per send) // Using a message converter
Avg time to read = 16 ms (per read) // Read 500 messages of
queue of
20,000 messages
** Default Journaling with Derby **
# of Messages = 20,000
Avg time to Send = Took over 20 minutes for test. Appox 1 min per
message
Avg time to read = Aborted Test after 30 minutes. Wasn't worth
completing.
** Using Kaha Persistence **
# of Messages = 20,000
Avg time to Send = ~ 500 ms (per send)
Avg time to read = ~ 1.2 seconds (per read)
My question is why is the default persistence appear to be SO SLOW...
Especially compared to the Kaha persistence.
Is Derby the bottleneck with the default journaling?
If not what is the difference between the standard Journaling and
Kaha?
Any guidance would be appreciated..
Thank you,
Darrell
--
View this message in context: http://www.nabble.com/Poort-ActiveMQ-
Persistence-Performance-tf4535973s2354.html#a12945570
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
