Hi folks,
I managed to get rid of the index of JMSMessages in JBoss.
(Based on JBoss-3.2.4RC2)
With my modified version I can have queues with (nearly)
arbitrary size. So far I tested with up to two million
messages in the queues :-).
This work is in a very early stage and I will spend more time
on it to make it more nice and stable. As soon as it works for
us, I'll make it available to JBoss and maybe it will even make
its way into future versions. If anyone wants to, you can get
my modifications any time, however, at the moment you don't
really want to see it :-).
I'll describe what I did later on and I'd be happy to see some
feedback about my approach. I'd also like to hear whether my
modifications seem to fit into future JBoss versions. In any
case, there are probably some modifications necessary to fit
your requirements (whatever they are :-).
I tried to stick with Adrians proposal. However, keep in mind that
my main concern is a reliable JMS for our production environment
(that's what I'm paid for).
Why
===
We use JBoss and JBossMQ in production (Thanks to all developers).
Most of our workflows are asynchronously implemented with JMS.
Therefore we need a JMS provider which is able to support
arbitrary queue sizes. Additionally we would like to have a
clustered JMS.
In our current cluster, the messages are spread over the nodes.
JBoss-3.2.4 provides clustered JMS with the deploy-hasingleton.
As a consequence, all messages are now on the master node and
the index will be much bigger than before (multiplied by the
number of nodes in the cluster). This will sooner or later
cause OutOfMemories. Now if the master node fails with an
OutOfMemory (due to the large index of messages), another node
will become master and deploy the jms subsystem which will
lead to an OutOfMemory. This would probably cause the whole
cluster to go down!
Requirements in mind
====================
- The queues must be able to handle an arbitrary number of messages
- QueueBrowser.getEnumeration must not cause an OutOfMemory
How it works
============
- I added a new column PRIORITIES to JMS_MESSAGES.
- The PersistenceManager (currently only jdbc2) provides a new method
public SpyMessage[] getNextMessages(long messageId, int priority, String
destination) throws JMSException;
This method returns a small array (currently 100 entries) of the next messages
AFTER the message with the given messageId and priority.
- The TreeSet in BasicQueue contains only the first 1000 Messages.
Each new message is inserted into the TreeSet and if the size
exceeds 1000 the last is removed. If the TreeSet is empty,
I want to receive messages, and there are more messages in the queue,
I check the database by calling getNextMessages(0, 10, destination);
- The methode getEnumeration in SpyQueueBrowser implements just a small
buffer. Any time all messages in the buffer are "consumed", it
recalls the BasicQueue which in turn calls the PersistenceManager
to read the next messages from the database.
- The method restoreQueue in PersistenceManager does nothing.
Any initialization is delayed until the first access to the queues.
Without this lazy initialization the size and max(messageId) will
be queried for all queues. This took around one minute for about 20
queues and 2 million messages.
I didn't do too many tests yet. It seems to work fine. MessageSelectors
are a problem, because in general they won't operate on the buffer.
If I don't want to have buffers per selectors, I think I have to
query the whole queue for each receive with a selector, am I right?
Any ideas or hints? Is there something totally wrong?
To be done
==========
- Tests. I haven't even checked subscribers so far, just
external QueueSender, QueueReceiver and QueueBrowser.
- Some minor optimizations (e.g.):
- Make it more sconfigurable (e.g. buffersize)
- Review my modifications and straighten them out
- The Enumeration returned by the QueueBrowser must be able to
provide an identification (e.g. messageId) of the last message consumed
in order to query for the next messages. However, the messageId
has no meaning outside of the server.
- Implement getNextMessages for the other PersistenceManagers
(I don't even know if this is possible with an acceptable performance)
- Exception handling. E.g. in BasicQueue.getQueueDepth I might get
exceptions from the persistence manager due to lazy initialization of
the queue.
- Topics. I never used them before. All I did so far is queues, although
I guess that it just works for Topis too.
Drawbacks:
==========
The Enumeration returned by QueueBrowser.getEnumeration might or might
not return messages sent to the queue after calling getEnumeration.
However, due to the API this seems to be ok.
Messageselectors might be very slow, depending on the number of
messages in the queue and the ratio of messages that will be
accepted by the selector.
I can't say too much about performance yet. However, my first rude
and unoptimized version seems to perfrom well.
Thoughts for further improvements:
==================================
- Asynchronous initialization of the queues
(Currently the first access to the queues is slow)
- Translation of MessageSelectors in SQL
(Nothing I want to do)
- Special Buffers for consumers with selectors
(Might be a housekeeping problem)
Further plans:
================
We don't like the singleton-jms-solution too much, since
the whole messagehandling is bound to the master node.
Even if the generation and processing of messages is dispatched,
the master might still become a bottleneck.
Don't undestand me wrong. It's still an improvement compared
to the old (completely unclustered) queue.
We envision a solution with jms deployed on all nodes in the
cluster synchronizing themselves either with Notifications
or based on the persistence manager.
We didn't spend too much thoughts on this yet, but as soon as
I'm done with the stuff written here, I'll give it a try.
(If I won't be assigned to some other projects).
--
Dietmar Posselt / IT-Infrastructure / Unix Development / Schlund + Partner AG
View the original post :
http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3836463#3836463
Reply to the post :
http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3836463
-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
JBoss-Development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development
