Hi Marco,
Please see comments inline.
Regards
Martyn
On 08/06/15 20:48, Marco wrote:
Hi all,
first, thanks for the release of v1.0.0 of Artemis!
I'm currently experimenting with the use of message groups to quickly
distribute messages to workers while keeping order on items belonging
together. In order to avoid artificial grouping/sharding it would be perfect
for my use case if the UUID of an object could be used as the message group
ID (to keep the order of modifications to this object).
As mentioned e.g. here
<http://scott.cranton.com/2010/09/activemq-message-groups.html> , there
seems to be a need to reconfigure the "classic" ActiveMQ to use a different
"messageGroupMapFactory" in order to use more than 1024 message groups. Of
course, in theory, the UUID maps could be much larger than my memory can
possibly hold, but if the implementation would throw away message group
assignments if the map is full (i.e. roll over) while there is currently no
message for this group ID, then this should work fine. The next incoming
UUID that was thrown away before would just be reassigned to one of the
consumers with the "first for this consumer" flag set.
My questions are:
- Does Artemis use the same default map as "classic" ActiveMQ, i.e. is there
a limitation to 1024 group IDs?
We do not limit the number of groups IDs.
- If yes, how can it be reconfigured to allow for more values?
- If not, how is Artemis currently designed to handle this?
Each Queue has associated with it, a mapping from groups to consumer.
We do a lookup to decide which consumer to forward to when the message
is routed.
- Is this use case feasible / Are there any experiences with such a
grouping?
One thing to note with the approach you outlined above. The Group ID on
a Queue exists for the entire life time of the consumer that is
associated with it. In other words, Group IDs will not be removed from
a Queues internal mapping until the consumer is closed. If in your use
case above, you have long lived consumers and are creating new groups
regularly, the group -> consumer map could potentially grow quite
large. You can't explicitly remove groups from a queue, but you can get
around this by setting a Local Group Handler (example below), which will
clean up stale groups (Groups that have not received a message for a
certain period of time).
For information on setting up GroupHandlers in a clustered environment
see: https://activemq.apache.org/artemis/docs/1.0.0/message-grouping.html
GroupHandler config, (add to broker.xml):
|<grouping-handler name="my-grouping-handler">
<type>LOCAL</type>
<address>Foo</address> <!--||This is a required field but is only used in a
clustered topology, you can set this to anything for now. The Group Handler is global, and works
on all addresses. -->
<group-timeout>60000</group-timeout> <!-- The amount of time in ms before a
group goes stale -->
</grouping-handler>|
Thanks in advance,
Marco
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Artemis-JMSXGroupID-Message-groups-Large-number-of-groups-tp4697436.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
