[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202315=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202315 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 22/Feb/19 00:58 Start Date: 22/Feb/19 00:58 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466230700 tests are good.. .merged This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202315) Time Spent: 14.5h (was: 14h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 14.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202314 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 22/Feb/19 00:58 Start Date: 22/Feb/19 00:58 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202314) Time Spent: 14h 20m (was: 14h 10m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 14h 20m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202237=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202237 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 21/Feb/19 21:15 Start Date: 21/Feb/19 21:15 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466169435 @michaelandrepearce everything looks good. Let me just run testsuite one time to be sure.. and i will merge it if everything good. testsuite takes 3 hours on my box... I should be able to merge this either tognitght or early morning if no test issues. I don't foresee any major issues from what I looked.. but I wanted to make sure before merging a big PR like this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202237) Time Spent: 14h 10m (was: 14h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 14h 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202225=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202225
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 21/Feb/19 20:45
Start Date: 21/Feb/19 20:45
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r259108524
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Yes its similar to amq5.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 202225)
Time Spent: 14h (was: 13h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 14h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202223=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202223 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 21/Feb/19 20:43 Start Date: 21/Feb/19 20:43 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466158873 @michaelandrepearce man.. you put quite a lot of work here... nice one! What I liked here is that you configured the number of groups at queue level, during creation. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202223) Time Spent: 13h 50m (was: 13h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 13h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=20=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-20
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 21/Feb/19 20:41
Start Date: 21/Feb/19 20:41
Worklog Time Spent: 10m
Work Description: clebertsuconic commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r259106232
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
@jbertram pointed me this is the same on AMQ5
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 20)
Time Spent: 13h 40m (was: 13.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 13h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202221
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 21/Feb/19 20:41
Start Date: 21/Feb/19 20:41
Worklog Time Spent: 10m
Work Description: clebertsuconic commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r259106148
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
that's just stripping out negatives.. got it!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 202221)
Time Spent: 13.5h (was: 13h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 13.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202218=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202218
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 21/Feb/19 20:38
Start Date: 21/Feb/19 20:38
Worklog Time Spent: 10m
Work Description: clebertsuconic commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r259105263
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ohh.. you hash the GroupID... got it!
I feel dumb that I don't understand why the bitwise operation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 202218)
Time Spent: 13h 20m (was: 13h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 13h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202209=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202209
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 21/Feb/19 20:20
Start Date: 21/Feb/19 20:20
Worklog Time Spent: 10m
Work Description: clebertsuconic commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r259099336
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Did you implement this from scratch or did you base it on the ActiveMQ5
bucket?
We had an outstanding task to implement it based on AMQ5 version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 202209)
Time Spent: 13h 10m (was: 13h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 13h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202205 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 21/Feb/19 20:13 Start Date: 21/Feb/19 20:13 Worklog Time Spent: 10m Work Description: franz1981 commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466149045 I've asked the review of Clebert or Justin too: please wait until their "yes". I could have missed something so better 4 eyes then just 2... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202205) Time Spent: 13h (was: 12h 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 13h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202204=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202204 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 21/Feb/19 20:12 Start Date: 21/Feb/19 20:12 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466148675 Can I have a day to review this? Say tomorrow? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202204) Time Spent: 12h 50m (was: 12h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 12h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=202198=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202198 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 21/Feb/19 20:08 Start Date: 21/Feb/19 20:08 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-466147272 Answered your queries franz, Ill look to merge on monday if no further comments. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 202198) Time Spent: 12h 40m (was: 12.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 12h 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=201283=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201283
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 20/Feb/19 12:52
Start Date: 20/Feb/19 12:52
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258466675
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
@franz1981 there is also explicit tests for concat already in
org.apache.activemq.artemis.tests.util.SimpleStringTest.testConcat
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 201283)
Time Spent: 12.5h (was: 12h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 12.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=201282=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201282
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 20/Feb/19 12:51
Start Date: 20/Feb/19 12:51
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258466675
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
there is also explicit tests for concat in
org.apache.activemq.artemis.tests.util.SimpleStringTest.testConcat
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 201282)
Time Spent: 12h 20m (was: 12h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200937=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200937
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:17
Start Date: 19/Feb/19 22:17
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258253358
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
It basically avoiding a new object and a byte array being created and thrown
away unneededly, when concat.
If its contentious i can revert.
as not actually needed for this feature as just a perf improvement, just
something i noticed when looking at object creation when profiling during dev
of the feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200937)
Time Spent: 11h 50m (was: 11h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200940=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200940
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:18
Start Date: 19/Feb/19 22:18
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247750
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
Its existing method its already covered, so functionality is covered by
existing.
also i checked to be sure and it shows as being tested when using code
coverage.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200940)
Time Spent: 12h 10m (was: 12h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 12h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200939=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200939
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:17
Start Date: 19/Feb/19 22:17
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258253358
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
It basically avoiding a new object and a byte array being created and thrown
away unneededly, when concat.
If its contentious i can revert.
as not actually needed for this feature as just a perf improvement, just
something i noticed when looking at object creation when profiling during dev
of the feature. And thought it be welcomed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200939)
Time Spent: 12h (was: 11h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 12h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200936=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200936
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:16
Start Date: 19/Feb/19 22:16
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258253358
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
It basically avoiding a new object and a byte array being created and thrown
away uneededly.
If its contentious i can revert.
as not actually needed for this feature as just a perf improvement, just
something i noticed when looking at object creation when profiling during dev
of the feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200936)
Time Spent: 11h 40m (was: 11.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200935=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200935
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:15
Start Date: 19/Feb/19 22:15
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258253358
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
It basically avoiding a new object and a byte array being created and thrown
away uneededly. If its contentious i can revert, as not actually needed for
this feature, just something i noticed when looking at object creation when
profiling during dev of the feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200935)
Time Spent: 11.5h (was: 11h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200928=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200928
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:11
Start Date: 19/Feb/19 22:11
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247586
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ive responded to this same comment several times. The power of two trick
only works if it is power of two, we have to honour the bucket count set so it
may not be power of two meaning we cant use that trick. I actually added
specific test to ensure bucket count is honoured
The actual performance compared to everything is tiny (eg i ran this vs
simple and no actual noticable diff end2end e.g just normal variance) Also this
is optional feature. Lastly Note we actually have mod in journal code fyi.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200928)
Time Spent: 11h 10m (was: 11h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11h 10m
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200929=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200929
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:12
Start Date: 19/Feb/19 22:12
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247750
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
Its existing code its already covered,
also i checked to be sure and it shows as being tested when using code
coverage.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200929)
Time Spent: 11h 20m (was: 11h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200911=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200911
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 21:58
Start Date: 19/Feb/19 21:58
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247750
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
I assume as existing code its already covered
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200911)
Time Spent: 9h 50m (was: 9h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200926=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200926
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:08
Start Date: 19/Feb/19 22:08
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247586
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ive responded to this same comment several times. The power of two trick
only works if it is power of two, we have to honour the bucket count set so it
may not be power of two meaning we cant use that trick. I actually added
specific test to ensure bucket count is honoured
The actual performance compared to everything is tiny. Also this is
optional. Note we actually have mod in journal code fyi.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200926)
Time Spent: 11h (was: 10h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 11h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200923=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200923 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 22:07 Start Date: 19/Feb/19 22:07 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465330174 Ive run what i can of the CI and integration tests Tbh i have never managed to run a full one successfully. Its one of my biggest bugs it seems you need voodo magic. Would be nice if we had a once a night full ci run on master rather than relying on as special box that can run ci but isnt owned or visible to apache This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200923) Time Spent: 10h 50m (was: 10h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 10h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200922 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 22:06 Start Date: 19/Feb/19 22:06 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465330174 Ive run what i can of the CI and integration tests Tbh i have never managed to run a full one. Its one of my biggest bugs it seems you need voodo magic. Would be nice if we had a once a night full ci run on master rather than relying on as special box that can run ci but isnt owned or visible to apache This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200922) Time Spent: 10h 40m (was: 10.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 10h 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200921 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 22:06 Start Date: 19/Feb/19 22:06 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465330174 Ive run what i can of the CI and integration tests Tbh i have never managed to run a full one. Its one of my biggest bugs it seems you need voodo magic. Would be nice if we had a once a night full ci run on master rather than relying on cleberts special box. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200921) Time Spent: 10.5h (was: 10h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 10.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200918
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:02
Start Date: 19/Feb/19 22:02
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247750
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
Its existing code its already covered.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200918)
Time Spent: 10h 20m (was: 10h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 10h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200917=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200917
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:01
Start Date: 19/Feb/19 22:01
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247586
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ive responded to this same comment several times yes. The power of two trick
only works if it is power of two and we have to honour the bucket count set so
it may not be. I actually added specific test to ensure bucket count is
honoured
The actual performance compared to everything is tiny. Also this is
optional. Note we actually have mod in journal code fyi.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200917)
Time Spent: 10h 10m (was: 10h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200915=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200915
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 22:01
Start Date: 19/Feb/19 22:01
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247586
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ive responded to this same comment several times yes. The power of two trick
only works if it is power of two and we have to honour the bucket count set so
it may not be. I actually added specific test to ensure bucket count is
honoured
The actual performance compared to everything is tiny. Note we actually have
mod in journal code fyi.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200915)
Time Spent: 10h (was: 9h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 10h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200909
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 21:58
Start Date: 19/Feb/19 21:58
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258247586
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
Ive responded to this same comment several times yes. That only works if
buckets is power of two and we have to honour the bucket count set.
The actual performance compared to everything is tiny. Note we actually have
mod in journal code fyi.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200909)
Time Spent: 9h 40m (was: 9.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200779=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200779
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 17:19
Start Date: 19/Feb/19 17:19
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258143450
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/BucketMessageGroups.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Predicate;
+import org.apache.activemq.artemis.api.core.SimpleString;
+
+/**
+ * BucketMessageGroups, stores values against a bucket, where the bucket used
is based on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific key.
+ *
+ * The number of buckets is provided at construction.
+ */
+public class BucketMessageGroups implements MessageGroups {
+
+ //This _AMQ_GROUP_BUCKET_INT_KEY uses the post-fixed value after this key,
as an int, it is used for a few cases:
+ //1) For the admin screen we need to show a group key so we have to map
back from int to something, as it expects SimpleString.
+ //2) Admin users still need to interact with a specific bucket/group e.g.
they may need to reset a bucket.
+ //3) Choice of key is we want to avoid risk of clashing with users groups
keys.
+ //4) Actually makes testing a little easier as we know how the parsed int
will hash.
+ private static SimpleString _AMQ_GROUP_BUCKET_INT_KEY = new
SimpleString("_AMQ_GROUP_BUCKET_INT_KEY_");
+
+ private final int bucketCount;
+ private C[] buckets;
+ private int size = 0;
+
+ public BucketMessageGroups(int bucketCount) {
+ if (bucketCount < 1) {
+ throw new IllegalArgumentException("Bucket count must be greater than
0");
+ }
+ this.bucketCount = bucketCount;
+ }
+
+ private int getBucket(SimpleString key) {
+ Object bucketKey = key;
+ if (key.startsWith(_AMQ_GROUP_BUCKET_INT_KEY)) {
+ bucketKey = retrieveBucketIntFromKey(key);
+ }
+ return getHashBucket(bucketKey, bucketCount);
+ }
+
+ private static int getHashBucket(final Object key, final int bucketCount) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
% is a quite heavy operation: masking with a power of 2 bucketCount would be
better if possible.
This is suppose to be on the hot path?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200779)
Time Spent: 9.5h (was: 9h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200777=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200777 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 17:13 Start Date: 19/Feb/19 17:13 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465224306 @franz1981 seems no more comments and been now several days, if you're happy could you merge please? Or even let me know if you're happy for me to merge it myself. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200777) Time Spent: 9h 10m (was: 9h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 9h 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200778=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200778
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 19/Feb/19 17:19
Start Date: 19/Feb/19 17:19
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r258141968
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/api/core/SimpleString.java
##
@@ -454,7 +454,16 @@ public boolean contains(final char c) {
* @return the concatenated SimpleString
*/
public SimpleString concat(final String toAdd) {
- return concat(new SimpleString(toAdd));
+ int len = toAdd.length();
Review comment:
This concat method need test coverage IMO
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 200778)
Time Spent: 9h 20m (was: 9h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200775=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200775 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 17:13 Start Date: 19/Feb/19 17:13 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465224306 @franz1981 if you're happy could you merge please? Or even let me know if you're happy for me to merge it myself. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200775) Time Spent: 9h (was: 8h 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 9h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200774 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 17:12 Start Date: 19/Feb/19 17:12 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465224306 @franz1981 if you're happy could you merge please? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200774) Time Spent: 8h 50m (was: 8h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=200773=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200773 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 19/Feb/19 17:12 Start Date: 19/Feb/19 17:12 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-465224306 @franz1981 if you're happy could you merge please? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 200773) Time Spent: 8h 40m (was: 8.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8h 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199128 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 09:00 Start Date: 15/Feb/19 09:00 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463959867 @franz1981 done, now using a single array (thats lazy initiated) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199128) Time Spent: 8.5h (was: 8h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199114=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199114 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 08:21 Start Date: 15/Feb/19 08:21 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463949198 @franz1981 and now because of the refactor and simpler methods we need to support i think i can easily implement the buckets one with a single array ... its like a win win .. just implementing the array bit now for you. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199114) Time Spent: 8h 20m (was: 8h 10m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8h 20m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199054 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 01:21 Start Date: 15/Feb/19 01:21 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463867719 @franz1981 the needing to copy a IntObjectHashmap, still didnt fit right with me. As such have had a sit and think. Have done a small refactor that means we can still use IntObjectHashmap without needing to clone it over. Essentially in queueimpl for message groups, we only use, put get remove clear size iterate over the collection so we can for remove consumers. As such rather than using map interface i made a new Interface named 'MessageGroups' with just this subset, as such makes implementations that are not backed by a plain forward map a little easier as less methods to contend with and also we can add or tweek an methods signature if needed, e.g. i can still use IntObjectHashMap but i don't have to make it fully implement Map. Lastly should others be wanted in the future as less methods to implement if the backing store isnt a map will be easier to implement. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199054) Time Spent: 8h 10m (was: 8h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8h 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199053=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199053 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 01:15 Start Date: 15/Feb/19 01:15 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463867719 @franz1981 the needing to copy a IntObjectHashmap, still didnt fit right with me. As such have had a sit and think. Have done a small refactor that means we can still use IntObjectHashmap without needing to clone it over. Essentially in queueimpl for message groups, we only use, put get remove clear size iterate over the collection so we can for remove consumers. As such rather than using map interface i made a new Interface named 'MessageGroups' with just this subset, as such makes implementations that are not backed by a plain forward map a little easier as less methods to contend with and also we can add or tweek an methods signature if needed, e.g. i can still use IntObjectHashMap but i don't have to make it fully implement Map. Also should others be wanted in the future as less methods to implement if the backing store isnt a map will be easier to implement. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199053) Time Spent: 8h (was: 7h 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 8h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199052=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199052 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 01:14 Start Date: 15/Feb/19 01:14 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463867719 @franz1981 the needing to copy a IntObjectHashmap, still didnt fit right with me. As such have had a sit and think. Have done a small refactor that means we can still use IntObjectHashmap without needing to clone it over. Essentially in queueimpl for message groups, we only use, put get remove clear size iterate over the collection so we can for remove consumers. As such rather than using map interface i made a new Interface named 'MessageGroups' with just this subset, as such makes implementations that are not backed by a plain forward map a little easier as less methods to contend with and also we can add or tweek an methods signature if needed, e.g. i can still use IntObjectHashMap but i don't have to make it fully implement Map. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199052) Time Spent: 7h 50m (was: 7h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 7h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=199049=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199049 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 15/Feb/19 01:11 Start Date: 15/Feb/19 01:11 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463867719 @franz1981 the needing to copy a IntObjectHashmap, still didnt fit right with me. As such have had a sit and think. Have done a small refactor that means we can still use IntObjectHashmap without needing to clone it over. Essentially in queueimpl for message groups, we only use, put get remove clear size iterate over the collection so we can for remove consumers. As such rather than using map interface i made a new Interface named 'MessageGroups' with just this subset, as such makes implementations that are not backed by a plain forward map a little easier as less methods to contend with, e.g. i can still use IntObjectHashMap but i don't have to make it fully implement Map. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 199049) Time Spent: 7h 40m (was: 7.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 7h 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198690 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 14/Feb/19 14:11 Start Date: 14/Feb/19 14:11 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463640868 @franz1981 Hopefully fixed the checkstyle, netty used 4 space tabs we use 3, lol, i feel like an episode of "silicone valley" now. Anyhow once PR build completes and goes green are you happy? Also i sent an PR to netty, so fingers crossed we don't need to have this clone/fork too long. https://github.com/netty/netty/pull/8866 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 198690) Time Spent: 7.5h (was: 7h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 7.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198600=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198600
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:22
Start Date: 14/Feb/19 09:22
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256745053
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 just because of that netty thing, i may have to have our own
version again of IntObjectHashMap.
What im thinking of doing, is so we dont hold back this feature and it makes
the 2.7.0 cut, is clone in the netty varient with a fix to our collections
(they're apache license so no license issues in doing this, ill just add an
attribution to the java doc), and then if netty fix it, then we can remove it
again at some fututre point.
Ill make this on purpose a separate commit, so its very clear and also a
simple revert once netty fix, and we upgrade. Before you ask, yes i will be
submitting a PR to netty ;)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198600)
Time Spent: 7h 10m (was: 7h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198601=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198601
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:22
Start Date: 14/Feb/19 09:22
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256745053
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 just because of that netty thing, i may have to have our own
version again of IntObjectHashMap.
What im thinking of doing, is so we dont hold back this feature and it makes
the 2.7.0 cut, is clone in the netty varient with a fix to our collections
(they're apache license so no license issues in doing this, ill just add an
attribution to the java doc), and then if netty fix it, then we can remove it
again at some fututre point.
Ill make this on purpose a separate commit, so its very clear and also a
simple revert once netty fix, and we upgrade. Before you ask, yes i will be
submitting a PR to netty ;) to address this there. bloody annoying.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198601)
Time Spent: 7h 20m (was: 7h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198598=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198598
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:13
Start Date: 14/Feb/19 09:13
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256745053
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 just because of that netty thing, i may have to have our own
version again of IntObjectHashMap.
What im thinking of doing, is so we dont hold back this feature and it makes
the 2.7.0 cut, is clone in the netty varient with a fix to our collections
(they're apache license so no license issues in doing this, ill just add an
attribution to the java doc), and then if netty fix it, then we can remove it
again at some fututre point.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198598)
Time Spent: 7h (was: 6h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 7h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198596
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:12
Start Date: 14/Feb/19 09:12
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256745053
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 just because of that netty thing, i may have to have our own
version again.
What im thinking of doing, is so we dont hold back this feature and it makes
the 2.7.0 cut, is clone in the netty varient with a fix to our collections
(they're apache license so no license issues in doing this, ill just add an
attribution to the java doc), and then if netty fix it, then we can remove it
again at some fututre point.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198596)
Time Spent: 6h 50m (was: 6h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198590=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198590
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:04
Start Date: 14/Feb/19 09:04
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256741638
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
I could implement it on Netty bud, np :)
While about the issue of size...we can just use a null value and lazy
initialize it I suppose.
I have done something similar on transactions...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198590)
Time Spent: 6h 40m (was: 6.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198587=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198587
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:57
Start Date: 14/Feb/19 08:57
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256739594
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 issue with array is imaging someone on their broker (that someone
being me) wants to ensure group buckets by default across the broker, as such
sets address setting match = #.
Then for every queue will create a fixed large empty array, even if the
producers on some address doesn't set message groups.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198587)
Time Spent: 6h 10m (was: 6h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198589=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198589
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 09:02
Start Date: 14/Feb/19 09:02
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256741638
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
I could implement it on Netty bud, np :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198589)
Time Spent: 6.5h (was: 6h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198588=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198588
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:57
Start Date: 14/Feb/19 08:57
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256739594
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 issue with array is imaging someone on their broker (that someone
being me) wants to ensure group buckets by default across the broker, as such
makes use of setting default in address settings, and sets address setting
match = #.
Then for every queue will create a fixed large empty array, even if the
producers on some address doesn't set message groups.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198588)
Time Spent: 6h 20m (was: 6h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198585=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198585
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:56
Start Date: 14/Feb/19 08:56
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256739594
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
@franz1981 issue with array is imaging someone on their broker (that someone
being me) wants to ensure group buckets by default across the broker to say
20k, as such sets address setting match = #.
Then for every queue will create a 20k array, even if the producer on the
address doesnt set message groups.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198585)
Time Spent: 6h (was: 5h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 6h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198584=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198584
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:53
Start Date: 14/Feb/19 08:53
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256738673
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Ive actually found a small issue in Netty's implementation...they don't
support values().itertator().remove(), annoyingly theres no reason either, the
underlying iterator they facade would support it gr
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198584)
Time Spent: 5h 50m (was: 5h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198580=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198580
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:33
Start Date: 14/Feb/19 08:33
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket. Like wise if someone uses a non power of 2 bucket count
that should be honored, e.g. say 5 buckets was set, then we must only have use
5 buckets, not the next power of 2 buckets. I'll actually add an explicit test
for this.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
Also codally its cleaner as simply we hash mod the key and then delegate
method to underlying map, meaning it has single responsibility. Rather than
having to have alot of logic in this class that deals with other bits that are
unrelated, such as keep track of size, supporting make set implementation for
values etc. etc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198580)
Time Spent: 5h 40m (was: 5.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198577=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198577
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:30
Start Date: 14/Feb/19 08:30
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket. Like wise if someone uses a non power of 2 bucket count
that should be honored, e.g. say 5 buckets was set, then we must only have use
5 buckets, not the next power of 2 buckets. I'll actually add an explicit test
for this.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
Also codally its cleaner as simply we hash mod the key and then delegate
method to underlying map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198577)
Time Spent: 5.5h (was: 5h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues,
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198546=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198546
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:20
Start Date: 14/Feb/19 08:20
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256728547
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
I will have some time on the next week and I will look better into this. ATM
I have just my phone :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198546)
Time Spent: 5h 20m (was: 5h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198545
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:19
Start Date: 14/Feb/19 08:19
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket. Like wise if someone uses a non power of 2 bucket count
that should be honored, e.g. say 5 buckets was set, then we must only have use
5 buckets, not the next power of 2 buckets.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
Also codally its cleaner as simply we hash mod the key and then delegate
method to underlying map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198545)
Time Spent: 5h 10m (was: 5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198544=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198544
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 08:18
Start Date: 14/Feb/19 08:18
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256728127
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Not sure...consider that the map will allocate both the int and obj arrays
multiplied by the load factor. And we will pay the rehashing too..a fixed size
thing will be more "direct" (less pointer chasing) and simple (just obj array,
never resized).
Is just a personal opinion eh, I'm anyway happy about it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198544)
Time Spent: 5h (was: 4h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198523=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198523
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 06:49
Start Date: 14/Feb/19 06:49
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
Also codally its cleaner as simply we hash mod the key and then delegate
method to underlying map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198523)
Time Spent: 4h 50m (was: 4h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198522=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198522
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 06:48
Start Date: 14/Feb/19 06:48
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
Also codally its cleaner as simply we has key and then delegate method to
underlying map.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198522)
Time Spent: 4h 40m (was: 4.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198521
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 06:44
Start Date: 14/Feb/19 06:44
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256706799
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
Collisions are 100% expected we are bucketing. E.g if bucket size is two and
the key was integer with keys 1 and 3 i actually expect them to collide and
goto the same bucket.
Re map vs array, i was toying on the same thing, downside with array is up
front will need to allocate the memory for that array even if not all buckets
used. Benefit of using underlying map is it will grow as needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198521)
Time Spent: 4.5h (was: 4h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198508
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 06:06
Start Date: 14/Feb/19 06:06
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256700662
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
And..idea..given that you are % on bucket count is you could have
collisions. Maybe you don't need a map here but just an array bucketCount
sized...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198508)
Time Spent: 4h 20m (was: 4h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198507=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198507
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 14/Feb/19 06:04
Start Date: 14/Feb/19 06:04
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256700265
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import io.netty.util.collection.IntObjectHashMap;
+import io.netty.util.collection.IntObjectMap;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+/**
+ * BucketMap, stores values against a bucket, where the bucket used is based
on the provided key objects hash.
+ *
+ * As such where keys compute to the same bucket they will act on that stored
value, not the unique specific keys as you'd normally expect from a map.
+ *
+ * The number of buckets is provided at construction.
+ *
+ * Its initial use, is in QueueImpl, where we want to bucket message groups to
consumers.
+ *
+ * @param the key type.
+ * @param the value type.
+ */
+public class BucketMap implements Map {
+
+ private final IntObjectMap buckets = new IntObjectHashMap<>();
Review comment:
I'm not sure if I have already left this comment but % is really an heavy
operation to be done on hot path so I suggest to use the power of 2 + mask trick
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198507)
Time Spent: 4h 10m (was: 4h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198402=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198402 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 23:47 Start Date: 13/Feb/19 23:47 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463423401 @franz1981 as noted i found Netty has an int primitive hashmap implementation using open addressing implementation so can use that, so removed IntHashMap to avoid maintenance and testing burden. added tests for NoOpMap, BucketMap as well as docs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 198402) Time Spent: 3h 50m (was: 3h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198404=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198404 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 23:48 Start Date: 13/Feb/19 23:48 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463423401 @franz1981 as noted i found Netty has an int primitive hashmap implementation using open addressing much the same, so can use that instead of having our own IntHashMap, thus removed IntHashMap to avoid maintenance and testing burden. added tests for NoOpMap, BucketMap as well as docs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 198404) Time Spent: 4h (was: 3h 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 4h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198383
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 22:50
Start Date: 13/Feb/19 22:50
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256623980
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/IntHashMap.java
##
@@ -0,0 +1,878 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import java.io.Serializable;
+import java.util.AbstractCollection;
+import java.util.AbstractSet;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntFunction;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * {@link java.util.Map} implementation specialised for int keys using open
addressing and
+ * linear probing for cache efficient access.
+ *
+ * @param type of values stored in the {@link java.util.Map}
+ */
+public class IntHashMap implements Map, Serializable {
+ /**
+* Default load factor to be used in open addressing hashed data structures.
+*/
+ static final float DEFAULT_LOAD_FACTOR = 0.55f;
+ static final int MIN_CAPACITY = 8;
+
+ private final float loadFactor;
+ private int resizeThreshold;
+ private int size;
+ private final boolean shouldAvoidAllocation;
+
+ private int[] keys;
+ private Object[] values;
+
+ private ValueCollection valueCollection;
+ private KeySet keySet;
+ private EntrySet entrySet;
+
+ public IntHashMap() {
+ this(MIN_CAPACITY, DEFAULT_LOAD_FACTOR, true);
+ }
+
+ public IntHashMap(
+ final int initialCapacity,
+ final float loadFactor) {
+ this(initialCapacity, loadFactor, true);
+ }
+
+ /**
+* Construct a new map allowing a configuration for initial capacity and
load factor.
+* @param initialCapacity for the backing array
+* @param loadFactor limit for resizing on puts
+* @param shouldAvoidAllocation should allocation be avoided by caching
iterators and map entries.
+*/
+ public IntHashMap(
Review comment:
I actually dont need this class, just realised Netty has one, so can use it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198383)
Time Spent: 3h 40m (was: 3.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
>
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198378
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 22:34
Start Date: 13/Feb/19 22:34
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256617634
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+public class BucketMap implements Map {
+
+ private final Map buckets = new IntHashMap<>();
+ private final int bucketCount;
+
+ public BucketMap(int bucketCount) {
+ this.bucketCount = bucketCount;
+ }
+
+ @Override
+ public int size() {
+ return buckets.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+ return buckets.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+ return buckets.containsKey(getBucket(key));
+ }
+
+ private int getBucket(Object key) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
So its always positive. This is quite a typical way of bucketing on object
hashes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198378)
Time Spent: 3.5h (was: 3h 20m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198373=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198373
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 22:31
Start Date: 13/Feb/19 22:31
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256617634
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+public class BucketMap implements Map {
+
+ private final Map buckets = new IntHashMap<>();
+ private final int bucketCount;
+
+ public BucketMap(int bucketCount) {
+ this.bucketCount = bucketCount;
+ }
+
+ @Override
+ public int size() {
+ return buckets.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+ return buckets.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+ return buckets.containsKey(getBucket(key));
+ }
+
+ private int getBucket(Object key) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
So its always positive.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198373)
Time Spent: 3h 20m (was: 3h 10m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198371=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198371
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 22:30
Start Date: 13/Feb/19 22:30
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256617515
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/QueueImpl.java
##
@@ -3347,6 +3346,16 @@ public void onError(int errorCode, String errorMessage)
{
}
+ public static Map groupMap(int groupBuckets) {
+ if (groupBuckets == -1) {
+ return new HashMap<>();
Review comment:
No we cant the key is simplestring
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198371)
Time Spent: 3h (was: 2h 50m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198372=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198372
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 22:30
Start Date: 13/Feb/19 22:30
Worklog Time Spent: 10m
Work Description: michaelandrepearce commented on pull request #2548:
ARTEMIS-2118 Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256617634
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/BucketMap.java
##
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+
+public class BucketMap implements Map {
+
+ private final Map buckets = new IntHashMap<>();
+ private final int bucketCount;
+
+ public BucketMap(int bucketCount) {
+ this.bucketCount = bucketCount;
+ }
+
+ @Override
+ public int size() {
+ return buckets.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+ return buckets.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+ return buckets.containsKey(getBucket(key));
+ }
+
+ private int getBucket(Object key) {
+ return (key.hashCode() & Integer.MAX_VALUE) % bucketCount;
Review comment:
So its always positive
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198372)
Time Spent: 3h 10m (was: 3h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198345=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198345
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 21:56
Start Date: 13/Feb/19 21:56
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256599873
##
File path:
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/collections/IntHashMap.java
##
@@ -0,0 +1,878 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.utils.collections;
+
+import java.io.Serializable;
+import java.util.AbstractCollection;
+import java.util.AbstractSet;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntFunction;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * {@link java.util.Map} implementation specialised for int keys using open
addressing and
+ * linear probing for cache efficient access.
+ *
+ * @param type of values stored in the {@link java.util.Map}
+ */
+public class IntHashMap implements Map, Serializable {
+ /**
+* Default load factor to be used in open addressing hashed data structures.
+*/
+ static final float DEFAULT_LOAD_FACTOR = 0.55f;
+ static final int MIN_CAPACITY = 8;
+
+ private final float loadFactor;
+ private int resizeThreshold;
+ private int size;
+ private final boolean shouldAvoidAllocation;
+
+ private int[] keys;
+ private Object[] values;
+
+ private ValueCollection valueCollection;
+ private KeySet keySet;
+ private EntrySet entrySet;
+
+ public IntHashMap() {
+ this(MIN_CAPACITY, DEFAULT_LOAD_FACTOR, true);
+ }
+
+ public IntHashMap(
+ final int initialCapacity,
+ final float loadFactor) {
+ this(initialCapacity, loadFactor, true);
+ }
+
+ /**
+* Construct a new map allowing a configuration for initial capacity and
load factor.
+* @param initialCapacity for the backing array
+* @param loadFactor limit for resizing on puts
+* @param shouldAvoidAllocation should allocation be avoided by caching
iterators and map entries.
+*/
+ public IntHashMap(
Review comment:
If it is taken from agrona I would reference it somehow to both give proper
credits and track eventual issues on it...
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198345)
Time Spent: 2h 40m (was: 2.5h)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[
https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=198347=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198347
]
ASF GitHub Bot logged work on ARTEMIS-2118:
---
Author: ASF GitHub Bot
Created on: 13/Feb/19 21:56
Start Date: 13/Feb/19 21:56
Worklog Time Spent: 10m
Work Description: franz1981 commented on pull request #2548: ARTEMIS-2118
Enhanced Message Groups Support
URL: https://github.com/apache/activemq-artemis/pull/2548#discussion_r256605066
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/QueueImpl.java
##
@@ -3347,6 +3346,16 @@ public void onError(int errorCode, String errorMessage)
{
}
+ public static Map groupMap(int groupBuckets) {
+ if (groupBuckets == -1) {
+ return new HashMap<>();
Review comment:
You can use the IntObjMap here as well
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 198347)
Time Spent: 2h 50m (was: 2h 40m)
> Enhanced Message Groups Support
>
>
> Key: ARTEMIS-2118
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2118
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Reporter: Michael Andre Pearce
>Assignee: Michael Andre Pearce
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> This ticket is for supporting some Message Group Enhancements.
>
> Currently Artemis implements message groups in a simple way, equivalent to
> SimpleMessageGroupMap in activemq5.
> [http://activemq.apache.org/message-groups.html]
> "SimpleMessageGroupMap keeps track of every group but suffers from unbounded
> memory use."
>
> For use cases migrating from activemq5 its important artemis supports similar
> MessageGroupHashBucked feature.
> "MessageGroupHashBucked keeps track of every group and has bounded memory
> use."
> As such, this JIRA is to add support for being able to enable a semantic
> equivalent to MessageGroupHashBucket from activemq5, at the queue level.
>
> Also currently it is not possible for a specific queue disable grouping, this
> can be useful where a shared address by many queues, where some consumers
> care for ordering and others do not.
> Lastly add an ability to rebalance message groups when a consumer is added,
> this is useful where you want groups more evenly balanced when consumers come
> online, even after dispatching started.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197971 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:43 Start Date: 13/Feb/19 07:43 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling with out special handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily. RE: JMSXGroupLastForConsumer its not possible to retro-acitvely add a header, because consumers can be added at anypoint, and at that point the group may not have a message to dispatch onwards it would not be possible so it would not be possible to add. Also note for AMQP messages we cannot alter by spec. RE: consumer releasing a message it maybe already processing, again no, any already dispatched messages wont be affected, also there is no scope in JMS api to support such release semantic. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. Obviously if you find a way to add those further enhancements you're always welcome to contribute such features yourself. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197971) Time Spent: 1h 10m (was: 1h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197979=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197979 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:57 Start Date: 13/Feb/19 07:57 Worklog Time Spent: 10m Work Description: billpoole-mi commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463096446 Okay great! You are correct regarding a header for when a group is assigned to a consumer. That's the JMSXGroupFirstForConsumer header in ActiveMQ 5.x, described [here](http://activemq.apache.org/message-groups.html). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197979) Time Spent: 2.5h (was: 2h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197975=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197975 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:46 Start Date: 13/Feb/19 07:46 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463093738 @billpoole-mi as i said, if/when this merges, you're more than welcome to contribute further enhancements you want. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197975) Time Spent: 1h 50m (was: 1h 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197976 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:47 Start Date: 13/Feb/19 07:47 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463093738 @billpoole-mi as i said, if/when this merges, you're more than welcome to contribute further enhancements you want. Contributions are always welcome. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197976) Time Spent: 2h (was: 1h 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197978=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197978 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:53 Start Date: 13/Feb/19 07:53 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463095530 So yes i think if you find a way to elegantly achieve that without a performance impact i personally would be open to reviewing it. I like the idea. Another further enhancement could be to add header when a group is assigned to a consumer, for signal of firstmessage in group also, i think there is a jira in backlog for that one already. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197978) Time Spent: 2h 20m (was: 2h 10m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197973=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197973 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:44 Start Date: 13/Feb/19 07:44 Worklog Time Spent: 10m Work Description: billpoole-mi commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463093142 There may be a workable workflow for transferring ownership of a message group, where that workflow is cooperation between Artemis and the existing consumer assigned for that message group. What if a message group is flagged for rebalancing when the need arises. Then, the next message to arrive in that message group is sent to the existing consumer with a special header that informs the consumer that the message group is being reassigned. That consumer is then expected to "release" all messages it is working on in that message group back to Artemis, which then sends them to the newly assigned consumer for that message group. If no further message turns up, then it's not really a load balancing issue. What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197973) Time Spent: 1.5h (was: 1h 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197974=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197974 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:44 Start Date: 13/Feb/19 07:44 Worklog Time Spent: 10m Work Description: billpoole-mi commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463093142 There may be a workable workflow for transferring ownership of a message group, where that workflow is a cooperation between Artemis and the existing consumer assigned for that message group. What if a message group is flagged for rebalancing when the need arises. Then, the next message to arrive in that message group is sent to the existing consumer with a special header that informs the consumer that the message group is being reassigned. That consumer is then expected to "release" all messages it is working on in that message group back to Artemis, which then sends them to the newly assigned consumer for that message group. If no further message turns up, then it's not really a load balancing issue. What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197974) Time Spent: 1h 40m (was: 1.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197972=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197972 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:43 Start Date: 13/Feb/19 07:43 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling with out special handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily, and lazily expire out. RE: JMSXGroupLastForConsumer its not possible to retro-acitvely add a header, because consumers can be added at anypoint, and at that point the group may not have a message to dispatch onwards it would not be possible so it would not be possible to add. Also note for AMQP messages we cannot alter by spec. RE: consumer releasing a message it maybe already processing, again no, any already dispatched messages wont be affected, also there is no scope in JMS api to support such release semantic. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. Obviously if you find a way to add those further enhancements you're always welcome to contribute such features yourself. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197972) Time Spent: 1h 20m (was: 1h 10m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197977=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197977 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:48 Start Date: 13/Feb/19 07:48 Worklog Time Spent: 10m Work Description: billpoole-mi commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463094268 @michaelandrepearce no worries. I really appreciate all the work you guys are doing and just wanted to check whether something like I proposed would be seen by the team as valuable/worthwhile or not. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197977) Time Spent: 2h 10m (was: 2h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197970=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197970 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:41 Start Date: 13/Feb/19 07:41 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily. RE: JMSXGroupLastForConsumer its not possible to retro-acitvely add a header, because consumers can be added at anypoint, and at that point the group may not have a message to dispatch onwards it would not be possible so it would not be possible to add. Also note for AMQP messages we cannot alter by spec. RE: consumer releasing a message it maybe already processing, again no, any already dispatched messages wont be affected, also there is no scope in JMS api to support such release semantic. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. Obviously if you find a way to add those further enhancements you're always welcome to contribute such features yourself. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197970) Time Spent: 1h (was: 50m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197969 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:38 Start Date: 13/Feb/19 07:38 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily. RE: JMSXGroupLastForConsumer its not possible to retro-acitvely add a header, because consumers can be added at anypoint, and at that point the group may not have a message to dispatch onwards it would not be possible so it would not be possible to add. Also note for AMQP messages we cannot alter by spec. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197969) Time Spent: 50m (was: 40m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197968=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197968 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:37 Start Date: 13/Feb/19 07:37 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily. RE: JMSXGroupLastForConsumer its not possible to retro-acitvely add a header, because consumers can be added at anypoint, and at that point the group may not have a message to dispatch onwards it would not be possible so it would not be possible to add. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197968) Time Spent: 40m (was: 0.5h) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197967=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197967 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 07:36 Start Date: 13/Feb/19 07:36 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463091333 @billpoole-mi No there wont be any special message. The intent here is for apps where you want evenly load balanced groups but where apps can handle a slight change in which groups they are handling, e.g. this is very usefull where a stateless app that simply needs to process and enrich data, and produces onwards. Any data relating to the group can be loaded lazily. RE: JMSXGroupLastForConsumer Because an consumers can be added at anypoint, when the group may not have a message to dispatch onwards it would not be possible it would not be possible to add. This said if you have the need, you can though listen for notification CONSUMER_ADDED on notifications address, and as such could then if you know that queue is using that feature, could know rebalance will occur. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197967) Time Spent: 0.5h (was: 20m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197888=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197888 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 13/Feb/19 02:09 Start Date: 13/Feb/19 02:09 Worklog Time Spent: 10m Work Description: billpoole-mi commented on issue #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548#issuecomment-463025385 I have a couple of questions regarding the "support rebalancing groups when a consumer is added" aspect of this change: - When a message group X is rebalanced from consumer A to consumer B, how will consumer A be informed that it is no longer assigned message group X and shouldn't expect any further messages in that message group? Will Artemis send the consumer a message, perhaps with JMSXGroupSeq = -1, or perhaps some kind of special header like JMSXGroupLastForConsumer? - Can consumer A release possession of the messages in message group X that consumer A is still processing (i.e. hasn't yet acknowledged) by acknowledging those messages as "released" (as opposed to accepted/rejected), such that those messages are then resent by Artemis to consumer B? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197888) Time Spent: 20m (was: 10m) > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (ARTEMIS-2118) Enhanced Message Groups Support
[ https://issues.apache.org/jira/browse/ARTEMIS-2118?focusedWorklogId=197256=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-197256 ] ASF GitHub Bot logged work on ARTEMIS-2118: --- Author: ASF GitHub Bot Created on: 11/Feb/19 23:36 Start Date: 11/Feb/19 23:36 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on pull request #2548: ARTEMIS-2118 Enhanced Message Groups Support URL: https://github.com/apache/activemq-artemis/pull/2548 Support using group buckets on a queue for better local group scaling Support disabling message groups on a queue Support rebalancing groups when a consumer is added. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 197256) Time Spent: 10m Remaining Estimate: 0h > Enhanced Message Groups Support > > > Key: ARTEMIS-2118 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2118 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker >Reporter: Michael Andre Pearce >Assignee: Michael Andre Pearce >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This ticket is for supporting some Message Group Enhancements. > > Currently Artemis implements message groups in a simple way, equivalent to > SimpleMessageGroupMap in activemq5. > [http://activemq.apache.org/message-groups.html] > "SimpleMessageGroupMap keeps track of every group but suffers from unbounded > memory use." > > For use cases migrating from activemq5 its important artemis supports similar > MessageGroupHashBucked feature. > "MessageGroupHashBucked keeps track of every group and has bounded memory > use." > As such, this JIRA is to add support for being able to enable a semantic > equivalent to MessageGroupHashBucket from activemq5, at the queue level. > > Also currently it is not possible for a specific queue disable grouping, this > can be useful where a shared address by many queues, where some consumers > care for ordering and others do not. > Lastly add an ability to rebalance message groups when a consumer is added, > this is useful where you want groups more evenly balanced when consumers come > online, even after dispatching started. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
