[GitHub] [james-project] thanhbv200585 commented on a diff in pull request #1501: JAMES-4865 Add FutureRelease Extension
thanhbv200585 commented on code in PR #1501: URL: https://github.com/apache/james-project/pull/1501#discussion_r1173326060 ## server/apps/distributed-app/docs/modules/ROOT/pages/configure/smtp-hooks.adoc: ## @@ -301,4 +301,21 @@ Example configuration: + + +== FUTURERELEASE hooks + +The Distributed server has optional support for FUTURERELEASE (link:https://www.rfc-editor.org/rfc/rfc4865.html[RFC-4865]) + + + +<...> + + + + + + + + Review Comment: I fixed it up already -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[james-project] 02/06: JAMES-3777 Incremental change POJO for JMAP filtering: handle updates
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit cebc98c0afbbb1d67931a85abd4af52ecccf915b
Author: Benoit Tellier
AuthorDate: Tue Apr 18 18:33:28 2023 +0700
JAMES-3777 Incremental change POJO for JMAP filtering: handle updates
TODO:
- Plug it in the event source system (aggregate, cassadra
event serialization)
- Conditionally disable incremental change via ENV and
explain that it should be disable during rolling updates
of the upgrade.
- Rebase the subscriber JAMES-3777: subscriber first might
need to load the aggregate again?
---
.../api/filtering/impl/IncrementalRuleChange.java | 26
.../james/jmap/api/filtering/RuleFixture.java | 7 +++-
.../filtering/impl/IncrementalRuleChangeTest.java | 47 +-
3 files changed, 53 insertions(+), 27 deletions(-)
diff --git
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
index 792b0fb8b9..9207663b35 100644
---
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
+++
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
@@ -22,8 +22,6 @@ package org.apache.james.jmap.api.filtering.impl;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
-import java.util.Set;
-import java.util.stream.Collectors;
import org.apache.james.eventsourcing.AggregateId;
import org.apache.james.eventsourcing.Event;
@@ -34,7 +32,6 @@ import com.google.common.base.MoreObjects;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Sets;
public class IncrementalRuleChange implements Event {
@@ -54,6 +51,7 @@ public class IncrementalRuleChange implements Event {
List commonElements =
ImmutableList.copyOf(Sets.intersection(idsBefore, idsAfter).immutableCopy());
ImmutableList idsAfterList = idsAfter.asList();
+ImmutableList.Builder updatedRules = ImmutableList.builder();
int prependedItems = 0;
int postPendedItems = 0;
boolean inPrepended = true;
@@ -98,8 +96,9 @@ public class IncrementalRuleChange implements Event {
return Optional.empty();
}
if (!beforeIndexed.get(id).equals(afterIndexed.get(id))) {
-// Rule content changed
-return Optional.empty();
+updatedRules.add(afterIndexed.get(id));
+position++;
+continue;
}
// All fine
position++;
@@ -122,7 +121,7 @@ public class IncrementalRuleChange implements Event {
.reverse();
return Optional.of(new IncrementalRuleChange(aggregateId, eventId,
-preprended, postPended, deleted));
+preprended, postPended, deleted, updatedRules.build()));
}
private final FilteringAggregateId aggregateId;
@@ -130,13 +129,15 @@ public class IncrementalRuleChange implements Event {
private final ImmutableList rulesPrepended;
private final ImmutableList rulesPostpended;
private final ImmutableSet rulesDeleted;
+private final ImmutableList rulesUpdated;
-public IncrementalRuleChange(FilteringAggregateId aggregateId, EventId
eventId, ImmutableList rulesPrepended, ImmutableList
rulesPostpended, ImmutableSet rulesDeleted) {
+public IncrementalRuleChange(FilteringAggregateId aggregateId, EventId
eventId, ImmutableList rulesPrepended, ImmutableList
rulesPostpended, ImmutableSet rulesDeleted, ImmutableList
rulesUpdated) {
this.aggregateId = aggregateId;
this.eventId = eventId;
this.rulesPrepended = rulesPrepended;
this.rulesPostpended = rulesPostpended;
this.rulesDeleted = rulesDeleted;
+this.rulesUpdated = rulesUpdated;
}
@Override
@@ -162,10 +163,17 @@ public class IncrementalRuleChange implements Event {
}
public ImmutableList apply(ImmutableList rules) {
+ImmutableMap indexedUpdates = rulesUpdated.stream()
+.collect(ImmutableMap.toImmutableMap(
+Rule::getId,
+rule -> rule));
+
return ImmutableList.builder()
.addAll(rulesPrepended)
.addAll(rules.stream()
.filter(rule -> !rulesDeleted.contains(rule.getId()))
+.map(rule ->
Optional.ofNullable(indexedUpdates.get(rule.getId()))
+.orElse(rule))
[james-project] 06/06: JAMES-3777 Upgrade instructions for filter increments
This is an automated email from the ASF dual-hosted git repository. btellier pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/james-project.git commit 3361957762889ea2f85b58dc4dde7e140748c5e5 Author: Benoit Tellier AuthorDate: Wed Apr 19 16:40:21 2023 +0700 JAMES-3777 Upgrade instructions for filter increments --- upgrade-instructions.md | 15 +++ 1 file changed, 15 insertions(+) diff --git a/upgrade-instructions.md b/upgrade-instructions.md index 64ac011062..a7d8d80bed 100644 --- a/upgrade-instructions.md +++ b/upgrade-instructions.md @@ -28,6 +28,21 @@ Change list: - [Adding delegatedUser column to user_table](#adding-delegatedusers-column-to-user-table) - [DeletedMessageVaultHook should not be used on Cassandra based products](#deleted-message-vault-is-now-deactivated-by-default) +### JMAP filters event sourcing increments and snapshots + +Date: 19/04/2023 + +JIRA: https://issues.apache.org/jira/browse/JAMES-3777 + +Concerned products: Distributed James, Cassandra James Server + +We recommend disabling increments during rolling updates (as older nodes in the cluster won't support them). +This can be done with the following system property: + +``` +-Djames.jmap.filters.eventsource.increments.enabled=false +``` + ### DeletedMessageVaultHook should not be used on Cassandra based products Date: 10/02/2023 - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[james-project] branch master updated (e4f0fb8a2d -> 3361957762)
This is an automated email from the ASF dual-hosted git repository. btellier pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/james-project.git from e4f0fb8a2d JAMES-3440 RFC-8621 support for emailQueryView before + inMailbox sorted by receivedAt new 07545e0e20 JAMES-3777 Incremental change POJO for JMAP filtering new cebc98c0af JAMES-3777 Incremental change POJO for JMAP filtering: handle updates new 893a98d23f JAMES-3777 Filtering aggregate should rely on increments new 7f445cd95d JAMES-3777 Serialization for IncrementalRuleChange new 0b7f40b2fd JAMES-3777 JVM propoerty to disable JMAP filtering increments new 3361957762 JAMES-3777 Upgrade instructions for filter increments The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../sample-configuration/jvm.properties| 7 +- .../sample-configuration/jvm.properties| 5 + .../james/modules/data/CassandraJmapModule.java| 1 + .../FilteringIncrementalRuleChangeDTO.java | 141 + .../FilteringRuleSetDefineDTOModules.java | 11 ++ ...sandraEventSourcingFilteringManagementTest.java | 5 +- .../james/jmap/cassandra/filtering/DTOTest.java| 22 +++ .../src/test/resources/json/increment.json | 31 +++ .../api/filtering/impl/FilteringAggregate.java | 17 +- .../api/filtering/impl/IncrementalRuleChange.java | 219 + .../james/jmap/api/filtering/RuleFixture.java | 6 + .../filtering/impl/IncrementalRuleChangeTest.java | 218 upgrade-instructions.md| 15 ++ 13 files changed, 694 insertions(+), 4 deletions(-) create mode 100644 server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/filtering/FilteringIncrementalRuleChangeDTO.java create mode 100644 server/data/data-jmap-cassandra/src/test/resources/json/increment.json create mode 100644 server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java create mode 100644 server/data/data-jmap/src/test/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChangeTest.java - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[james-project] 05/06: JAMES-3777 JVM propoerty to disable JMAP filtering increments
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 0b7f40b2fd00019eedd68b435fe2fb8b00bae675
Author: Benoit Tellier
AuthorDate: Wed Apr 19 11:45:42 2023 +0700
JAMES-3777 JVM propoerty to disable JMAP filtering increments
---
.../cassandra-app/sample-configuration/jvm.properties| 7 ++-
.../distributed-app/sample-configuration/jvm.properties | 5 +
.../jmap/api/filtering/impl/FilteringAggregate.java | 16
3 files changed, 23 insertions(+), 5 deletions(-)
diff --git a/server/apps/cassandra-app/sample-configuration/jvm.properties
b/server/apps/cassandra-app/sample-configuration/jvm.properties
index 96e4286ee1..7055fc8731 100644
--- a/server/apps/cassandra-app/sample-configuration/jvm.properties
+++ b/server/apps/cassandra-app/sample-configuration/jvm.properties
@@ -52,4 +52,9 @@ james.jmx.credential.generation=true
# Disable Remote Code Execution feature from JMX
# CF
https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/19fb8f93c59dfd791f62d41f332db9e306bc1422/src/java.management/share/classes/com/sun/jmx/remote/security/MBeanServerAccessController.java#L646
-jmx.remote.x.mlet.allow.getMBeansFromURL=false
\ No newline at end of file
+jmx.remote.x.mlet.allow.getMBeansFromURL=false
+
+# Disabling JMAP filters event source increments is necessary during rolling
adoption of this change.
+# Defaults to true, meaning James will use JMAP filters event source
increments, thus transparently and significantly
+# improving JMAP filter storage efficiency.
+# james.jmap.filters.eventsource.increments.enabled=true
\ No newline at end of file
diff --git a/server/apps/distributed-app/sample-configuration/jvm.properties
b/server/apps/distributed-app/sample-configuration/jvm.properties
index 3c62bafd29..2d5c47318e 100644
--- a/server/apps/distributed-app/sample-configuration/jvm.properties
+++ b/server/apps/distributed-app/sample-configuration/jvm.properties
@@ -53,3 +53,8 @@ james.jmx.credential.generation=true
# Disable Remote Code Execution feature from JMX
# CF
https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/19fb8f93c59dfd791f62d41f332db9e306bc1422/src/java.management/share/classes/com/sun/jmx/remote/security/MBeanServerAccessController.java#L646
jmx.remote.x.mlet.allow.getMBeansFromURL=false
+
+# Disabling JMAP filters event source increments is necessary during rolling
adoption of this change.
+# Defaults to true, meaning James will use JMAP filters event source
increments, thus transparently and significantly
+# improving JMAP filter storage efficiency.
+# james.jmap.filters.eventsource.increments.enabled=true
diff --git
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
index 51648f2e2b..d5ce85418d 100644
---
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
+++
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
@@ -34,6 +34,7 @@ import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
public class FilteringAggregate {
+private static final boolean ENABLE_INCREMENTS =
Boolean.parseBoolean(System.getProperty("james.jmap.filters.eventsource.increments.enabled",
"true"));
public static FilteringAggregate load(FilteringAggregateId aggregateId,
History eventsOfAggregate) {
return new FilteringAggregate(aggregateId, eventsOfAggregate);
@@ -70,14 +71,21 @@ public class FilteringAggregate {
public List defineRules(DefineRulesCommand storeCommand) {
Preconditions.checkArgument(shouldNotContainDuplicates(storeCommand.getRules()));
StateMismatchException.checkState(expectedState(storeCommand.getIfInState()),
"Provided state must be as same as the current state");
-
-ImmutableList events =
IncrementalRuleChange.ofDiff(aggregateId, history.getNextEventId(),
state.rules, storeCommand.getRules())
-.map(ImmutableList::of)
-.orElseGet(() -> ImmutableList.of(new RuleSetDefined(aggregateId,
history.getNextEventId(), ImmutableList.copyOf(storeCommand.getRules();
+ImmutableList events = generateEvents(storeCommand);
events.forEach(this::apply);
return events;
}
+private ImmutableList generateEvents(DefineRulesCommand
storeCommand) {
+if (ENABLE_INCREMENTS) {
+return IncrementalRuleChange.ofDiff(aggregateId,
history.getNextEventId(), state.rules, storeCommand.getRules())
+.map(ImmutableList::of)
+.orElseGet(() -> ImmutableList.of(new
RuleSetDefined(aggregateId, history.getNextEventId(),
ImmutableList.copyOf(storeCommand.getRules();
+} else {
+
[james-project] 04/06: JAMES-3777 Serialization for IncrementalRuleChange
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 7f445cd95ded31f6949bcbe761ccff21d7ef276f
Author: Benoit Tellier
AuthorDate: Wed Apr 19 11:38:56 2023 +0700
JAMES-3777 Serialization for IncrementalRuleChange
---
.../james/modules/data/CassandraJmapModule.java| 1 +
.../FilteringIncrementalRuleChangeDTO.java | 141 +
.../FilteringRuleSetDefineDTOModules.java | 11 ++
...sandraEventSourcingFilteringManagementTest.java | 5 +-
.../james/jmap/cassandra/filtering/DTOTest.java| 22
.../src/test/resources/json/increment.json | 31 +
6 files changed, 210 insertions(+), 1 deletion(-)
diff --git
a/server/container/guice/cassandra/src/main/java/org/apache/james/modules/data/CassandraJmapModule.java
b/server/container/guice/cassandra/src/main/java/org/apache/james/modules/data/CassandraJmapModule.java
index 77e64336b9..1dd7a16301 100644
---
a/server/container/guice/cassandra/src/main/java/org/apache/james/modules/data/CassandraJmapModule.java
+++
b/server/container/guice/cassandra/src/main/java/org/apache/james/modules/data/CassandraJmapModule.java
@@ -101,6 +101,7 @@ public class CassandraJmapModule extends AbstractModule {
Multibinder>
eventDTOModuleBinder = Multibinder.newSetBinder(binder(), new TypeLiteral<>()
{});
eventDTOModuleBinder.addBinding().toInstance(FilteringRuleSetDefineDTOModules.FILTERING_RULE_SET_DEFINED);
+
eventDTOModuleBinder.addBinding().toInstance(FilteringRuleSetDefineDTOModules.FILTERING_INCREMENT);
Multibinder.newSetBinder(binder(), UsernameChangeTaskStep.class)
diff --git
a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/filtering/FilteringIncrementalRuleChangeDTO.java
b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/filtering/FilteringIncrementalRuleChangeDTO.java
new file mode 100644
index 00..92cadb461d
--- /dev/null
+++
b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/filtering/FilteringIncrementalRuleChangeDTO.java
@@ -0,0 +1,141 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the*
+ * specific language governing permissions and limitations *
+ * under the License. *
+ /
+
+package org.apache.james.jmap.cassandra.filtering;
+
+import java.util.Objects;
+
+import org.apache.james.eventsourcing.EventId;
+import org.apache.james.eventsourcing.eventstore.cassandra.dto.EventDTO;
+import org.apache.james.jmap.api.filtering.Rule;
+import org.apache.james.jmap.api.filtering.impl.FilteringAggregateId;
+import org.apache.james.jmap.api.filtering.impl.IncrementalRuleChange;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+
+public class FilteringIncrementalRuleChangeDTO implements EventDTO {
+
+public static FilteringIncrementalRuleChangeDTO from(IncrementalRuleChange
event, String type) {
+return new FilteringIncrementalRuleChangeDTO(
+type, event.eventId().serialize(),
+event.getAggregateId().asAggregateKey(),
+RuleDTO.from(event.getRulesPrepended()),
+RuleDTO.from(event.getRulesPostPended()),
+RuleDTO.from(event.getRulesUpdated()),
+event.getRulesDeleted().stream()
+.map(id -> id.asString())
+.collect(ImmutableSet.toImmutableSet()));
+}
+
+public static FilteringIncrementalRuleChangeDTO from(IncrementalRuleChange
event) {
+return from(event, FilteringRuleSetDefineDTOModules.TYPE_INCREMENTAL);
+}
+
+private final String type;
+private
[james-project] 01/06: JAMES-3777 Incremental change POJO for JMAP filtering
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 07545e0e201d8da02a799cd3b15fb34e739cecb4
Author: Benoit Tellier
AuthorDate: Tue Apr 18 17:10:25 2023 +0700
JAMES-3777 Incremental change POJO for JMAP filtering
TODO:
- Incremental changes for rule modification
- Plug it in the event source system (aggregate, cassadra
event serialization)
- Conditionally disable incremental change via ENV and
explain that it should be disable during rolling updates
of the upgrade.
- Rebase the subscriber JAMES-3777: subscriber first might
need to load the aggregate again?
---
.../api/filtering/impl/IncrementalRuleChange.java | 205
.../james/jmap/api/filtering/RuleFixture.java | 1 +
.../filtering/impl/IncrementalRuleChangeTest.java | 207 +
3 files changed, 413 insertions(+)
diff --git
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
new file mode 100644
index 00..792b0fb8b9
--- /dev/null
+++
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
@@ -0,0 +1,205 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the*
+ * specific language governing permissions and limitations *
+ * under the License. *
+ /
+
+package org.apache.james.jmap.api.filtering.impl;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.james.eventsourcing.AggregateId;
+import org.apache.james.eventsourcing.Event;
+import org.apache.james.eventsourcing.EventId;
+import org.apache.james.jmap.api.filtering.Rule;
+
+import com.google.common.base.MoreObjects;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.ImmutableSortedSet;
+import com.google.common.collect.Sets;
+
+public class IncrementalRuleChange implements Event {
+public static Optional ofDiff(FilteringAggregateId
aggregateId, EventId eventId, List before, List after) {
+ImmutableSet idsBefore =
before.stream().map(Rule::getId).collect(ImmutableSet.toImmutableSet());
+ImmutableSet idsAfter =
after.stream().map(Rule::getId).collect(ImmutableSet.toImmutableSet());
+
+ImmutableMap beforeIndexed = before.stream()
+.collect(ImmutableMap.toImmutableMap(Rule::getId, rule -> rule));
+
+ImmutableMap afterIndexed = after.stream()
+.collect(ImmutableMap.toImmutableMap(Rule::getId, rule -> rule));
+
+// Deleted elements appears in
+ImmutableSet deleted = Sets.difference(idsBefore,
idsAfter).immutableCopy();
+
+List commonElements =
ImmutableList.copyOf(Sets.intersection(idsBefore, idsAfter).immutableCopy());
+
+ImmutableList idsAfterList = idsAfter.asList();
+int prependedItems = 0;
+int postPendedItems = 0;
+boolean inPrepended = true;
+boolean inCommonSection = false;
+boolean inPostpended = false;
+int position = 0;
+while (position < idsAfter.size()) {
+Rule.Id id = idsAfterList.get(position);
+if (inPrepended) {
+if (commonElements.contains(id)) {
+inPrepended = false;
+inCommonSection = true;
+continue;
+} else {
+prependedItems++;
+position++;
+continue;
+}
+}
+if (inPostpended) {
+if
[james-project] 03/06: JAMES-3777 Filtering aggregate should rely on increments
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 893a98d23f75f631c2397c407601c0fbba4b9061
Author: Benoit Tellier
AuthorDate: Wed Apr 19 10:09:15 2023 +0700
JAMES-3777 Filtering aggregate should rely on increments
---
.../apache/james/jmap/api/filtering/impl/FilteringAggregate.java | 9 +++--
.../james/jmap/api/filtering/impl/IncrementalRuleChange.java | 4
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
index 6e3baf1878..51648f2e2b 100644
---
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
+++
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/FilteringAggregate.java
@@ -70,8 +70,10 @@ public class FilteringAggregate {
public List defineRules(DefineRulesCommand storeCommand) {
Preconditions.checkArgument(shouldNotContainDuplicates(storeCommand.getRules()));
StateMismatchException.checkState(expectedState(storeCommand.getIfInState()),
"Provided state must be as same as the current state");
-ImmutableList events = ImmutableList.of(
-new RuleSetDefined(aggregateId, history.getNextEventId(),
ImmutableList.copyOf(storeCommand.getRules(;
+
+ImmutableList events =
IncrementalRuleChange.ofDiff(aggregateId, history.getNextEventId(),
state.rules, storeCommand.getRules())
+.map(ImmutableList::of)
+.orElseGet(() -> ImmutableList.of(new RuleSetDefined(aggregateId,
history.getNextEventId(), ImmutableList.copyOf(storeCommand.getRules();
events.forEach(this::apply);
return events;
}
@@ -101,5 +103,8 @@ public class FilteringAggregate {
if (event instanceof RuleSetDefined) {
state = state.set(((RuleSetDefined)event).getRules());
}
+if (event instanceof IncrementalRuleChange) {
+state =
state.set(((IncrementalRuleChange)event).apply(state.rules));
+}
}
}
diff --git
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
index 9207663b35..8d3c7d26e8 100644
---
a/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
+++
b/server/data/data-jmap/src/main/java/org/apache/james/jmap/api/filtering/impl/IncrementalRuleChange.java
@@ -162,6 +162,10 @@ public class IncrementalRuleChange implements Event {
return rulesDeleted;
}
+public ImmutableList getRulesUpdated() {
+return rulesUpdated;
+}
+
public ImmutableList apply(ImmutableList rules) {
ImmutableMap indexedUpdates = rulesUpdated.stream()
.collect(ImmutableMap.toImmutableMap(
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa merged pull request #1530: JAMES-3777 Incremental events for Filters
chibenwa merged PR #1530: URL: https://github.com/apache/james-project/pull/1530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] Arsnael commented on a diff in pull request #1533: JAMES-3900 Snapshots for polled updates
Arsnael commented on code in PR #1533: URL: https://github.com/apache/james-project/pull/1533#discussion_r1173287134 ## server/task/task-memory/src/main/scala/org/apache/james/task/eventsourcing/DecisionProjection.scala: ## @@ -32,7 +32,7 @@ case class DecisionProjection(status: Status, latestUpdateAdditionalInformationU case event: Cancelled => DecisionProjection(Status.CANCELLED, event.additionalInformation.map(_.timestamp)) case event: Completed => DecisionProjection(Status.COMPLETED, event.additionalInformation.map(_.timestamp)) case event: Failed => DecisionProjection(Status.FAILED, event.additionalInformation.map(_.timestamp)) -case event: AdditionalInformationUpdated => DecisionProjection(status, Some(event.additionalInformation.timestamp)) +case event: AdditionalInformationUpdated => DecisionProjection(Status.IN_PROGRESS, Some(event.additionalInformation.timestamp)) Review Comment: Case class indeed, my bad -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa commented on a diff in pull request #1501: JAMES-4865 Add FutureRelease Extension
chibenwa commented on code in PR #1501: URL: https://github.com/apache/james-project/pull/1501#discussion_r1173278941 ## server/apps/distributed-app/docs/modules/ROOT/pages/configure/smtp-hooks.adoc: ## @@ -301,4 +301,21 @@ Example configuration: + + +== FUTURERELEASE hooks + +The Distributed server has optional support for FUTURERELEASE (link:https://www.rfc-editor.org/rfc/rfc4865.html[RFC-4865]) + + + +<...> + + + + + + + + Review Comment: ```suggestion <...> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #1522: POC Code coverage for Apache James
chibenwa commented on PR #1522: URL: https://github.com/apache/james-project/pull/1522#issuecomment-1517215731 Code coverage: 227,121 of 846,037 | 73% Modulo scala issues: likely 120,000 of 846,037 | 85% -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #1522: POC Code coverage for Apache James
chibenwa commented on PR #1522: URL: https://github.com/apache/james-project/pull/1522#issuecomment-1517212357  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #1522: POC Code coverage for Apache James
chibenwa commented on PR #1522: URL: https://github.com/apache/james-project/pull/1522#issuecomment-1517209589  Looks like scala support is so-so ... Also, do we get a global aggregate? Per module view is nice, but an aggregate would be better -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] chibenwa commented on a diff in pull request #1533: JAMES-3900 Snapshots for polled updates
chibenwa commented on code in PR #1533: URL: https://github.com/apache/james-project/pull/1533#discussion_r1173268219 ## server/task/task-memory/src/main/scala/org/apache/james/task/eventsourcing/DecisionProjection.scala: ## @@ -32,7 +32,7 @@ case class DecisionProjection(status: Status, latestUpdateAdditionalInformationU case event: Cancelled => DecisionProjection(Status.CANCELLED, event.additionalInformation.map(_.timestamp)) case event: Completed => DecisionProjection(Status.COMPLETED, event.additionalInformation.map(_.timestamp)) case event: Failed => DecisionProjection(Status.FAILED, event.additionalInformation.map(_.timestamp)) -case event: AdditionalInformationUpdated => DecisionProjection(status, Some(event.additionalInformation.timestamp)) +case event: AdditionalInformationUpdated => DecisionProjection(Status.IN_PROGRESS, Some(event.additionalInformation.timestamp)) Review Comment: > I think I get the change, but what's the point of keeping status as an argument to the method update then? Status is not an argument of the update method but part of the state of DecisionProjection and as such we need it. This change is that the event fully determines the Decision projection disregardless of previous state (prerequisite for snapshot) but do not change the fact that status is needed to take a decision. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] Arsnael commented on pull request #1522: POC Code coverage for Apache James
Arsnael commented on PR #1522: URL: https://github.com/apache/james-project/pull/1522#issuecomment-1517200636 It seems the newer modules are the less covered in terms of tests (jmap, rate limiter, redis, rspamd, ...) Good job though :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] quantranhong1999 commented on pull request #1522: POC Code coverage for Apache James
quantranhong1999 commented on PR #1522: URL: https://github.com/apache/james-project/pull/1522#issuecomment-1517196057 At least from 1 green build, we had a beautiful code coverage report: [code coverage James.webm](https://user-images.githubusercontent.com/55171818/233531240-ab61ebde-f993-40d2-a827-f4edcbfcfb66.webm) The archive: [archive (1).zip](https://github.com/apache/james-project/files/11291907/archive.1.zip) Now I would polish the commits and expect a green build... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] Arsnael commented on a diff in pull request #1533: JAMES-3900 Snapshots for polled updates
Arsnael commented on code in PR #1533: URL: https://github.com/apache/james-project/pull/1533#discussion_r1172292146 ## server/task/task-memory/src/main/scala/org/apache/james/task/eventsourcing/DecisionProjection.scala: ## @@ -32,7 +32,7 @@ case class DecisionProjection(status: Status, latestUpdateAdditionalInformationU case event: Cancelled => DecisionProjection(Status.CANCELLED, event.additionalInformation.map(_.timestamp)) case event: Completed => DecisionProjection(Status.COMPLETED, event.additionalInformation.map(_.timestamp)) case event: Failed => DecisionProjection(Status.FAILED, event.additionalInformation.map(_.timestamp)) -case event: AdditionalInformationUpdated => DecisionProjection(status, Some(event.additionalInformation.timestamp)) +case event: AdditionalInformationUpdated => DecisionProjection(Status.IN_PROGRESS, Some(event.additionalInformation.timestamp)) Review Comment: I think I get the change, but what's the point of keeping status as an argument to the method update then? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] Arsnael commented on a diff in pull request #1532: JAMES-3777 Snapshots for eventSourcing JMAP filters
Arsnael commented on code in PR #1532:
URL: https://github.com/apache/james-project/pull/1532#discussion_r1172286945
##
event-sourcing/event-store-cassandra/src/test/scala/org/apache/james/eventsourcing/eventstore/cassandra/CassandraEventStoreTest.scala:
##
@@ -18,8 +18,41 @@
/
package org.apache.james.eventsourcing.eventstore.cassandra
-import org.apache.james.eventsourcing.eventstore.EventStoreContract
+import org.apache.james.eventsourcing.eventstore.cassandra.dto.SnapshotEvent
+import org.apache.james.eventsourcing.{EventId, TestEvent}
+import org.apache.james.eventsourcing.eventstore.{EventStore,
EventStoreContract, History}
+import org.assertj.core.api.Assertions.assertThat
+import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
+import reactor.core.scala.publisher.SMono
@ExtendWith(Array(classOf[CassandraEventStoreExtensionForTestEvents]))
-class CassandraEventStoreTest extends EventStoreContract
\ No newline at end of file
+class CassandraEventStoreTest extends EventStoreContract {
+ @Test
+ def test(testee: EventStore) : Unit = {
Review Comment:
a more meaningful test name maybe?
##
event-sourcing/event-store-cassandra/src/test/scala/org/apache/james/eventsourcing/eventstore/cassandra/CassandraEventStoreTest.scala:
##
@@ -18,8 +18,41 @@
/
package org.apache.james.eventsourcing.eventstore.cassandra
-import org.apache.james.eventsourcing.eventstore.EventStoreContract
+import org.apache.james.eventsourcing.eventstore.cassandra.dto.SnapshotEvent
+import org.apache.james.eventsourcing.{EventId, TestEvent}
+import org.apache.james.eventsourcing.eventstore.{EventStore,
EventStoreContract, History}
+import org.assertj.core.api.Assertions.assertThat
+import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
+import reactor.core.scala.publisher.SMono
@ExtendWith(Array(classOf[CassandraEventStoreExtensionForTestEvents]))
-class CassandraEventStoreTest extends EventStoreContract
\ No newline at end of file
+class CassandraEventStoreTest extends EventStoreContract {
+ @Test
+ def test(testee: EventStore) : Unit = {
+val event1 = TestEvent(EventId.first, EventStoreContract.AGGREGATE_1,
"first")
+val event2 = SnapshotEvent(EventId.first.next,
EventStoreContract.AGGREGATE_1, "second")
+val event3 = TestEvent(EventId.first.next.next,
EventStoreContract.AGGREGATE_1, "third")
+
+SMono(testee.append(event1)).block()
+SMono(testee.append(event2)).block()
+SMono(testee.append(event3)).block()
+
+
assertThat(SMono(testee.getEventsOfAggregate(EventStoreContract.AGGREGATE_1)).block())
+ .isEqualTo(History.of(event2, event3))
+ }
+
+ @Test
+ def test1(testee: EventStore) : Unit = {
Review Comment:
idem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] Arsnael commented on pull request #1519: JAMES-3777 Mitigate performance problems with JMAP filters
Arsnael commented on PR #1519: URL: https://github.com/apache/james-project/pull/1519#issuecomment-1515908022 Just realising... last commit is a fixup correct? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[GitHub] [james-project] quantranhong1999 commented on a diff in pull request #1501: JAMES-4865 Add FutureRelease Extension
quantranhong1999 commented on code in PR #1501: URL: https://github.com/apache/james-project/pull/1501#discussion_r1172121321 ## server/apps/distributed-app/docs/modules/ROOT/pages/configure/smtp-hooks.adoc: ## @@ -305,7 +305,7 @@ Example configuration: == FUTURE-RELEASE hooks -The Distributed server has optional support for DSN (link:https://www.rfc-editor.org/rfc/rfc4865.html[RFC-4865]) +The Distributed server has optional support for FUTURE-RELEASE (link:https://www.rfc-editor.org/rfc/rfc4865.html[RFC-4865]) Review Comment: IMO we should respect the specification ``` 1) The Extended Hello (EHLO) keyword associated with this service extension is "FUTURERELEASE". ``` No hyphen in between ## server/apps/distributed-app/docs/modules/ROOT/pages/configure/smtp-hooks.adoc: ## @@ -305,7 +305,7 @@ Example configuration: == FUTURE-RELEASE hooks Review Comment: this too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
