[james-project] 10/13: JAMES-2807 Simplify maven structure for the vault

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 092e127f8f324b1f51bce87b58882856c517227b
Author: Benoit Tellier 
AuthorDate: Mon Jul 1 14:17:53 2019 +0700

JAMES-2807 Simplify maven structure for the vault

Define the metadata storage API within the vault API.

As the memory implementation is shipped as part of the vault API, include
the memory implementation for metadata storage there too.
---
 .../pom.xml| 87 --
 .../deleted-messages-vault-blobstore/pom.xml   | 86 -
 mailbox/plugin/deleted-messages-vault/pom.xml  |  4 +
 .../MemoryDeletedMessageMetadataVault.java |  5 +-
 .../metadata}/DeletedMessageMetadataVault.java |  2 +-
 .../DeletedMessageWithStorageInformation.java  |  4 +-
 .../james/vault/metadata}/StorageInformation.java  |  2 +-
 .../MemoryDeletedMessageMetadataVaultTest.java |  4 +-
 .../DeletedMessageMetadataVaultContract.java   |  9 ++-
 .../DeletedMessageVaultMetadataFixture.java|  2 +-
 .../DeletedMessageWithStorageInformationTest.java  |  5 +-
 .../vault/metadata}/StorageInformationTest.java|  6 +-
 mailbox/pom.xml|  2 -
 pom.xml| 11 ---
 14 files changed, 28 insertions(+), 201 deletions(-)

diff --git a/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
deleted file mode 100644
index faf54ee..000
--- a/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
+++ /dev/null
@@ -1,87 +0,0 @@
-
-
-http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
-4.0.0
-
-
-apache-james-mailbox
-org.apache.james
-3.4.0-SNAPSHOT
-../../pom.xml
-
-
-
apache-james-mailbox-deleted-messages-vault-blobstore-memory
-Apache James :: Mailbox :: Plugin :: Deleted Messages Vault :: 
BlobStore :: Memory
-Memory implementation of the metadata storage API required by 
the Deleted Messages Vault implementation on top of BlobStore
-
-
-
-${james.groupId}
-apache-james-mailbox-api
-test-jar
-test
-
-
-${james.groupId}
-
apache-james-mailbox-deleted-messages-vault
-test-jar
-test
-
-
-${james.groupId}
-
apache-james-mailbox-deleted-messages-vault-blobstore
-
-
-${james.groupId}
-
apache-james-mailbox-deleted-messages-vault-blobstore
-test-jar
-test
-
-
-${james.groupId}
-apache-james-mailbox-memory
-test
-
-
-${james.groupId}
-apache-james-mailbox-memory
-test
-test-jar
-
-
-com.google.guava
-guava
-
-
-org.assertj
-assertj-core
-test
-
-
-org.junit.jupiter
-junit-jupiter-engine
-test
-
-
-org.junit.platform
-junit-platform-launcher
-test
-
-
-
-
\ No newline at end of file
diff --git a/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml 
b/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml
deleted file mode 100644
index 5575f57..000
--- a/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml
+++ /dev/null
@@ -1,86 +0,0 @@
-
-
-http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
-4.0.0
-
-
-apache-james-mailbox
-org.apache.james
-3.4.0-SNAPSHOT
-../../pom.xml
-
-
-
apache-james-mailbox-deleted-messages-vault-blobstore
-Apache James :: Mailbox :: Plugin :: Deleted Messages Vault :: 
BlobStore
-Apache James Mailbox Deleted Messages Vault implementation on 
top of BlobStore
-
-
-
-${james.groupId}
-apache-james-mailbox-api
-test-jar
-test
-
-
-${james.groupId}
-
apache-james-mailbox-deleted-messages-vault
-
-
-${james.groupId}
-
apache-james-mailbox-deleted-messages-vault
-test-jar
-test
-
-
-${james.groupId}
-apache-james-mailbox-memory
-test
-
-
-${james.groupId}
-apache-james-mai

[james-project] 08/13: JAMES-2807 DeletedMessageMetadataVault API and memory skeleton

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit c7177df2f5be728270516db4580cba6e4acefa13
Author: Benoit Tellier 
AuthorDate: Tue Jun 25 11:31:16 2019 +0700

JAMES-2807 DeletedMessageMetadataVault API and memory skeleton
---
 .../pom.xml| 40 +++
 .../vault/MemoryDeletedMessageMetadataVault.java   | 58 ++
 .../james/vault/DeletedMessageMetadataVault.java   | 39 +++
 mailbox/pom.xml|  1 +
 pom.xml| 11 
 5 files changed, 149 insertions(+)

diff --git a/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
new file mode 100644
index 000..d5ae54e
--- /dev/null
+++ b/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
@@ -0,0 +1,40 @@
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+4.0.0
+
+
+apache-james-mailbox
+org.apache.james
+3.4.0-SNAPSHOT
+../../pom.xml
+
+
+
apache-james-mailbox-deleted-messages-vault-blobstore-memory
+Apache James :: Mailbox :: Plugin :: Deleted Messages Vault :: 
BlobStore :: Memory
+Memory implementation of the metadata storage API required by 
the Deleted Messages Vault implementation on top of BlobStore
+
+
+
+org.apache.james
+
apache-james-mailbox-deleted-messages-vault-blobstore
+
+
+
+
+
\ No newline at end of file
diff --git 
a/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
new file mode 100644
index 000..33f11f6
--- /dev/null
+++ 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
@@ -0,0 +1,58 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.vault;
+
+import org.apache.commons.lang3.NotImplementedException;
+import org.apache.james.blob.api.BucketName;
+import org.apache.james.core.User;
+import org.apache.james.mailbox.model.MessageId;
+import org.reactivestreams.Publisher;
+
+public class MemoryDeletedMessageMetadataVault implements 
DeletedMessageMetadataVault {
+@Override
+public Publisher store(DeletedMessageWithStorageInformation 
deletedMessage) {
+throw new NotImplementedException("Not yet implemented");
+}
+
+@Override
+public Publisher removeBucket(BucketName bucketName) {
+throw new NotImplementedException("Not yet implemented");
+}
+
+@Override
+public Publisher remove(BucketName bucketName, User user, MessageId 
messageId) {
+throw new NotImplementedException("Not yet implemented");
+}
+
+@Override
+public Publisher retrieveStorageInformation(User user, 
MessageId messageId) {
+throw new NotImplementedException("Not yet implemented");
+}
+
+@Override
+public Publisher 
listMessages(BucketName bucketName, User user) {
+throw new NotImplementedException("Not yet implemented");
+}
+
+@Override
+public Publisher listBuckets() {
+throw new NotImplementedException("Not yet implemented");
+}
+}
diff --git 
a/mailbox/plugin/deleted-messages-vault-blobstore/src/main/java/org/apache/james/vault/DeletedMessageMetadataVault.java
 
b/mailbox/plugin/

[james-project] 01/13: JAMES-2806 add deleteBucket method in BlobStore

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 7257bc83bb487a22f96449fa48ae7696d83bf322
Author: Rene Cordier 
AuthorDate: Wed Jun 26 15:55:31 2019 +0700

JAMES-2806 add deleteBucket method in BlobStore
---
 .../src/main/java/org/apache/james/blob/api/BlobStore.java |  2 ++
 .../java/org/apache/james/blob/api/MetricableBlobStore.java|  5 +
 .../org/apache/james/blob/cassandra/CassandraBlobsDAO.java |  6 ++
 .../java/org/apache/james/blob/memory/MemoryBlobStore.java |  6 ++
 .../apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java |  6 ++
 .../main/java/org/apache/james/blob/union/UnionBlobStore.java  |  7 +++
 .../java/org/apache/james/blob/union/UnionBlobStoreTest.java   | 10 ++
 7 files changed, 42 insertions(+)

diff --git 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/BlobStore.java 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/BlobStore.java
index ed7f6f7..3302203 100644
--- 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/BlobStore.java
+++ 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/BlobStore.java
@@ -36,4 +36,6 @@ public interface BlobStore {
 default Mono save(BucketName bucketName, String data) {
 return save(bucketName, data.getBytes(StandardCharsets.UTF_8));
 }
+
+Mono deleteBucket(BucketName bucketName);
 }
diff --git 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
index 3976bcd..89548d8 100644
--- 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
+++ 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
@@ -70,4 +70,9 @@ public class MetricableBlobStore implements BlobStore {
 return metricFactory
 .runPublishingTimerMetric(READ_TIMER_NAME, () -> 
blobStoreImpl.read(bucketName, blobId));
 }
+
+@Override
+public Mono deleteBucket(BucketName bucketName) {
+return blobStoreImpl.deleteBucket(bucketName);
+}
 }
diff --git 
a/server/blob/blob-cassandra/src/main/java/org/apache/james/blob/cassandra/CassandraBlobsDAO.java
 
b/server/blob/blob-cassandra/src/main/java/org/apache/james/blob/cassandra/CassandraBlobsDAO.java
index 23be4e5..38edc21 100644
--- 
a/server/blob/blob-cassandra/src/main/java/org/apache/james/blob/cassandra/CassandraBlobsDAO.java
+++ 
b/server/blob/blob-cassandra/src/main/java/org/apache/james/blob/cassandra/CassandraBlobsDAO.java
@@ -34,6 +34,7 @@ import java.util.stream.Stream;
 import javax.inject.Inject;
 
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.NotImplementedException;
 import org.apache.commons.lang3.tuple.Pair;
 import 
org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
 import org.apache.james.backends.cassandra.utils.CassandraAsyncExecutor;
@@ -218,4 +219,9 @@ public class CassandraBlobsDAO implements BlobStore {
 return Mono.fromCallable(() -> IOUtils.toByteArray(data))
 .flatMap(this::saveAsMono);
 }
+
+@Override
+public Mono deleteBucket(BucketName bucketName) {
+throw new NotImplementedException("not implemented");
+}
 }
diff --git 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
index 173ce78..154a9b5 100644
--- 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
+++ 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
@@ -28,6 +28,7 @@ import java.util.concurrent.ConcurrentHashMap;
 import javax.inject.Inject;
 
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.NotImplementedException;
 import org.apache.james.blob.api.BlobId;
 import org.apache.james.blob.api.BlobStore;
 import org.apache.james.blob.api.BucketName;
@@ -78,6 +79,11 @@ public class MemoryBlobStore implements BlobStore {
 return new ByteArrayInputStream(retrieveStoredValue(blobId));
 }
 
+@Override
+public Mono deleteBucket(BucketName bucketName) {
+throw new NotImplementedException("not implemented");
+}
+
 private byte[] retrieveStoredValue(BlobId blobId) {
 return Optional.ofNullable(blobs.get(blobId))
 .orElseThrow(() -> new ObjectStoreException("unable to find blob 
with id " + blobId));
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index 7bbd2da..48840f8 100644
--- 
a/server/blob/blob-objectstorage/src/m

[james-project] 04/13: JAMES-2806 Add deleteBucket implementation for memory and its contract

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 1a7e0588d24a86728dd04289c899fead15c78355
Author: Rene Cordier 
AuthorDate: Mon Jul 1 10:55:21 2019 +0700

JAMES-2806 Add deleteBucket implementation for memory and its contract
---
 .../james/blob/api/BucketBlobStoreContract.java| 36 ++
 .../apache/james/blob/memory/MemoryBlobStore.java  | 11 ---
 2 files changed, 43 insertions(+), 4 deletions(-)

diff --git 
a/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
index 4655bb0..d52d703 100644
--- 
a/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
+++ 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
@@ -20,6 +20,7 @@
 package org.apache.james.blob.api;
 
 import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatCode;
 import static org.assertj.core.api.Assertions.assertThatThrownBy;
 
 import java.io.ByteArrayInputStream;
@@ -39,6 +40,30 @@ public interface BucketBlobStoreContract {
 BlobId.Factory blobIdFactory();
 
 @Test
+default void deleteBucketShouldThrowWhenNullBucketName() {
+assertThatThrownBy(() -> testee().deleteBucket(null).block())
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void deleteBucketShouldDeleteExistingBucketWithItsData() {
+BlobId blobId = testee().save(CUSTOM, SHORT_BYTEARRAY).block();
+testee().deleteBucket(CUSTOM).block();
+
+assertThatThrownBy(() -> testee().read(CUSTOM, blobId))
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+default void deleteBucketShouldBeIdempotent(){
+testee().save(CUSTOM, SHORT_BYTEARRAY).block();
+testee().deleteBucket(CUSTOM).block();
+
+assertThatCode(() -> testee().deleteBucket(CUSTOM).block())
+.doesNotThrowAnyException();
+}
+
+@Test
 default void saveBytesShouldThrowWhenNullBucketName() {
 assertThatThrownBy(() -> testee().save(null, SHORT_BYTEARRAY).block())
 .isInstanceOf(NullPointerException.class);
@@ -103,4 +128,15 @@ public interface BucketBlobStoreContract {
 .operationCount(10)
 .runSuccessfullyWithin(Duration.ofMinutes(1));
 }
+
+@Test
+default void deleteBucketConcurrentlyShouldNotFail() throws Exception {
+testee().save(CUSTOM, SHORT_BYTEARRAY).block();
+
+ConcurrentTestRunner.builder()
+.operation(((threadNumber, step) -> 
testee().deleteBucket(CUSTOM).block()))
+.threadCount(10)
+.operationCount(10)
+.runSuccessfullyWithin(Duration.ofMinutes(1));
+}
 }
diff --git 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
index b7f907a..812ea3d 100644
--- 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
+++ 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
@@ -27,7 +27,6 @@ import java.util.Optional;
 import javax.inject.Inject;
 
 import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang3.NotImplementedException;
 import org.apache.james.blob.api.BlobId;
 import org.apache.james.blob.api.BlobStore;
 import org.apache.james.blob.api.BucketName;
@@ -35,10 +34,8 @@ import org.apache.james.blob.api.ObjectStoreException;
 
 import com.google.common.base.Preconditions;
 import com.google.common.collect.HashBasedTable;
-import com.google.common.collect.ImmutableList;
 import com.google.common.collect.Table;
 
-import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
 
 public class MemoryBlobStore implements BlobStore {
@@ -92,7 +89,13 @@ public class MemoryBlobStore implements BlobStore {
 
 @Override
 public Mono deleteBucket(BucketName bucketName) {
-throw new NotImplementedException("not implemented");
+Preconditions.checkNotNull(bucketName);
+
+return Mono.fromRunnable(() -> {
+synchronized (blobs) {
+blobs.row(bucketName).clear();
+}
+});
 }
 
 private byte[] retrieveStoredValue(BucketName bucketName, BlobId blobId) {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 09/13: JAMES-2807 Implement Memory DeletedMessageMetadataVault and related contract

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 5063bfca0d28010a508fa7b132a0b6b4b199ab7d
Author: Benoit Tellier 
AuthorDate: Tue Jun 25 14:14:26 2019 +0700

JAMES-2807 Implement Memory DeletedMessageMetadataVault and related contract
---
 .../pom.xml|  51 +-
 .../vault/MemoryDeletedMessageMetadataVault.java   |  68 ++-
 .../MemoryDeletedMessageMetadataVaultTest.java}|  22 ++-
 .../DeletedMessageWithStorageInformation.java  |  16 +-
 .../vault/DeletedMessageMetadataVaultContract.java | 198 +
 .../vault/DeletedMessageVaultMetadataFixture.java  |   1 +
 6 files changed, 331 insertions(+), 25 deletions(-)

diff --git a/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
index d5ae54e..faf54ee 100644
--- a/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
+++ b/mailbox/plugin/deleted-messages-vault-blobstore-memory/pom.xml
@@ -31,10 +31,57 @@
 
 
 
-org.apache.james
+${james.groupId}
+apache-james-mailbox-api
+test-jar
+test
+
+
+${james.groupId}
+
apache-james-mailbox-deleted-messages-vault
+test-jar
+test
+
+
+${james.groupId}
+
apache-james-mailbox-deleted-messages-vault-blobstore
+
+
+${james.groupId}
 
apache-james-mailbox-deleted-messages-vault-blobstore
+test-jar
+test
+
+
+${james.groupId}
+apache-james-mailbox-memory
+test
+
+
+${james.groupId}
+apache-james-mailbox-memory
+test
+test-jar
+
+
+com.google.guava
+guava
+
+
+org.assertj
+assertj-core
+test
+
+
+org.junit.jupiter
+junit-jupiter-engine
+test
+
+
+org.junit.platform
+junit-platform-launcher
+test
 
 
 
-
 
\ No newline at end of file
diff --git 
a/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
index 33f11f6..4bafe68 100644
--- 
a/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
+++ 
b/mailbox/plugin/deleted-messages-vault-blobstore-memory/src/main/java/org/apache/james/vault/MemoryDeletedMessageMetadataVault.java
@@ -19,40 +19,94 @@
 
 package org.apache.james.vault;
 
-import org.apache.commons.lang3.NotImplementedException;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Optional;
+
 import org.apache.james.blob.api.BucketName;
 import org.apache.james.core.User;
 import org.apache.james.mailbox.model.MessageId;
 import org.reactivestreams.Publisher;
 
+import com.google.common.collect.HashBasedTable;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Table;
+
+import reactor.core.publisher.Flux;
+import reactor.core.publisher.Mono;
+
 public class MemoryDeletedMessageMetadataVault implements 
DeletedMessageMetadataVault {
+private final Table> table;
+
+public MemoryDeletedMessageMetadataVault() {
+table = HashBasedTable.create();
+}
+
 @Override
 public Publisher store(DeletedMessageWithStorageInformation 
deletedMessage) {
-throw new NotImplementedException("Not yet implemented");
+BucketName bucketName = 
deletedMessage.getStorageInformation().getBucketName();
+User owner = deletedMessage.getDeletedMessage().getOwner();
+MessageId messageId = 
deletedMessage.getDeletedMessage().getMessageId();
+
+return Mono.fromRunnable(() -> {
+synchronized (table) {
+Map userVault 
= userVault(bucketName, owner);
+userVault.put(messageId, deletedMessage);
+table.put(bucketName, owner, userVault);
+}
+});
 }
 
 @Override
 public Publisher removeBucket(BucketName bucketName) {
-throw new NotImplementedException("Not yet implemented");
+return Mono.fromRunnable(() -> {
+synchronized (table) {
+table.row(bucketName).clear();
+}
+});
 }
 
 @Override
 public Publisher remove(BucketName bucketName, User user, MessageId 
messageId) {
-throw new NotImplementedException("

[james-project] 03/13: JAMES-2806 add contract for BlobStore with custom buckets

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 2672ea9c839bbaca795ebe0146bcfdaab7014a75
Author: Rene Cordier 
AuthorDate: Tue Jun 25 16:49:46 2019 +0700

JAMES-2806 add contract for BlobStore with custom buckets
---
 .../james/blob/api/BucketBlobStoreContract.java| 106 +
 .../james/blob/memory/MemoryBlobStoreTest.java |   3 +-
 2 files changed, 108 insertions(+), 1 deletion(-)

diff --git 
a/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
new file mode 100644
index 000..4655bb0
--- /dev/null
+++ 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketBlobStoreContract.java
@@ -0,0 +1,106 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.blob.api;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import java.io.ByteArrayInputStream;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+
+import org.apache.james.util.concurrency.ConcurrentTestRunner;
+import org.junit.jupiter.api.Test;
+
+public interface BucketBlobStoreContract {
+String SHORT_STRING = "toto";
+byte[] SHORT_BYTEARRAY = SHORT_STRING.getBytes(StandardCharsets.UTF_8);
+BucketName CUSTOM = BucketName.of("custom");
+
+BlobStore testee();
+
+BlobId.Factory blobIdFactory();
+
+@Test
+default void saveBytesShouldThrowWhenNullBucketName() {
+assertThatThrownBy(() -> testee().save(null, SHORT_BYTEARRAY).block())
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void saveStringShouldThrowWhenNullBucketName() {
+assertThatThrownBy(() -> testee().save(null, SHORT_STRING).block())
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void saveInputStreamShouldThrowWhenNullBucketName() {
+assertThatThrownBy(() -> testee().save(null, new 
ByteArrayInputStream(SHORT_BYTEARRAY)).block())
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void readShouldThrowWhenNullBucketName() {
+BlobId blobId = testee().save(BucketName.DEFAULT, 
SHORT_BYTEARRAY).block();
+assertThatThrownBy(() -> testee().read(null, blobId))
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void readBytesStreamShouldThrowWhenNullBucketName() {
+BlobId blobId = testee().save(BucketName.DEFAULT, 
SHORT_BYTEARRAY).block();
+assertThatThrownBy(() -> testee().readBytes(null, blobId).block())
+.isInstanceOf(NullPointerException.class);
+}
+
+@Test
+default void readStringShouldThrowWhenBucketDoesNotExist() {
+BlobId blobId = testee().save(BucketName.DEFAULT, 
SHORT_BYTEARRAY).block();
+assertThatThrownBy(() -> testee().read(CUSTOM, blobId))
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+default void readBytesStreamShouldThrowWhenBucketDoesNotExist() {
+BlobId blobId = testee().save(BucketName.DEFAULT, 
SHORT_BYTEARRAY).block();
+assertThatThrownBy(() -> testee().readBytes(CUSTOM, blobId).block())
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+default void shouldBeAbleToSaveDataInMultipleBuckets() {
+BlobId blobIdDefault = testee().save(BucketName.DEFAULT, 
SHORT_BYTEARRAY).block();
+BlobId blobIdCustom = testee().save(CUSTOM, SHORT_BYTEARRAY).block();
+
+byte[] bytesDefault = testee().readBytes(BucketName.DEFAULT, 
blobIdDefault).block();

[james-project] 02/13: JAMES-2806 implement buckets with memory BlobStore

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 53d3ed7317abf7f541f2fa01c1b6d0d8473da996
Author: Rene Cordier 
AuthorDate: Tue Jun 25 14:50:54 2019 +0700

JAMES-2806 implement buckets with memory BlobStore
---
 .../apache/james/blob/memory/MemoryBlobStore.java  | 35 +++---
 1 file changed, 24 insertions(+), 11 deletions(-)

diff --git 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
index 154a9b5..b7f907a 100644
--- 
a/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
+++ 
b/server/blob/blob-memory/src/main/java/org/apache/james/blob/memory/MemoryBlobStore.java
@@ -23,7 +23,6 @@ import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.util.Optional;
-import java.util.concurrent.ConcurrentHashMap;
 
 import javax.inject.Inject;
 
@@ -35,31 +34,41 @@ import org.apache.james.blob.api.BucketName;
 import org.apache.james.blob.api.ObjectStoreException;
 
 import com.google.common.base.Preconditions;
+import com.google.common.collect.HashBasedTable;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Table;
 
+import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
 
 public class MemoryBlobStore implements BlobStore {
-private final ConcurrentHashMap blobs;
 private final BlobId.Factory factory;
+private final Table blobs;
 
 @Inject
 public MemoryBlobStore(BlobId.Factory factory) {
 this.factory = factory;
-blobs = new ConcurrentHashMap<>();
+blobs = HashBasedTable.create();
 }
 
 @Override
 public Mono save(BucketName bucketName, byte[] data) {
+Preconditions.checkNotNull(bucketName);
 Preconditions.checkNotNull(data);
-BlobId blobId = factory.forPayload(data);
 
-blobs.put(blobId, data);
+BlobId blobId = factory.forPayload(data);
 
-return Mono.just(blobId);
+return Mono.fromCallable(() -> {
+synchronized (blobs) {
+blobs.put(bucketName, blobId, data);
+return blobId;
+}
+});
 }
 
 @Override
 public Mono save(BucketName bucketName, InputStream data) {
+Preconditions.checkNotNull(bucketName);
 Preconditions.checkNotNull(data);
 try {
 byte[] bytes = IOUtils.toByteArray(data);
@@ -71,12 +80,14 @@ public class MemoryBlobStore implements BlobStore {
 
 @Override
 public Mono readBytes(BucketName bucketName, BlobId blobId) {
-return Mono.fromCallable(() -> retrieveStoredValue(blobId));
+Preconditions.checkNotNull(bucketName);
+return Mono.fromCallable(() -> retrieveStoredValue(bucketName, 
blobId));
 }
 
 @Override
 public InputStream read(BucketName bucketName, BlobId blobId) {
-return new ByteArrayInputStream(retrieveStoredValue(blobId));
+Preconditions.checkNotNull(bucketName);
+return new ByteArrayInputStream(retrieveStoredValue(bucketName, 
blobId));
 }
 
 @Override
@@ -84,8 +95,10 @@ public class MemoryBlobStore implements BlobStore {
 throw new NotImplementedException("not implemented");
 }
 
-private byte[] retrieveStoredValue(BlobId blobId) {
-return Optional.ofNullable(blobs.get(blobId))
-.orElseThrow(() -> new ObjectStoreException("unable to find blob 
with id " + blobId));
+private byte[] retrieveStoredValue(BucketName bucketName, BlobId blobId) {
+synchronized (blobs) {
+return Optional.ofNullable(blobs.get(bucketName, blobId))
+.orElseThrow(() -> new ObjectStoreException("Unable to find 
blob with id " + blobId + " in bucket " + bucketName.asString()));
+}
 }
 }


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 07/13: JAMES-2807 Create POJOs for DeletedMessageMetadataVault API

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 9ebaa14a0ace03e0b2b19b7cee18b15e95376183
Author: Benoit Tellier 
AuthorDate: Tue Jun 25 11:21:36 2019 +0700

JAMES-2807 Create POJOs for DeletedMessageMetadataVault API
---
 .../deleted-messages-vault-blobstore/pom.xml   | 86 ++
 .../DeletedMessageWithStorageInformation.java  | 61 +++
 .../org/apache/james/vault/StorageInformation.java | 64 
 .../vault/DeletedMessageVaultMetadataFixture.java  | 30 
 .../DeletedMessageWithStorageInformationTest.java  | 47 
 .../apache/james/vault/StorageInformationTest.java | 47 
 mailbox/pom.xml|  4 +
 7 files changed, 339 insertions(+)

diff --git a/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml 
b/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml
new file mode 100644
index 000..5575f57
--- /dev/null
+++ b/mailbox/plugin/deleted-messages-vault-blobstore/pom.xml
@@ -0,0 +1,86 @@
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+4.0.0
+
+
+apache-james-mailbox
+org.apache.james
+3.4.0-SNAPSHOT
+../../pom.xml
+
+
+
apache-james-mailbox-deleted-messages-vault-blobstore
+Apache James :: Mailbox :: Plugin :: Deleted Messages Vault :: 
BlobStore
+Apache James Mailbox Deleted Messages Vault implementation on 
top of BlobStore
+
+
+
+${james.groupId}
+apache-james-mailbox-api
+test-jar
+test
+
+
+${james.groupId}
+
apache-james-mailbox-deleted-messages-vault
+
+
+${james.groupId}
+
apache-james-mailbox-deleted-messages-vault
+test-jar
+test
+
+
+${james.groupId}
+apache-james-mailbox-memory
+test
+
+
+${james.groupId}
+apache-james-mailbox-memory
+test
+test-jar
+
+
+${james.groupId}
+blob-api
+
+
+nl.jqno.equalsverifier
+equalsverifier
+test
+
+
+org.assertj
+assertj-core
+test
+
+
+org.junit.jupiter
+junit-jupiter-engine
+test
+
+
+org.junit.platform
+junit-platform-launcher
+test
+
+
+
+
\ No newline at end of file
diff --git 
a/mailbox/plugin/deleted-messages-vault-blobstore/src/main/java/org/apache/james/vault/DeletedMessageWithStorageInformation.java
 
b/mailbox/plugin/deleted-messages-vault-blobstore/src/main/java/org/apache/james/vault/DeletedMessageWithStorageInformation.java
new file mode 100644
index 000..669b3a6
--- /dev/null
+++ 
b/mailbox/plugin/deleted-messages-vault-blobstore/src/main/java/org/apache/james/vault/DeletedMessageWithStorageInformation.java
@@ -0,0 +1,61 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.vault;
+
+import java.util.Objects;
+
+import com.google.common.base.Preconditions;
+
+public class DeletedMessageWithStorageInformation {
+private final DeletedMessage deletedmessage;
+private final StorageInformation storageInformation;
+
+public DeletedMessageWithStorageInformation(DeletedMessage deletedmessage, 
StorageInformation storageInformation) {
+Preconditions.checkNotNull(deletedmessage);
+

[james-project] 05/13: JAMES-2806 add a metric for deleteBucket method and test in its contract

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 6ec43fb87bd37317da03574c3a5939aa50263563
Author: Rene Cordier 
AuthorDate: Tue Jul 2 10:35:48 2019 +0700

JAMES-2806 add a metric for deleteBucket method and test in its contract
---
 .../org/apache/james/blob/api/MetricableBlobStore.java |  4 +++-
 .../apache/james/blob/api/MetricableBlobStoreContract.java | 14 ++
 .../apache/james/blob/cassandra/CassandraBlobsDAOTest.java |  7 +++
 .../blob/objectstorage/ObjectStorageBlobsDAOTest.java  |  7 +++
 4 files changed, 31 insertions(+), 1 deletion(-)

diff --git 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
index 89548d8..11fd997 100644
--- 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
+++ 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/MetricableBlobStore.java
@@ -36,6 +36,7 @@ public class MetricableBlobStore implements BlobStore {
 static final String SAVE_INPUT_STREAM_TIMER_NAME = 
BLOB_STORE_METRIC_PREFIX + "saveInputStream";
 static final String READ_BYTES_TIMER_NAME = BLOB_STORE_METRIC_PREFIX + 
"readBytes";
 static final String READ_TIMER_NAME = BLOB_STORE_METRIC_PREFIX + "read";
+static final String DELETE_TIMER_NAME = BLOB_STORE_METRIC_PREFIX + 
"delete";
 
 private final MetricFactory metricFactory;
 private final BlobStore blobStoreImpl;
@@ -73,6 +74,7 @@ public class MetricableBlobStore implements BlobStore {
 
 @Override
 public Mono deleteBucket(BucketName bucketName) {
-return blobStoreImpl.deleteBucket(bucketName);
+return metricFactory
+.runPublishingTimerMetric(DELETE_TIMER_NAME, () -> 
blobStoreImpl.deleteBucket(bucketName));
 }
 }
diff --git 
a/server/blob/blob-api/src/test/java/org/apache/james/blob/api/MetricableBlobStoreContract.java
 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/MetricableBlobStoreContract.java
index 74a30ff..983c516 100644
--- 
a/server/blob/blob-api/src/test/java/org/apache/james/blob/api/MetricableBlobStoreContract.java
+++ 
b/server/blob/blob-api/src/test/java/org/apache/james/blob/api/MetricableBlobStoreContract.java
@@ -19,6 +19,7 @@
 
 package org.apache.james.blob.api;
 
+import static org.apache.james.blob.api.MetricableBlobStore.DELETE_TIMER_NAME;
 import static 
org.apache.james.blob.api.MetricableBlobStore.READ_BYTES_TIMER_NAME;
 import static org.apache.james.blob.api.MetricableBlobStore.READ_TIMER_NAME;
 import static 
org.apache.james.blob.api.MetricableBlobStore.SAVE_BYTES_TIMER_NAME;
@@ -101,4 +102,17 @@ public interface MetricableBlobStoreContract extends 
BlobStoreContract {
 
assertThat(metricsTestExtension.getMetricFactory().executionTimesFor(READ_TIMER_NAME))
 .hasSize(2);
 }
+
+@Test
+default void deleteBucketShouldPublishDeleteTimerMetrics() {
+BucketName bucketName = BucketName.of("custom");
+testee().save(BucketName.DEFAULT, BYTES_CONTENT).block();
+testee().save(bucketName, BYTES_CONTENT).block();
+
+testee().deleteBucket(BucketName.DEFAULT);
+testee().deleteBucket(bucketName);
+
+
assertThat(metricsTestExtension.getMetricFactory().executionTimesFor(DELETE_TIMER_NAME))
+.hasSize(2);
+}
 }
\ No newline at end of file
diff --git 
a/server/blob/blob-cassandra/src/test/java/org/apache/james/blob/cassandra/CassandraBlobsDAOTest.java
 
b/server/blob/blob-cassandra/src/test/java/org/apache/james/blob/cassandra/CassandraBlobsDAOTest.java
index 758ad7e..23e3375 100644
--- 
a/server/blob/blob-cassandra/src/test/java/org/apache/james/blob/cassandra/CassandraBlobsDAOTest.java
+++ 
b/server/blob/blob-cassandra/src/test/java/org/apache/james/blob/cassandra/CassandraBlobsDAOTest.java
@@ -35,6 +35,7 @@ import org.apache.james.blob.api.MetricableBlobStore;
 import org.apache.james.blob.api.MetricableBlobStoreContract;
 import org.apache.james.util.ZeroedInputStream;
 import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Disabled;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
 
@@ -70,6 +71,12 @@ public class CassandraBlobsDAOTest implements 
MetricableBlobStoreContract {
 return new HashBlobId.Factory();
 }
 
+@Override
+@Disabled("JAMES-2806: delete bucket not implemented yet for Cassandra")
+public void deleteBucketShouldPublishDeleteTimerMetrics() {
+
+}
+
 @Test
 void readBytesShouldReturnSplitSavedDataByChunk() {
 String longString = Strings.repeat("0123456789\n", 
MULTIPLE_CHUNK_SIZE);
diff --git 
a/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOTest.java
 
b/server

[james-project] branch master updated (9d244d5 -> aec776e)

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.


from 9d244d5  JAMES-2813 small fixes on TaskManagerContract
 new 7257bc8  JAMES-2806 add deleteBucket method in BlobStore
 new 53d3ed7  JAMES-2806 implement buckets with memory BlobStore
 new 2672ea9  JAMES-2806 add contract for BlobStore with custom buckets
 new 1a7e058  JAMES-2806 Add deleteBucket implementation for memory and its 
contract
 new 6ec43fb  JAMES-2806 add a metric for deleteBucket method and test in 
its contract
 new 2a2af71  JAMES-2806 add tests for deleteBucket with UnionBlobStore
 new 40292be  Merge remote-tracking branch 
'rcordier/james-2806-bucket-name-memory'
 new 9ebaa14  JAMES-2807 Create POJOs for DeletedMessageMetadataVault API
 new c7177df  JAMES-2807 DeletedMessageMetadataVault API and memory skeleton
 new 5063bfc  JAMES-2807 Implement Memory DeletedMessageMetadataVault and 
related contract
 new 092e127  JAMES-2807 Simplify maven structure for the vault
 new e7b5683  JAMES-2807 Make it clear that DeletedMessageMetadataVault do 
not actually store buckets
 new aec776e  Merge remote-tracking branch 'btellier/JAMES-2807'

The 13 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 mailbox/plugin/deleted-messages-vault/pom.xml  |   4 +
 .../MemoryDeletedMessageMetadataVault.java | 115 
 .../DeletedMessageMetadataVault.java}  |  21 +--
 .../DeletedMessageWithStorageInformation.java} |  51 +++---
 .../james/vault/metadata/StorageInformation.java   |  44 ++---
 .../MemoryDeletedMessageMetadataVaultTest.java |  19 +-
 .../DeletedMessageMetadataVaultContract.java   | 199 +
 .../DeletedMessageVaultMetadataFixture.java|  23 +--
 .../DeletedMessageWithStorageInformationTest.java  |  24 ++-
 .../vault/metadata/StorageInformationTest.java |  30 ++--
 mailbox/pom.xml|   3 +
 .../java/org/apache/james/blob/api/BlobStore.java  |   2 +
 .../apache/james/blob/api/MetricableBlobStore.java |   7 +
 .../james/blob/api/BucketBlobStoreContract.java| 142 +++
 .../blob/api/MetricableBlobStoreContract.java  |  14 ++
 .../james/blob/cassandra/CassandraBlobsDAO.java|   6 +
 .../blob/cassandra/CassandraBlobsDAOTest.java  |   7 +
 .../apache/james/blob/memory/MemoryBlobStore.java  |  44 +++--
 .../james/blob/memory/MemoryBlobStoreTest.java |   3 +-
 .../blob/objectstorage/ObjectStorageBlobsDAO.java  |   6 +
 .../objectstorage/ObjectStorageBlobsDAOTest.java   |   7 +
 .../apache/james/blob/union/UnionBlobStore.java|   7 +
 .../james/blob/union/UnionBlobStoreTest.java   |  49 +
 23 files changed, 692 insertions(+), 135 deletions(-)
 create mode 100644 
mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVault.java
 copy 
mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/{DeletedMessageVault.java
 => metadata/DeletedMessageMetadataVault.java} (68%)
 copy 
mailbox/plugin/{quota-search/src/main/java/org/apache/james/quota/search/Limit.java
 => 
deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/DeletedMessageWithStorageInformation.java}
 (56%)
 copy 
server/data/data-library/src/main/java/org/apache/james/dlp/eventsourcing/commands/ClearCommand.java
 => 
mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/StorageInformation.java
 (62%)
 copy 
server/mailrepository/mailrepository-memory/src/test/java/org/apache/james/mailrepository/memory/MemoryMailRepositoryTest.java
 => 
mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVaultTest.java
 (68%)
 create mode 100644 
mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageMetadataVaultContract.java
 copy 
server/container/guice/blob-memory-guice/src/main/java/org/apache/james/modules/BlobMemoryModule.java
 => 
mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageVaultMetadataFixture.java
 (69%)
 copy 
server/data/data-jmap/src/test/java/org/apache/james/jmap/api/filtering/impl/DefineRulesCommandTest.java
 => 
mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageWithStorageInformationTest.java
 (67%)
 copy mailet/api/src/test/java/org/apache/mailet/AttributeNameTest.java => 
mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/StorageInformationTest.java
 (70%)
 create mode 100644 
server/blob/blob-api/src/test/java/org

[james-project] 12/13: Merge remote-tracking branch 'rcordier/james-2806-bucket-name-memory'

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 40292bec751517ba5c64a0026dea219a9e939e43
Merge: 9d244d5 2a2af71
Author: Antoine DUPRAT 
AuthorDate: Thu Jul 4 10:32:39 2019 +0200

Merge remote-tracking branch 'rcordier/james-2806-bucket-name-memory'

 .../java/org/apache/james/blob/api/BlobStore.java  |   2 +
 .../apache/james/blob/api/MetricableBlobStore.java |   7 +
 .../james/blob/api/BucketBlobStoreContract.java| 142 +
 .../blob/api/MetricableBlobStoreContract.java  |  14 ++
 .../james/blob/cassandra/CassandraBlobsDAO.java|   6 +
 .../blob/cassandra/CassandraBlobsDAOTest.java  |   7 +
 .../apache/james/blob/memory/MemoryBlobStore.java  |  44 +--
 .../james/blob/memory/MemoryBlobStoreTest.java |   3 +-
 .../blob/objectstorage/ObjectStorageBlobsDAO.java  |   6 +
 .../objectstorage/ObjectStorageBlobsDAOTest.java   |   7 +
 .../apache/james/blob/union/UnionBlobStore.java|   7 +
 .../james/blob/union/UnionBlobStoreTest.java   |  49 +++
 12 files changed, 282 insertions(+), 12 deletions(-)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 13/13: Merge remote-tracking branch 'btellier/JAMES-2807'

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit aec776ebb4bce831dd4ae08ab38e28b0b9d6c4e7
Merge: 40292be e7b5683
Author: Antoine DUPRAT 
AuthorDate: Thu Jul 4 10:33:15 2019 +0200

Merge remote-tracking branch 'btellier/JAMES-2807'

 mailbox/plugin/deleted-messages-vault/pom.xml  |   4 +
 .../MemoryDeletedMessageMetadataVault.java | 115 
 .../metadata/DeletedMessageMetadataVault.java  |  39 
 .../DeletedMessageWithStorageInformation.java  |  63 +++
 .../james/vault/metadata/StorageInformation.java   |  64 +++
 .../MemoryDeletedMessageMetadataVaultTest.java |  38 
 .../DeletedMessageMetadataVaultContract.java   | 199 +
 .../DeletedMessageVaultMetadataFixture.java|  31 
 .../DeletedMessageWithStorageInformationTest.java  |  48 +
 .../vault/metadata/StorageInformationTest.java |  47 +
 mailbox/pom.xml|   3 +
 11 files changed, 651 insertions(+)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 06/13: JAMES-2806 add tests for deleteBucket with UnionBlobStore

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 2a2af7101ec83fa49173412499ea569f84200b36
Author: Rene Cordier 
AuthorDate: Tue Jul 2 10:39:04 2019 +0700

JAMES-2806 add tests for deleteBucket with UnionBlobStore
---
 .../james/blob/union/UnionBlobStoreTest.java   | 39 ++
 1 file changed, 39 insertions(+)

diff --git 
a/server/blob/blob-union/src/test/java/org/apache/james/blob/union/UnionBlobStoreTest.java
 
b/server/blob/blob-union/src/test/java/org/apache/james/blob/union/UnionBlobStoreTest.java
index a44fd0d..1789129 100644
--- 
a/server/blob/blob-union/src/test/java/org/apache/james/blob/union/UnionBlobStoreTest.java
+++ 
b/server/blob/blob-union/src/test/java/org/apache/james/blob/union/UnionBlobStoreTest.java
@@ -489,4 +489,43 @@ class UnionBlobStoreTest implements BlobStoreContract {
 assertThat(pushBackIS)
 .hasSameContentAs(new ByteArrayInputStream(new byte[0]));
 }
+
+@Test
+void deleteBucketShouldDeleteBothCurrentAndLegacyBuckets() {
+BlobId legacyBlobId = legacyBlobStore.save(BucketName.DEFAULT, 
BLOB_CONTENT).block();
+BlobId currentBlobId = currentBlobStore.save(BucketName.DEFAULT, 
BLOB_CONTENT).block();
+
+unionBlobStore.deleteBucket(BucketName.DEFAULT).block();
+
+assertThatThrownBy(() -> legacyBlobStore.readBytes(BucketName.DEFAULT, 
legacyBlobId).block())
+.isInstanceOf(ObjectStoreException.class);
+assertThatThrownBy(() -> 
currentBlobStore.readBytes(BucketName.DEFAULT, currentBlobId).block())
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+void deleteBucketShouldDeleteCurrentBucketEvenWhenLegacyDoesNotExist() {
+BlobId currentBlobId = currentBlobStore.save(BucketName.DEFAULT, 
BLOB_CONTENT).block();
+
+unionBlobStore.deleteBucket(BucketName.DEFAULT).block();
+
+assertThatThrownBy(() -> 
currentBlobStore.readBytes(BucketName.DEFAULT, currentBlobId).block())
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+void deleteBucketShouldDeleteLegacyBucketEvenWhenCurrentDoesNotExist() {
+BlobId legacyBlobId = legacyBlobStore.save(BucketName.DEFAULT, 
BLOB_CONTENT).block();
+
+unionBlobStore.deleteBucket(BucketName.DEFAULT).block();
+
+assertThatThrownBy(() -> legacyBlobStore.readBytes(BucketName.DEFAULT, 
legacyBlobId).block())
+.isInstanceOf(ObjectStoreException.class);
+}
+
+@Test
+void deleteBucketShouldNotThrowWhenCurrentAndLegacyBucketsDoNotExist() {
+assertThatCode(() -> 
unionBlobStore.deleteBucket(BucketName.DEFAULT).block())
+.doesNotThrowAnyException();
+}
 }


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 11/13: JAMES-2807 Make it clear that DeletedMessageMetadataVault do not actually store buckets

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit e7b56830237716782a7dac3dc2476a44452f379f
Author: Benoit Tellier 
AuthorDate: Mon Jul 1 16:00:12 2019 +0700

JAMES-2807 Make it clear that DeletedMessageMetadataVault do not actually 
store buckets

It only exposes primitives to take actions on metadata related to a bucket 
used for the
DeletedMessageVault
---
 .../metadata/MemoryDeletedMessageMetadataVault.java|  6 +++---
 .../vault/metadata/DeletedMessageMetadataVault.java|  4 ++--
 .../metadata/DeletedMessageMetadataVaultContract.java  | 18 +-
 3 files changed, 14 insertions(+), 14 deletions(-)

diff --git 
a/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVault.java
 
b/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVault.java
index ccf5fc6..59ede6d 100644
--- 
a/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVault.java
+++ 
b/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/memory/metadata/MemoryDeletedMessageMetadataVault.java
@@ -62,7 +62,7 @@ public class MemoryDeletedMessageMetadataVault implements 
DeletedMessageMetadata
 }
 
 @Override
-public Publisher removeBucket(BucketName bucketName) {
+public Publisher removeMetadataRelatedToBucket(BucketName 
bucketName) {
 return Mono.fromRunnable(() -> {
 synchronized (table) {
 table.row(bucketName).clear();
@@ -81,7 +81,7 @@ public class MemoryDeletedMessageMetadataVault implements 
DeletedMessageMetadata
 
 @Override
 public Publisher retrieveStorageInformation(User user, 
MessageId messageId) {
-return Flux.from(listBuckets())
+return Flux.from(listRelatedBuckets())
 .concatMap(bucket -> {
 synchronized (table) {
 return Mono.justOrEmpty(userVault(bucket, 
user).get(messageId));
@@ -102,7 +102,7 @@ public class MemoryDeletedMessageMetadataVault implements 
DeletedMessageMetadata
 }
 
 @Override
-public Publisher listBuckets() {
+public Publisher listRelatedBuckets() {
 synchronized (table) {
 return Flux.fromIterable(ImmutableSet.copyOf(table.rowKeySet()));
 }
diff --git 
a/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/DeletedMessageMetadataVault.java
 
b/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/DeletedMessageMetadataVault.java
index 25b5a4a..3aab7aa 100644
--- 
a/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/DeletedMessageMetadataVault.java
+++ 
b/mailbox/plugin/deleted-messages-vault/src/main/java/org/apache/james/vault/metadata/DeletedMessageMetadataVault.java
@@ -27,7 +27,7 @@ import org.reactivestreams.Publisher;
 public interface DeletedMessageMetadataVault {
 Publisher store(DeletedMessageWithStorageInformation deletedMessage);
 
-Publisher removeBucket(BucketName bucketName);
+Publisher removeMetadataRelatedToBucket(BucketName bucketName);
 
 Publisher remove(BucketName bucketName, User user, MessageId 
messageId);
 
@@ -35,5 +35,5 @@ public interface DeletedMessageMetadataVault {
 
 Publisher listMessages(BucketName 
bucketName, User user);
 
-Publisher listBuckets();
+Publisher listRelatedBuckets();
 }
diff --git 
a/mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageMetadataVaultContract.java
 
b/mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageMetadataVaultContract.java
index 3b85bc5..0052f0e 100644
--- 
a/mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageMetadataVaultContract.java
+++ 
b/mailbox/plugin/deleted-messages-vault/src/test/java/org/apache/james/vault/metadata/DeletedMessageMetadataVaultContract.java
@@ -79,7 +79,7 @@ public interface DeletedMessageMetadataVaultContract {
 
 @Test
 default void listBucketsShouldBeEmptyWhenNoMessageInserted() {
-Stream messages = 
Flux.from(metadataVault().listBuckets()).toStream();
+Stream messages = 
Flux.from(metadataVault().listRelatedBuckets()).toStream();
 assertThat(messages).isEmpty();
 }
 
@@ -88,7 +88,7 @@ public interface DeletedMessageMetadataVaultContract {
 Mono.from(metadataVault().store(DELETED_MESSAGE)).block();
 
Mono.from(metadataVault().store(DELETED_MESSAGE_2_OTHER_BUCKET)).block();
 
-Stream messages = 
Flux.from(metadataVault().listBuckets()).toStream();
+Stream messages = 
Flux.from(metadataVault().listRelatedBuckets()).toStream();
 assertT

[jira] [Resolved] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat resolved JAMES-2806.
---
   Resolution: Fixed
Fix Version/s: 3.4.0

merged

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat closed JAMES-2806.
-

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Resolved] (JAMES-2807) DeletedMessageMetadataVault API + contract +memory

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat resolved JAMES-2807.
---
   Resolution: Fixed
Fix Version/s: 3.4.0

merged

> DeletedMessageMetadataVault API + contract +memory
> --
>
> Key: JAMES-2807
> URL: https://issues.apache.org/jira/browse/JAMES-2807
> Project: James Server
>  Issue Type: Sub-task
>  Components: deletedMessageVault
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> We need a sub API for storing per Container metadata of Deleted Messages.
> Here is the API:
> {code:java}
> class StorageInformation {
> private final BucketName bucketName;
> private final BlobId blobId;
> }
> class DeletedMessageWithStorageInformation {
> private final DeletedMessage deletedmessage;
> private final StorageInformation storageInformation;
> }
> interface DeletedMessageMetadataVault {
> Publisher store(BucketName, DeletedMessageWithStorageInformation);
> Publisher removeBucket(BucketName);
> Publisher remove(BucketName, User, MessageId);
> Publisher retrieveStorageInformation(User, MessageId);
> Publisher listMessages(BucketName, 
> User);
> Publisher listBuckets();
> }
> {code}
> You will provide the API + contract in 
> `mailbox/plugin/deleted-messages-vault`.
> You will provide a memory implementation of it in 
> `mailbox/plugin/deleted-messages-vault-memory`



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2807) DeletedMessageMetadataVault API + contract +memory

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat closed JAMES-2807.
-

> DeletedMessageMetadataVault API + contract +memory
> --
>
> Key: JAMES-2807
> URL: https://issues.apache.org/jira/browse/JAMES-2807
> Project: James Server
>  Issue Type: Sub-task
>  Components: deletedMessageVault
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> We need a sub API for storing per Container metadata of Deleted Messages.
> Here is the API:
> {code:java}
> class StorageInformation {
> private final BucketName bucketName;
> private final BlobId blobId;
> }
> class DeletedMessageWithStorageInformation {
> private final DeletedMessage deletedmessage;
> private final StorageInformation storageInformation;
> }
> interface DeletedMessageMetadataVault {
> Publisher store(BucketName, DeletedMessageWithStorageInformation);
> Publisher removeBucket(BucketName);
> Publisher remove(BucketName, User, MessageId);
> Publisher retrieveStorageInformation(User, MessageId);
> Publisher listMessages(BucketName, 
> User);
> Publisher listBuckets();
> }
> {code}
> You will provide the API + contract in 
> `mailbox/plugin/deleted-messages-vault`.
> You will provide a memory implementation of it in 
> `mailbox/plugin/deleted-messages-vault-memory`



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Created] (JAMES-2816) Append messages optimization

2019-07-04 Thread Gautier DI FOLCO (JIRA)
Gautier DI FOLCO created JAMES-2816:
---

 Summary: Append messages optimization
 Key: JAMES-2816
 URL: https://issues.apache.org/jira/browse/JAMES-2816
 Project: James Server
  Issue Type: Improvement
Reporter: Gautier DI FOLCO


We tend to copy the whole message when only meta-data are needed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Created] (JAMES-2817) Various EventBus fixes

2019-07-04 Thread Gautier DI FOLCO (JIRA)
Gautier DI FOLCO created JAMES-2817:
---

 Summary: Various EventBus fixes
 Key: JAMES-2817
 URL: https://issues.apache.org/jira/browse/JAMES-2817
 Project: James Server
  Issue Type: Improvement
Reporter: Gautier DI FOLCO






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 01/03: JAMES-2773 Fix CI benchmark glowroot creation

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit f7cf9679f0d0437fa5745619251956d5f7e84ad0
Author: Gautier DI FOLCO 
AuthorDate: Tue Jul 2 15:00:08 2019 +0200

JAMES-2773 Fix CI benchmark glowroot creation
---
 JenkinsfileStressTests| 88 ---
 JenkinsfileStressTests.groovy |  1 +
 2 files changed, 1 insertion(+), 88 deletions(-)

diff --git a/JenkinsfileStressTests b/JenkinsfileStressTests
deleted file mode 100644
index 9bf9425..000
--- a/JenkinsfileStressTests
+++ /dev/null
@@ -1,88 +0,0 @@
-// This file should respect the Jenkinsfile format describe here: 
-// https://jenkins.io/doc/book/pipeline/jenkinsfile/
-// 
-// It may be used by any Jenkins instance you own.
-//
-// In order to work properly, it requires the following parameters:
-// - SENARIO: the Gatling scenrio you want to play (ex. 
com.linagora.gatling.imap.scenario.ImapSimpleScenario)
-
-// Method in order to retry a command.
-// It may be used to wait for a service to be available.
-def maxRetries = 120
-def waitForCommandSuccess(command, maxRetries) {
-def tries = 0;
-while (tries++ < maxRetries) {
-try {
-sh command
-return true
-} catch (Exception e) {
-if (tries >= maxRetries) {
-throw new Exception("${maxRetries} tries exceeded. ${command} 
failed.", e)
-}
-sleep 1 SECONDS
-}
-}
-}
-
-pipeline {
-agent { label 'my-slave' }
-
-tools {
-maven 'maven'
-}
-
-stages {
-stage('Compile') {
-steps {
-sh "mvn clean install -T1C -DskipTests"
-}
-}
-stage('Build image') {
-steps {
-script {
-docker.withTool('docker') {
-sh "cp 
server/container/guice/cassandra-rabbitmq-guice/target/james-server-cassandra-rabbitmq-guice.jar
 dockerfiles/run/guice/cassandra-rabbitmq/destination"
-sh "cp -r 
server/container/guice/cassandra-rabbitmq-guice/target/james-server-cassandra-rabbitmq-guice.lib
 dockerfiles/run/guice/cassandra-rabbitmq/destination"
-sh "cp 
server/container/cli/target/james-server-cli.jar 
dockerfiles/run/guice/cassandra-rabbitmq/destination"
-sh "cp -r 
server/container/cli/target/james-server-cli.lib 
dockerfiles/run/guice/cassandra-rabbitmq/destination"
-sh "docker build -t james_run 
dockerfiles/run/guice/cassandra-rabbitmq"
-}
-}
-}
-}
-stage('Start James') {
-steps {
-script {
-docker.withTool('docker') {
-sh "docker run -d --name=cassandra cassandra:3.11.3"
-sh "docker run -d --name=rabbitmq 
rabbitmq:3.7.7-management"
-sh "docker run -d --name=elasticsearch --env 
'discovery.type=single-node' 
docker.elastic.co/elasticsearch/elasticsearch:6.3.2"
-
-sh "docker run -d --hostname HOSTNAME -p 25:25 -p 
8081:80 -p 110:110 -p 143:143 -p 465:465 -p 587:587 -p 993:993 --link 
cassandra:cassandra --link rabbitmq:rabbitmq --link elasticsearch:elasticsearch 
--name james_run -t james_run"
-try {
-def jamesCliWithOptions = 'java -jar 
/root/james-cli.jar -h 127.0.0.1'
-waitForCommandSuccess("docker exec james_run 
${jamesCliWithOptions} listusers", maxRetries)
-} catch (Exception e) {
-sh "docker logs james_run"
-}
-}
-}
-}
-}
-stage('Run Gatling test') {
-steps {
-build job: 'Gatling job', parameters: [[$class: 
'StringParameterValue', name: 'SBT_ACTION', value: "gatling:testOnly  
${params.SCENARIO}"]]
-}
-}
-}
-
-post {
-always {
-script {
-docker.withTool('docker') {
-sh "docker rm -f cassandra rabbitmq elasticsearch 
james_run"
-}
-}
-}
-}
-}
diff --git a/JenkinsfileStressTests.groovy b/JenkinsfileStressTests.groovy
index f9fb851..076cbdb 100644
--- a/JenkinsfileStressTests.groovy
+++ b/JenkinsfileStressTests.groovy
@@ -38,6 +38,7 @@ pipeline {
 sh "cp 
server/container/cli/target/james-server-cli.jar 
dockerfiles/run/guice/cassandra-rabbitmq/destination"
 sh "cp -r 
server/container/cli/target/james-server-cli.lib 
dockerfiles/run/guice/cassandra-rabbitmq/destination"
 sh 'cp 
server/protocols/jmap-integration-testing/rabbitmq-jmap-integra

[james-project] 03/03: JAMES-2771 Fix dirty provisionning for CI benchmarks

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 78813a88b126936a38180547a089aa5fbc47f113
Author: Gautier DI FOLCO 
AuthorDate: Wed Jul 3 14:13:25 2019 +0200

JAMES-2771 Fix dirty provisionning for CI benchmarks
---
 JenkinsfileStressTests.groovy | 4 
 1 file changed, 4 insertions(+)

diff --git a/JenkinsfileStressTests.groovy b/JenkinsfileStressTests.groovy
index 737f690..b621958 100644
--- a/JenkinsfileStressTests.groovy
+++ b/JenkinsfileStressTests.groovy
@@ -50,6 +50,10 @@ pipeline {
 stage('Start James') {
 steps {
 script {
+if (fileExists('/srv/bench-running-docker')) {
+echo 'Last build failed, cleaning 
provisionning'
+sh 'sudo btrfs subvolume delete 
/srv/bench-running-docker'
+}
 sh "cd /srv && sudo btrfs subvolume snapshot 
bench-snapshot bench-running-docker"
 sh 'docker run -d --name=cassandra -p 9042:9042 -v 
/srv/bench-running-docker/cassandra:/var/lib/cassandra cassandra:3.11.3'
 sh 'docker run -d --name=elasticsearch -p 
9200:9200 -v 
/srv/bench-running-docker/elasticsearch:/usr/share/elasticsearch/data/elasticsearch
  --env "discovery.type=single-node" 
docker.elastic.co/elasticsearch/elasticsearch:6.3.2'


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 02/03: JAMES-2771 Fix concurrency issues on CI benchmark

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit bc257bc82ab29aad7a1e7e4b2f5f096261a07495
Author: Gautier DI FOLCO 
AuthorDate: Wed Jul 3 13:42:18 2019 +0200

JAMES-2771 Fix concurrency issues on CI benchmark
---
 JenkinsfileStressTests.groovy | 4 
 1 file changed, 4 insertions(+)

diff --git a/JenkinsfileStressTests.groovy b/JenkinsfileStressTests.groovy
index 076cbdb..737f690 100644
--- a/JenkinsfileStressTests.groovy
+++ b/JenkinsfileStressTests.groovy
@@ -10,6 +10,10 @@
 // It may be used to wait for a service to be available.
 
 pipeline {
+options {
+disableConcurrentBuilds()
+}
+
 agent none
 
 stages {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] branch master updated (aec776e -> 78813a8)

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.


from aec776e  Merge remote-tracking branch 'btellier/JAMES-2807'
 new f7cf967  JAMES-2773 Fix CI benchmark glowroot creation
 new bc257bc  JAMES-2771 Fix concurrency issues on CI benchmark
 new 78813a8  JAMES-2771 Fix dirty provisionning for CI benchmarks

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 JenkinsfileStressTests| 88 ---
 JenkinsfileStressTests.groovy |  9 +
 2 files changed, 9 insertions(+), 88 deletions(-)
 delete mode 100644 JenkinsfileStressTests


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Resolved] (JAMES-2773) Add glowroot in the docker images

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat resolved JAMES-2773.
---
   Resolution: Fixed
Fix Version/s: 3.4.0

merged

> Add glowroot in the docker images
> -
>
> Key: JAMES-2773
> URL: https://issues.apache.org/jira/browse/JAMES-2773
> Project: James Server
>  Issue Type: Bug
>Reporter: Rémi Kowalski
>Priority: Major
> Fix For: 3.4.0
>
>
> In order to have instrumentation on the JMAP and IMAP transactions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 02/15: Position data length when possible

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit ed91ea3e94f57cf14719a199dcd3f15f7ad38f01
Author: Benoit Tellier 
AuthorDate: Fri Dec 7 14:47:30 2018 +0700

Position data length when possible
---
 .../james/blob/objectstorage/AESPayloadCodec.java  |  5 +
 .../james/blob/objectstorage/DefaultPayloadCodec.java  |  9 +
 .../blob/objectstorage/ObjectStorageBlobsDAO.java  | 18 ++
 .../apache/james/blob/objectstorage/PayloadCodec.java  |  2 ++
 4 files changed, 26 insertions(+), 8 deletions(-)

diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/AESPayloadCodec.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/AESPayloadCodec.java
index 2b0eaee..e19e51d 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/AESPayloadCodec.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/AESPayloadCodec.java
@@ -72,6 +72,11 @@ public class AESPayloadCodec implements PayloadCodec {
 }
 
 @Override
+public Payload write(byte[] bytes) {
+return write(new ByteArrayInputStream(bytes));
+}
+
+@Override
 public Payload write(InputStream inputStream) {
 try (FileBackedOutputStream outputStream = new 
FileBackedOutputStream(MAX_BYTES.intValue())) {
 outputStream.write(aead.encrypt(IOUtils.toByteArray(inputStream), 
EMPTY_ASSOCIATED_DATA));
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/DefaultPayloadCodec.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/DefaultPayloadCodec.java
index 6f56b26..9a6fcc3 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/DefaultPayloadCodec.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/DefaultPayloadCodec.java
@@ -19,6 +19,7 @@
 
 package org.apache.james.blob.objectstorage;
 
+import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.util.Optional;
@@ -32,6 +33,14 @@ public class DefaultPayloadCodec implements PayloadCodec {
 }
 
 @Override
+public Payload write(byte[] bytes) {
+if (bytes.length == 0) {
+return write(new ByteArrayInputStream(bytes));
+}
+return new Payload(Payloads.newByteArrayPayload(bytes), 
Optional.of(new Long(bytes.length)));
+}
+
+@Override
 public InputStream read(Payload payload) throws IOException {
 return payload.getPayload().openStream();
 }
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index fb22164..c38ea12 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
@@ -19,7 +19,6 @@
 
 package org.apache.james.blob.objectstorage;
 
-import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.util.Optional;
@@ -93,15 +92,18 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 }
 
 @Override
-public Mono save(byte[] data) {
+public Mono save(BucketName bucketName, byte[] data) {
+Preconditions.checkNotNull(data);
 BlobId blobId = blobIdFactory.forPayload(data);
+Payload payload = payloadCodec.write(data);
 
 Blob blob = blobStore.blobBuilder(blobId.asString())
-.payload(payloadCodec.write(new ByteArrayInputStream(data)))
+.payload(payload.getPayload())
+.contentLength(payload.getLength().orElse(new Long(data.length)))
 .build();
 
 return save(bucketName, blob)
-.thenApply(any -> blobId);
+.thenReturn(blobId);
 }
 
 @Override
@@ -109,7 +111,7 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 Preconditions.checkNotNull(data);
 
 BlobId tmpId = blobIdFactory.randomId();
-return save(data, tmpId)
+return save(bucketName, data, tmpId)
 .flatMap(id -> updateBlobId(tmpId, id));
 }
 
@@ -121,18 +123,18 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 .thenReturn(to);
 }
 
-private Mono save(InputStream data, BlobId id) {
+private Mono save(BucketName bucketName, InputStream data, BlobId 
id) {
 HashingInputStream hashingInputStream = new 
HashingInputStream(Hashing.sha256(), data);
 Payload payload = payloadCodec.write(hashingInputStream);
  

[james-project] 01/15: Avoid blob copy when saving byte arrays

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 862dd154133317eacffc8e5434083ac0d0452b1d
Author: Benoit Tellier 
AuthorDate: Fri Dec 7 14:14:26 2018 +0700

Avoid blob copy when saving byte arrays

We can iterate the in memory byte array first to compute the blobId to use,
saving a blob copy.
---
 .../blob/objectstorage/ObjectStorageBlobsDAO.java | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index 7bbd2da..fb22164 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
@@ -93,8 +93,15 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 }
 
 @Override
-public Mono save(BucketName bucketName, byte[] data) {
-return save(bucketName, new ByteArrayInputStream(data));
+public Mono save(byte[] data) {
+BlobId blobId = blobIdFactory.forPayload(data);
+
+Blob blob = blobStore.blobBuilder(blobId.asString())
+.payload(payloadCodec.write(new ByteArrayInputStream(data)))
+.build();
+
+return save(bucketName, blob)
+.thenApply(any -> blobId);
 }
 
 @Override
@@ -121,10 +128,15 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 .payload(payload.getPayload())
 .build();
 
-return Mono.fromRunnable(() -> putBlobFunction.putBlob(blob))
+return save(blob)
 .then(Mono.fromCallable(() -> 
blobIdFactory.from(hashingInputStream.hash().toString(;
 }
 
+private Mono save(Blob blob) {
+String containerName = this.containerName.value();
+return Mono.fromCallable(() -> blobStore.putBlob(containerName, blob));
+}
+
 @Override
 public Mono readBytes(BucketName bucketName, BlobId blobId) {
 return Mono.fromCallable(() -> IOUtils.toByteArray(read(bucketName, 
blobId)));
@@ -145,7 +157,6 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 "Failed to readBytes blob " + blobId.asString(),
 cause);
 }
-
 }
 
 public void deleteContainer() {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 05/15: JAMES-2816 Avoid copying the whole message when we only need to copy metadata

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 6c20dff5a87926ba0657fec8a185c925ce000c26
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 15:15:34 2019 +0100

JAMES-2816 Avoid copying the whole message when we only need to copy 
metadata
---
 mailbox/memory/src/test/resources/test.eml  |  0
 .../mailbox/spamassassin/SpamAssassinListenerTest.java  |  4 ++--
 .../apache/james/mailbox/store/StoreMessageIdManager.java   |  2 +-
 .../org/apache/james/mailbox/store/StoreMessageManager.java | 13 ++---
 .../org/apache/james/mailbox/store/event/EventFactory.java  |  4 
 .../james/mailbox/store/MessageIdManagerTestSystem.java |  2 +-
 6 files changed, 6 insertions(+), 19 deletions(-)

diff --git a/mailbox/memory/src/test/resources/test.eml 
b/mailbox/memory/src/test/resources/test.eml
new file mode 100644
index 000..e69de29
diff --git 
a/mailbox/plugin/spamassassin/src/test/java/org/apache/james/mailbox/spamassassin/SpamAssassinListenerTest.java
 
b/mailbox/plugin/spamassassin/src/test/java/org/apache/james/mailbox/spamassassin/SpamAssassinListenerTest.java
index 36922be..72d8b39 100644
--- 
a/mailbox/plugin/spamassassin/src/test/java/org/apache/james/mailbox/spamassassin/SpamAssassinListenerTest.java
+++ 
b/mailbox/plugin/spamassassin/src/test/java/org/apache/james/mailbox/spamassassin/SpamAssassinListenerTest.java
@@ -241,7 +241,7 @@ public class SpamAssassinListenerTest {
 .randomEventId()
 .mailboxSession(MAILBOX_SESSION)
 .mailbox(inbox)
-.addMessage(message)
+.addMetaData(message.metaData())
 .build();
 
 listener.event(addedEvent);
@@ -257,7 +257,7 @@ public class SpamAssassinListenerTest {
 .randomEventId()
 .mailboxSession(MAILBOX_SESSION)
 .mailbox(mailbox1)
-.addMessage(message)
+.addMetaData(message.metaData())
 .build();
 
 listener.event(addedEvent);
diff --git 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageIdManager.java
 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageIdManager.java
index 1b28585..20a5b0f 100644
--- 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageIdManager.java
+++ 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageIdManager.java
@@ -395,7 +395,7 @@ public class StoreMessageIdManager implements 
MessageIdManager {
 .randomEventId()
 .mailboxSession(mailboxSession)
 .mailbox(mailboxMapper.findMailboxById(mailboxId))
-.addMessage(copy)
+.addMetaData(copy.metaData())
 .build(),
 new MailboxIdRegistrationKey(mailboxId))
 .block();
diff --git 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
index 5d63d3b..8c32c55 100644
--- 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
+++ 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
@@ -451,12 +451,12 @@ public class StoreMessageManager implements 
MessageManager {
 MessageMetaData data = appendMessageToStore(message, 
attachments, mailboxSession);
 
 Mailbox mailbox = getMailboxEntity();
-MailboxMessage copy = copyMessage(message);
+
 eventBus.dispatch(EventFactory.added()
 .randomEventId()
 .mailboxSession(mailboxSession)
 .mailbox(mailbox)
-.addMessage(copy)
+.addMetaData(message.metaData())
 .build(),
 new 
MailboxIdRegistrationKey(mailbox.getMailboxId()))
 .block();
@@ -500,15 +500,6 @@ public class StoreMessageManager implements MessageManager 
{
 return new SimpleMailboxMessage(messageIdFactory.generate(), 
internalDate, size, bodyStartOctet, content, flags, propertyBuilder, 
getMailboxEntity().getMailboxId(), attachments);
 }
 
-private MailboxMessage copyMessage(MailboxMessage message) throws 
MailboxException {
-return SimpleMailboxMessage
-.from(message)
-.mailboxId(message.getMailboxId())
-.uid(message.getUid())
-.modseq(message.getModSeq())
-.build();
-}
-
 @Override
 public boolean isWriteable(MailboxSession session) throws MailboxException 
{
 return storeRightManager.isReadWrite(session, mailbox, 
getSharedPermanentFlags(session));
diff --git 
a/mailbox/st

[james-project] 04/15: fixup! Avoid blob copy when saving byte arrays

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 088123b1cc97da4b48bc277a04f27604971615be
Author: Gautier DI FOLCO 
AuthorDate: Tue Jul 2 11:33:56 2019 +0200

fixup! Avoid blob copy when saving byte arrays
---
 .../org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java   | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index c38ea12..c80f23e 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
@@ -134,9 +134,8 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 .then(Mono.fromCallable(() -> 
blobIdFactory.from(hashingInputStream.hash().toString(;
 }
 
-private Mono save(BucketName bucketName, Blob blob) {
-String containerName = this.containerName.value();
-return Mono.fromCallable(() -> blobStore.putBlob(containerName, blob));
+private Mono save(BucketName bucketName, Blob blob) {
+return Mono.fromRunnable(() -> putBlobFunction.putBlob(blob));
 }
 
 @Override


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 12/15: JAMES-2817 GroupRegistration deliver doesn't need to use subscribeWith

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 2338e6461779ec114a9eb13f1da887a2f6030a64
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 14:26:33 2019 +0100

JAMES-2817 GroupRegistration deliver doesn't need to use subscribeWith
---
 .../main/java/org/apache/james/mailbox/events/GroupRegistration.java | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
index c704ad3..8d477e7 100644
--- 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
+++ 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
@@ -40,7 +40,6 @@ import com.rabbitmq.client.Connection;
 import reactor.core.Disposable;
 import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
-import reactor.core.publisher.MonoProcessor;
 import reactor.core.scheduler.Schedulers;
 import reactor.rabbitmq.AcknowledgableDelivery;
 import reactor.rabbitmq.BindingSpecification;
@@ -143,9 +142,7 @@ class GroupRegistration implements Registration {
 .publishOn(Schedulers.elastic())
 .flatMap(any -> Mono.fromRunnable(Throwing.runnable(() -> 
runListener(event
 .onErrorResume(throwable -> retryHandler.handleRetry(event, 
currentRetryCount, throwable))
-.then(Mono.fromRunnable(acknowledgableDelivery::ack))
-.subscribeWith(MonoProcessor.create())
-.then();
+.then(Mono.fromRunnable(acknowledgableDelivery::ack));
 }
 
 Mono reDeliver(Event event) {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 13/15: Merge remote-tracking branch 'btellier/improve-blob-store-efficiency-v4'

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 859161869055c9ec79d9ab851ff6a9362885da63
Merge: 78813a8 088123b
Author: Antoine DUPRAT 
AuthorDate: Thu Jul 4 11:28:31 2019 +0200

Merge remote-tracking branch 'btellier/improve-blob-store-efficiency-v4'

 .../main/java/org/apache/james/blob/api/Store.java | 24 ++
 .../james/blob/objectstorage/AESPayloadCodec.java  |  5 +
 .../blob/objectstorage/DefaultPayloadCodec.java|  9 
 .../blob/objectstorage/ObjectStorageBlobsDAO.java  | 24 --
 .../james/blob/objectstorage/PayloadCodec.java |  2 ++
 .../apache/james/blob/mail/MimeMessageStore.java   |  6 +++---
 6 files changed, 57 insertions(+), 13 deletions(-)

diff --cc 
server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index 48840f8,c80f23e..aa209b5
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
@@@ -146,14 -158,8 +159,13 @@@ public class ObjectStorageBlobsDAO impl
  "Failed to readBytes blob " + blobId.asString(),
  cause);
  }
- 
  }
  
 +@Override
 +public Mono deleteBucket(BucketName bucketName) {
 +throw new NotImplementedException("not implemented");
 +}
 +
  public void deleteContainer() {
  blobStore.deleteContainer(containerName.value());
  }


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 07/15: JAMES-2816 Fix Intelij warnings in SimpleMailboxMessage

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 398594db2e6869c37d8d65357ff82b1f3c3ab60b
Author: Benoit Tellier 
AuthorDate: Mon Feb 18 10:12:45 2019 +0700

JAMES-2816 Fix Intelij warnings in SimpleMailboxMessage
---
 .../mailbox/store/mail/model/impl/SimpleMailboxMessage.java| 10 +++---
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMailboxMessage.java
 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMailboxMessage.java
index 479053d..34b3b5c 100644
--- 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMailboxMessage.java
+++ 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMailboxMessage.java
@@ -138,13 +138,9 @@ public class SimpleMailboxMessage extends 
DelegatingMailboxMessage {
 SimpleMailboxMessage simpleMailboxMessage = new 
SimpleMailboxMessage(messageId, internalDate, size,
 bodyStartOctet, content, flags, propertyBuilder, mailboxId, 
attachments.build());
 
-if (uid.isPresent()) {
-simpleMailboxMessage.setUid(uid.get());
-}
+uid.ifPresent(simpleMailboxMessage::setUid);
+modseq.ifPresent(simpleMailboxMessage::setModSeq);
 
-if (modseq.isPresent()) {
-simpleMailboxMessage.setModSeq(modseq.get());
-}
 return simpleMailboxMessage;
 }
 }
@@ -218,7 +214,7 @@ public class SimpleMailboxMessage extends 
DelegatingMailboxMessage {
 PropertyBuilder propertyBuilder, MailboxId 
mailboxId) {
 this(messageId, internalDate, size, bodyStartOctet,
 content, flags,
-propertyBuilder, mailboxId, 
ImmutableList.of());
+propertyBuilder, mailboxId, ImmutableList.of());
 }
 
 @Override


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] branch master updated (78813a8 -> 2548451)

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.


from 78813a8  JAMES-2771 Fix dirty provisionning for CI benchmarks
 new 862dd15  Avoid blob copy when saving byte arrays
 new ed91ea3  Position data length when possible
 new 30dc0b9  Allow Store.Impl to cary along a byte[] representation
 new 088123b  fixup! Avoid blob copy when saving byte arrays
 new 8591618  Merge remote-tracking branch 
'btellier/improve-blob-store-efficiency-v4'
 new 6c20dff  JAMES-2816 Avoid copying the whole message when we only need 
to copy metadata
 new 55257ba  JAMES-2816 Use buffered streams to handle mails
 new 398594d  JAMES-2816 Fix Intelij warnings in SimpleMailboxMessage
 new 02fb91a  JAMES-2816 Refactor appendMessage method to make it more 
readable
 new 031775a  Merge remote-tracking branch 
'mbaechler/optimize-append-message'
 new 87136c3  JAMES-2817 Rename subscribeWorkQueue to consumeWorkQueue and 
subscribe in start()
 new 37b0b39  JAMES-2817 Avoid C-like cast syntax in GroupRegistration
 new d3c5a61  JAMES-2817 WaitDelayGenerator jitter could remove time from 
next iteration
 new 2338e64  JAMES-2817 GroupRegistration deliver doesn't need to use 
subscribeWith
 new 2548451  Merge remote-tracking branch 
'mbaechler/some-small-eventBus-fix'

The 15 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../mailbox/events/ErrorHandlingContract.java  |   4 +-
 .../james/mailbox/events/GroupRegistration.java|  25 +-
 .../james/mailbox/events/WaitDelayGenerator.java   |  24 +-
 .../mailbox/events/WaitDelayGeneratorTest.java |   8 +-
 .../memory/src/test/resources/test.eml |   0
 .../spamassassin/SpamAssassinListenerTest.java |   4 +-
 .../james/mailbox/store/StoreMessageIdManager.java |   2 +-
 .../james/mailbox/store/StoreMessageManager.java   | 290 -
 .../james/mailbox/store/event/EventFactory.java|   4 -
 .../mail/model/impl/SimpleMailboxMessage.java  |  10 +-
 .../mailbox/store/MessageIdManagerTestSystem.java  |   2 +-
 .../main/java/org/apache/james/blob/api/Store.java |  24 +-
 .../james/blob/objectstorage/AESPayloadCodec.java  |   5 +
 .../blob/objectstorage/DefaultPayloadCodec.java|   9 +
 .../blob/objectstorage/ObjectStorageBlobsDAO.java  |  24 +-
 .../james/blob/objectstorage/PayloadCodec.java |   2 +
 .../apache/james/blob/mail/MimeMessageStore.java   |   6 +-
 17 files changed, 259 insertions(+), 184 deletions(-)
 copy 
server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/service/CassandraMigrationService.java
 => mailbox/memory/src/test/resources/test.eml (100%)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 14/15: Merge remote-tracking branch 'mbaechler/optimize-append-message'

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 031775a2051f4b9f92dbb6cd2cc7377c8f049ab9
Merge: 8591618 02fb91a
Author: Antoine DUPRAT 
AuthorDate: Thu Jul 4 11:29:09 2019 +0200

Merge remote-tracking branch 'mbaechler/optimize-append-message'

 mailbox/memory/src/test/resources/test.eml |   0
 .../spamassassin/SpamAssassinListenerTest.java |   4 +-
 .../james/mailbox/store/StoreMessageIdManager.java |   2 +-
 .../james/mailbox/store/StoreMessageManager.java   | 290 -
 .../james/mailbox/store/event/EventFactory.java|   4 -
 .../mail/model/impl/SimpleMailboxMessage.java  |  10 +-
 .../mailbox/store/MessageIdManagerTestSystem.java  |   2 +-
 7 files changed, 171 insertions(+), 141 deletions(-)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 10/15: JAMES-2817 Avoid C-like cast syntax in GroupRegistration

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 37b0b39571f9f50268bdfa88979ace839bd84369
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 09:47:20 2019 +0100

JAMES-2817 Avoid C-like cast syntax in GroupRegistration
---
 .../main/java/org/apache/james/mailbox/events/GroupRegistration.java| 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
index 7730efc..c704ad3 100644
--- 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
+++ 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
@@ -164,7 +164,7 @@ class GroupRegistration implements Registration {
 return 
Optional.ofNullable(acknowledgableDelivery.getProperties().getHeaders())
 .flatMap(headers -> Optional.ofNullable(headers.get(RETRY_COUNT)))
 .filter(object -> object instanceof Integer)
-.map(object -> (Integer) object)
+.map(Integer.class::cast)
 .orElse(DEFAULT_RETRY_COUNT);
 }
 


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 15/15: Merge remote-tracking branch 'mbaechler/some-small-eventBus-fix'

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 2548451538098db2912872d06c0421b38be5f372
Merge: 031775a 2338e64
Author: Antoine DUPRAT 
AuthorDate: Thu Jul 4 11:29:38 2019 +0200

Merge remote-tracking branch 'mbaechler/some-small-eventBus-fix'

 .../mailbox/events/ErrorHandlingContract.java  |  4 ++--
 .../james/mailbox/events/GroupRegistration.java| 25 ++
 .../james/mailbox/events/WaitDelayGenerator.java   | 24 -
 .../mailbox/events/WaitDelayGeneratorTest.java |  8 +++
 4 files changed, 31 insertions(+), 30 deletions(-)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 08/15: JAMES-2816 Refactor appendMessage method to make it more readable

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 02fb91af65229e1ccd4f714c55682b7abe4673f5
Author: Rémi Kowalski 
AuthorDate: Mon Jul 1 17:44:22 2019 +0200

JAMES-2816 Refactor appendMessage method to make it more readable
---
 .../james/mailbox/store/StoreMessageManager.java   | 276 -
 1 file changed, 160 insertions(+), 116 deletions(-)

diff --git 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
index 5015d67..3cf9e06 100644
--- 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
+++ 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
@@ -133,6 +133,16 @@ public class StoreMessageManager implements MessageManager 
{
  */
 protected static final Flags MINIMAL_PERMANET_FLAGS;
 
+private static class MediaType {
+final String mediaType;
+final String subType;
+
+private MediaType(String mediaType, String subType) {
+this.mediaType = mediaType;
+this.subType = subType;
+}
+}
+
 static {
 MINIMAL_PERMANET_FLAGS = new Flags();
 MINIMAL_PERMANET_FLAGS.add(Flags.Flag.ANSWERED);
@@ -327,7 +337,6 @@ public class StoreMessageManager implements MessageManager {
 
 @Override
 public ComposedMessageId appendMessage(InputStream msgIn, Date 
internalDate, final MailboxSession mailboxSession, boolean isRecent, Flags 
flagsToBeSet) throws MailboxException {
-
 File file = null;
 
 if (!isWriteable(mailboxSession)) {
@@ -346,126 +355,20 @@ public class StoreMessageManager implements 
MessageManager {
 // Disable line length... This should be handled by the smtp 
server
 // component and not the parser itself
 // https://issues.apache.org/jira/browse/IMAP-122
-
-final MimeTokenStream parser = new 
MimeTokenStream(MimeConfig.PERMISSIVE, new DefaultBodyDescriptorBuilder());
-
-parser.setRecursionMode(RecursionMode.M_NO_RECURSE);
-parser.parse(bIn);
-final HeaderImpl header = new HeaderImpl();
-
-EntityState next = parser.next();
-while (next != EntityState.T_BODY && next != 
EntityState.T_END_OF_STREAM && next != EntityState.T_START_MULTIPART) {
-if (next == EntityState.T_FIELD) {
-header.addField(parser.getField());
-}
-next = parser.next();
-}
+final MimeTokenStream parser = getParser(bIn);
+final HeaderImpl header = readHeader(parser);
 final MaximalBodyDescriptor descriptor = 
(MaximalBodyDescriptor) parser.getBodyDescriptor();
-final PropertyBuilder propertyBuilder = new PropertyBuilder();
-final String mediaType;
-final String mediaTypeFromHeader = descriptor.getMediaType();
-final String subType;
-if (mediaTypeFromHeader == null) {
-mediaType = "text";
-subType = "plain";
-} else {
-mediaType = mediaTypeFromHeader;
-subType = descriptor.getSubType();
-}
-propertyBuilder.setMediaType(mediaType);
-propertyBuilder.setSubType(subType);
-propertyBuilder.setContentID(descriptor.getContentId());
-
propertyBuilder.setContentDescription(descriptor.getContentDescription());
-
propertyBuilder.setContentLocation(descriptor.getContentLocation());
-propertyBuilder.setContentMD5(descriptor.getContentMD5Raw());
-
propertyBuilder.setContentTransferEncoding(descriptor.getTransferEncoding());
-
propertyBuilder.setContentLanguage(descriptor.getContentLanguage());
-
propertyBuilder.setContentDispositionType(descriptor.getContentDispositionType());
-
propertyBuilder.setContentDispositionParameters(descriptor.getContentDispositionParameters());
-
propertyBuilder.setContentTypeParameters(descriptor.getContentTypeParameters());
-// Add missing types
-final String codeset = descriptor.getCharset();
-if (codeset == null) {
-if ("TEXT".equalsIgnoreCase(mediaType)) {
-propertyBuilder.setCharset("us-ascii");
-}
-} else {
-propertyBuilder.setCharset(codeset);
-}
-
-final String boundary = descriptor.getBoundary();
-if (boundary != null) {
-   

[james-project] 06/15: JAMES-2816 Use buffered streams to handle mails

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 55257ba20f52d0d91043e2f6733d77209520e2d7
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 15:16:34 2019 +0100

JAMES-2816 Use buffered streams to handle mails

x3 improvement in micro-benchmark, should be around 5% improvement in 
real life
appendMessage reported to take 8.51% in a flamegraph
---
 .../java/org/apache/james/mailbox/store/StoreMessageManager.java   | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
index 8c32c55..5015d67 100644
--- 
a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
+++ 
b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreMessageManager.java
@@ -22,6 +22,8 @@ package org.apache.james.mailbox.store;
 import static 
org.apache.james.mailbox.extension.PreDeletionHook.DeleteOperation;
 import static 
org.apache.james.mailbox.store.mail.AbstractMessageMapper.UNLIMITED;
 
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
 import java.io.File;
 import java.io.FileOutputStream;
 import java.io.IOException;
@@ -102,7 +104,6 @@ import com.github.steveash.guavate.Guavate;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.ImmutableSortedMap;
-
 import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
 import reactor.core.scheduler.Schedulers;
@@ -339,7 +340,8 @@ public class StoreMessageManager implements MessageManager {
 // source for the InputStream
 file = File.createTempFile("imap", ".msg");
 try (FileOutputStream out = new FileOutputStream(file);
- TeeInputStream tmpMsgIn = new TeeInputStream(msgIn, out);
+ BufferedOutputStream bufferedOut = new 
BufferedOutputStream(out);
+ BufferedInputStream tmpMsgIn = new BufferedInputStream(new 
TeeInputStream(msgIn, bufferedOut));
  BodyOffsetInputStream bIn = new 
BodyOffsetInputStream(tmpMsgIn)) {
 // Disable line length... This should be handled by the smtp 
server
 // component and not the parser itself
@@ -433,6 +435,7 @@ public class StoreMessageManager implements MessageManager {
 // the file now
 // via the TeeInputStream
 }
+bufferedOut.close();
 int bodyStartOctet = (int) bIn.getBodyStartOffset();
 if (bodyStartOctet == -1) {
 bodyStartOctet = 0;


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 11/15: JAMES-2817 WaitDelayGenerator jitter could remove time from next iteration

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit d3c5a61f0591a9538be670188c59894c8f5862c8
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 12:31:45 2019 +0100

JAMES-2817 WaitDelayGenerator jitter could remove time from next iteration
---
 .../mailbox/events/ErrorHandlingContract.java  |  4 ++--
 .../james/mailbox/events/WaitDelayGenerator.java   | 24 +-
 .../mailbox/events/WaitDelayGeneratorTest.java |  8 
 3 files changed, 20 insertions(+), 16 deletions(-)

diff --git 
a/mailbox/api/src/test/java/org/apache/james/mailbox/events/ErrorHandlingContract.java
 
b/mailbox/api/src/test/java/org/apache/james/mailbox/events/ErrorHandlingContract.java
index c6f4494..d0b3923 100644
--- 
a/mailbox/api/src/test/java/org/apache/james/mailbox/events/ErrorHandlingContract.java
+++ 
b/mailbox/api/src/test/java/org/apache/james/mailbox/events/ErrorHandlingContract.java
@@ -167,8 +167,8 @@ interface ErrorHandlingContract extends EventBusContract {
 softly.assertThat(timeElapsed).hasSize(4);
 
 long minFirstDelayAfter = 100; // first backOff
-long minSecondDelayAfter = 100; // 200 * jitter factor (200 * 0.5)
-long minThirdDelayAfter = 200; // 400 * jitter factor (400 * 0.5)
+long minSecondDelayAfter = 50; // 50 * jitter factor (50 * 0.5)
+long minThirdDelayAfter = 100; // 100 * jitter factor (100 * 0.5)
 
 softly.assertThat(timeElapsed.get(1))
 
.isAfterOrEqualTo(timeElapsed.get(0).plusMillis(minFirstDelayAfter));
diff --git 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/WaitDelayGenerator.java
 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/WaitDelayGenerator.java
index ff0378b..33a3586 100644
--- 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/WaitDelayGenerator.java
+++ 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/WaitDelayGenerator.java
@@ -24,7 +24,7 @@ import java.time.Duration;
 
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Preconditions;
-
+import com.google.common.primitives.Ints;
 import reactor.core.publisher.Mono;
 import reactor.core.scheduler.Schedulers;
 
@@ -34,12 +34,14 @@ class WaitDelayGenerator {
 return new WaitDelayGenerator(retryBackoff);
 }
 
-private static int randomBetween(int lowest, int highest) {
-Preconditions.checkArgument(lowest <= highest, "lowest always has to 
be less than or equals highest");
-if (lowest == highest) {
-return lowest;
+private static Duration randomBetween(Duration base, Duration jitter) {
+Preconditions.checkArgument(!jitter.isNegative(), "jitter value should 
always be positive");
+if (jitter.isZero()) {
+return base;
 }
-return SECURE_RANDOM.nextInt(highest - lowest) + lowest;
+long maxJitterAsMillis = jitter.toMillis();
+long jitterAsMillis = 
SECURE_RANDOM.nextInt(Ints.checkedCast(maxJitterAsMillis * 2)) / 2;
+return base.plusMillis(jitterAsMillis);
 }
 
 private static final SecureRandom SECURE_RANDOM = new SecureRandom();
@@ -65,11 +67,13 @@ class WaitDelayGenerator {
 if (!shouldDelay(retryCount)) {
 return Duration.ZERO;
 }
-int exponentialFactor = Double.valueOf(Math.pow(2, retryCount - 
1)).intValue();
-int minDelay = exponentialFactor * (int) 
retryBackoff.getFirstBackoff().toMillis();
-int maxDelay = Double.valueOf(minDelay + minDelay * 
retryBackoff.getJitterFactor()).intValue();
+long exponentialFactor = Double.valueOf(Math.pow(2, retryCount - 
1)).longValue();
+Duration minDelay = 
retryBackoff.getFirstBackoff().multipliedBy(exponentialFactor);
+Duration jitterDelay = retryBackoff.getFirstBackoff()
+.multipliedBy(Double.valueOf(retryBackoff.getJitterFactor() * 
100).intValue())
+.dividedBy(100);
 
-return Duration.ofMillis(randomBetween(minDelay, maxDelay));
+return randomBetween(minDelay, jitterDelay);
 }
 
 private boolean shouldDelay(int retryCount) {
diff --git 
a/mailbox/event/event-rabbitmq/src/test/java/org/apache/james/mailbox/events/WaitDelayGeneratorTest.java
 
b/mailbox/event/event-rabbitmq/src/test/java/org/apache/james/mailbox/events/WaitDelayGeneratorTest.java
index 9cb2d62..e7f1fb6 100644
--- 
a/mailbox/event/event-rabbitmq/src/test/java/org/apache/james/mailbox/events/WaitDelayGeneratorTest.java
+++ 
b/mailbox/event/event-rabbitmq/src/test/java/org/apache/james/mailbox/events/WaitDelayGeneratorTest.java
@@ -46,13 +46,13 @@ class WaitDelayGeneratorTest {
 
 SoftAssertions.assertSoftly(softly -> {
 softly.assertThat(generator.generateDelay(1)

[james-project] 09/15: JAMES-2817 Rename subscribeWorkQueue to consumeWorkQueue and subscribe in start()

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 87136c39e011e6d0197e3a0e85a8203c1e861e35
Author: Matthieu Baechler 
AuthorDate: Fri Feb 15 09:14:26 2019 +0100

JAMES-2817 Rename subscribeWorkQueue to consumeWorkQueue and subscribe in 
start()
---
 .../apache/james/mailbox/events/GroupRegistration.java | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
index bbdd46b..7730efc 100644
--- 
a/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
+++ 
b/mailbox/event/event-rabbitmq/src/main/java/org/apache/james/mailbox/events/GroupRegistration.java
@@ -37,7 +37,6 @@ import org.apache.james.util.MDCBuilder;
 import com.github.fge.lambdas.Throwing;
 import com.google.common.base.Preconditions;
 import com.rabbitmq.client.Connection;
-
 import reactor.core.Disposable;
 import reactor.core.publisher.Flux;
 import reactor.core.publisher.Mono;
@@ -105,10 +104,11 @@ class GroupRegistration implements Registration {
 }
 
 GroupRegistration start() {
-createGroupWorkQueue()
-.then(retryHandler.createRetryExchange(queueName))
-.doOnSuccess(any -> this.subscribeWorkQueue())
-.block();
+receiverSubscriber = Optional
+.of(createGroupWorkQueue()
+.then(retryHandler.createRetryExchange(queueName))
+.then(Mono.fromCallable(() -> this.consumeWorkQueue()))
+.block());
 return this;
 }
 
@@ -126,12 +126,12 @@ class GroupRegistration implements Registration {
 .then();
 }
 
-private void subscribeWorkQueue() {
-receiverSubscriber = 
Optional.of(receiver.consumeManualAck(queueName.asString(), new 
ConsumeOptions().qos(EventBus.EXECUTION_RATE))
+private Disposable consumeWorkQueue() {
+return receiver.consumeManualAck(queueName.asString(), new 
ConsumeOptions().qos(EventBus.EXECUTION_RATE))
+.publishOn(Schedulers.parallel())
 .filter(delivery -> Objects.nonNull(delivery.getBody()))
 .flatMap(this::deliver)
-.subscribeOn(Schedulers.elastic())
-.subscribe());
+.subscribe();
 }
 
 private Mono deliver(AcknowledgableDelivery acknowledgableDelivery) {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[james-project] 03/15: Allow Store.Impl to cary along a byte[] representation

2019-07-04 Thread aduprat
This is an automated email from the ASF dual-hosted git repository.

aduprat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 30dc0b9f04d5bd3fc7121aedf8af8a972a356d35
Author: Benoit Tellier 
AuthorDate: Fri Dec 7 15:08:00 2018 +0700

Allow Store.Impl to cary along a byte[] representation

Bytes are available but hidded behing an inputStream for implementation 
purposes.

We propose in this commit a new level of indirection allowing to save an 
object as bytes or as InputSteam.
---
 .../main/java/org/apache/james/blob/api/Store.java | 24 ++
 .../apache/james/blob/mail/MimeMessageStore.java   |  6 +++---
 2 files changed, 23 insertions(+), 7 deletions(-)

diff --git 
a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/Store.java 
b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/Store.java
index 6a4f859..ce640d5 100644
--- a/server/blob/blob-api/src/main/java/org/apache/james/blob/api/Store.java
+++ b/server/blob/blob-api/src/main/java/org/apache/james/blob/api/Store.java
@@ -19,7 +19,6 @@
 
 package org.apache.james.blob.api;
 
-import java.io.InputStream;
 import java.util.Collection;
 import java.util.Objects;
 import java.util.stream.Stream;
@@ -66,8 +65,25 @@ public interface Store {
 
 class Impl implements Store {
 
+public interface ValueToSave {
+Mono saveIn(BucketName bucketName, BlobStore blobStore);
+}
+
+public static class BytesToSave implements ValueToSave {
+private final byte[] bytes;
+
+public BytesToSave(byte[] bytes) {
+this.bytes = bytes;
+}
+
+@Override
+public Mono saveIn(BucketName bucketName, BlobStore 
blobStore) {
+return blobStore.save(bucketName, bytes);
+}
+}
+
 public interface Encoder {
-Stream> encode(T t);
+Stream> encode(T t);
 }
 
 public interface Decoder {
@@ -94,9 +110,9 @@ public interface Store {
 .map(idFactory::generate);
 }
 
-private Mono> saveEntry(Pair entry) {
+private Mono> saveEntry(Pair entry) {
 return Mono.just(entry.getLeft())
-.zipWith(blobStore.save(BucketName.DEFAULT, entry.getRight()));
+.zipWith(entry.getRight().saveIn(BucketName.DEFAULT, 
blobStore));
 }
 
 @Override
diff --git 
a/server/blob/mail-store/src/main/java/org/apache/james/blob/mail/MimeMessageStore.java
 
b/server/blob/mail-store/src/main/java/org/apache/james/blob/mail/MimeMessageStore.java
index 12683f9..f0c2480 100644
--- 
a/server/blob/mail-store/src/main/java/org/apache/james/blob/mail/MimeMessageStore.java
+++ 
b/server/blob/mail-store/src/main/java/org/apache/james/blob/mail/MimeMessageStore.java
@@ -69,15 +69,15 @@ public class MimeMessageStore {
 
 static class MimeMessageEncoder implements Store.Impl.Encoder 
{
 @Override
-public Stream> encode(MimeMessage message) 
{
+public Stream> 
encode(MimeMessage message) {
 try {
 byte[] messageAsArray = messageToArray(message);
 int bodyStartOctet = computeBodyStartOctet(messageAsArray);
 byte[] headerBytes = getHeaderBytes(messageAsArray, 
bodyStartOctet);
 byte[] bodyBytes = getBodyBytes(messageAsArray, 
bodyStartOctet);
 return Stream.of(
-Pair.of(HEADER_BLOB_TYPE, new 
ByteArrayInputStream(headerBytes)),
-Pair.of(BODY_BLOB_TYPE, new 
ByteArrayInputStream(bodyBytes)));
+Pair.of(HEADER_BLOB_TYPE, new 
Store.Impl.BytesToSave(headerBytes)),
+Pair.of(BODY_BLOB_TYPE, new 
Store.Impl.BytesToSave(bodyBytes)));
 } catch (MessagingException | IOException e) {
 throw new RuntimeException(e);
 }


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Resolved] (JAMES-2816) Append messages optimization

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat resolved JAMES-2816.
---
   Resolution: Fixed
Fix Version/s: 3.4.0

merged

> Append messages optimization
> 
>
> Key: JAMES-2816
> URL: https://issues.apache.org/jira/browse/JAMES-2816
> Project: James Server
>  Issue Type: Improvement
>Reporter: Gautier DI FOLCO
>Priority: Trivial
> Fix For: 3.4.0
>
>
> We tend to copy the whole message when only meta-data are needed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Resolved] (JAMES-2817) Various EventBus fixes

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat resolved JAMES-2817.
---
   Resolution: Fixed
Fix Version/s: 3.4.0

merged

> Various EventBus fixes
> --
>
> Key: JAMES-2817
> URL: https://issues.apache.org/jira/browse/JAMES-2817
> Project: James Server
>  Issue Type: Improvement
>Reporter: Gautier DI FOLCO
>Priority: Minor
> Fix For: 3.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2817) Various EventBus fixes

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat closed JAMES-2817.
-

> Various EventBus fixes
> --
>
> Key: JAMES-2817
> URL: https://issues.apache.org/jira/browse/JAMES-2817
> Project: James Server
>  Issue Type: Improvement
>Reporter: Gautier DI FOLCO
>Priority: Minor
> Fix For: 3.4.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2816) Append messages optimization

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat closed JAMES-2816.
-

> Append messages optimization
> 
>
> Key: JAMES-2816
> URL: https://issues.apache.org/jira/browse/JAMES-2816
> Project: James Server
>  Issue Type: Improvement
>Reporter: Gautier DI FOLCO
>Priority: Trivial
> Fix For: 3.4.0
>
>
> We tend to copy the whole message when only meta-data are needed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Reopened] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat reopened JAMES-2806:
---

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2773) Add glowroot in the docker images

2019-07-04 Thread Antoine Duprat (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Antoine Duprat closed JAMES-2773.
-

> Add glowroot in the docker images
> -
>
> Key: JAMES-2773
> URL: https://issues.apache.org/jira/browse/JAMES-2773
> Project: James Server
>  Issue Type: Bug
>Reporter: Rémi Kowalski
>Priority: Major
> Fix For: 3.4.0
>
>
> In order to have instrumentation on the JMAP and IMAP transactions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Commented] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Rene Cordier (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878946#comment-16878946
 ] 

Rene Cordier commented on JAMES-2806:
-

Work for the ObjectStorage implementation has been divided into 3 PRs :
*  https://github.com/linagora/james-project/pull/2489 `[JAMES-2806] [Part 1] 
ObjectStorageBlobsDAO with BucketName`
*  https://github.com/linagora/james-project/pull/2492 `[JAMES-2806] [Part 2] 
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
*  https://github.com/linagora/james-project/pull/2497 `[JAMES-2806] [Part 3] 
ObjectStorageBlobsDAO BucketPrefix implementation`

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Comment Edited] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Rene Cordier (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878946#comment-16878946
 ] 

Rene Cordier edited comment on JAMES-2806 at 7/5/19 3:22 AM:
-

Work for the ObjectStorage implementation has been divided into 3 PRs :
 *  [https://github.com/linagora/james-project/pull/2489] `[Part 1] 
ObjectStorageBlobsDAO with BucketName`
 *  [https://github.com/linagora/james-project/pull/2492] `[Part 2] 
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
 *  [https://github.com/linagora/james-project/pull/2497] `[Part 3] 
ObjectStorageBlobsDAO BucketPrefix implementation`


was (Author: rcordier):
Work for the ObjectStorage implementation has been divided into 3 PRs :
*  https://github.com/linagora/james-project/pull/2489 `[JAMES-2806] [Part 1] 
ObjectStorageBlobsDAO with BucketName`
*  https://github.com/linagora/james-project/pull/2492 `[JAMES-2806] [Part 2] 
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
*  https://github.com/linagora/james-project/pull/2497 `[JAMES-2806] [Part 3] 
ObjectStorageBlobsDAO BucketPrefix implementation`

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Comment Edited] (JAMES-2806) Adding bucket capability into BlobStore

2019-07-04 Thread Rene Cordier (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878946#comment-16878946
 ] 

Rene Cordier edited comment on JAMES-2806 at 7/5/19 3:23 AM:
-

Work for the ObjectStorage implementation has been divided into 3 PRs :
* [https://github.com/linagora/james-project/pull/2489] `[Part 1] 
ObjectStorageBlobsDAO with BucketName`
* [https://github.com/linagora/james-project/pull/2492] `[Part 2] 
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
* [https://github.com/linagora/james-project/pull/2497] `[Part 3] 
ObjectStorageBlobsDAO BucketPrefix implementation`


was (Author: rcordier):
Work for the ObjectStorage implementation has been divided into 3 PRs :
 *  [https://github.com/linagora/james-project/pull/2489] `[Part 1] 
ObjectStorageBlobsDAO with BucketName`
 *  [https://github.com/linagora/james-project/pull/2492] `[Part 2] 
ObjectStorageBlobsDAO creating bucket when saving fails (Swfit + S3 )`
 *  [https://github.com/linagora/james-project/pull/2497] `[Part 3] 
ObjectStorageBlobsDAO BucketPrefix implementation`

> Adding bucket capability into BlobStore
> ---
>
> Key: JAMES-2806
> URL: https://issues.apache.org/jira/browse/JAMES-2806
> Project: James Server
>  Issue Type: Sub-task
>  Components: Blob
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.4.0
>
>
> You should update current BlobStore API to be able to customize the way we 
> storeBlob:
>  - We want to store a blob in a specified bucket
> {code:java}
> public interface BlobStore {
> Publisher save(BucketName, byte[] data);
> Publisher save(BucketName, InputStream data);
> Publisher readBytes(BucketName, BlobId blobId);
> InputStream read(BucketName, BlobId blobId);
> }
> {code}
> And to implement the feature of deleting buckets exceed retention time, we 
> need to have another additional method of deleting bucket:
> {code:java}
> public interface BlobStore {
> Publisher delete(BucketName bucket);
> }
> {code}
> Scope: API change, contract test, memory + object storage implementation - 
> cassandra will be done in a later task



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org