Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
vttranlina commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1842164539 rebase master & squash -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] Little polish of the PG implementation [james-project]
Arsnael commented on PR #1841: URL: https://github.com/apache/james-project/pull/1841#issuecomment-1842073777 ``` 15:30:08,662 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.3.0:check (check-style) on project apache-james-mailbox-api: You have 1 Checkstyle violation. -> [Help 1] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
(james-project) branch postgresql updated: JAMES-2586 [PGSQL] Initialization to configure users repository
This is an automated email from the ASF dual-hosted git repository.
rcordier pushed a commit to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
The following commit(s) were added to refs/heads/postgresql by this push:
new 698454a765 JAMES-2586 [PGSQL] Initialization to configure users
repository
698454a765 is described below
commit 698454a765f3b02d0c8c8d88e127a365a0c508ea
Author: Tung Tran
AuthorDate: Tue Dec 5 10:15:34 2023 +0700
JAMES-2586 [PGSQL] Initialization to configure users repository
---
.../james/modules/data/PostgresUsersRepositoryModule.java | 10 ++
.../java/org/apache/james/user/lib/UsersRepositoryImpl.java| 2 ++
2 files changed, 12 insertions(+)
diff --git
a/server/container/guice/postgres-common/src/main/java/org/apache/james/modules/data/PostgresUsersRepositoryModule.java
b/server/container/guice/postgres-common/src/main/java/org/apache/james/modules/data/PostgresUsersRepositoryModule.java
index 99289c5ce4..575f7621f0 100644
---
a/server/container/guice/postgres-common/src/main/java/org/apache/james/modules/data/PostgresUsersRepositoryModule.java
+++
b/server/container/guice/postgres-common/src/main/java/org/apache/james/modules/data/PostgresUsersRepositoryModule.java
@@ -28,12 +28,15 @@ import org.apache.james.user.postgres.PostgresUserModule;
import org.apache.james.user.postgres.PostgresUsersDAO;
import org.apache.james.user.postgres.PostgresUsersRepository;
import org.apache.james.user.postgres.PostgresUsersRepositoryConfiguration;
+import org.apache.james.utils.InitializationOperation;
+import org.apache.james.utils.InitilizationOperationBuilder;
import com.google.inject.AbstractModule;
import com.google.inject.Provides;
import com.google.inject.Scopes;
import com.google.inject.Singleton;
import com.google.inject.multibindings.Multibinder;
+import com.google.inject.multibindings.ProvidesIntoSet;
public class PostgresUsersRepositoryModule extends AbstractModule {
@Override
@@ -54,4 +57,11 @@ public class PostgresUsersRepositoryModule extends
AbstractModule {
return PostgresUsersRepositoryConfiguration.from(
configurationProvider.getConfiguration("usersrepository"));
}
+
+@ProvidesIntoSet
+InitializationOperation configureInitialization(ConfigurationProvider
configurationProvider, PostgresUsersRepository usersRepository) {
+return InitilizationOperationBuilder
+.forClass(PostgresUsersRepository.class)
+.init(() ->
usersRepository.configure(configurationProvider.getConfiguration("usersrepository")));
+}
}
diff --git
a/server/data/data-library/src/main/java/org/apache/james/user/lib/UsersRepositoryImpl.java
b/server/data/data-library/src/main/java/org/apache/james/user/lib/UsersRepositoryImpl.java
index c338618532..f118d34cbf 100644
---
a/server/data/data-library/src/main/java/org/apache/james/user/lib/UsersRepositoryImpl.java
+++
b/server/data/data-library/src/main/java/org/apache/james/user/lib/UsersRepositoryImpl.java
@@ -78,6 +78,8 @@ public class UsersRepositoryImpl
implements UsersRepository,
verifyFailureDelay =
Optional.ofNullable(configuration.getString("verifyFailureDelay"))
.map(string -> DurationParser.parse(string,
ChronoUnit.SECONDS).toMillis())
.orElse(0L);
+LOGGER.debug("Init configure users repository with virtualHosting {},
administratorId {}, verifyFailureDelay {}",
+virtualHosting, administratorId, verifyFailureDelay);
}
public void setEnableVirtualHosting(boolean virtualHosting) {
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Initialization to configure users repository [james-project]
Arsnael merged PR #1837: URL: https://github.com/apache/james-project/pull/1837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
Arsnael commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1842072118 > A lot of docker crash on my pc when try provision 5k users :D don't hesitate to use james-charge-01 instead then :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 Postgres app - Fixup missing guice binding quota module & clean test [james-project]
Arsnael commented on PR #1838: URL: https://github.com/apache/james-project/pull/1838#issuecomment-1842059504 Need a rebase anyways :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
(james-project) branch postgresql updated: JAMES-2586 Implement PostgresPerUserMaxQuotaManager (#1839)
This is an automated email from the ASF dual-hosted git repository.
rcordier pushed a commit to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
The following commit(s) were added to refs/heads/postgresql by this push:
new c5bd9ab41f JAMES-2586 Implement PostgresPerUserMaxQuotaManager (#1839)
c5bd9ab41f is described below
commit c5bd9ab41f9e521af78bf82b25ad35a7c24f710f
Author: hungphan227 <45198168+hungphan...@users.noreply.github.com>
AuthorDate: Wed Dec 6 11:16:38 2023 +0700
JAMES-2586 Implement PostgresPerUserMaxQuotaManager (#1839)
---
.../org/apache/james/mailbox}/quota/Limits.java| 2 +-
.../apache/james/mailbox}/quota/QuotaCodec.java| 8 +-
.../quota/CassandraGlobalMaxQuotaDao.java | 2 +
.../quota/CassandraPerDomainMaxQuotaDao.java | 2 +
.../quota/CassandraPerUserMaxQuotaDao.java | 2 +
.../quota/CassandraPerUserMaxQuotaManagerV1.java | 1 +
.../quota/CassandraPerUserMaxQuotaManagerV2.java | 2 +
mailbox/postgres/pom.xml | 21 --
.../postgres/quota/JPAPerUserMaxQuotaDAO.java | 238 -
.../postgres/quota/JPAPerUserMaxQuotaManager.java | 292 -
.../quota/PostgresPerUserMaxQuotaManager.java} | 141 +-
.../quota/model/MaxDomainMessageCount.java | 54
.../postgres/quota/model/MaxDomainStorage.java | 55
.../quota/model/MaxGlobalMessageCount.java | 54
.../postgres/quota/model/MaxGlobalStorage.java | 54
.../postgres/quota/model/MaxUserMessageCount.java | 52
.../postgres/quota/model/MaxUserStorage.java | 53
.../james/mailbox/postgres/JPAMailboxFixture.java | 51
...ava => PostgresPerUserMaxQuotaManagerTest.java} | 22 +-
.../postgres/src/test/resources/persistence.xml| 42 ---
.../postgres/host/PostgresHostSystem.java | 26 +-
.../src/main/resources/META-INF/persistence.xml| 9 -
.../james/modules/mailbox/PostgresQuotaModule.java | 10 +-
23 files changed, 104 insertions(+), 1089 deletions(-)
diff --git
a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/Limits.java
b/mailbox/api/src/main/java/org/apache/james/mailbox/quota/Limits.java
similarity index 97%
rename from
mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/Limits.java
rename to mailbox/api/src/main/java/org/apache/james/mailbox/quota/Limits.java
index 3ef7aec097..f278d03ed7 100644
---
a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/Limits.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/quota/Limits.java
@@ -17,7 +17,7 @@
* under the License. *
/
-package org.apache.james.mailbox.cassandra.quota;
+package org.apache.james.mailbox.quota;
import java.util.Optional;
diff --git
a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/QuotaCodec.java
b/mailbox/api/src/main/java/org/apache/james/mailbox/quota/QuotaCodec.java
similarity index 90%
rename from
mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/QuotaCodec.java
rename to
mailbox/api/src/main/java/org/apache/james/mailbox/quota/QuotaCodec.java
index 87b6cdcef7..d3d9b5cd67 100644
---
a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/QuotaCodec.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/quota/QuotaCodec.java
@@ -16,7 +16,7 @@
* specific language governing permissions and limitations *
* under the License. *
/
-package org.apache.james.mailbox.cassandra.quota;
+package org.apache.james.mailbox.quota;
import java.util.Optional;
import java.util.function.Function;
@@ -30,18 +30,18 @@ public class QuotaCodec {
private static final long INFINITE = -1;
private static final long NO_RIGHT = 0L;
-static Long quotaValueToLong(QuotaLimitValue value) {
+public static Long quotaValueToLong(QuotaLimitValue value) {
if (value.isUnlimited()) {
return INFINITE;
}
return value.asLong();
}
-static Optional longToQuotaSize(Long value) {
+public static Optional longToQuotaSize(Long value) {
return longToQuotaValue(value, QuotaSizeLimit.unlimited(),
QuotaSizeLimit::size);
}
-static Optional longToQuotaCount(Long value) {
+public static Optional longToQuotaCount(Long value) {
return longToQuotaValue(value, QuotaCountLimit.unlimited(),
QuotaCountLimit::count);
}
diff --git
a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/CassandraGlobalMaxQuotaDao.java
b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/quota/CassandraGlobalMaxQuotaDao.java
index
Re: [PR] JAMES-2586 Implement PostgresPerUserMaxQuotaManager [james-project]
Arsnael merged PR #1839: URL: https://github.com/apache/james-project/pull/1839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
vttranlina commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1841960541 > Just a remark: did you warm up James before testing? I ran the Gatling test with duration=5m before real test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
quantranhong1999 commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1841955971 Just a remark: did you warm up James before testing? BTW 5k users looks too easy on your "super" machine :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
vttranlina commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1841950324 ## Benchmarks - Env: on my local PC (both run gatling & james in the same PC) - Sample data:  - The first time: Duration: 40m  - The second time. Duration 15m  ### Conclusion The better is not clear?, it will be more trust if Gatling and James are on separate servers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
[PR] Little polish of the PG implementation [james-project]
chibenwa opened a new pull request, #1841: URL: https://github.com/apache/james-project/pull/1841 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] Various polish for the PGSQL implementation [james-project]
chibenwa closed pull request #1821: Various polish for the PGSQL implementation URL: https://github.com/apache/james-project/pull/1821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] Various polish for the PGSQL implementation [james-project]
chibenwa commented on PR #1821: URL: https://github.com/apache/james-project/pull/1821#issuecomment-1840747333 Can be safely closed as thos commits were integrated in @vttranlina work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
vttranlina commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1840540066 > I'll approuve when provided benchmarks backing this change ;-) yes, I'm doing it. It takes time to fix some guice app and provision sample data. A lot of docker crash on my pc when try provision 5k users :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 [PGSQL] Optimize Mailbox/Message Mapper [james-project]
chibenwa commented on PR #1834: URL: https://github.com/apache/james-project/pull/1834#issuecomment-1840516532 I'll approuve when provided benchmarks backing this change ;-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
(james-project) 03/03: JAMES-2586 PostgresExecutor: Retry only upon PreparedStatement conflict exception
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 9081a2f5a0aedc0bc921a88d340200e36902a381
Author: Quan Tran
AuthorDate: Tue Dec 5 12:25:00 2023 +0700
JAMES-2586 PostgresExecutor: Retry only upon PreparedStatement conflict
exception
io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: [42P05]
prepared statement "S_0" already exists
Should not retry upon other fatal exception e.g. database failure, invalid
authorization...
---
.../backends/postgres/utils/PostgresExecutor.java | 18 ++
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git
a/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
b/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
index b8c39ae88f..b530405f09 100644
---
a/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
+++
b/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
@@ -22,6 +22,7 @@ package org.apache.james.backends.postgres.utils;
import java.time.Duration;
import java.util.Optional;
import java.util.function.Function;
+import java.util.function.Predicate;
import javax.inject.Inject;
@@ -37,6 +38,7 @@ import org.jooq.impl.DSL;
import com.google.common.annotations.VisibleForTesting;
import io.r2dbc.spi.Connection;
+import io.r2dbc.spi.R2dbcBadGrammarException;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.util.retry.Retry;
@@ -82,26 +84,30 @@ public class PostgresExecutor {
public Mono executeVoid(Function> queryFunction)
{
return dslContext()
.flatMap(queryFunction)
-.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF))
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF)
+.filter(preparedStatementConflictException()))
.then();
}
public Flux executeRows(Function>
queryFunction) {
return dslContext()
.flatMapMany(queryFunction)
-.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF));
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF)
+.filter(preparedStatementConflictException()));
}
public Mono executeRow(Function>
queryFunction) {
return dslContext()
.flatMap(queryFunction)
-.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF));
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF)
+.filter(preparedStatementConflictException()));
}
public Mono executeCount(Function>> queryFunction) {
return dslContext()
.flatMap(queryFunction)
-.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF))
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF)
+.filter(preparedStatementConflictException()))
.map(Record1::value1);
}
@@ -113,4 +119,8 @@ public class PostgresExecutor {
public Mono dispose() {
return connection.flatMap(con -> Mono.from(con.close()));
}
+
+private Predicate preparedStatementConflictException() {
+return throwable -> throwable.getCause() instanceof
R2dbcBadGrammarException;
+}
}
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
(james-project) 02/03: JAMES-2586 PostgresExecutor: Retry upon PreparedStatement conflicts
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit dfb5d803795083b01a472bc0db42bb081f3f0ccd
Author: Quan Tran
AuthorDate: Tue Dec 5 12:15:33 2023 +0700
JAMES-2586 PostgresExecutor: Retry upon PreparedStatement conflicts
PreparedStatement id is unique per PG connection.
We share a PG connection across multi threads leads to PreparedStatement id
conflicts.
We can retry upon PreparedStatement id conflicts.
---
.../james/backends/postgres/utils/PostgresExecutor.java | 12 ++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git
a/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
b/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
index 1fa3ccb410..b8c39ae88f 100644
---
a/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
+++
b/backends-common/postgres/src/main/java/org/apache/james/backends/postgres/utils/PostgresExecutor.java
@@ -19,6 +19,7 @@
package org.apache.james.backends.postgres.utils;
+import java.time.Duration;
import java.util.Optional;
import java.util.function.Function;
@@ -38,10 +39,13 @@ import com.google.common.annotations.VisibleForTesting;
import io.r2dbc.spi.Connection;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
+import reactor.util.retry.Retry;
public class PostgresExecutor {
public static final String DEFAULT_INJECT = "default";
+public static final int MAX_RETRY_ATTEMPTS = 5;
+public static final Duration MIN_BACKOFF = Duration.ofMillis(1);
public static class Factory {
@@ -78,22 +82,26 @@ public class PostgresExecutor {
public Mono executeVoid(Function> queryFunction)
{
return dslContext()
.flatMap(queryFunction)
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF))
.then();
}
public Flux executeRows(Function>
queryFunction) {
return dslContext()
-.flatMapMany(queryFunction);
+.flatMapMany(queryFunction)
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF));
}
public Mono executeRow(Function>
queryFunction) {
return dslContext()
-.flatMap(queryFunction);
+.flatMap(queryFunction)
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF));
}
public Mono executeCount(Function>> queryFunction) {
return dslContext()
.flatMap(queryFunction)
+.retryWhen(Retry.backoff(MAX_RETRY_ATTEMPTS, MIN_BACKOFF))
.map(Record1::value1);
}
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
(james-project) 01/03: JAMES-2586 Rework ConnectionThreadSafetyTest -> PostgresExecutorThreadSafetyTest
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 07986593001814dc58f85af090198f29f36da0b9
Author: Quan Tran
AuthorDate: Tue Dec 5 10:53:52 2023 +0700
JAMES-2586 Rework ConnectionThreadSafetyTest ->
PostgresExecutorThreadSafetyTest
We do not control directly r2dbc-postgresql Connection but library owner,
thus we can do nothing upon tests failure.
But we can handle library failure at James layer using PostgresExecutor
wrapper.
---
java => PostgresExecutorThreadSafetyTest.java} | 135 +
1 file changed, 59 insertions(+), 76 deletions(-)
diff --git
a/backends-common/postgres/src/test/java/org/apache/james/backends/postgres/ConnectionThreadSafetyTest.java
b/backends-common/postgres/src/test/java/org/apache/james/backends/postgres/PostgresExecutorThreadSafetyTest.java
similarity index 55%
rename from
backends-common/postgres/src/test/java/org/apache/james/backends/postgres/ConnectionThreadSafetyTest.java
rename to
backends-common/postgres/src/test/java/org/apache/james/backends/postgres/PostgresExecutorThreadSafetyTest.java
index 4cdecdc86d..e8c3d6a9f8 100644
---
a/backends-common/postgres/src/test/java/org/apache/james/backends/postgres/ConnectionThreadSafetyTest.java
+++
b/backends-common/postgres/src/test/java/org/apache/james/backends/postgres/PostgresExecutorThreadSafetyTest.java
@@ -23,79 +23,65 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.time.Duration;
import java.util.List;
-import java.util.Optional;
import java.util.Set;
import java.util.Vector;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.Function;
import java.util.stream.Stream;
-import
org.apache.james.backends.postgres.utils.DomainImplPostgresConnectionFactory;
-import org.apache.james.core.Domain;
+import org.apache.james.backends.postgres.utils.PostgresExecutor;
import org.apache.james.util.concurrency.ConcurrentTestRunner;
-import org.jetbrains.annotations.NotNull;
+import org.jooq.Record;
+import org.jooq.impl.DSL;
+import org.jooq.impl.SQLDataType;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
-import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
-import io.r2dbc.postgresql.api.PostgresqlConnection;
-import io.r2dbc.postgresql.api.PostgresqlResult;
-import io.r2dbc.spi.Connection;
-import io.r2dbc.spi.Result;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
-public class ConnectionThreadSafetyTest {
+class PostgresExecutorThreadSafetyTest {
static final int NUMBER_OF_THREAD = 100;
-static final String CREATE_TABLE_STATEMENT = "CREATE TABLE IF NOT EXISTS
person (\n" +
-"\tid serial PRIMARY KEY,\n" +
-"\tname VARCHAR ( 50 ) UNIQUE NOT NULL\n" +
-");";
@RegisterExtension
static PostgresExtension postgresExtension = PostgresExtension.empty();
-private static PostgresqlConnection postgresqlConnection;
-private static DomainImplPostgresConnectionFactory
jamesPostgresConnectionFactory;
+private static PostgresExecutor postgresExecutor;
@BeforeAll
static void beforeAll() {
-jamesPostgresConnectionFactory = new
DomainImplPostgresConnectionFactory(postgresExtension.getConnectionFactory());
-postgresqlConnection = (PostgresqlConnection)
postgresExtension.getConnection().block();
+postgresExecutor = postgresExtension.getPostgresExecutor();
}
@BeforeEach
void beforeEach() {
-postgresqlConnection.createStatement(CREATE_TABLE_STATEMENT)
-.execute()
-.flatMap(PostgresqlResult::getRowsUpdated)
-.then()
+postgresExecutor.executeVoid(dslContext ->
Mono.from(dslContext.createTableIfNotExists("person")
+.column("id", SQLDataType.INTEGER.identity(true))
+.column("name", SQLDataType.VARCHAR(50).nullable(false))
+.constraints(DSL.constraint().primaryKey("id"))
+.unique("name")))
.block();
}
@AfterEach
void afterEach() {
-postgresqlConnection.createStatement("DROP TABLE person")
-.execute()
-.flatMap(PostgresqlResult::getRowsUpdated)
-.then()
+postgresExecutor.executeVoid(dslContext ->
Mono.from(dslContext.dropTableIfExists("person")))
.block();
}
@Test
-void
connectionShouldWorkWellWhenItIsUsedByMultipleThreadsAndAllQueriesAreSelect()
throws Exception {
-createData(NUMBER_OF_THREAD);
-
-Connection connection =
jamesPostgresConnectionFactory.getConnection(Domain.of("james")).block();
+void
(james-project) branch postgresql updated (eb773f4f44 -> 9081a2f5a0)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch postgresql
in repository https://gitbox.apache.org/repos/asf/james-project.git
from eb773f4f44 JAMES-2586 Fix Postgres Mailbox Annotation mpt imap test
new 0798659300 JAMES-2586 Rework ConnectionThreadSafetyTest ->
PostgresExecutorThreadSafetyTest
new dfb5d80379 JAMES-2586 PostgresExecutor: Retry upon PreparedStatement
conflicts
new 9081a2f5a0 JAMES-2586 PostgresExecutor: Retry only upon
PreparedStatement conflict exception
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../backends/postgres/utils/PostgresExecutor.java | 22 +++-
java => PostgresExecutorThreadSafetyTest.java} | 135 +
2 files changed, 79 insertions(+), 78 deletions(-)
rename
backends-common/postgres/src/test/java/org/apache/james/backends/postgres/{ConnectionThreadSafetyTest.java
=> PostgresExecutorThreadSafetyTest.java} (55%)
-
To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org
For additional commands, e-mail: notifications-h...@james.apache.org
Re: [PR] JAMES-2586 PostgresExecutor: Retry upon PreparedStatement conflicts [james-project]
chibenwa merged PR #1840: URL: https://github.com/apache/james-project/pull/1840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@james.apache.org For additional commands, e-mail: notifications-h...@james.apache.org
