Re: Jenkins CI setup
Hello everyone, The current jenkins setup won't run builds on PRs from people who don't have write access to the repository. This means that even though Benoit fixed the cassandra-related flaky test[1] and I fixed a rabbitmq-related flaky test[2] andI then rebased it all to account for Benoit's changes, I can't go any further because I can't trigger builds to detect more flaky tests or reach a successful build :) I need help from one of the project's committers to duplicate the PR again (as matthieu did in https://github.com/apache/james-project/pull/265) to get jenkins to build it (or to change the settings in jenkins to let it build PRs from "untrusted" users or somehow whitelist PR #265 to be built I'm not sure exactly what can be done there). Of note, the apache ci platform seems reasonably powerful when running the builds, I don't have much comparison points but a full project build (without tests) at T1C completes in about 5 minutes. I have no idea how that compares to the private linagora CI but it would be so much better than the nothing non linagora user currently enjoy ;) It is unclear from the documentation[3] whether there really are compute quotas enabled or not, I don't think there are by default and watching the activemq/maven builds run along for hours on the shared nodes I feel that james will be just fine :) The documentation [4] also mentions that apache CI boasts a free SonarQube integration with sonarcloud.io, I think it would be a nice next step after CI is enabled on the project Thanks in advance Jean As a side note I'm currently upgrading the MPT SMTP tests to use junit 5 jupiter APIS (with extensions and the like) instead of rules. [1] https://github.com/apache/james-project/pull/267 [2] https://github.com/apache/james-project/pull/264/commits/c2218b1eff07245294ee976507b9cda013a1b0b7 [3] https://cwiki.apache.org/confluence/display/INFRA/Jenkins [4] https://cwiki.apache.org/confluence/display/INFRA/SonarQube+Analysis On Thu, Nov 26, 2020 at 12:30 PM Jean Helou wrote: > > > Likely "unstable". >> >> I will have a look at this tomorrow. >> > > Ok thanks > >>
[jira] [Commented] (JAMES-3451) james 3.5.0 OutOfMemoryError
[ https://issues.apache.org/jira/browse/JAMES-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240913#comment-17240913 ] Simon Levesque commented on JAMES-3451: --- On my side, I am using the MaridDB driver. I switched the DataSource that I provide in Guice from MariaDbPoolDataSource to BasicDataSource from the commons-dbcp. It has been running for the last 2 days and it didn't have to restart. Normally, I reach 4G of memory used in a bit less than 24 hours. So, thanks for the tip. I just have one question. There is still one place where I must put the driver directly: the james-database.properties file ( [https://github.com/foilen/foilen-email-server/blob/dev/src/main/resources/com/foilen/email/server/james/conf/james-database.properties.ftl] ) Is that file: * used only once? (if yes, then that is fine since it won't build up any memory leak) * if used more than once, is there a way to provide a datapool? thanks > james 3.5.0 OutOfMemoryError > > > Key: JAMES-3451 > URL: https://issues.apache.org/jira/browse/JAMES-3451 > Project: James Server > Issue Type: Bug > Components: mailbox >Affects Versions: 3.5.0 > Environment: aliyun linux & mysql 8 & jpa-guice >Reporter: owenzhu >Priority: Major > Attachments: bigobject.png, thread.png > > > database: mysql8 > platform: aliyun linux > jvm params: -Xms128m -Xmx2560m > When I run the James for a long time, the james server used more and more > heap memory, eventually it runs out of memory and refuse to receive email. > only restart the james will work. > java.lang.OutOfMemoryError: Java heap spacejava.lang.OutOfMemoryError: Java > heap space at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2173) at > com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1992) at > com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3413) at > com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:471) at > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3115) at > com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2344) at > com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2739) at > com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2486) at > com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858) > at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1966) > at > org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:302) > at > org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeQuery(LoggingConnectionDecorator.java:1169) > at > org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:300) > at > org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1866) > at > org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:290) > at org.apache.openjpa.jdbc.sql.SelectImpl.executeQuery(SelectImpl.java:530) > at org.apache.openjpa.jdbc.sql.SelectImpl.execute(SelectImpl.java:455) at > org.apache.openjpa.jdbc.sql.SelectImpl.execute(SelectImpl.java:422) at > org.apache.openjpa.jdbc.sql.LogicalUnion$UnionSelect.execute(LogicalUnion.java:472) > at org.apache.openjpa.jdbc.sql.LogicalUnion.execute(LogicalUnion.java:254) > at org.apache.openjpa.jdbc.sql.LogicalUnion.execute(LogicalUnion.java:243) at > org.apache.openjpa.jdbc.kernel.SelectResultObjectProvider.open(SelectResultObjectProvider.java:95) > at > org.apache.openjpa.lib.rop.EagerResultList.(EagerResultList.java:36) at > org.apache.openjpa.kernel.QueryImpl.toResult(QueryImpl.java:1311) at > org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1062) at > org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:912) at > org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:843) at > org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:601) > at org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:297) at > org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:314) at > org.apache.james.mailbox.jpa.mail.JPAMessageMapper.findMessagesInMailbox(JPAMessageMapper.java:421) > at > org.apache.james.mailbox.jpa.mail.JPAMessageMapper.findAsList(JPAMessageMapper.java:113) > I used jProfile to parse the dump file and found that many transactions were > waiting.See the attached screenshot for details. > In addition, I will find the following error in the log, I don't know whether > it is the cause of OOM, ApplicableFlags is null in debug : > 2020-11-17 21:41:00.390 [ERROR] [elastic-1226] > (o.a.j.i.p.base.SelectedMailboxImpl:367) - applicableFlags is null, > boxId=130656, mail=udysk@88mail.vip2020-11-17 21:41:00.390 [ERROR] >
[jira] [Closed] (JAMES-3459) Implement a MailboxChangeRepository
[ https://issues.apache.org/jira/browse/JAMES-3459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benoit Tellier closed JAMES-3459. - Fix Version/s: 3.6.0 Resolution: Fixed https://github.com/linagora/james-project/pull/4088 solved this and is merged. > Implement a MailboxChangeRepository > --- > > Key: JAMES-3459 > URL: https://issues.apache.org/jira/browse/JAMES-3459 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > Fix For: 3.6.0 > > > From the spec: [https://jmap.io/spec-core.html#changes] > ``` > When the state of the set of Foo records in an account changes on the server > (whether due to creation, updates, or deletion), the state property of the > Foo/get response will change. > ``` > We need to store all the changes made to mailbox objects in a table name > MailboxChangeRepository. First we will implement a memory version of it. > h1. How > * Define MailboxChange model. > * Define a contract for the MailboxChangeRepository. > * Implement MemoryMailboxChangeRepository APIs (save, getSinceState) base on > the contract. > h1. DoD > Write unit tests to show that MemoryMailboxChangesRepository is functioning > properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Closed] (JAMES-3458) [IMAP] GetQuotaRoot commands is issuing more than 10% of cassandra qurey time!
[ https://issues.apache.org/jira/browse/JAMES-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benoit Tellier closed JAMES-3458. - Resolution: Fixed > [IMAP] GetQuotaRoot commands is issuing more than 10% of cassandra qurey time! > -- > > Key: JAMES-3458 > URL: https://issues.apache.org/jira/browse/JAMES-3458 > Project: James Server > Issue Type: Improvement > Components: cassandra, IMAPServer >Reporter: Benoit Tellier >Priority: Major > Labels: perf > Fix For: 3.6.0 > > Attachments: Capture d’écran de 2020-11-27 10-58-05.png, Capture > d’écran de 2020-11-27 10-58-28.png > > > # Why > IMAP clients performs a GETQUOTAROOT command for each mailboxes. > Untagged responses also include the actual quota; we end up because of IMAP > design reading the quotas for each mailboxes! This is an issue eg when a > given user have hundreds of mailboxes! (Who said IMAP is a BAD protocol?) > # How > While patching IMAP design is not doable, we can optimize our queries to > reduce our query count: storage and count quota parts (current usage, max > user storage, max domain storage) are hosted on the same row, that we are > reading two times due to bad readpath design. > Unlocking grouped read for collocated data would likely make this much less > of an issue. > **Definition of done**: GetQuotaRoot cassandra query volume should be reduced > of 40% > Glowroot captures attached. The first shows queries executed by GetQuotaroot, > as well as the percentage of total IMAP time spent on GetQuotaRoot queries > resolution. > As a comparison the second capture shows all the IMAP queries being issued... > # Impact > Note that this work would also enhance slightly append speed (SMTP, IMAP > append, etc) as quota needs to be checked, backend quota processing (quota > updated events are less expensive to generate), and will slightly help > regarding JMAP Mailbox/get call (which reads the quota). Though the > enhancement would be less noticeable than for GetQuotaRoot IMAP command. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3458) [IMAP] GetQuotaRoot commands is issuing more than 10% of cassandra qurey time!
[ https://issues.apache.org/jira/browse/JAMES-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240722#comment-17240722 ] Benoit Tellier commented on JAMES-3458: --- https://github.com/linagora/james-project/pull/4092 solved this. > [IMAP] GetQuotaRoot commands is issuing more than 10% of cassandra qurey time! > -- > > Key: JAMES-3458 > URL: https://issues.apache.org/jira/browse/JAMES-3458 > Project: James Server > Issue Type: Improvement > Components: cassandra, IMAPServer >Reporter: Benoit Tellier >Priority: Major > Labels: perf > Fix For: 3.6.0 > > Attachments: Capture d’écran de 2020-11-27 10-58-05.png, Capture > d’écran de 2020-11-27 10-58-28.png > > > # Why > IMAP clients performs a GETQUOTAROOT command for each mailboxes. > Untagged responses also include the actual quota; we end up because of IMAP > design reading the quotas for each mailboxes! This is an issue eg when a > given user have hundreds of mailboxes! (Who said IMAP is a BAD protocol?) > # How > While patching IMAP design is not doable, we can optimize our queries to > reduce our query count: storage and count quota parts (current usage, max > user storage, max domain storage) are hosted on the same row, that we are > reading two times due to bad readpath design. > Unlocking grouped read for collocated data would likely make this much less > of an issue. > **Definition of done**: GetQuotaRoot cassandra query volume should be reduced > of 40% > Glowroot captures attached. The first shows queries executed by GetQuotaroot, > as well as the percentage of total IMAP time spent on GetQuotaRoot queries > resolution. > As a comparison the second capture shows all the IMAP queries being issued... > # Impact > Note that this work would also enhance slightly append speed (SMTP, IMAP > append, etc) as quota needs to be checked, backend quota processing (quota > updated events are less expensive to generate), and will slightly help > regarding JMAP Mailbox/get call (which reads the quota). Though the > enhancement would be less noticeable than for GetQuotaRoot IMAP command. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2884) Update JMAP implementation to conform to RFC 8620/8621
[ https://issues.apache.org/jira/browse/JAMES-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240721#comment-17240721 ] Benoit Tellier commented on JAMES-2884: --- Thanks Lan! https://github.com/linagora/james-project/pull/4093 better documented Authentication on top of the new Jmap specification following https://github.com/linagora/jmap-client/issues/87 which showed this was needed ;-) > Update JMAP implementation to conform to RFC 8620/8621 > -- > > Key: JAMES-2884 > URL: https://issues.apache.org/jira/browse/JAMES-2884 > Project: James Server > Issue Type: Improvement > Components: JMAP >Reporter: cketti >Assignee: Antoine Duprat >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > Historically, James is an early adopter for the JMAP specification, and a > first partial implementation was conducted when JMAP was just a draft. IETF > draft undergo radical changes and the community could not keep this > implementation up to date with the spec changes. > As off summer 2019, JMAP core ([RFC > 8620|https://tools.ietf.org/html/rfc8620]) and JMAP mail ([RFC > 8621|https://tools.ietf.org/html/rfc8621]) had been officially published > (will not change anymore). Thus we should implement these new specifications. > Point of attention: part of the community actively rely on the actual 'draft' > implementation of JMAP existing in James. We should ensure no changes is done > to that 'draft' protocol is done while implementing the new one. > The proposed approach is to keep the current implementation under the > `jmap-draft` name, and implement step by step a `jmap` compliant > implementation, that will be exposed on a separate port. No modification in > `jmap-draft` integration test should be counducted. > This will allow existing `jmap-draft` clients to smoothly transition to > `jmap`, then trigger the classic "deprecation-then-removal" process. > For now, as a first implementation step, we will only support `jmap` on top > of memory-guice (ease testing, speed of development). To ensure a > `storage-compliant` behavior of newly introduced storage APIs, we should use > persistent datastructures (like the one in vavr) and always deep-copy objects > at the storage boundaries. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-2884) Update JMAP implementation to conform to RFC 8620/8621
[ https://issues.apache.org/jira/browse/JAMES-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240664#comment-17240664 ] Lan Khuat commented on JAMES-2884: -- Hi everyone, I'm Lan, a developer of the Apache James project. In the coming month, i will be working on the JMAP Push for James, starting with a POC for the Mailbox/changes part. First we want to have a repository to track all the changes made to Mailbox objects, in memory implemented. See: https://issues.apache.org/jira/browse/JAMES-3459 You can take a look at the PR which handle the aforementioned ticket here: [https://github.com/linagora/james-project/pull/4088 |https://github.com/linagora/james-project/pull/4088] Tickets need to be handled: * Implement a MailboxChangeListener: https://issues.apache.org/jira/browse/JAMES-3460 * Implement Mailbox/changes method: https://issues.apache.org/jira/browse/JAMES-3461 * Cassandra version of the MailboxChangeRepository: https://issues.apache.org/jira/browse/JAMES-3462 * *state* property handling: https://issues.apache.org/jira/browse/JAMES-3463 * *oldState* & *newState* handling: https://issues.apache.org/jira/browse/JAMES-3464 * differentiate changes between a Mailbox and messages belong to it: https://issues.apache.org/jira/browse/JAMES-3465 > Update JMAP implementation to conform to RFC 8620/8621 > -- > > Key: JAMES-2884 > URL: https://issues.apache.org/jira/browse/JAMES-2884 > Project: James Server > Issue Type: Improvement > Components: JMAP >Reporter: cketti >Assignee: Antoine Duprat >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > Historically, James is an early adopter for the JMAP specification, and a > first partial implementation was conducted when JMAP was just a draft. IETF > draft undergo radical changes and the community could not keep this > implementation up to date with the spec changes. > As off summer 2019, JMAP core ([RFC > 8620|https://tools.ietf.org/html/rfc8620]) and JMAP mail ([RFC > 8621|https://tools.ietf.org/html/rfc8621]) had been officially published > (will not change anymore). Thus we should implement these new specifications. > Point of attention: part of the community actively rely on the actual 'draft' > implementation of JMAP existing in James. We should ensure no changes is done > to that 'draft' protocol is done while implementing the new one. > The proposed approach is to keep the current implementation under the > `jmap-draft` name, and implement step by step a `jmap` compliant > implementation, that will be exposed on a separate port. No modification in > `jmap-draft` integration test should be counducted. > This will allow existing `jmap-draft` clients to smoothly transition to > `jmap`, then trigger the classic "deprecation-then-removal" process. > For now, as a first implementation step, we will only support `jmap` on top > of memory-guice (ease testing, speed of development). To ensure a > `storage-compliant` behavior of newly introduced storage APIs, we should use > persistent datastructures (like the one in vavr) and always deep-copy objects > at the storage boundaries. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3462) Implement CassandraMailboxChangeRepository
[ https://issues.apache.org/jira/browse/JAMES-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3462: - Description: Same purpose with the MemoryMailboxChangeRepository, we now need to implement it in Cassandra. h1. How 1. Define a MailboxChange table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the CassandraMailboxChangeRepository APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the MailboxChangeRepository is working properly. was: Same purpose with the MemoryMailboxChangeRepository, we now need to implement it in Cassandra. h1. How 1. Define a MailboxChange table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the CassandraMailboxChangeRepository APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the MailboxChangeStore is working properly. > Implement CassandraMailboxChangeRepository > -- > > Key: JAMES-3462 > URL: https://issues.apache.org/jira/browse/JAMES-3462 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > Same purpose with the MemoryMailboxChangeRepository, we now need to implement > it in Cassandra. > h1. How > 1. Define a MailboxChange table with these fields: > - accountId > - state > - created > - updated > - destroyed > - date > 2. Implement the CassandraMailboxChangeRepository APIs, based on the contract > that has already been defined. > h1. DoD > Write unit tests to show that the MailboxChangeRepository is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3462) Implement CassandraMailboxChangeRepository
[ https://issues.apache.org/jira/browse/JAMES-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3462: - Summary: Implement CassandraMailboxChangeRepository (was: Implement CassandraMailboxChangeStore) > Implement CassandraMailboxChangeRepository > -- > > Key: JAMES-3462 > URL: https://issues.apache.org/jira/browse/JAMES-3462 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > Same purpose with the MemoryMailboxChangeStore, we now need to implement it > in Cassandra. > h1. How > 1. Define a MailboxChange table with these fields: > - accountId > - state > - created > - updated > - destroyed > - date > 2. Implement the CassandraMailboxChangeStore APIs, based on the contract that > has already been defined. > h1. DoD > Write unit tests to show that the MailboxChangeStore is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3462) Implement CassandraMailboxChangeRepository
[ https://issues.apache.org/jira/browse/JAMES-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3462: - Description: Same purpose with the MemoryMailboxChangeRepository, we now need to implement it in Cassandra. h1. How 1. Define a MailboxChange table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the CassandraMailboxChangeRepository APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the MailboxChangeStore is working properly. was: Same purpose with the MemoryMailboxChangeStore, we now need to implement it in Cassandra. h1. How 1. Define a MailboxChange table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the CassandraMailboxChangeStore APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the MailboxChangeStore is working properly. > Implement CassandraMailboxChangeRepository > -- > > Key: JAMES-3462 > URL: https://issues.apache.org/jira/browse/JAMES-3462 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > Same purpose with the MemoryMailboxChangeRepository, we now need to implement > it in Cassandra. > h1. How > 1. Define a MailboxChange table with these fields: > - accountId > - state > - created > - updated > - destroyed > - date > 2. Implement the CassandraMailboxChangeRepository APIs, based on the contract > that has already been defined. > h1. DoD > Write unit tests to show that the MailboxChangeStore is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3464) Mailbox/set should handle oldState & newState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240657#comment-17240657 ] Lan Khuat commented on JAMES-3464: -- Sorry i was mistaken. Description fixed now. > Mailbox/set should handle oldState & newState > - > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > oldState: The state string that would have been returned by Foo/get before > making the requested changes, or null if the server doesn’t know what the > previous state string was. > newState: The state string that will now be returned by Foo/get. > {code} > h1. How > * When a Mailbox/set request is received, we need to fetch the current state > of the Mailbox objects. This should be returned as the *oldState* property in > the response. > * After all the changes in the Mailbox/set request have been applied > successfully, we should create a new state, store it in the > MailboxChangeRepository and return it with the response as the *newState* > property. > * If all the methodCalls in the request end up failing then no new state > should be generated. > h1. DoD > Integration tests to show that the Mailbox/set method can return oldState & > newState property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3464) Mailbox/set should handle oldState & newState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3464: - Description: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} oldState: The state string that would have been returned by Foo/get before making the requested changes, or null if the server doesn’t know what the previous state string was. newState: The state string that will now be returned by Foo/get. {code} h1. How * When a Mailbox/set request is received, we need to fetch the current state of the Mailbox objects. This should be returned as the *oldState* property in the response. * After all the changes in the Mailbox/set request have been applied successfully, we should create a new state, store it in the MailboxChangeRepository and return it with the response as the *newState* property. * If all the methodCalls in the request end up failing then no new state should be generated. h1. DoD Integration tests to show that the Mailbox/set method can return oldState & newState property. was: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} oldState: The state string that would have been returned by Foo/get before making the requested changes, or null if the server doesn’t know what the previous state string was. newState: The state string that will now be returned by Foo/get. {code} h1. How * When a Mailbox/set request is received, we need to fetch the current state of the Mailbox objects. This should be returned as `oldState` property in the response. * After all the changes in the Mailbox/set request have been applied successfully, we should create a new state, store it in the MailboxChangeRepository and return it with the response as `newState` property. * If all the methodCalls in the request end up failing then no new state should be generated. h1. DoD Integration tests to show that the Mailbox/set method can return oldState & newState property. > Mailbox/set should handle oldState & newState > - > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > oldState: The state string that would have been returned by Foo/get before > making the requested changes, or null if the server doesn’t know what the > previous state string was. > newState: The state string that will now be returned by Foo/get. > {code} > h1. How > * When a Mailbox/set request is received, we need to fetch the current state > of the Mailbox objects. This should be returned as the *oldState* property in > the response. > * After all the changes in the Mailbox/set request have been applied > successfully, we should create a new state, store it in the > MailboxChangeRepository and return it with the response as the *newState* > property. > * If all the methodCalls in the request end up failing then no new state > should be generated. > h1. DoD > Integration tests to show that the Mailbox/set method can return oldState & > newState property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3464) Mailbox/set should handle oldState & newState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3464: - Description: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} oldState: The state string that would have been returned by Foo/get before making the requested changes, or null if the server doesn’t know what the previous state string was. newState: The state string that will now be returned by Foo/get. {code} h1. How * When a Mailbox/set request is received, we need to fetch the current state of the Mailbox objects. This should be returned as `oldState` property in the response. * After all the changes in the Mailbox/set request have been applied successfully, we should create a new state, store it in the MailboxChangeRepository and return it with the response as `newState` property. * If all the methodCalls in the request end up failing then no new state should be generated. h1. DoD Integration tests to show that the Mailbox/set method can return oldState & newState property. was: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} oldState: The state string that would have been returned by Foo/get before making the requested changes, or null if the server doesn’t know what the previous state string was. newState: The state string that will now be returned by Foo/get. {code} h1. How - When a Mailbox/set request is received, we need to fetch the current state of the Mailbox objects. This should be returned as `oldState` property in the response. - After all the changes in the Mailbox/set request have been applied successfully, we should create a new state, store it in the MailboxChangeRepository and return it with the response as `newState` property. - If all the methodCalls in the request end up failing then no new state should be generated. h1. DoD Integration tests to show that the Mailbox/set method can return oldState & newState property. > Mailbox/set should handle oldState & newState > - > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > oldState: The state string that would have been returned by Foo/get before > making the requested changes, or null if the server doesn’t know what the > previous state string was. > newState: The state string that will now be returned by Foo/get. > {code} > h1. How > * When a Mailbox/set request is received, we need to fetch the current state > of the Mailbox objects. This should be returned as `oldState` property in the > response. > * After all the changes in the Mailbox/set request have been applied > successfully, we should create a new state, store it in the > MailboxChangeRepository and return it with the response as `newState` > property. > * If all the methodCalls in the request end up failing then no new state > should be generated. > h1. DoD > Integration tests to show that the Mailbox/set method can return oldState & > newState property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3464) Mailbox/set should handle oldState & newState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3464: - Description: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} oldState: The state string that would have been returned by Foo/get before making the requested changes, or null if the server doesn’t know what the previous state string was. newState: The state string that will now be returned by Foo/get. {code} h1. How - When a Mailbox/set request is received, we need to fetch the current state of the Mailbox objects. This should be returned as `oldState` property in the response. - After all the changes in the Mailbox/set request have been applied successfully, we should create a new state, store it in the MailboxChangeRepository and return it with the response as `newState` property. - If all the methodCalls in the request end up failing then no new state should be generated. h1. DoD Integration tests to show that the Mailbox/set method can return oldState & newState property. was: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} ifInState: This is a state string as returned by the Foo/get method (representing the state of all objects of this type in the account). If supplied, the string must match the current state; otherwise, the method will be aborted and a stateMismatch error returned. If null, any changes will be applied to the current state. {code} h1. How When Email/set request is received, if the ifInState property is present we need to compare it with the current Mailbox state stored in MailboxChangeRepository. * if mismatched, a stateMismatch error should be returned. * if matched, all the changes in the request will be applied and should create a new state, unless all the methodCalls in the request end up failing. If the ifInState property is omitted then all the changes will be applied to the current state. h1. DoD Integration tests to show that the Mailbox/set method can handle ifInState property. Summary: Mailbox/set should handle oldState & newState (was: Mailbox/set should handle ifInState) > Mailbox/set should handle oldState & newState > - > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > oldState: The state string that would have been returned by Foo/get before > making the requested changes, or null if the server doesn’t know what the > previous state string was. > newState: The state string that will now be returned by Foo/get. > {code} > h1. How > - When a Mailbox/set request is received, we need to fetch the current state > of the Mailbox objects. This should be returned as `oldState` property in the > response. > - After all the changes in the Mailbox/set request have been applied > successfully, we should create a new state, store it in the > MailboxChangeRepository and return it with the response as `newState` > property. > - If all the methodCalls in the request end up failing then no new state > should be generated. > > h1. DoD > Integration tests to show that the Mailbox/set method can return oldState & > newState property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3465) Mailbox/changes updatedProperties handling
[ https://issues.apache.org/jira/browse/JAMES-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240646#comment-17240646 ] Benoit Tellier commented on JAMES-3465: --- We need to adapt MailboxChanges storage to add a boolean indicating wether the changes concerns the mailboxes, or the messages contained in it - count properties. Based on this we could refine Mailbox/changes updatedProperties field. > Mailbox/changes updatedProperties handling > -- > > Key: JAMES-3465 > URL: https://issues.apache.org/jira/browse/JAMES-3465 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > We need to distinguish Mailbox "counts" updates from others. > We should have storage for "updated" distinct from "countUpdated" changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3464) Mailbox/set should handle ifInState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240645#comment-17240645 ] Benoit Tellier commented on JAMES-3464: --- I don't see how this fit with the Distributed James consistency model. I don't see how to easily and efficiently implement this. I don't see how this helps regarding Mailbox/changes. Did you confused it with the newState / oldState properties in the Mailbox/set response object? > Mailbox/set should handle ifInState > --- > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > ifInState: This is a state string as returned by the Foo/get method > (representing the state of all objects of this type in the account). If > supplied, the string must match the current state; otherwise, the method will > be aborted and a stateMismatch error returned. If null, any changes will be > applied to the current state. > {code} > h1. How > When Email/set request is received, if the ifInState property is present we > need to compare it with the current Mailbox state stored in > MailboxChangeRepository. > * if mismatched, a stateMismatch error should be returned. > * if matched, all the changes in the request will be applied and should > create a new state, unless all the methodCalls in the request end up failing. > If the ifInState property is omitted then all the changes will be applied to > the current state. > h1. DoD > Integration tests to show that the Mailbox/set method can handle ifInState > property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3465) Mailbox/changes updatedProperties handling
Lan Khuat created JAMES-3465: Summary: Mailbox/changes updatedProperties handling Key: JAMES-3465 URL: https://issues.apache.org/jira/browse/JAMES-3465 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat We need to distinguish Mailbox "counts" updates from others. We should have storage for "updated" distinct from "countUpdated" changes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3464) Mailbox/set should handle ifInState
[ https://issues.apache.org/jira/browse/JAMES-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3464: - Description: >From spec: [https://jmap.io/spec-core.html#set] (section 5.3) {code:java} ifInState: This is a state string as returned by the Foo/get method (representing the state of all objects of this type in the account). If supplied, the string must match the current state; otherwise, the method will be aborted and a stateMismatch error returned. If null, any changes will be applied to the current state. {code} h1. How When Email/set request is received, if the ifInState property is present we need to compare it with the current Mailbox state stored in MailboxChangeRepository. * if mismatched, a stateMismatch error should be returned. * if matched, all the changes in the request will be applied and should create a new state, unless all the methodCalls in the request end up failing. If the ifInState property is omitted then all the changes will be applied to the current state. h1. DoD Integration tests to show that the Mailbox/set method can handle ifInState property. was: >From spec: https://jmap.io/spec-core.html#set (section 5.3) {code:java} ifInState: This is a state string as returned by the Foo/get method (representing the state of all objects of this type in the account). If supplied, the string must match the current state; otherwise, the method will be aborted and a stateMismatch error returned. If null, any changes will be applied to the current state. {code} h1. How When Email/set request is received, if the ifInState property is present we need to compare it with the current Mailbox state stored in MailboxChangeRepository. - if mismatched, a stateMismatch error should be returned. - if matched, all the changes in the request will be applied and should create a new state, unless all the methodCalls in the request end up failing. If the ifInState property is omitted then all the changes will be applied to the current state. h1. DoD Integration tests to show that the Mailbox/set method can handle ifInState property. > Mailbox/set should handle ifInState > --- > > Key: JAMES-3464 > URL: https://issues.apache.org/jira/browse/JAMES-3464 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > From spec: [https://jmap.io/spec-core.html#set] (section 5.3) > {code:java} > ifInState: This is a state string as returned by the Foo/get method > (representing the state of all objects of this type in the account). If > supplied, the string must match the current state; otherwise, the method will > be aborted and a stateMismatch error returned. If null, any changes will be > applied to the current state. > {code} > h1. How > When Email/set request is received, if the ifInState property is present we > need to compare it with the current Mailbox state stored in > MailboxChangeRepository. > * if mismatched, a stateMismatch error should be returned. > * if matched, all the changes in the request will be applied and should > create a new state, unless all the methodCalls in the request end up failing. > If the ifInState property is omitted then all the changes will be applied to > the current state. > h1. DoD > Integration tests to show that the Mailbox/set method can handle ifInState > property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3464) Mailbox/set should handle ifInState
Lan Khuat created JAMES-3464: Summary: Mailbox/set should handle ifInState Key: JAMES-3464 URL: https://issues.apache.org/jira/browse/JAMES-3464 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat >From spec: https://jmap.io/spec-core.html#set (section 5.3) {code:java} ifInState: This is a state string as returned by the Foo/get method (representing the state of all objects of this type in the account). If supplied, the string must match the current state; otherwise, the method will be aborted and a stateMismatch error returned. If null, any changes will be applied to the current state. {code} h1. How When Email/set request is received, if the ifInState property is present we need to compare it with the current Mailbox state stored in MailboxChangeRepository. - if mismatched, a stateMismatch error should be returned. - if matched, all the changes in the request will be applied and should create a new state, unless all the methodCalls in the request end up failing. If the ifInState property is omitted then all the changes will be applied to the current state. h1. DoD Integration tests to show that the Mailbox/set method can handle ifInState property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3463) Mailbox/get should handle state property
Lan Khuat created JAMES-3463: Summary: Mailbox/get should handle state property Key: JAMES-3463 URL: https://issues.apache.org/jira/browse/JAMES-3463 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat >From spec: https://jmap.io/spec-core.html#get (section 5.1) {code:java} State: A string representing the state on the server for all the data of this type in the account (not just the objects returned in this call). If the data changes, this string MUST change. If the data is unchanged, servers SHOULD return the same state string on subsequent requests. {code} h1. How Whenever a Mailbox/get request is received, we need to fetch the most recent state of Mailbox objects stored in the MailboxChangeRepository and return it in the response. h1. DoD Integration tests to show that Mailbox/get method can handle the state property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3462) Implement CassandraMailboxChangeStore
[ https://issues.apache.org/jira/browse/JAMES-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3462: - Description: Same purpose with the MemoryMailboxChangeStore, we now need to implement it in Cassandra. h1. How 1. Define a MailboxChange table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the CassandraMailboxChangeStore APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the MailboxChangeStore is working properly. was: Same purpose with the *MemoryMailboxChangeStore*, we now need to implement it in Cassandra. h1. How 1. Define a *MailboxChange* table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the *CassandraMailboxChangeStore* APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the *MailboxChangeStore* is working properly. > Implement CassandraMailboxChangeStore > - > > Key: JAMES-3462 > URL: https://issues.apache.org/jira/browse/JAMES-3462 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > Same purpose with the MemoryMailboxChangeStore, we now need to implement it > in Cassandra. > h1. How > 1. Define a MailboxChange table with these fields: > - accountId > - state > - created > - updated > - destroyed > - date > 2. Implement the CassandraMailboxChangeStore APIs, based on the contract that > has already been defined. > h1. DoD > Write unit tests to show that the MailboxChangeStore is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Updated] (JAMES-3462) Implement CassandraMailboxChangeStore
[ https://issues.apache.org/jira/browse/JAMES-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lan Khuat updated JAMES-3462: - Description: Same purpose with the *MemoryMailboxChangeStore*, we now need to implement it in Cassandra. h1. How 1. Define a *MailboxChange* table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the *CassandraMailboxChangeStore* APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the *MailboxChangeStore* is working properly. was: Same purpose with the `MemoryMailboxChangeStore`, we now need to implement it in Cassandra. h1. How 1. Define a `MailboxChange` table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the `CassandraMailboxChangeStore` APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the `MailboxChangeStore` is working properly. > Implement CassandraMailboxChangeStore > - > > Key: JAMES-3462 > URL: https://issues.apache.org/jira/browse/JAMES-3462 > Project: James Server > Issue Type: Sub-task >Reporter: Lan Khuat >Priority: Major > > Same purpose with the *MemoryMailboxChangeStore*, we now need to implement it > in Cassandra. > h1. How > 1. Define a *MailboxChange* table with these fields: > - accountId > - state > - created > - updated > - destroyed > - date > 2. Implement the *CassandraMailboxChangeStore* APIs, based on the contract > that has already been defined. > h1. DoD > Write unit tests to show that the *MailboxChangeStore* is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3462) Implement CassandraMailboxChangeStore
Lan Khuat created JAMES-3462: Summary: Implement CassandraMailboxChangeStore Key: JAMES-3462 URL: https://issues.apache.org/jira/browse/JAMES-3462 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat Same purpose with the `MemoryMailboxChangeStore`, we now need to implement it in Cassandra. h1. How 1. Define a `MailboxChange` table with these fields: - accountId - state - created - updated - destroyed - date 2. Implement the `CassandraMailboxChangeStore` APIs, based on the contract that has already been defined. h1. DoD Write unit tests to show that the `MailboxChangeStore` is working properly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3461) Implement Mailbox/changes method and related contract tests
Lan Khuat created JAMES-3461: Summary: Implement Mailbox/changes method and related contract tests Key: JAMES-3461 URL: https://issues.apache.org/jira/browse/JAMES-3461 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat >From the spec: [https://jmap.io/spec-core.html#changes] {code:java} The Foo/changes method allows a client to efficiently update the state of its Foo cache to match the new state on the server. {code} h1. How 1. Write a serializer to deserialize/serialize Mailbox/changes request/response. 2. Implement Mailbox/changes method + tests. h1. Example *Request* {code:java} [[ "Mailbox/changes", { "accountId": "29883977c13473ae7cb7678ef767cbfbaffc8a44a6e463d971d23a65c1dc4af6", "sinceState": "01" }, "t0" ]] {code} *Response* {code:java} [[ "Mailbox/changes", { "accountId": "29883977c13473ae7cb7678ef767cbfbaffc8a44a6e463d971d23a65c1dc4af6", "oldState": "01", "newState": "02", "hasMoreChanges": false, "created": [ "1", "2" ], "updated": [], "destroyed": [] }, "t0" ]] {code} h1. DoD Write integration tests to show that we can retrieve the changes to mailbox(es) from a particular state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Created] (JAMES-3460) Implement a JMAP MailboxChangeListener
Lan Khuat created JAMES-3460: Summary: Implement a JMAP MailboxChangeListener Key: JAMES-3460 URL: https://issues.apache.org/jira/browse/JAMES-3460 Project: James Server Issue Type: Sub-task Reporter: Lan Khuat We need to implement a listener in order to track the changes made to objects whenever a JMAP operation complete successfully (create/update/destroy). h1. How * Define a MailboxChangesListener that implements MailboxListener. * This component will handle MailboxAdded, MailboxDeletion, MailboxRenamed, MailboxACLUpdated events, storing them as MailboxChange(s) in the MailboxChangeRepository. h1. DoD Integration tests to show that the listener is able to record all changes accurately in the MailboxChangesRepository. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org