Re: svn commit: r48925 - in /release/james: hupa/0.0.2/ jdkim/ jsieve/0.7/ jspf/1.0.1/ mailets/3.6.0/ mime4j/0.8.5/ mpt/0.1/ server/ server/3.6.0/

2021-07-21 Thread btell...@linagora.com 
Hello all,

Just a word about this,

The INFRA requests us to armor our signature on the download page,

I received the following message:

|> Sorry, but the download page is still wrong. |||> | |||> |The links for the 
hupa artifacts are missing the /hupa/ subdirectory |||> |segment. > || 
> ||Also, I've just noticed that all the .ASC files are binary. > 
||They are supposed to be ascii-armoured. > ||As it stands, the wrong 
content-type will be applied which may
cause issues > ||with downloads. > ||Please fix for future releases, 
and if necessary for current
releases. > ||Please check what needs to be done with INFRA ASAP.> || 
> ||Sebb|

I did the update.

See also this PR to put ourselves in conformity.

This prevents us from sending anything to Apache announce mailing list.

No downloadable artifact had been changed. Only signatures.

Cheers,

Benoit

On 21/07/2021 17:58, btell...@apache.org wrote:
> Author: btellier
> Date: Wed Jul 21 10:58:42 2021
> New Revision: 48925
>
> Log:
> Follow ASF policy regarding artifact signature: use ascii-armoured instead of 
> binaries
>
> Modified:
> release/james/hupa/0.0.2/hupa-0.0.2.war.asc
> release/james/hupa/0.0.2/hupa-parent-0.0.2-source-release.zip.asc
> release/james/jdkim/apache-jdkim-0.2-bin.tar.gz.asc
> release/james/jdkim/apache-jdkim-0.2-bin.zip.asc
> release/james/jdkim/apache-jdkim-project-0.2-source-release.zip.asc
> release/james/jsieve/0.7/apache-jsieve-0.7-all.tar.gz.asc
> release/james/jsieve/0.7/apache-jsieve-core-0.7.jar.asc
> release/james/jsieve/0.7/apache-jsieve-sources-0.7.zip.asc
> release/james/jspf/1.0.1/apache-jspf-1.0.1-bin.zip.asc
> release/james/jspf/1.0.1/apache-jspf-sources-1.0.1.zip.asc
> release/james/mailets/3.6.0/apache-mailet-api-3.6.0-sources.jar.asc
> release/james/mailets/3.6.0/apache-mailet-api-3.6.0.jar.asc
> release/james/mailets/3.6.0/apache-mailet-base-3.6.0-sources.jar.asc
> release/james/mailets/3.6.0/apache-mailet-base-3.6.0.jar.asc
> release/james/mailets/3.6.0/apache-mailet-crypto-3.6.0-sources.jar.asc
> release/james/mailets/3.6.0/apache-mailet-crypto-3.6.0.jar.asc
> release/james/mailets/3.6.0/apache-mailet-standard-3.6.0-sources.jar.asc
> release/james/mailets/3.6.0/apache-mailet-standard-3.6.0.jar.asc
> release/james/mime4j/0.8.5/apache-mime4j-0.8.5-bin.tar.gz.asc
> release/james/mime4j/0.8.5/apache-mime4j-0.8.5-bin.zip.asc
> release/james/mime4j/0.8.5/apache-mime4j-core-0.8.5.jar.asc
> release/james/mime4j/0.8.5/apache-mime4j-dom-0.8.5.jar.asc
> release/james/mime4j/0.8.5/james-mime4j-sources-0.8.5.zip.asc
> release/james/mpt/0.1/apache-james-mpt-0.1-src.tar.gz.asc
> release/james/mpt/0.1/apache-james-mpt-0.1-src.zip.asc
> release/james/server/3.6.0/james-project-3.6.0-source-release.zip.asc
> release/james/server/3.6.0/james-server-app-3.6.0-app.zip.asc
> release/james/server/james-2.3.2.1-src.tar.gz.asc
> release/james/server/james-2.3.2.1-src.zip.asc
> release/james/server/james-binary-2.3.2.1.tar.gz.asc
> release/james/server/james-binary-2.3.2.1.zip.asc
>
> Modified: release/james/hupa/0.0.2/hupa-0.0.2.war.asc
> ==
> Binary files - no diff available.
>
> Modified: release/james/hupa/0.0.2/hupa-parent-0.0.2-source-release.zip.asc
> ==
> Binary files - no diff available.
>
> Modified: release/james/jdkim/apache-jdkim-0.2-bin.tar.gz.asc
> ==
> --- release/james/jdkim/apache-jdkim-0.2-bin.tar.gz.asc (original)
> +++ release/james/jdkim/apache-jdkim-0.2-bin.tar.gz.asc Wed Jul 21 10:58:42 
> 2021
> @@ -1,7 +1,11 @@
>  -BEGIN PGP SIGNATURE-
> -Version: GnuPG/MacGPG2 v2.0.11 (Darwin)
>  
> -iEYEABECAAYFAk4tph0ACgkQBWrKdNRgAL+8dwCfVJjLjVAM8AZg5X/ahKaEzEM4
> -cy4An2SAMph1a4ItQ/NYNSOfpNxLCN2h
> -=4pPC
> +iQEyBAABCAAdFiEEFnhshRITEEQ+vk9LDhxDw/s1ts4FAmD3+18ACgkQDhxDw/s1
> +ts7/9Qf4k4ERCZpPZ00zkzNOkTR52awf7UbxDElStTQZR14s9N3tqjrUROVDtTHt
> +Fk5/bqOp4LHRdEwSvU8Xni34VuYYUviUzey4zJe0YVDNWqN8nbntsxsD+TopAqo5
> +/kvkGKXDwsBPVK0QRfhTjmXroA/RykgouuTVsR7K0pkgCqaLrLrzpIzpXSVGaPQ+
> +nPXoSt2vmoglFuCxd4M+9Sqm8AahIbcddptycc5P1pQOES9be0AGhJEpUSYblO9O
> +YqOjHAwuyFnx1pJK3++N5oP7PSZYkClVLG8dNbWmkrlvv4wwN7NGdFhlqL95m5ey
> +26lVgM/XkZNsOCkQ7KmnX1OmzcqF
> +=wsvH
>  -END PGP SIGNATURE-
>
> Modified: release/james/jdkim/apache-jdkim-0.2-bin.zip.asc
> ==
> --- release/james/jdkim/apache-jdkim-0.2-bin.zip.asc (original)
> +++ release/james/jdkim/apache-jdkim-0.2-bin.zip.asc Wed Jul 21 10:58:42 2021
> @@ -1,7 +1,11 @@
>  -BEGIN PGP SIGNATURE-
> -Version: GnuPG/MacGPG2 v2.0.11 (Darwin)
>  
> -iEYEABECAAYFAk4tph0ACgkQBWrKdNRgAL/5bwCaAscqpCVk2SM8NZ25VeuzBZYV
> -R

Re: Implement ThreadIdGuessingAlgorithm for the distributed module

2021-07-21 Thread btell...@linagora.com 


On 21/07/2021 18:06, Quan tran hong wrote:
> Hi Benoit,
> Thanks for your great reviews.
>
>> Or we can just do several selects, one for each mimeMessageIds.I
> personaly likely prefer this option.
>
> This will make the upper logic layer more complicated. But I agree with
> your suggestion cause I would prefer balance and performance also.
>
>> Remember to do the delete of ThreadTable before threadtable_lookup,
> otherwise if you delete the pointer before the data you might end up in
> a case where the actual data is never deleted.
>
> Sure. Thanks for your reminder.
>
>> When/How do we call it?
> StoringThreadIdGuessingAlgorithm maybe?
Not really, this would only be called when appending messages.

Likely this would need to be hooked in DeleteMessageListener.
>
> Cheers,
> Quan
>
> Vào Th 4, 21 thg 7, 2021 vào lúc 17:03 btell...@apache.org <
> btell...@apache.org> đã viết:
>
>> Hello quan,
>>
>> Nice proposal! I think the Cassandra data model you propose is evolved
>> enough so that we start implementing it.
>>
>> Some comments inlined...
>>
>> On 21/07/2021 15:55, Quan tran hong wrote:
>>> Hi Benoit,
>>> I did have another try on this. Please have a look.
>>> CREATE threadtable
>>>
>>> CREATE TABLE ThreadTable (messageId timeuuid, threadId timeuuid, username
>>> text, mimeMessageId text, baseSubject text, PRIMARY KEY((username,
>>> mimemessageid), messageid));
>>>
>>> => Partition key: (username, mimemessageid), clustering key: messageid.
>>>
>>> [...]
>>>
>>> SELECT basesubject, threadId FROM threadtable WHERE username = 'quan'
>> AND
>>> mimeMessageId IN ('MimeMessageID2', 'MimeMessageID3');
>> This looks way better to me.
>>
>> IN usage is PRIMARY KEY is discouraged as it leads to coordination
>> across partitions.
>>
>> Read more for instance in
>>
>> https://stackoverflow.com/questions/55604857/cassandra-query-performance-using-in-clause-for-one-portion-of-the-composite-pa
>>
>> Either we should move "mimeMessageId" to the clustering key (but all the
>> messages of a user, including their subject would end up in a single
>> partition, which could be quite large... for instance 1 million messages
>> x size of the mime message ids and subject could be too much, as
>> partitions are recommended to be 100MBs at most).
>>
>> Or we can just do several selects, one for each mimeMessageIds.I
>> personaly likely prefer this option.
>>> [...]
>>>
>>> CREATE threadtable_lookup for deletion purpose
>>>
>>> Supposed when we delete a message, we would need to delete that message’s
>>> thread-related data in the threadtable also. I guess we just need
>> messageId
>>> to delete that message.
>>>
>>> We would need to have another table similar to the threadtable but looked
>>> up by messageId to get the needed params for deletion query.
>>>
>>> CREATE TABLE ThreadTable_lookup (messageId timeuuid, username text,
>>> mimeMessageIds set, PRIMARY KEY(messageid));
>> The set should be frozen. We will never add or remove data in it, so we
>> do not need a CRDT (Commutative Replicated Data Type).
>>
>> Forbidding adding - removing individual elements avoids lots of issues,
>> and lead to a more compact storage.
>>
>>> To ease testing, I create a table with messageId’s type is int instead.
>>>
>>> We will insert the data as same as the original table.
>>>
>>> insert into ThreadTable_lookup (messageId, username, mimeMessageIds)
>>> values (1, 'quan', {'MimeMessageID1', 'MimeMessageID2',
>> 'MimeMessageID3'});
>>> [...]
>>>
>>>
>>> SELECT * FROM threadtable_lookup;
>>>
>>>
>>> Now we will do a query selection by messageId to get the needed username,
>>> mimeMessageIds for original threadtable deletion.
>>>
>>> Supposed we want to delete message 4.
>>>
>>> SELECT username, mimemessageids FROM threadtable_lookup WHERE messageid
>> = 4;
>>>
>>> Then we will use these results to do a deletion query on threadtable.
>>>
>> Remember to do the delete of ThreadTable before threadtable_lookup,
>> otherwise if you delete the pointer before the data you might end up in
>> a case where the actual data is never deleted.
>>
>> The algorithm that you propose looks good.
>>
>> When/How do we call it?
>>
>>
>> Cheers,
>>
>> Benoit
>>> The data of messageId 4 deleted.
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Quan
>>>
>>>
>>> Vào Th 3, 20 thg 7, 2021 vào lúc 18:39 btell...@apache.org <
>>> btell...@apache.org> đã viết:
>>>
 Hello Quan,

 On 20/07/2021 17:24, Quan tran hong wrote:
> [...]
>
> SELECT threadId FROM threadtable WHERE username = 'quan' AND
>> baseSubject
 =
> 'baseSubject1' AND mimeMessageId IN ('MimeMessageID2',
>> 'MimeMessageID3')
> LIMIT 1 ALLOW FILTERING;
 ALLOW FILTERING should not be used as it will result in a full scan and
 is thus a performance disaster.

 If you need it, this means you do not have the right table structure and
 likely should rework the CREATE TABLE statement.
> => This new message should have this threadId.
> New unrelated message

Re: [VOTE] Retire Apache James HUPA

2021-07-23 Thread btell...@linagora.com 
+1

On 23/07/2021 16:00, btell...@apache.org wrote:
> Hello all,
>
> Following a first email on the topic [1] I would like to call for a
> formal vote on Apache James Hupa retirement.
>
> [1] https://www.mail-archive.com/server-dev@james.apache.org/msg70575.html
>
> Rationnals:
>  - The latest release (0.3.0) dates from 2012 which is an eternity in
> computing.
>  - The latest tag on Github is 0.0.3
>  - The pom references 0.0.5-SNAPSHOT suggesting that 0.0.4 release is
> lost :-(
>  - This repository is crippled by multiple CVEs (quick dependabot review):
>   - CVE-2021-29425 (commons-io)
>       - GHSA-m6cp-vxjx-65j6 CVE-2017-7656 CVE-2015-2080 CVE-2017-7657
> CVE-2019-10241 CVE-2019-10247 (Jetty server)
>   - CVE-2020-9447 (gwtupload)
>       - GHSA-g3wg-6mcf-8jj6 (jetty-webapp)
>   - CVE-2019-17571 (log4j)
>   - CVE-2016-131 CVE-2016-3092 (commons-fileupload)
>  - Sporadic activity since 2012
>  - Zero to no exchanges for several years on the mailing lists.
>
> Given that alternatives exists, given that the project is
> likely not mature, unmaintained and unsecure, I propose to retire this
> Apache James subproject.
>
> |Voting rules: - This is a majority vote as stated in [2] for procedural
> issues. - The vote starts at Friday 23rd of July 2021, 4pm UTC+7 - The
> vote ends at Friday 30th of July 2021, 4pm UTC+7 [2]
> https://www.apache.org/foundation/voting.html Following this retirement,
> follow up steps are to be taken as described in [3] [3]
> https://www.mail-archive.com/server-dev@james.apache.org/msg70585.html | - 1. 
> Get a formal vote on server-dev mailing list
>  - 2. Place a RETIRED_PROJECT file marker in the git
>  - 3. Add a note in the project README
>  - 4. Retire the ISSUE trackers (Project names HUPA and POSTAGE)
>  - 5. Announce it on gene...@james.apache.org and announce@apache
>  - 6. Add a notice to the Apache website, if present
>  - 7. Remove releases from downloads.apache.org
>  - 8. Add notices on the Apache release archives (example
> https://archive.apache.org/dist/ant/antidote/ 
> )
>
> Best regards,
>
> Benoit Tellier
> ||
>
>
> -
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>
>

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Akquinet contributions to James

2021-11-02 Thread btell...@linagora.com 
Hello Tung,

Thanks for the feedback.

On 03/11/2021 11:24, Tung Tran Van wrote:
> Hello Otto,
> Very interesting,
>
>>  Delay on authentication failure (S)
> I will follow how it will be implemented. Right now, I think about Redis
> with expire time key-value, with key is the fingerprint of the client.
>
I think what Otto proposes is "if the authentication fails then just
Thead.sleep(1000)" which is a simple business rule mitigating
disctionary attacks using James as an Oracle.

To implement it you do not need any kind of synchronisations. So it is
easy to implement.

Of course this is not as efficient as (say) fail2ban [1] (tool scrapping
the logs and shadowing ips at the firewall level in case of too many bad
logins...) but of course such a set up would need to be distributed (and
things would start being complicated with some state synchronization eg
through REDIS). Fail2ban stuff is inefficient against a bot farm where
the pool of ips used allows to bypass the checks

[1] https://www.fail2ban.org/wiki/index.php/Main_Page

My take is that we could implement the easy thing first :-)
>> Check user credentials via WebAdmin (M)
> What is the key difference between webAdmin endpoint and /jmap/session
> endpoint?

1. Not every admin deploys JMAP

2. Discoverability: given that I'm an admin I am likely familiar with
WebAdmin /users endpoints familly. To think of using JMAP session (basic
auth) to check a password is complex, non straightforward. Smells like a
roundabout way to do things ;-)

>
> Regards,
> Tung, Tran Van
>
> On Tue, Nov 2, 2021 at 11:42 PM Otto, Karsten Andreas
>  wrote:
>
>> Dear James Community,
>>
>> Over the last two years, we at Akquinet have developed an email solution
>> for the medical and healthcare sector. We chose Apache James for this
>> project because it provides
>> - a robust clustering solution out of the box,
>> - a comprehensive WebAdmin REST interface for integration with other
>> product components,
>> - a flexible Mailet architecture easily adapting to our specific
>> requirements,
>> - an open source solution with an active community and commercial
>> support where needed.
>>
>> Our deployment uses the distributed-pop3 app variant, with multiple
>> James server instances running on top of a Cassandra cluster,
>> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
>> strange, but our customers typically employ third party systems for
>> semi-automatic mail processing. For this use case, POP3 enables much
>> simpler systems integration than the more complex protocols.
>>
>> Despite Apache James being a very flexible solution, we encountered a
>> few situations during development where we had to change the original
>> codebase in order to meet our requirements. We believe these changes
>> might well be of interest to the James community at large, and in the
>> spirit of open source we would like to share them with you!
>>
>> Over the next few weeks we plan to create Jira tickes and pull requests
>> for the features below. Let us know what you think, and which you would
>> like to see first!
>>
>> (The list includes a complexity estimate in T-Shirt size, where S is
>> just a few / localized changes and L is a lot of code / all over the
>> place.)
>>
>>
>> SECURITY ENHANCEMENTS
>>
>> # TLS authentication via client certificate (M)
>>
>> Add options to network server configurations to set certificate-based
>> authentication mode (none, want, need), and the associated trust store
>> to validate these client certificates. Useful to limit server access to
>> trusted partners/users.
>>
>> # Separate trust store for S3 (M)
>>
>> Extend blob store configuration to specify a trust store, which is used
>> to validate the S3 server certificate. Useful if also using TLS client
>> cert authentication (see above) to keep the security realms separate.
>>
>> # Delay on authentication failure (S)
>>
>> Add an option verifyFailureDelay to usersrepository.xml to delay the
>> response if someone tries to authenticate with a non-existing user or
>> wrong password. Basic protection against people using James as a
>> password oracle for brute-force/dictionary attacks.
>>
>> # Support password salting (M) *
>>
>> Add extra hashingMode choices in usersrepository.xml ("salted",
>> "legacy_salted") to include the user name in the password hash. Basic
>> protection against rainbow table cracking if someone ever manages to
>> steal the password database.
>>
>> # Gradual migration of password hash settings (L) *
>>
>> Add a hashingMode column to the user table, use it together with the
>> algorithm column to verify password hashes. But use the configured
>> algorithm and hashingMode from usersrepository.xml when updating the
>> password. This way, the user database can gradually migrate to a
>> different (hopefully stronger) security setting. Useful to get rid of
>> the legacy hashing mode, and to introduce password salting (see above).
>>
>> *NOTE: Currently only works with Cassandr

Re: Authentication in James: Enable/require/disable OpenID for JMAP, IMAP and SMTP

2021-12-01 Thread btell...@linagora.com 
Hopefully rendered correctly in plain text...

-

Hello Jamers!

As part of my work at Linagora, we are looking toward
 - better integrating James within our product suite and for us this
means support "Single Sign On" and "Single Log Out" for JMAP following
the OpenId connect standard [0].
 - Improving security standards used to opperate James. (We have a
growing activity within the health care market, sensible to security
arguments)

Regarding security standards we should ideally:
 - Be able to NOT advertise AUTH / LOGIN capabilities of unencrypted
channels (Correspond to IMAP plainAuthDissallowed but generalized to
other protocols)
 - Be able to require OAUTH/OIDC authentication for all protocols (IMAP,
SMTP, JMAP) - this is what some of the health care providers we spoke to
desired to do.
 - For our collaborative suite usage, while OIDC only for JMAP makes
sense, we still wish to maintain PLAIN AUTH for IMAP and SMTP.
 - Of course settings for regular users not interested in OIDC should
not change (no breaking changes, OIDC adoption is opt-in only).

As such we propose ourselves to:
 - Contribute IMAP and SMTP SASL OAUTH extension described in [RFC-7628]
 - Modularize JMAP authentication mechanisms (letting the admin choose
which one she wishes to use)
 - Enable authentication through a header mechanism eg `X-USER:
btell...@apache.org`, which can be used to delegate OIDC authentication
through a third party API gateway. We have Krackend [1] in mind.
 - Share documentation and a docker-compose of OIDC setup for IMAP, SMTP
and JMAP in
https://github.com/apache/james-project/tree/master/examples/oidc. This
would include:
    - A LDAP still used by James UsersRepository. Provisionned with a
testing user.
    - A pre-configured Keycloack [2] OpenID connect provider.
    - A pre-configured Krakend API gateway proxying JMAP
    - And finally a James server configured to only accept OIDC as an
authentication mechanism for IMAP, SMTP and JMAP.
 - Unit tests for existing IMAP `plainAuthDissallowed` configuration
parameter.
   
Finally this is a good opportunity to restructure authentication related
settings in imapserver.xml and smtpserver.xml file.

Here are proposals for both files:

[imapserver.xml]

    
    imapserver-ssl
    0.0.0.0:993
    
    file://conf/private.key
    file://conf/certs.self-signed.csr
    
    
    
    true|false 
     
    file://conf/imapJWT.pub
   
https://example.com/.well-known/openid-config
    mailAddress
    
    
    


[smtpserver.xml]

    
    smtpserver-ssl
    0.0.0.0:465
    
    file://conf/private.key
    file://conf/certs.self-signed.csr
    
    
    true|false
    
    true|false 
     
    file://conf/imapJWT.pub
   
https://example.com/.well-known/openid-config
    mailAddress
    
    
    


You can see that:
 - The `plainAuthDissallowed` parameter is proposed to be renamed to
`auth.requireSSL`. (of course we should support fallback NOT to have a
breaking change)
 - `auth.plainAuthEnabled` enable turning on/off plain auth, which
allows having OIDC only mechanism.
 - `auth.requireSSL` will be generalized to SMTP.
 - In SMTP `requireAuth` setting is very misleading as it rather is
`requireAuthForRelay`. I propose we rename this configuration option (of
course we should support fallback NOT to have a breaking change).
 
Here is the implementation strategies we would follow:

 - For JMAP have Krakend doing all the hard job for us, and use a
dedicated header to carry the information over to James.
   - Our code contributions aims at easing such a setup (that would only
require configuration)
   - Provide an informative example using krakend. We understand that
this choice is ours, yet sharing it could allow reuse or similar setup
   - If some people wishes to write an OIDC authentication strategy
directly in James then they perfectly can! (Reusing the modularization
of JMAP authentication strategies we would provide).
  
 - For IMAP and SMTP then we proposes to check the bearer payload
against a statically configured public key for these protocols. (Sadly
there is no API gateway for those protocols)
   - Drawbacks includes no revocation of access tokens (once it's signed
it is always valid), revocation do not shut down existing connections
authenticated with the revocated token, and key rotation would require
reconfiguration and reboot.
   - One alternative would be to systematically ask the OpenID server to
validate the bearer. This might be acceptable as IMAP and SMTP are long
lived protocols that do not often establish new connections. While this
do not change anything regarding already opened connection management,
this solves revocation and public key rotation at the price of exposing
more the ide

Re: iPhone/Android mail apps with JAMES

2022-04-26 Thread btell...@linagora.com 

Hello Jerry,

I confirm that different email applications uses the IMAP protocol in 
different ways.


I confirm that some clients uses the IMAP SEARCH results upon 
resynchronisation, and thus that inacurate results could result in bad 
synchronisation.


That being said, having traffic capture for bith apps would be valuable 
to diagnose what is going on. Things like wireshark of James debug logs 
(that include IMAP command).


FYI I succeeded to reinde email on the JPA Guice based distribution on 
the 3.7.0 demo image.


Regards,

Benoit

On 4/26/22 11:41, Jerry Malcolm wrote:
This is a critical problem.  I really need some direction on this. 
Please!!!


I started this thread about two and half years ago when I moved my 
James installation to AWS EC2.  All of my clients lost all of their 
mail on their mobile devices, even though all of the mail still shows 
up fine on desktop Thunderbird.  I tried all of the suggestions, but 
never could get the older emails restored.  Since then I have added 
another very large customer account on a completely different james 
installation, and I've upgraded to pretty much the latest GitHub 
version of James on my original installation.  For the new client, 
there was no migration.  It was completely start from scratch on a new 
domain.  Yet across the board, NONE of my clients on either system can 
reliably get their mail on iPhone or Android.  On my own iPhone, I 
have replaced the native iPhone mail app with Edison mail app and 
later with Outlook mail app.  I get one or two emails downloaded 
periodically on each app on my various email accounts, and some email 
accounts just start saying there's no mail in the last month or two 
(and there's actually typically 10-20 emails each day on those 
accounts). Today, a client configured their mail account on an 
iPhone.  It immediately said 21 unread emails, and the inbox then 
downloaded 3 emails.  Open TBird on the same account.  There's there's 
the 21 emails.


I've tried to re-index the Lucene cache.  But I'm still getting the 
same error that I got in Oct 2019 about wrong parameters or something 
when I try to do that.  So I completely erased the Lucene cache 
folder.  No change. I'm running Spring.  I tried to move to Guice a 
month or so ago, but gave up when I couldn't fix the errors I was 
getting , so I moved back to Spring.  Is this whole iPhone problem due 
to me using the Spring build?  I have no problem trying again to get 
Guice up and running.  But I don't want to waste a week trying to get 
Guice up and find out the same problem exists in the Guice build.


It's obvious that for some reason all of the mobile email apps 
(native, Edison, Outlook) ask for email differently than Thunderbird.  
But I'm at a loss to explain why JAMES refuses to send the mail that 
is there when these same email client apps have no problem getting 
mail from other mail servers.


It's hard for me to understand how every other JAMES user in the world 
is working totally successfully with mobile phones when I have two 
completely independent JAMES environments with a huge number of 
clients on each and NOT ONE of them can get more than 5% of their real 
mail on their phone.


If somebody can just educate me just a little on the differences 
between how JAMES responds to IMAP queries on mobile devices vs. IMAP 
queries from Thunderbird, and point me to the handling code, I'll 
start seeing what I can do to resolve this.  Or better yet, is there 
someway to disable all of the Lucene or whatever caching completely 
and just make JAMES think it's talking to Thunderbird instead of an 
iPhone?


 I'm to the point that my major client is refusing to use the JAMES 
environment for their company mail accounts since their phone email 
apps are not receiving most of their critical corporate emails. To say 
the least, they are NOT happy.  Somebody PLEASE respond before my 
customer fires me.


Please HELP Give me SOMETHING I can work with  I just want to 
get a conversation going.


Thanks

Jerry


On 11/8/2019 8:03 AM, Matthieu Baechler wrote:

Hi Jerry,

On Tue, 2019-10-29 at 15:12 -0500, Jerry Malcolm wrote:

Ok, I need an IMAP expert Below is a very brief trace of the
communications between iPhone mail and JAMES (3.4).  I completely
deleted an account on my iPhone, then recreated it while in airplane
mode to make sure I didn't miss any communications in my trace.  I
started the trace, exited airplane mode and let the iPhone do an
initial
sync with the account.  The inbox folder in this account has over
1000
emails going back to early 2019.

I'm not an expert in IMAP.  But it appears that the iPhone mail app
requests all of the emails 1:* (see line 812), but JAMES returns a
single id plus two ranges (line 813).  But the total count JAMES
reports
is nowhere near the full 1000.  Subsequently (line 822), iPhone
requests
the emails JAMES told it about in line 813.  From what I can tell,
the
problem is in line 813.  JAMES d

Re: Call for vote: Apache James 3.7.5

2024-01-09 Thread btell...@linagora.com 

+1

On 09/01/2024 09:15, Benoit TELLIER wrote:

Hi,

I would like to propose a new vote for 3.7.5 release of the Apache 
James server.


You can find:

 - The maven release staged in repository.apache.org as the artifact 
#1079:

https://repository.apache.org/content/repositories/orgapachejames-1079/
 - The changelog for 3.7.5:
https://github.com/apache/james-project/blob/master/CHANGELOG.md

This release mostly ships bug fixes.

Voting rules:
 - This is a majority approval: this release may not be vetoed.
 - A quorum of 3 binding votes is required
 - The vote starts at Tuesday 9th of January 2024, 9am UTC+2
 - The vote ends at Tuesday 16th of January 2024, 9am UTC+2

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Cheers,

Benoit TELLIER


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Call for vote: Apache James 3.8.1

2024-01-09 Thread btell...@linagora.com 

+1

On 09/01/2024 09:17, Benoit TELLIER wrote:

Hi,

I would like to propose a new vote for 3.8.1 release of the Apache 
James server.


You can find:

 - The maven release staged in repository.apache.org as the artifact 
#1077:

https://repository.apache.org/content/repositories/orgapachejames-1077/
 - The changelog for 3.8.1:
https://github.com/apache/james-project/blob/master/CHANGELOG.md

This release mostly ships bug fixes.

Voting rules:
 - This is a majority approval: this release may not be vetoed.
 - A quorum of 3 binding votes is required
 - The vote starts at Tuesday 9th of January 2024, 9am UTC+2
 - The vote ends at Tuesday 16th of January 2024, 9am UTC+2

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Cheers,

Benoit TELLIER


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Call for vote: Apache James 3.8.1

2024-01-10 Thread btell...@linagora.com 

I just merged the changelog updates, this should be better now.

On 10/01/2024 11:26, Jean Helou wrote:

hmm the linked changelog doesn't list any unreleased changes is that normal
?

Le mar. 9 janv. 2024 à 09:17, Benoit TELLIER  a écrit :


Hi,

I would like to propose a new vote for 3.8.1 release of the Apache James
server.

You can find:

   - The maven release staged in repository.apache.org as the artifact
#1077:
https://repository.apache.org/content/repositories/orgapachejames-1077/
   - The changelog for 3.8.1:
https://github.com/apache/james-project/blob/master/CHANGELOG.md

This release mostly ships bug fixes.

Voting rules:
   - This is a majority approval: this release may not be vetoed.
   - A quorum of 3 binding votes is required
   - The vote starts at Tuesday 9th of January 2024, 9am UTC+2
   - The vote ends at Tuesday 16th of January 2024, 9am UTC+2

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Cheers,

Benoit TELLIER


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Call for vote: Apache James JSPF 1.0.4

2024-06-07 Thread btell...@linagora.com 

+1

On 07/06/2024 15:39, Benoit TELLIER wrote:

Hi,

I would like to propose a new vote for 1.0.4 release of the Apache 
James JSPF library.


You can find:

 - The maven release staged in repository.apache.org as the artifact 
#1083: 
https://repository.apache.org/content/repositories/orgapachejames-1083/
 - The blog post for this release: 
https://github.com/apache/james-project/pull/2286


This release fixes asynchronous JSPF executors.

Voting rules:
 - This is a majority approval: this release may not be vetoed.
 - A quorum of 3 binding votes is required
 - The vote starts at Friday 7th of June 2024, 3pm UTC+7
 - The vote ends at Friday 14th of June 2024, 3pm UTC+7

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Cheers,

Benoit TELLIER


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Call for vote: Apache James JSPF 1.0.5

2024-07-16 Thread btell...@linagora.com 

+1

On 16/07/2024 17:13, Benoit TELLIER wrote:

Hi,

I would like to propose a new vote for 1.0.5 release of the Apache 
James JSPF library.


You can find:

 - The maven release staged in repository.apache.org as the artifact 
#1084: 
https://repository.apache.org/content/repositories/orgapachejames-1084/
 - The blog post for this release: 
https://github.com/apache/james-project/pull/2352


This release fixes asynchronous JSPF executors error management 
(JSPF-111).


Voting rules:
 - This is a majority approval: this release may not be vetoed.
 - A quorum of 3 binding votes is required
 - The vote starts at Tuesday 16th of July 2024, 5pm UTC+7
 - The vote ends at Tuesday 23rd of July 2024, 5pm UTC+7

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Thanks to "aleksey" for the report.

Thanks
Cheers,

Benoit TELLIER


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: [VOTE] Call for vote: Apache James JSPF 1.0.5 #3

2024-08-22 Thread btell...@linagora.com 

+1

On 22/08/2024 09:28, Benoit TELLIER wrote:

Sorry I screw up the voting dates:

 - The vote starts at Thurdsay 22nd of August 2024, 9am UTC+2
 - The vote ends at Thurdsay 29th of August 2024, 9am UTC+2

Best regards,

Benoit

On 22/08/2024 09:26, Benoit TELLIER wrote:

Hi,

I would like to propose a new vote for 1.0.5 release of the Apache 
James JSPF library - third edition.


You can find:

 - The maven release staged in repository.apache.org as the artifact 
#1087: 
https://repository.apache.org/content/repositories/orgapachejames-1087/ 
 
 - The blog post for this release: 
https://github.com/apache/james-project/pull/2352


This release fixes asynchronous JSPF executors error management 
(JSPF-111) and upgrades dnsjava to a non vulnerable release.


Voting rules:
 - This is a majority approval: this release may not be vetoed.
 - A quorum of 3 binding votes is required
 - The vote starts at Friday 26th of July 2024, 2pm UTC+2
 - The vote ends at Friday 2nd of August 2024, 2pm UTC+2

You can answer to it just with +1 and -1. Down-votes may be motivated.

3 binding votes are expected move forward on this release.

Thanks to "aleksey" and "HABA" for the report. Thanks to all people 
involved in making this release happen.


Thanks
Cheers,

Benoit TELLIER



-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org




-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: James documentation: Roadmap

2024-08-28 Thread btell...@linagora.com 

Hello,

Thanks for bringing the roadmap refresher topic up!


  Should  we add as well a roadmap section in Antora doc for example?


Duplicated information is a recipe for partial updates and consistency 
issues.


(True with Cassandra, even more with humans)

So IMO -1.

On 28/08/2024 08:20, Jean Helou wrote:

hello !

suggestions inline



I would think regarding main goals that:

- the distributed mail server is already stable and might not need to
keep figuring in this section?


maybe switch from providing (done) to maintaining  like for JPA
IMO we already maintain all parts of Apache James including JPA and 
distributed server.


Is this relevant to explicitly state what we maintain?




- we are working on a postgres reactive based implementation of James
for smaller deployments as an alternative to JPA


that´s for incoming work isn't it ?

+1




Incoming work:

- is Antora doc still a point? Maybe some bits still missing before
really releasing it officially


It feels like the documentation situation is not so great atm. the maven
site still holds a lot of information but has quite a few dead links, the
antora also has information but also lots of empty placeholders. its
unclear to me where to invest my time and I end up not doing anything wrt
to documentation :/ if there are Epics/Parent Jira issues for the various
items could it be worth linking to them from the roadmap ?


+1

I would be in favor of eventually completing the Antora documentation 
and dropping the maven-site one.


> if there are Epics/Parent Jira issues for the various items could it 
be worth linking to them from the roadmap ?


Let's create it?



- Guice based applications: been promoted, just still need the removal

step on James 4.0.0


I think the deprecation notice should remain on the roadmap

+1




- anything else worth mentioning?


pulsar integration ? its slow but it's getting there (my SMTP server has
been running on the internet for a couple months now)

+1




Should we add as well a roadmap section in Antora doc for example?


Regards,

Rene.


-
To unsubscribe, e-mail:server-dev-unsubscr...@james.apache.org
For additional commands, e-mail:server-dev-h...@james.apache.org

Re: Jira :D

2024-08-30 Thread btell...@linagora.com 

Hello Jean!

Thanks a lot for the initiative.

I and Eugen had been doing that several time in the last few years, 
though I agree the JIRA backlog is still messy.


Answers inlined.

Best regards

On 30/08/2024 09:42, Jean Helou wrote:

Hello,

You might wonder what I'm doing with jira. I'm trying to get a handle on
- what's actively being worked on
- what's backlog material
- what's dead weight and clear that out

I have always found it hard to find issues in the james jira, even the
issues matthieu and I were actively working on in the james jira. I have
read the (completely off topic) discussion on jira as issue tracker versus
jira as project management tool in the dedicated pg backend jira
  and will not close
unresolved issues except for really outdated bugs  or bugs that are lacking
reproduction information.
To go about this, I created a kanban board



Thanks!

I had a lok and did triage a couple of tickets...


in the james server project, using the default configuration which creates
a fastlane for blocker level bugs.
This surfaced 6 bugs (4 of which have now been closed), 2 issues remain
that still seem relevant
-https://issues.apache.org/jira/browse/JAMES-3580


3.6.0 is old now, unsure it is still relevant on newer versions of james


-https://issues.apache.org/jira/browse/JAMES-3969

I then started reviewing the top issues in the everything else lane. This
led me to close very old bugs opened against very old versions since I
don't see anyone backporting fixes that far. There are still quite a few
left 😢

I also noticed some wish issues, I filtered them out of the board and
intend to setup a backlog board with only the wishes.
That leaves
- improvement requests such as
https://issues.apache.org/jira/browse/JAMES-1409  i have no idea if they are
actually relevant at the moment.
This one likely is: this field takes a coma separated list of address, 
stored in a string field of size 100...


A good example of limitations that make me claim JPA implementation 
isn't great ;-)


(thanks for pointing this out!)

And clearly it is not an improvment but clearly a *bug*!

- bugs that sound more like wishes
https://issues.apache.org/jira/browse/JAMES-1405


log4j specific and old. I would disregard it.

Agreed it is a wish ...


- new features that sound interesting and include years old patches that
probably wouldn't apply anymore
https://issues.apache.org/jira/browse/JAMES-1357

I'll continue to review the ~300 issues that are left open on my own time.
Don't hesitate to tell me if I'm overstepping here and I'll reopen the
issues I closed.


+1 you are doing a great job, I'll try to keep up with it!



jean

Re: JMS & File mail queue still rely on serializable.

2020-12-03 Thread btell...@linagora.com (OpenPaaS) 
Nice to see we are on the same wave length!

Regarding the upgrade path, I'm in favor of requiring an empty mailQueue.

Rationals: if there is any sort of retro-compatibility, then an attacker
controlling ie the JMS solution would still be able to control the
deserialized payload (and trigger all sort of not funny stuff!)

To achieve that, downtime on the SMTP service + MailQueue flush.

Would it be resonable to people around here?

Cheers,

Benoit

Le 03/12/2020 à 14:58, Matthieu Baechler a écrit :
> Hi Benoit,
> 
> On Thu, 2020-12-03 at 11:54 +0700, Benoit Tellier wrote:
>> While working on https://issues.apache.org/jira/browse/JAMES-3431 I
>> discovered that JMS & File mailqueue do still rely on serialization.
>>
>> This is what motivates to re-open this ticket:
>> https://issues.apache.org/jira/browse/JAMES-2578
>>
>> Please kindly note that all of the MailRepository implementation no
>> longer uses Java serialization.
>>
>> Our AttributeValue adoption is partial; I would like to finish the
>> job.
>>
>> Here are the options we have:
>>
>>     Accept DSN feature do not work on tese implementations (not my
>> prefered at all!)
>>     Re-implement DSNParameters attribute mapping to not use
>> collection
>> attributeValues. This work around the main issue for this specific
>> use
>> case of attribute values. (I feel okay with that)
>>     Try to fix collection attributeValue java serialization is likely
>> hard to do, but also keeps java serialization around for longer in
>> the
>> code base. Likely a dead-end.
>>     No longer rely on Java serialization for "JMS" & "File" mail
>> queues.
>> This means either smart fallback code, or at worst an upgrade path
>> with
>> an empty mail queue. That is by far my preferred option, and I will
>> start community discussions in that direction.
>>
>> Do we got consensus around this topic?
> 
> This Java serialization compat has been crippling our developpements
> for too long. I'm in favor of complete removal and a migration
> strategy.
> 
> Cheers,
> 
> -- Matthieu
> 
> 
> -
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

About our usage of LWT in Cassandra related code

2020-12-03 Thread btell...@linagora.com (OpenPaaS) 
Hi,

I'm currently trying to increase overall efficiency of the Distributed
James server.

As such, I'm pocking around for improvement areas and found a huge topic
around LWT.

My conclusions so far are that we should keep LWT and SERIAL consistency
level out of the most common use cases.

I know that this is a massive change in regard of the way the project
had been working with Cassandra in the past few years. I would
definitely, in the middle term, would like to reach LWT free reads on
the Cassandra Mailbox to scale the deployments I am responsible of as
part of my Linagora job (my long term goal being to decrease the total
cost of ownership of a "Distributed James" based solution). While I am
not opposed to diverge from the Apache James project on this point, if
needed, I do believe an efficient distributed server (with the
consequences it implies in term of eventual consistency) might be a
strong asset for the Apache project as well, and would prefer to see
this work lending on the James project.

I've been ambitious on the ADR writing, especially in the complementary
work section. Let's see which consensual ground we find on that! (the ML
version here below serving as a public, immutable reference of my thinking!)

Cheers,

Benoit

---

## Context

As any kind of server James needs to provide some level of consistencies.

Strong consistency can be achieved with Cassandra by relying on
LightWeight transactions. This enables
optimistic transactions on a single partition key.

Under the hood, Cassandra relies on the PAXOS algorithm to achieve
consensus across replica allowing us
to achieve linearizable consistency at the entry level. To do so,
Cassandra tracks consensus in a system.paxos
table. This `system.paxos` table needs to be checked upon reads as well
in order to ensure the latest state of the ongoing
consensus is known. This can be achieved by using the SERIAL consistency
level.

Experiments on a distributed James cluster (4 James nodes, having 4 CPU
and 8 GB of RAM each, and a 3 node Cassandra
cluster of 32 GB of RAM, 8 CPUs, and SSD disks) demonstrated that the
system.paxos table was by far the most read
and compacted table (ratio 5).

The table triggering the most reads to the `system.paxos` table was the
`acl` table. Deactivating LWT on this table alone
(lightweight transactions & SERIAL consistency level) enabled an instant
80% throughput, latencies reductions
as well as softer degradations when load breaking point is exceeded.

## Decision

Rely on `event sourcing` to maintain a projection of ACLs that do not
rely on LWT or SERIAL consistency level.

Event sourcing is thus responsible of handling concurrency and race
conditions as well as governing denormalization
for ACLs. It can be used as a source of truth to re-build ACL projections.

Note that the ACL projection tables can end up being out of
synchronization from the aggregate but we still have a
non-questionable source of truth handled via event sourcing.

## Consequences

We expect a better load handling, better response time, and cheaper
operation costs for Distributed James while not
compromising the data safety of ACL operations.

ACL updates being a rare operation, we do not expect significant
degradation of write performance by relying on
`eventSourcing`.

We need to implement a corrective task to fix the ACL denormalization
projections. Applicative read repairs could be
implemented as well, offering both diagnostic and on-the-fly corrections
without admin actions (a low probability should
however be used as loading an event sourcing aggregate is not a cheap
thing).

## Complementary work

There are several other places where we rely on Lightweight transaction
in the Cassandra code base and
that we might want to challenge:

 - `users` we rely on LWT for throwing "AlreadyExist" exceptions. LWT
are likely unnecessary as the webadmin
presentation layer is offering an idempotent API (and silents the
AlreadyExist exceptions). Only the CLI
(soon to be deprecated for Guice products) makes this distinction.
Discussions have started on the topic and a proof of
concept is available.
 - `domains` we rely on LWT for throwing "AlreadyExist" exceptions. LWT
are likely unnecessary as the webadmin
presentation layer is offering an idempotent API (and silents the
AlreadyExist exceptions). Only the CLI
(soon to be deprecated for Guice products) makes this distinction.
Discussions have started on the topic and a proof of
concept is available.
 - `mailboxes` relies on LWT to enforce name unicity. We hit the same
pitfalls than for ACLs as this is a very often
 read table (however mailboxes of a given user being grouped together,
primary key read are more limited hence this is
 less critical). Similar results could be expected. Discussions on this
topic have not been started yet. Further
 impact studies on performance needs to be conducted.
 - `messages` as flags update is so far transa

Re: Jenkins CI setup

2020-12-03 Thread btell...@linagora.com (OpenPaaS) 


Le 04/12/2020 à 03:21, Jean Helou a écrit :
> Hello fellow jamers !
>
> The Jenkinsfile in the PR works, up until the test suite fails, the tests
> failures are from seemingly "unstable" tests that fail because of timing
> issues. Benoit fixed the first one in
> https://github.com/apache/james-project/pull/267 by disabling read repairs
> during consistency checks (I have no idea what it means but it sounds
> awesome :) ), I fixed the second one in
> https://github.com/apache/james-project/pull/269 where the event bus sender
> and receivers where closed out of order on shutdown sometimes leading up to
> events being sent to a closed receiver.
>
> After some cleanup, Matthieu recreated a buildable PR which lead to yet
> another unstable test in
> https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/1/tests
We have been encoutering this for a while. Thanks for digging in.
>
> I started investigating the issue and ended up roping in Matthieu since the
> symptoms for the issue left me completely puzzled. Matthieu managed to
> pinpoint the root cause to a NPE sometimes thrown from
> within org.apache.james.server.core.MimeMessageCopyOnWriteProxy which in
> turn triggered further NullPointerExceptions in the mailet pipeline error
> handling code.
> We finally confirmed a concurrency issue in the refcounting management of
> the proxy which if I understand correctly can lead to unrecoverable data
> loss. We wrote a test to trigger it [1] in an almost deterministic manner.
I'm in favor for opening a dedicated ticket and merge a disabled version
of this test in order to document the problem.
>
> Once we had a test to reproduce the race condition, we tried to fix the
> issue only to realize that it led to even more concurrency issues. The
> rather depressing conclusion we reached yesterday was that the whole
> implementation is currently unsound with regard to concurrency. I am unable
> to estimate the resolution effort at this point, Matthieu has some ideas
> and will work on it (as well as I) when time allows.
>
> Which leads me to my current interrogations: I feel that fixing such long
> standing issues in the test suite is not actually part of configuring the
> apache CI but I am unsure how to proceed.
+1
>
> Here is what I would like to do at this stage :
> - Isolate the unstable tests under with an unstable tag (akin to "feature
> tags")
I'd advocate a @Disabled tag, referencing both a JIRA ticket specific to
the bugfix needed, and the JIRA of the CI build.

Having a list of such issues in the JIRA (CI setup) ticket would be
valuable. I'd even advise doing subtickets to have a nice checklist.
> - exclude these tests from the default surefire execution profile,
> - add a parallel pipeline step for these tests where the step failure
> doesn't fail the pipeline [2]
> - ensure that the build is green
> - merge so the project finally has a working public CI
>
> I intend to start working on this quickly so we can all enjoy a functional
> public CI.
+1 I agree on the approach.
>
> Alternatives:
> - Merge the jenkinsfile after the whole pipeline has been tested in the PR
> branch, which may not happen in a short-medium term...
> - Merging as is, means that many builds on PRs will end up failing and the
> last steps (snapshot publish) might fail even if the testsuite succeeds
> since it never ran.
> - Something I haven't thought of ?
>
> Another issue I want to raise is the availability of the CI builds. As you
> have seen from my experiments, the CI triggers configuration will only
> build commits from :
> - all branches of the main repository
> - all PRs opened from the main repository
> - all PRs opened by someone with write access to the main repository
>
> Which means that PRs for external contributors will not be built at all.
>
> I tried adding the  issueCommentTrigger to the jenkins file but neither my
> comments nor those of someone with commit access were able to trigger the
> build.
>
> I think that one of the project members should revise the current settings
> to make it possible to build external contributors PR one way or another.
> (only project members have access or can have access to the jenkins project
> configuration).
> Here are two options:
> - the easiest and quickest modification is to let the CI build all and
> every PR, there are relatively few PRs on james so the burden on the CI
> platform shouldn't be too bad.
> - alternatively it may be possible to configure jenkins to require a
> comment for someone with write access to trigger a build. unfortunately I
> am not certain how to set this up, maybe INFRA can help.
Having a build in the first place, even with the restrictions you
describe sounds like a good progress to me.

I agree we need to see what other Apache projects are doing, and if
needed ask the INFRA.
>
> I know this was a long piece, I look forward to reading your opinions !
Thanks for your involvement on this topic.

Benoit
> Jean
>
> [1] see
> h

Re: JMS & File mail queue still rely on serializable.

2020-12-07 Thread btell...@linagora.com (OpenPaaS) 
A quick follow up to state that
https://github.com/linagora/james-project/pull/4110 implemented that ;-)

Cheers,

Benoit

Le 03/12/2020 à 09:08, btell...@linagora.com (OpenPaaS) a écrit :
> Nice to see we are on the same wave length!
> 
> Regarding the upgrade path, I'm in favor of requiring an empty mailQueue.
> 
> Rationals: if there is any sort of retro-compatibility, then an attacker
> controlling ie the JMS solution would still be able to control the
> deserialized payload (and trigger all sort of not funny stuff!)
> 
> To achieve that, downtime on the SMTP service + MailQueue flush.
> 
> Would it be resonable to people around here?
> 
> Cheers,
> 
> Benoit
> 
> Le 03/12/2020 à 14:58, Matthieu Baechler a écrit :
>> Hi Benoit,
>>
>> On Thu, 2020-12-03 at 11:54 +0700, Benoit Tellier wrote:
>>> While working on https://issues.apache.org/jira/browse/JAMES-3431 I
>>> discovered that JMS & File mailqueue do still rely on serialization.
>>>
>>> This is what motivates to re-open this ticket:
>>> https://issues.apache.org/jira/browse/JAMES-2578
>>>
>>> Please kindly note that all of the MailRepository implementation no
>>> longer uses Java serialization.
>>>
>>> Our AttributeValue adoption is partial; I would like to finish the
>>> job.
>>>
>>> Here are the options we have:
>>>
>>>     Accept DSN feature do not work on tese implementations (not my
>>> prefered at all!)
>>>     Re-implement DSNParameters attribute mapping to not use
>>> collection
>>> attributeValues. This work around the main issue for this specific
>>> use
>>> case of attribute values. (I feel okay with that)
>>>     Try to fix collection attributeValue java serialization is likely
>>> hard to do, but also keeps java serialization around for longer in
>>> the
>>> code base. Likely a dead-end.
>>>     No longer rely on Java serialization for "JMS" & "File" mail
>>> queues.
>>> This means either smart fallback code, or at worst an upgrade path
>>> with
>>> an empty mail queue. That is by far my preferred option, and I will
>>> start community discussions in that direction.
>>>
>>> Do we got consensus around this topic?
>>
>> This Java serialization compat has been crippling our developpements
>> for too long. I'm in favor of complete removal and a migration
>> strategy.
>>
>> Cheers,
>>
>> -- Matthieu
>>
>>
>> -
>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-dev-h...@james.apache.org
>>
> 
> -
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org