Re: Setting "administratorid"?

2019-11-04 Thread Jerry Malcolm
Yes, I will definitely contribute to this.  A whole section on how to 
use delegation will be very helpful to other users.


On 11/4/2019 3:38 AM, Tellier Benoit wrote:

I just created an issue regarding this.

https://issues.apache.org/jira/browse/JAMES-2963

Jerry, would you have time to contribute this missing documentation?

Regards,

Benoit

On 04/11/2019 16:27, Tellier Benoit wrote:

Answers inlined,

Regards,

Benoit

On 30/10/2019 11:37, Jerry Malcolm wrote:

"enableVirtualHosting" is another config parameter that's set in this
same method.  So I decided to search around and see where that is set.
I found it, and just taking a wild guess, is this correct?

Maybe we need to further explain this.

When virtual hosting is enabled, usernames are composed of a local part
and a domain part. al...@domain.tld and al...@company.org are two
distinct users, with different login and distinct mailboxes. Creating a
user without domain part is forbidden.


When virtual hosting is turned of, usernames are only composed of a
local part. All domains handled by James (check domain list) can be used
as domain part of their mail address. al...@domain.tld and
al...@company.org are two mail address belonging to a same "alice" user,
with same login and mailboxes. Creating users with domain parts is
forbidden.

I will add the following indication within the documentation as it might
get really useful to understand this. Thanks for highlighting this weak
point!



     MD5

Try more secure algorithms, you should give a go to SHA-512.

We should update default configuration accordingly as MD5 hashing is not
acceptable.


     true
ad...@myhost.com


This took a whole lot of digging to figure out.  I really think the
default usersrepository.xml should have at least commented-out lines
defining where/how to set the administrator id.

I would be glad to have you contribute this!


I'll add it to my list if this is indeed the correct implementation.

Jerry


On 10/29/2019 11:27 PM, Jerry Malcolm wrote:

I see in AbstractUsersRepository where "adminstratorid" is pulled from
somewhere in the configuration:

     administratorId =
Optional.ofNullable(configuration.getString("administratorId"));

But I can't find any place to set it in the shipped configuration
files.  Where is the administrator id supposed to be defined, and what
is the correct syntax to define it?

Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: AdministratorId Documentation

2019-11-04 Thread Jerry Malcolm
This is a followup to an earlier discussion.  I have a very large number 
of imap utilities that I use to maintain my clients' accounts and 
mailboxes.  They were all written using javax.mail's version of the IMAP 
interface.   Your code reference below was very helpful.  But I realized 
I would have to change all of my javax.mail code to commons.net.  It was 
going to be worth it if I could get the delegate function.  But I still 
was dreading the amount of migration/testing work.


So I did a bit of playing around looking at what was really 
happening to send the admin id, admin pw, and target account in the 
login.  Turns out in the javax.mail package, if you use:


      store.connect(host, port, user + "\0" + adminId, adminPW);

it logs in correctly with delegation.  So as much as I appreciate your 
code :-),  a one-line change to my javax.mail login code won out over 
almost a full rewrite of all my utilities.


Jerry


On 9/12/2019 11:01 PM, Tellier Benoit wrote:

If you are using commons-net, I succeeded to write:

https://gist.github.com/chibenwa/abd12fd6c0b06cadd1de591e3ac792b9

That should be helping you!

On 13/09/2019 10:40, Tellier Benoit wrote:

Wich library are you using?

On 13/09/2019 10:33, Jerry Malcolm wrote:

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I
was looking for how to change the following code to include an
administrator id.  I only have one field in the store.connect() method
for a user id, but I have an administrator id and the userid for the
target mailbox.  How do I pass in both?

    try
    {
   session = Session.getInstance(props, null);
   session.setDebug(debug);
   store = session.getStore(protocol);
   store.connect(host, port, user, password);
    }
    catch( Exception e)
    {
   log( "IMAPClient.connect() exception: " + e );
   throw(e);
    }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related "reading
config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it straight
away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test


Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My environment is
based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:

Since it appears I'm not going to be able to build James 3.3.0 in the
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've
lost count.

My goal now is to assess my possibilities of using James 3.3.0 binaries
as-is and discarding the functionality I had hoped to re-add to 3.3.0
from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  From
what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of the
master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the case, then
that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define it?  And
how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: 

Re: Setting "administratorid"?

2019-11-04 Thread Jerry Malcolm

Hi Benoit,

I wasn't actually questioning enableVirtualHosting.  I was simply trying 
to figure out where to put the admin id.  I noticed that the adminId was 
pulled from the configuration in the next line after the 
enableVirtualHosting flag was pulled.  So I was just deducing that 
perhaps they should be side by side in the same config file.  I tried 
that, and it worked.


My only recommendation was that we add a 'dummy' commented-only 
 line to the default config file, so others can easily 
find where to set it.


Jerry

On 11/4/2019 3:27 AM, Tellier Benoit wrote:

Answers inlined,

Regards,

Benoit

On 30/10/2019 11:37, Jerry Malcolm wrote:

"enableVirtualHosting" is another config parameter that's set in this
same method.  So I decided to search around and see where that is set.
I found it, and just taking a wild guess, is this correct?

Maybe we need to further explain this.

When virtual hosting is enabled, usernames are composed of a local part
and a domain part. al...@domain.tld and al...@company.org are two
distinct users, with different login and distinct mailboxes. Creating a
user without domain part is forbidden.


When virtual hosting is turned of, usernames are only composed of a
local part. All domains handled by James (check domain list) can be used
as domain part of their mail address. al...@domain.tld and
al...@company.org are two mail address belonging to a same "alice" user,
with same login and mailboxes. Creating users with domain parts is
forbidden.

I will add the following indication within the documentation as it might
get really useful to understand this. Thanks for highlighting this weak
point!



     MD5

Try more secure algorithms, you should give a go to SHA-512.

We should update default configuration accordingly as MD5 hashing is not
acceptable.


     true
ad...@myhost.com


This took a whole lot of digging to figure out.  I really think the
default usersrepository.xml should have at least commented-out lines
defining where/how to set the administrator id.

I would be glad to have you contribute this!


I'll add it to my list if this is indeed the correct implementation.

Jerry


On 10/29/2019 11:27 PM, Jerry Malcolm wrote:

I see in AbstractUsersRepository where "adminstratorid" is pulled from
somewhere in the configuration:

     administratorId =
Optional.ofNullable(configuration.getString("administratorId"));

But I can't find any place to set it in the shipped configuration
files.  Where is the administrator id supposed to be defined, and what
is the correct syntax to define it?

Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



(Resolved): Moving Servers Wipes Out iPhone's Mail Accounts

2019-11-04 Thread Jerry Malcolm
Finally, after 2 weeks, iPhones have all of their mail.  The problem was 
that my Lucene index was corrupted. Or to be more precise, only emails 
from the last two weeks since I first started up James on the new 
environment were indexed. Even after renaming (effectively removing) the 
/var/store/lucene folder, the index would not fully rebuild.  Same 
problem... but worse now... now it showed only mail that came in over 
the last 10 minutes since I cleaned the lucene index folder and lost the 
last 2 weeks' mail that actually was indexed.  On a whim, I changed 
indexer.xml to use lazyIndex instead of luceneIndex... voila all of 
the folders were fully indexed, and iPhone has all of the mail for each 
folder.  Couldn't find much documentation on lazyIndex vs. luceneIndex.  
But at this point, I think I'll take having mail back on my client's 
iPhones over any downside to using lazyIndex. But educate me if 
there are serious downsides to using lazyIndex.


Thunderbird populates folders using a different imap command which, 
apparently, goes straight to the JPA database instead of using the 
Lucene index.  So that's why the problem didn't appear in TBird.


I'm going to start another thread in the dev group specifically to 
discuss the Lucene issue.


Jerry

On 10/31/2019 12:35 PM, Jerry Malcolm wrote:
(I'm moving this thread to the dev forum since it appears now to not 
be a user problem)


More data now... Since iPhone doesn't load the correct email list and 
TBird does, I traced the precise same scenario with the same email 
account with TBird as the previous iPhone trace in the post below.  As 
I suspected, in order to do an initial full sync on a folder, TBird is 
asking for the list of UIDs in a different way. Note from the two traces:


TBird: UID fetch 1:* (FLAGS)
    Response: 1 FETCH (FLAGS (\\Seen) UID 5801)  (...followed by many 
other UIDs including the ones in iPhone search below)

---
iPhone: UID SEARCH RETURN (COUNT) 1:* NOT DELETED
 Response: ESEARCH (TAG "39") UID ALL 7776,7955:7970,7975:8033

The two queries above are for the same email account/folder. Again, 
I'm not an IMAP expert.  But it appears that both should return the 
same list of UIDs even though they ask for it differently.


Summary... the TBird query gets all UIDs for the folder starting at 
5801 as expected.  The iPhone query gets a subset of UIDs starting at 
7776.  This continues to support my current theory that the code that 
processes UID SEARCH RETURN (COUNT) 1:* is not correct.


If someone knows what class processes these imap queries, please let 
me know.  I'm going to dig into the code now.  But a few pointers to 
the package/classes would be helpful.


Thunderbird trace:

127 Request: 19 select "INBOX"
129,Response: * FLAGS (\\Answered \\Deleted \\Draft \\Flagged \\Seen 
"MailFlagBit1 "MailFlagBit0 NonJunk "label3 "label1 "Forwarded)

130,Response: 19 OK [READ-WRITE] SELECT completed.
132 Request: 20 getquotaroot "INBOX"
134 Response: * QUOTAROOT "INBOX" #private@malcolms.com
135,Response: 20 OK GETQUOTAROOT completed.
137,Request: 21 UID fetch 1:* (FLAGS)
139,Response: * 1 FETCH (FLAGS (\\Seen) UID 5801)
140,Response: * 2 FETCH (FLAGS (\\Seen) UID 5802)
141,Response: * 4 FETCH (FLAGS (\\Seen) UID 5804)

On 10/29/2019 3:12 PM, Jerry Malcolm wrote:
Ok, I need an IMAP expert Below is a very brief trace of the 
communications between iPhone mail and JAMES (3.4).  I completely 
deleted an account on my iPhone, then recreated it while in airplane 
mode to make sure I didn't miss any communications in my trace.  I 
started the trace, exited airplane mode and let the iPhone do an 
initial sync with the account.  The inbox folder in this account has 
over 1000 emails going back to early 2019.


I'm not an expert in IMAP.  But it appears that the iPhone mail app 
requests all of the emails 1:* (see line 812), but JAMES returns a 
single id plus two ranges (line 813).  But the total count JAMES 
reports is nowhere near the full 1000.  Subsequently (line 822), 
iPhone requests the emails JAMES told it about in line 813.  From 
what I can tell, the problem is in line 813. JAMES did not report all 
of the actual mail that is in the INBOX, which is exactly what I'm 
seeing on the phone.  Shouldn't JAMES return all 1000+ email ids that 
exist in INBOX?  Or am I reading the IMAP trace incorrectly?


Note that the only emails JAMES reports came in AFTER I migrated to 
the new server and to the James 3.4 from v3b5.  It appears that there 
is something different about pre-existing emails in the mailbox 
folder that is causing JAMES not to recognize them. But this problem 
ONLY exists on the iPhone mail app. Thunderbird gets all of the mail 
for the inbox.  So is iPhone must be sending a slightly different 
request command syntax than other clients do (??)  Just speculating


 798 IMAP Request: 37 SELECT INBOX
 802 IMAP Response: 37 OK [READ-WRITE] SELECT completed.
 806 IMAP Request: 38 UID SEARCH RETURN 

Re: Setting "administratorid"?

2019-11-04 Thread Tellier Benoit
I just created an issue regarding this.

https://issues.apache.org/jira/browse/JAMES-2963

Jerry, would you have time to contribute this missing documentation?

Regards,

Benoit

On 04/11/2019 16:27, Tellier Benoit wrote:
> Answers inlined,
> 
> Regards,
> 
> Benoit
> 
> On 30/10/2019 11:37, Jerry Malcolm wrote:
>> "enableVirtualHosting" is another config parameter that's set in this
>> same method.  So I decided to search around and see where that is set. 
>> I found it, and just taking a wild guess, is this correct?
> 
> Maybe we need to further explain this.
> 
> When virtual hosting is enabled, usernames are composed of a local part
> and a domain part. al...@domain.tld and al...@company.org are two
> distinct users, with different login and distinct mailboxes. Creating a
> user without domain part is forbidden.
> 
> 
> When virtual hosting is turned of, usernames are only composed of a
> local part. All domains handled by James (check domain list) can be used
> as domain part of their mail address. al...@domain.tld and
> al...@company.org are two mail address belonging to a same "alice" user,
> with same login and mailboxes. Creating users with domain parts is
> forbidden.
> 
> I will add the following indication within the documentation as it might
> get really useful to understand this. Thanks for highlighting this weak
> point!
> 
>>
>> > class="org.apache.james.user.jpa.JPAUsersRepository">
>>     MD5
> 
> Try more secure algorithms, you should give a go to SHA-512.
> 
> We should update default configuration accordingly as MD5 hashing is not
> acceptable.
> 
>>     true
>> ad...@myhost.com
>> 
>>
>> This took a whole lot of digging to figure out.  I really think the
>> default usersrepository.xml should have at least commented-out lines
>> defining where/how to set the administrator id.
> 
> I would be glad to have you contribute this!
> 
>>
>> I'll add it to my list if this is indeed the correct implementation.
>>
>> Jerry
>>
>>
>> On 10/29/2019 11:27 PM, Jerry Malcolm wrote:
>>> I see in AbstractUsersRepository where "adminstratorid" is pulled from
>>> somewhere in the configuration:
>>>
>>>     administratorId =
>>> Optional.ofNullable(configuration.getString("administratorId"));
>>>
>>> But I can't find any place to set it in the shipped configuration
>>> files.  Where is the administrator id supposed to be defined, and what
>>> is the correct syntax to define it?
>>>
>>> Thx
>>>
>>> Jerry
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>
>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: Setting "administratorid"?

2019-11-04 Thread Tellier Benoit
Answers inlined,

Regards,

Benoit

On 30/10/2019 11:37, Jerry Malcolm wrote:
> "enableVirtualHosting" is another config parameter that's set in this
> same method.  So I decided to search around and see where that is set. 
> I found it, and just taking a wild guess, is this correct?

Maybe we need to further explain this.

When virtual hosting is enabled, usernames are composed of a local part
and a domain part. al...@domain.tld and al...@company.org are two
distinct users, with different login and distinct mailboxes. Creating a
user without domain part is forbidden.


When virtual hosting is turned of, usernames are only composed of a
local part. All domains handled by James (check domain list) can be used
as domain part of their mail address. al...@domain.tld and
al...@company.org are two mail address belonging to a same "alice" user,
with same login and mailboxes. Creating users with domain parts is
forbidden.

I will add the following indication within the documentation as it might
get really useful to understand this. Thanks for highlighting this weak
point!

> 
>  class="org.apache.james.user.jpa.JPAUsersRepository">
>     MD5

Try more secure algorithms, you should give a go to SHA-512.

We should update default configuration accordingly as MD5 hashing is not
acceptable.

>     true
> ad...@myhost.com
> 
> 
> This took a whole lot of digging to figure out.  I really think the
> default usersrepository.xml should have at least commented-out lines
> defining where/how to set the administrator id.

I would be glad to have you contribute this!

> 
> I'll add it to my list if this is indeed the correct implementation.
> 
> Jerry
> 
> 
> On 10/29/2019 11:27 PM, Jerry Malcolm wrote:
>> I see in AbstractUsersRepository where "adminstratorid" is pulled from
>> somewhere in the configuration:
>>
>>     administratorId =
>> Optional.ofNullable(configuration.getString("administratorId"));
>>
>> But I can't find any place to set it in the shipped configuration
>> files.  Where is the administrator id supposed to be defined, and what
>> is the correct syntax to define it?
>>
>> Thx
>>
>> Jerry
>>
>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: Setting "administratorid"?

2019-11-04 Thread Tellier Benoit
Please have a look at http://james.apache.org/server/config-users.html

ad...@plateform.com in
usersrepository.xml

Will enable ad...@plateform.com to login as f...@plateform.com using imap.

On 30/10/2019 11:27, Jerry Malcolm wrote:
> I see in AbstractUsersRepository where "adminstratorid" is pulled from
> somewhere in the configuration:
> 
>     administratorId =
> Optional.ofNullable(configuration.getString("administratorId"));
> 
> But I can't find any place to set it in the shipped configuration
> files.  Where is the administrator id supposed to be defined, and what
> is the correct syntax to define it?
> 
> Thx
> 
> Jerry
> 
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org