Re: Can't authenticate using ARGON2ID crypt scheme from doveadm pw w/rounds != default

[Keith Amidon]

On 9/26/18 2:42 AM, Aki Tuomi wrote:

The reason it fails is because auth process runs out of memory and there
is a bug in libsodium which causes it to crash in this scenario. The fix
is to do

service auth {

   vsz_limit = 0 # or 2G at least

}

After testing this evening, I can confirm that this did
indeed resolve the problem. Thank you for the quick
response and accurate diagnosis!

Re: Best way to move mail from one server to another

[Admin]

I used (had to) imapsync two weeks ago to move Mails from dovecote 2.1 to 2.2, 
as doveadm failed (different versions issue).

Worked as expected. But all mails 60GB got resynced (summed up to approx. 250GB 
due to several accounts being used across multiple clients).

Tried both - without and with uid option. With latter one I did not have any 
duplicate mails being skipped; resync however seemed to be unavoidable.

-M

> Am 26.09.2018 um 16:18 schrieb Tanstaafl :
> 
> Never mind, should have waited and read the entire thread...
> 
> On Wed Sep 26 2018 09:52:26 GMT-0400 (Eastern Standard Time), Tanstaafl
>  wrote:
>> Finally have some time to review list emails...
>> 
>> On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami
>> Ketola  wrote:
>>> imapsync always loses data.
>> 
>> Hi Sami,
>> 
>> Can you expand on this?
>> 
>> I used ImapSync to migrate from Dovecot to Office365 a couple of years
>> ago, and didn't notice any issues with it at all.
> 

Re: What causes folders to be reported as noselect?

[Daniel Miller]

On 2018-09-26 10:14, Aki Tuomi wrote:

On 26 September 2018 at 18:42 Daniel Miller  wrote:


As the subject says.  This may be a bit open-ended - but it would 
really

help troubleshooting some obscure folder issues.

In my case, I happen to have both some "real" folders and also some
"virtual" folders that respond to IMAP LIST commands with the
"\NoSelect" flag - and I don't know why.  Via telnet, I can manually
issue SELECT, SEARCH, and FETCH for such folders without errors.

--

Daniel



\NoSelect folders are usually namespace boundaries and non-existing
folders, such as parents for children in systems where the parents do
not need to exist for real.

You should not be able to SELECT a \NoSelect folder.

Aki


At the moment, the folders in question:

My primary namespace "inbox", with no prefix, has a folder INBOX, with a 
child folder "Other", which in turn has two children.  "INBOX/Other" 
shows as \NoSelect - the two children are normal.


In my "virtual" namespace, I had a virtual folder defined as "Archives". 
 I created a new folder "Archive-Search" and copied the dovecot-virtual 
file over - and it works fine.


I don't see anything wrong via filesystem permissions or ownership - so 
I'm assuming either there are reserved words I'm not allowed to use with 
IMAP folders (but I can't find any documented), or something in my 
namespace or folder setup is applying some kind of mask (or something is 
corrupted...more on this below), or...there's a bug.  But I'm willing to 
assume the flaw lies with me.  Or at least my ever wonderful server - 
which continues to keep me entertained instead of simply operating 
quietly and consistently without endearing quirks...


As far as selectability...

I was going to post a telnet session to prove I could...but when I 
tested previously I was using the "virtual/Archives" folder and it 
worked manually - before I created the "virtual/Archive-Search" folder 
and deleted the other.  So I tried the "INBOX/Other" folder - and I do 
get the expected "NO Mailbox doesn't exist: INBOX/Other".  So...


Just for fun...I created "virtual/Archives" again, copied the 
dovecot-virtual, set the permissions...and it works fine! And just in 
case...I also tried "virtual/Archive" - also now selectable.  And to be 
clear - I create these folders directly in the filesystem, manually copy 
the dovecot-virtual file, and set the owner/permission.


Let's try another experiment...other email>


Ok...moving on.  "INBOX/Other" isn't selectable.  Let's experiment a 
little more carefully.  Using RoundCubeMail, view the folder list, 
rename "INBOX/Other" to "INBOX/Other-Old".  Same conditions.  Using 
RoundCube - create a new folder "INBOX/Other" - this is now selectable!  
Using RoundCube - move the first child of "INBOX/Other-Old" to 
"INBOX/Other".


Now it's weird.  INBOX/Other is present and selectable, 
INBOX/Other/Child1 is present and selectable - INBOX/Other-Old has 
disappeared and the former INBOX/Other-Old/Child2 is now at 
INBOX/Child2.  Move that to INBOX/Other/Child2...now everything is 
selectable as expected.


Which leaves me wondering...what the  was broken - and was there 
any other way to see it?  The on-disk structure looked right and the 
IMAP folder lists looked right other than the non-selectability.

--
Daniel

Possible bug - otherwise a public admission of oops

[Daniel Miller]

While trying to identify possible causes of wrong mail folder creation I 
did something...bad.


Normally, I would recognize that deleting a mail folder would naturally 
delete all the contained mails.  However...somehow my imaginative self 
decided that deleting a virtual folder via IMAP would only delete the 
virtual folder...and not proceed to delete every referenced email via 
the virtual mapping.


So note to self, and possible reminder to others, deleting a virtual 
folder via a filesystem command is just tiny bit different than via 
IMAP...


Fortunately I keep my feathers numbered for just such an emergency...

--
Daniel

Re: Local access to IMAP mailboxes

[Joseph Tam]

On Wed, 26 Sep 2018, Victor Sudakov wrote:


However, I often read and modify the mailboxes locally with mutt (e.g.
append and delete mails).

Should I expect any problems wit Dovecot indexes etc? What if I even
do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?


I do it all the time.  Works fine.

As others have written, you may see performance degradation as Dovecot
will have to rebuild indices, but if you have small mailboxes, this won't
be too bad.  The only reason I use direct file access rather than IMAP
is that I'm too lazy to work out a passwordless access method.  If this
doesn't bother you or you can configure this (e.g. Kerberos, keyring,
etc.), IMAP access is preferable since you won't pull the indices out
from Dovecot's feet.

You'll also get a lot of innocuous griping in the log files about
UIDVALIDITY and mailbox corruption, but they can be safely ignored.

Joseph Tam 

Re: What causes folders to be reported as noselect?

[Aki Tuomi]

> On 26 September 2018 at 18:42 Daniel Miller  wrote:
> 
> 
> As the subject says.  This may be a bit open-ended - but it would really 
> help troubleshooting some obscure folder issues.
> 
> In my case, I happen to have both some "real" folders and also some 
> "virtual" folders that respond to IMAP LIST commands with the 
> "\NoSelect" flag - and I don't know why.  Via telnet, I can manually 
> issue SELECT, SEARCH, and FETCH for such folders without errors.
> 
> -- 
> 
> Daniel
>

\NoSelect folders are usually namespace boundaries and non-existing folders, 
such as parents for children in systems where the parents do not need to exist 
for real.

You should not be able to SELECT a \NoSelect folder.

Aki

What causes folders to be reported as noselect?

[Daniel Miller]

As the subject says.  This may be a bit open-ended - but it would really 
help troubleshooting some obscure folder issues.


In my case, I happen to have both some "real" folders and also some 
"virtual" folders that respond to IMAP LIST commands with the 
"\NoSelect" flag - and I don't know why.  Via telnet, I can manually 
issue SELECT, SEARCH, and FETCH for such folders without errors.


--

Daniel

Re: split auth from other logging

[Admin]

This is great, thank you!!

-M

> Am 26.09.2018 um 15:53 schrieb Kai Schaetzl :
> 
> This works for splitting off lmtp traffic, for instance.
> 
> syslog_facility = uucp
> 
> rsyslog:
> 
> :msg, contains, "lmtp(" -/var/log/dovecot/lmtp.log
> & stop
> 
> uucp.=debug -/var/log/dovecot/debug.log
> uucp.=info  -/var/log/dovecot/dovecot.log
> uucp.=warn  -/var/log/dovecot/warn.log
> uucp.=err   -/var/log/dovecot/error.log
> uucp.=crit  -/var/log/warn.log
> 
> plus:
> auth,authpriv,cron,daemon,mail,uucp,news.none   -/var/log/syslog
> (whatever you don't want to see in syslog)
> 
> Kai
> 
> 

Re: split auth from other logging

[Kai Schaetzl]

I forgot to mention that you have to change owner for the /var/log/dovecot 
directory before rsyslog can log.

Kai

Re: Best way to move mail from one server to another

[Tanstaafl]

Never mind, should have waited and read the entire thread...

On Wed Sep 26 2018 09:52:26 GMT-0400 (Eastern Standard Time), Tanstaafl
 wrote:
> Finally have some time to review list emails...
> 
> On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami
> Ketola  wrote:
>> imapsync always loses data.
> 
> Hi Sami,
> 
> Can you expand on this?
> 
> I used ImapSync to migrate from Dovecot to Office365 a couple of years
> ago, and didn't notice any issues with it at all.
> 

Re: Best way to move mail from one server to another

[Reio Remma]

On 26/09/2018 16:52, Tanstaafl wrote:

Finally have some time to review list emails...

On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami
Ketola  wrote:

imapsync always loses data.

Hi Sami,

Can you expand on this?

I used ImapSync to migrate from Dovecot to Office365 a couple of years
ago, and didn't notice any issues with it at all.


Keep reading, the topic was discussed at length. :D

Good luck,
Reio

Re: split auth from other logging

[Kai Schaetzl]

This works for splitting off lmtp traffic, for instance.

syslog_facility = uucp

rsyslog:

:msg, contains, "lmtp(" -/var/log/dovecot/lmtp.log
& stop

uucp.=debug -/var/log/dovecot/debug.log
uucp.=info  -/var/log/dovecot/dovecot.log
uucp.=warn  -/var/log/dovecot/warn.log
uucp.=err   -/var/log/dovecot/error.log
uucp.=crit  -/var/log/warn.log

plus:
auth,authpriv,cron,daemon,mail,uucp,news.none   -/var/log/syslog
(whatever you don't want to see in syslog)

Kai

Re: Best way to move mail from one server to another

[Tanstaafl]

Finally have some time to review list emails...

On Tue Sep 04 2018 03:41:50 GMT-0400 (Eastern Standard Time), Sami
Ketola  wrote:
> imapsync always loses data.

Hi Sami,

Can you expand on this?

I used ImapSync to migrate from Dovecot to Office365 a couple of years
ago, and didn't notice any issues with it at all.

Re: split auth from other logging

[Kai Schaetzl]

I'm not going to log dovecot to mail, that creates only the same mixup as 
before, even worse, now postfix and dovecot mixed. I had to stop/start 
(force-reload would also work) rsyslogd to pick up the changed config. A 
restart doesn't change the config as with other daemons. Now I can filter 
lmtp out.

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

Re: split auth from other logging

[Kai Schaetzl]

Kai Schaetzl wrote on Wed, 26 Sep 2018 12:43:28 +0200:

> But it logs only to /var/log/syslog

It seems that "service rsyslog restart" doesn't correctly restart 
rsyslogd. You have to stop and start it. Then it picks up the changed 
config.

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

Re: split auth from other logging

[ad...@awib.it]

# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info   -/var/log/mail.info
mail.warn   -/var/log/mail.warn
mail.err/var/log/mail.err

> Am 26.09.2018 um 12:43 schrieb Kai Schaetzl :
> 
> I hoped I don't have to switch to syslog logging. Well, anyway.
> 
> I changed 
> 10-logging.conf:
> syslog_facility = uucp
> and commented out the other log lines.
> 
> rsyslog.d/50-default.conf:
> uucp.debug  -/var/log/dovecot/debug.log
> uucp.info   -/var/log/dovecot/dovecot.log
> uucp.warn   -/var/log/dovecot/warn.log
> uucp.err-/var/log/dovecot/error.log
> uucp.crit   -/var/log/warn.log
> 
> No fancy redirects with rsyslog yet, plain logging by facility and level.
> 
> But it logs only to /var/log/syslog. As if dovecot sets another facility.
> I've used uucp in the past with success (not with rsyslog, but with 
> syslog). AFAIK, uucp still exists as a facility in rsyslog.
> Shouldn't the above work?
> 
> (Yes, I restarted both daemons.)
> 
> Kai
> 
> 

Re: Local access to IMAP mailboxes

[Thomas Leuxner]

* Victor Sudakov  2018.09.26 12:17:

> > >> However, I often read and modify the mailboxes locally with Mutt (e.g.
> > >> append and delete mails).

Why not use Mutt's IMAP capabilities and keep the indexes nice and clean?

Regards
Thomas


signature.asc
Description: PGP signature

Re: split auth from other logging

[Kai Schaetzl]

I hoped I don't have to switch to syslog logging. Well, anyway.

I changed 
10-logging.conf:
syslog_facility = uucp
and commented out the other log lines.

rsyslog.d/50-default.conf:
uucp.debug  -/var/log/dovecot/debug.log
uucp.info   -/var/log/dovecot/dovecot.log
uucp.warn   -/var/log/dovecot/warn.log
uucp.err-/var/log/dovecot/error.log
uucp.crit   -/var/log/warn.log

No fancy redirects with rsyslog yet, plain logging by facility and level.

But it logs only to /var/log/syslog. As if dovecot sets another facility.
I've used uucp in the past with success (not with rsyslog, but with 
syslog). AFAIK, uucp still exists as a facility in rsyslog.
Shouldn't the above work?

(Yes, I restarted both daemons.)

Kai

Re: Local access to IMAP mailboxes

[Victor Sudakov]

Sami Ketola wrote:
> > On 26 Sep 2018, at 13.07, Aki Tuomi  wrote:
> > 
> >> 
> >> I have made mailboxes in ~/Mail available via IMAP (Dovecot 2.3.2.1),
> >> that is: "mail_location = mbox:~/Mail:INBOX=/var/mail/%u"
> >> 
> >> However, I often read and modify the mailboxes locally with Mutt (e.g.
> >> append and delete mails).
> >> 
> >> Should I expect any problems wit Dovecot indexes etc? What if I even
> >> do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?
> >> 
> > 
> > Dovecot is tolerant to changes with mbox and maildir. It will reindex if
> > it detects someone changing them outside.

That is good news! Thank you Aki!

> 
> That is true but it however has performance impacts if the caching
> is impaired but still does works.

Ah, is this perhaps the reason why my Android IMAP client sometimes takes so
long to list a folder's contents? I must have touched the folder
locally by Mutt and Dovecot is reindexing, is that right?

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: Local access to IMAP mailboxes

[Sami Ketola]

> On 26 Sep 2018, at 13.07, Aki Tuomi  wrote:
> 
> 
> 
> On 26.09.2018 12:22, Victor Sudakov wrote:
>> Dear Colleagues,
>> 
>> I have made mailboxes in ~/Mail available via IMAP (Dovecot 2.3.2.1),
>> that is: "mail_location = mbox:~/Mail:INBOX=/var/mail/%u"
>> 
>> However, I often read and modify the mailboxes locally with mutt (e.g.
>> append and delete mails).
>> 
>> Should I expect any problems wit Dovecot indexes etc? What if I even
>> do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?
>> 
> 
> Dovecot is tolerant to changes with mbox and maildir. It will reindex if
> it detects someone changing them outside.


That is true but it however has performance impacts if the caching is impaired 
but still does works.

Sami

Re: Local access to IMAP mailboxes

[Aki Tuomi]

On 26.09.2018 12:22, Victor Sudakov wrote:
> Dear Colleagues,
>
> I have made mailboxes in ~/Mail available via IMAP (Dovecot 2.3.2.1),
> that is: "mail_location = mbox:~/Mail:INBOX=/var/mail/%u"
>
> However, I often read and modify the mailboxes locally with mutt (e.g.
> append and delete mails).
>
> Should I expect any problems wit Dovecot indexes etc? What if I even
> do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?
>

Dovecot is tolerant to changes with mbox and maildir. It will reindex if
it detects someone changing them outside.

Aki

Re: Can't authenticate using ARGON2ID crypt scheme from doveadm pw w/rounds != default

[Aki Tuomi]

The reason it fails is because auth process runs out of memory and there
is a bug in libsodium which causes it to crash in this scenario. The fix
is to do

service auth {

  vsz_limit = 0 # or 2G at least

}

Aki


On 26.09.2018 09:12, Keith Amidon wrote:
> I'm using dovecot version 2.3.2.1 (0719df592) and trying to use the
> ARGON2ID crypt scheme for authentication using the passdb passwd-file
> driver. My passdb config is very simple:
>
>     passdb {
>   driver = passwd-file
>   args = username_format=%u 
>     }
>
> If I generate a password this way:
>
>     doveadm pw -s ARGON2ID -p 'This is a test'
>
> I get a crypt value for the password that I can place in the password
> file like:
>
> testuser:{ARGON2ID}$argon2id$v=19$m=65536,t=3,p=1$UuqF25QtumNBe9R2FmUZvA$5avvHY5TIaj5Wl5C4k8BOI4bcmNei7BwPLlXYQVybMc
>
>
> And if I test authentication with this command:
>
>     doveadm auth login testuser 'This is a test'
>
> It works as shown by the (lightly redacted) log:
>
>     Sep 25 22:46:01 myhost dovecot[17538]: auth: Debug: auth client
> connected (pid=55417)
>     Sep 25 22:46:01 myhost dovecot[17538]: auth: Debug: client in:
> AUTH    1    PLAIN service=doveadm    debug   
> resp=
>     Sep 25 22:46:01 myhost dovecot[17538]: auth: Debug:
> passwd-file(testuser): lookup: user=testuser file=
>     Sep 25 22:46:01 myhost dovecot[17538]: auth: Debug: client passdb
> out: OK    1    user=testuser
>
> However, if I instead specify a non-default number of rounds this way:
>
>  doveadm pw -s ARGON2ID -p 'This is a test' -r 7
>
> and place the result in an entry in the password file like:
>
> testuser:{ARGON2ID}$argon2id$v=19$m=1048576,t=7,p=1$kIhnUR13GrtOvvpbJNJmnQ$o7O6Whxs3s8IE09yY9S2dPkJjJyEVc78GRFilYVS9fU
>
>
> Then testing authentication using the same command (repeated here):
>
>     doveadm auth login testuser 'This is a test'
>
> then authentication fails, as shown by this (lightly redacted) log:
>
>     Sep 25 22:52:05 myhost dovecot[17538]: auth: Debug: auth client
> connected (pid=7)
>     Sep 25 22:52:05 myhost dovecot[17538]: auth: Debug: client in:
> AUTH    1    PLAIN service=doveadm    debug   
> resp=
>     Sep 25 22:52:05 myhost dovecot[17538]: auth: Debug: passwd-file
> : Read 3 users in 0 secs
>     Sep 25 22:52:05 myhost dovecot[17538]: auth: Debug:
> passwd-file(testuser): lookup: user=testuser file=
>     Sep 25 22:52:05 myhost dovecot[17538]: auth:
> passwd-file(testuser): Password mismatch
>     Sep 25 22:52:07 myhost dovecot[17538]: auth: Debug: client passdb
> out: FAIL    1    user=testuser
>
> Experimentation with other values for the -r option has not produced
> a value that works for me. Using the exact same procedure but the
> BLF-CRYPT scheme, with varying number of rounds, does work.
>
> Am I doing something wrong or is there a bug in either the
> doveadm pw generation or the auth evaluation of the password?
>
> I'd like to use ARGON2ID with ~6 rounds if I can make this work.
> Any help would be greatly appreciated.
>
> Thanks, Keith
>

Local access to IMAP mailboxes

[Victor Sudakov]

Dear Colleagues,

I have made mailboxes in ~/Mail available via IMAP (Dovecot 2.3.2.1),
that is: "mail_location = mbox:~/Mail:INBOX=/var/mail/%u"

However, I often read and modify the mailboxes locally with mutt (e.g.
append and delete mails).

Should I expect any problems wit Dovecot indexes etc? What if I even
do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/

Re: split auth from other logging

[ad...@awib.it]

Hi,

yesterday I had something similar.

I would like to skip the login & logout lines from being logged.

This is something I found, but did not get it working, as I had
no time fiddling with the rsyslog config yet:

https://serverfault.com/questions/253418/force-dovecot-not-to-log-connect-disconnect-messages
 


-M

> Am 26.09.2018 um 09:21 schrieb Kai Schaetzl :
> 
> Is there a way to split the auth logging (logins and failed logins) from 
> the other logging that goes to 
> info_log_path = /var/log/dovecot/dovecot.log
> ?
> This log gets a lot of other info as well, most notably the lmtp 
> notifications about every filed mail (with no level stamping, btw).
> This makes it really hard to find authentication errors quickly and 
> comfortably.
> It would be nice to be able to split at least the lmtp messages away.
> 
> Kai
> 
> 
> 

split auth from other logging

[Kai Schaetzl]

Is there a way to split the auth logging (logins and failed logins) from 
the other logging that goes to 
info_log_path = /var/log/dovecot/dovecot.log
?
This log gets a lot of other info as well, most notably the lmtp 
notifications about every filed mail (with no level stamping, btw).
This makes it really hard to find authentication errors quickly and 
comfortably.
It would be nice to be able to split at least the lmtp messages away.

Kai