Re: Dovecot auth error

[Aki Tuomi]

On 05.11.2017 03:02, Mathieu R. wrote:
> I just tried to configure a new dovecot/postfix server, and i end up with a
> dovecot auth error at startup.
> I can't find a solution by myself.
> Below are details, thanks in advance for your precious help, and excuse my
> poor english :
>
> dovecot --version
> 2.2.27 (c0f36b0) (Debian)
>
> Dovecot -n :
> https://400iso.net/public/dov.txt
>
>
> grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf
> driver = mysql
> connect = host=127.0.0.1 dbname=postfix user=postfix password=password
> default_pass_scheme = MD5-CRYPT
> user_query = SELECT '/srv/vmail/%d/%n' AS home, 3000 AS uid, 3000 AS gid,
> CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE
> username = '%u' AND active='1'
> password_query = SELECT password FROM mailbox WHERE username = '%u'
>
>
> Here is part of the server's log :
>
> Nov  4 20:57:49 vps81550 postfix/postscreen[21578]: CONNECT from
> [209.85.215.51]:47485 to [149.56.x.x]:25
> Nov  4 20:57:49 vps81550 postfix/dnsblog[21583]: addr 209.85.215.51 listed
> by domain dnsbl.sorbs.net as 127.0.0.6
> Nov  4 20:57:55 vps81550 postfix/postscreen[21578]: PASS OLD
> [209.85.215.51]:47485
> Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: connect from
> mail-lf0-f51.google.com[209.85.215.51]
> Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: Untrusted TLS connection
> established from mail-lf0-f51.google.com[209.85.215.51] TLSv1.2 with cipher
> ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
> Nov  4 20:57:55 vps81550 dovecot: auth: Fatal: sql: driver not set in
> configuration file /etc/dovecot/dovecot-sql.conf.ext
> Nov  4 20:57:55 vps81550 dovecot: master: Error: service(auth): command
> startup failed, throttling for 2 secs
> Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: fatal: no SASL
> authentication mechanisms
> Nov  4 20:57:56 vps81550 postfix/master[21528]: warning: process
> /usr/lib/postfix/sbin/smtpd pid 21585 exit status 1
> Nov  4 20:57:56 vps81550 postfix/master[21528]: warning:
> /usr/lib/postfix/sbin/smtpd: bad command startup -- throttling
This usually means that you have not installed mysql support for
dovecot. In dovecot, it's usually a separate package,
called dovecot-mysql.

Aki

Re: mail_crypt plugin, few questions

[Aki Tuomi]

On 04.11.2017 20:52, Zbyszek Żółkiewski wrote:
> Hi,
>
> I have few questions regarding mail_crypt:
>
> 1) Is mail_crypt_global_private_key file read upon dovecot start/restart only 
> or it is/can be read in any other time? I have made few tests by starting 
> dovecot and removing master key for decryption - therefore it is not 
> available on the platform - it only reside in memory, removing one of attack 
> vectors
It can be given from config file, or from user database. It is read on
use. You can also encrypt the key using a password, but in the end, the
password or the key needs to be provided by something.
> 2) Is there planned any “rollout” for key rotation in the feature ?
There is already *some* key rotation availability. Mail re-encryption is
not supported, but you can roll the user key, and you can take new
folder keys into use. The old ones need to be retained unless you move
emails out and back to the folder.
>
> 3) is there any better way to encrypt mails that were sent before enabling 
> mail_crypt ? I have made some simple script to automate and in-place 
> encryption: https://gist.github.com/kolargol/d551d132949068ce6efce7bc85a317cb 
> but maybe there is better way? If someone want to use it, please read code 
> before as it require updating magic(5) local database
>
> thanks!
>
> _
> Zbyszek Żółkiewski

The Best Practice, supported way, is to move the emails around, so you
move all your mails from INBOX to FooBox, and then back. This can have
some side-effects, and you might want to test this. Most notably it will
consume UIDs, unless you reset the folder by removing indexes.

Aki

Re: stats module

[Aki Tuomi]

On 05.11.2017 22:29, Jeff Abrahamson wrote:
> Doh.  Yes, that is why the file wasn't being read.  Thanks.
>
> It turned out, moveover, that to get the inet listener to start, I
> needed to do a restart, not just a reload (sudo service dovecot
> reload).  I'm not sure why that should be.  But it's now working.
>
>
> Meanwhile, I started seeing errors like this when I check mail:
>
>     Nov  5 20:57:06 nantes-1 dovecot: imap(jeff): Error: stats:
> open(/var/run/dovecot/stats-mail) failed: Permission denied
>
> The comments on the wiki page tell me to make this socket mode 600 and
> owned by the user that my mail process runs as.  I think this means
> dovecot.  (I tried dovenull, that was wrong.)
>
It means 'vmail', not 'dovecot'. If you are using real system users, you
probably need to set it as 0666.

Aki

Re: dovecot-lda without starting dovecot?

[Sami Ketola]

> On 5 Nov 2017, at 12.55, Stephan von Krawczynski  wrote:
> Sorry to say this setup works flawlessly for years. The only addition we
> will make now is to do the delivery with dovecot-lda. Everything else
> (including multiple dovecot pop/imap servers) will stay as is.
> Hopefully dovecot-lda does not fiddle around with the indexes too much, as we
> then would have to delete this part of the code out. It is not needed as we
> found out during the last 10 years of delivering mails into the maildirs by
> atomic rename action while dovecot is presenting them over imap.


Feel free to do anything you like. I'm just going to mention to people later 
reading these
from the achives not to take this kind of strange hack as an example of 
recommended
dovecot clustering. Instead consider it as an opposite of any best practices 
cluster setup.

Sami

Re: stats module

[Jeff Abrahamson]

Doh.  Yes, that is why the file wasn't being read.  Thanks.

It turned out, moveover, that to get the inet listener to start, I
needed to do a restart, not just a reload (sudo service dovecot
reload).  I'm not sure why that should be.  But it's now working.


Meanwhile, I started seeing errors like this when I check mail:

    Nov  5 20:57:06 nantes-1 dovecot: imap(jeff): Error: stats:
open(/var/run/dovecot/stats-mail) failed: Permission denied

The comments on the wiki page tell me to make this socket mode 600 and
owned by the user that my mail process runs as.  I think this means
dovecot.  (I tried dovenull, that was wrong.)

    ╭╴ (master=)╶╮
    ╰ [T] jeff@nantes-1:p27 $ ll /var/run/dovecot/stats*
    srw--- 1 root    root 0 Nov  5 21:02 /var/run/dovecot/stats
    prw--- 1 dovecot root 0 Nov  5 21:02 /var/run/dovecot/stats-mail
    prw--- 1 root    root 0 Nov  5 21:02 /var/run/dovecot/stats-user
    ╭╴ (master=)╶╮
    ╰ [T] jeff@nantes-1:p27 $

But actually this might be tricky, since there are many such users,
depending on what we mean.

    root  7553  0.0  0.2  18244  3000 ?    Ss   Nov03   0:00
/usr/sbin/dovecot
    dovecot   7559  0.0  0.0   9520   960 ?    S    Nov03   0:00  \_
dovecot/anvil
    root 26686  0.0  0.2   9652  2376 ?    S    21:02   0:00  \_
dovecot/log
    root 26690  0.0  0.3  25300  4008 ?    S    21:02   0:00  \_
dovecot/config
    dovenull 26806  0.0  0.5  18820  5488 ?    S    21:02   0:00  \_
dovecot/imap-login
    root 26808  0.0  0.0   9516   948 ?    S    21:02   0:00  \_
dovecot/ssl-params
    jeff 26810  0.0  0.3  25852  3860 ?    S    21:02   0:00  \_
dovecot/imap

With a bit of trial-and-error, I see that the error goes away if I
change the owner to jeff.  It sounds like I need to create an imap-users
group and make the socket accessible to everyone in that group (mode
660).  But that's not really what the docs say.  Am I confused or is
this something I should suggest we change in the docs?

    https://wiki2.dovecot.org/Statistics

Jeff


On 04/11/17 21:06, Steffen wrote:
> Jeff Abrahamson wrote: > > On 03/11/17 17:43, Mark Moseley wrote: > >> > >> > 
> >> On Fri, Nov
3, 2017 at 9:35 AM, Jeff Abrahamson  >>
> wrote: > >> > >> Sorry, Aki, I don't follow you.
Did I do it wrong in the file > >> 91-stats that I shared in my original
mail (attached here)? > >> > >  -rw-r--r-- 1 root root 1856 Nov 3
16:11 91-stats -rw-r--r-- >  1 root root 1430 Oct 31 16:33 > >>
99-mail-stack-delivery.conf > >  The file 91-stats contains the
contents I pasted in my >  earlier > >> mail. >  I'm a bit
unclear what to check next. > > > Thanks. I tried simply removing the
socket listener, as I don't > > need it. Still no error, still no port
24242 listener. > > IMHO: the default dovecot.conf includes *.conf files
only. If you > haven't changed it, hence, try to rename the file into >
> 91-stats.conf > ^ >
-- 

Jeff Abrahamson
+33 6 24 40 01 57
+44 7920 594 255

http://p27.eu/jeff/

Re: dovecot-lda without starting dovecot?

[Stephan von Krawczynski]

On Sun, 5 Nov 2017 10:44:25 +0200
Sami Ketola  wrote:

> > On 4 Nov 2017, at 10.31, Stephan von Krawczynski 
> > wrote:
> > 
> > On Sat, 4 Nov 2017 01:57:31 +0200
> > Sami Ketola  wrote:  
> >> Again that does not answer my question why? Why do you want all the
> >> locking problems and multi-access problems that come with setup like
> >> that? What is the actual problem that you are trying to solve?
> >> 
> >> Sami  
> > 
> > Really, I can hardly believe you don't now large loadbalancing ISP setups
> > with multiple nodes per single service ...?
> > The simple problem: massive numbers of emails  
> 
> Nope. Has never been done. Has never been recommended way. You will get more
> problems with that setup that you are seeking to solve.
> 
> Use multiple dovecot backends with director ring in front and switch to lmtp
> delivery via the director ring if you have scalability problems. Then you
> can just increase number of backends in case they are overloaded.
> 
> Sami

Sorry to say this setup works flawlessly for years. The only addition we
will make now is to do the delivery with dovecot-lda. Everything else
(including multiple dovecot pop/imap servers) will stay as is.
Hopefully dovecot-lda does not fiddle around with the indexes too much, as we
then would have to delete this part of the code out. It is not needed as we
found out during the last 10 years of delivering mails into the maildirs by
atomic rename action while dovecot is presenting them over imap.

-- 
Regards,
Stephan

Dovecot auth error

[Mathieu R.]

I just tried to configure a new dovecot/postfix server, and i end up with a
dovecot auth error at startup.
I can't find a solution by myself.
Below are details, thanks in advance for your precious help, and excuse my
poor english :

dovecot --version
2.2.27 (c0f36b0) (Debian)

Dovecot -n :
https://400iso.net/public/dov.txt


grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-sql.conf
driver = mysql
connect = host=127.0.0.1 dbname=postfix user=postfix password=password
default_pass_scheme = MD5-CRYPT
user_query = SELECT '/srv/vmail/%d/%n' AS home, 3000 AS uid, 3000 AS gid,
CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE
username = '%u' AND active='1'
password_query = SELECT password FROM mailbox WHERE username = '%u'


Here is part of the server's log :

Nov  4 20:57:49 vps81550 postfix/postscreen[21578]: CONNECT from
[209.85.215.51]:47485 to [149.56.x.x]:25
Nov  4 20:57:49 vps81550 postfix/dnsblog[21583]: addr 209.85.215.51 listed
by domain dnsbl.sorbs.net as 127.0.0.6
Nov  4 20:57:55 vps81550 postfix/postscreen[21578]: PASS OLD
[209.85.215.51]:47485
Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: connect from
mail-lf0-f51.google.com[209.85.215.51]
Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: Untrusted TLS connection
established from mail-lf0-f51.google.com[209.85.215.51] TLSv1.2 with cipher
ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
Nov  4 20:57:55 vps81550 dovecot: auth: Fatal: sql: driver not set in
configuration file /etc/dovecot/dovecot-sql.conf.ext
Nov  4 20:57:55 vps81550 dovecot: master: Error: service(auth): command
startup failed, throttling for 2 secs
Nov  4 20:57:55 vps81550 postfix/smtpd[21585]: fatal: no SASL
authentication mechanisms
Nov  4 20:57:56 vps81550 postfix/master[21528]: warning: process
/usr/lib/postfix/sbin/smtpd pid 21585 exit status 1
Nov  4 20:57:56 vps81550 postfix/master[21528]: warning:
/usr/lib/postfix/sbin/smtpd: bad command startup -- throttling
-- 

Mathieu R.

Re: Migrating from Dovecot 1 to Dovecot 2

[Sean Kamath]

On Nov 3, 2017, at 4:23 PM, Dovecot list  wrote:
> 
> Hello.
> I try to migrate about 200G of mails from one server to another.
> On the old i have Dovecot1 with Maildirs (without master pass etc.), on the
> new one i setup dovecot2 with mdbox. I need now to migrate (partialy, not
> all at once) mails from one to another.
> I can't find any solution that i can use? I dont have master password, and
> i want to mikgrate all mailaccont each other. Can anyone use me a working
> config for this ? Best will be that migratet dont want to be downloaded by
> mail client one more time.
> Thanks for any help.
> Best regards.

Hi.  I just did this.

All things considered, it went without impack.

It wasn’t perfect, as when someone was accessing their mailbox I’d get errors.  
So, for the final sync, I turned off the imap service for everything but the 
transfer (blocked at the firewall).

All things considered, it went well.  I’m sure this is jacked up and wrong and 
whatnot, but got my wife and kids (and me!) moved over with no hiccups hardly 
at all!

I still happen to have the config files I used:

I ended up doing some random crap that probably didn’t need to be done: Taking 
the output of dovecot -n and putting it in a template file (mostly to get the 
location of the sdbox’s I use).  Then I added the following after it:

imapc_host = old_dovecot1_host
imapc_port = 993
imapc_ssl = imaps
imapc_user = %%USER%%
imapc_password = %%PASSWORD%%
mail_fsync = never
imapc_ssl_verify = no
imapc_features = rfc822.size fetch-headers
# Read multiple mails in parallel, improves performance
mail_prefetch_count = 20

Now, I don’t know if this is all correct or not, but generally worked well.   
The imap_ssl_verify = no bit was because my cert expired in the middle of the 
migration.  Prior to that I had:

ssl_client_ca_file = /home//dovecot/certs/huh.crt

So, basically, I then wrote a script (I called it ‘synchrotron’ because that’s 
me):

#!/bin/sh

ACCTS=/home//conf.template # (the template file)

if [ `whoami` != "root" ]
then
echo "You should be root, my friend."
exit 1
fi

CONF=$(mktemp)

do_sync() {
UNAME=$1
PASS=$(grep "^$UNAME:" $ACCTS | cut -d: -f2)
if [ -z "$PASS" ]
then
echo "Unknown user $UNAME: No Password!"
return
fi
sed -e "s/%%USER%%/$UNAME/" -e "s/%%PASSWORD%%/$PASS/" $TEMPL > $CONF
START=`date +%s`
CMD="/usr/local/bin/doveadm -v -c $CONF backup -R -u $UNAME imapc:"
echo "START: $(date)"
echo "Running $CMD"
$CMD
ret=$?
STOP=`date +%s`
echo "STOP: Returned $ret $(date)"
rm -f $CONF
echo "Duration: $(expr $STOP '-' $START) seconds”
}

# This allows you to specify a user on the command line. . .
if [ $# -gt 0 ]
then
echo "Only doing a few users"
while [ $# -gt 0 ]
do
do_sync $1
shift
done
exit
fi

# Otherwise, do 'em all

# I like to log everything
exec >> /tmp/synchrotron.out 2>&1

for user in $(cut -d: -f1 $ACCTS)
do
do_sync $user
done

Re: stats module

[Steffen]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Jeff Abrahamson wrote:
> On 03/11/17 17:43, Mark Moseley wrote:
>> 
>> 
>> On Fri, Nov 3, 2017 at 9:35 AM, Jeff Abrahamson > > wrote:
>> 
>> Sorry, Aki, I don't follow you.  Did I do it wrong in the file 
>> 91-stats that I shared in my original mail (attached here)?
>> 

 -rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats -rw-r--r--
 1 root root  1430 Oct 31 16:33
>> 99-mail-stack-delivery.conf

 The file 91-stats contains the contents I pasted in my
 earlier
>> mail.
 I'm a bit unclear what to check next.

> Thanks.  I tried simply removing the socket listener, as I don't
> need it.  Still no error, still no port 24242 listener.

IMHO: the default dovecot.conf includes *.conf files only. If you
haven't changed it, hence, try to rename the file into

91-stats.conf
^

- -- 
Steffen
-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iQEcBAEBCgAGBQJZ/h3hAAoJEHz1H7kL/d9r1doIALrx38iwGoI4k7L6KYpS7IgS
1mdVDwuR05MoAUkkhHT+B9/wtYI4gdDaGS/R7m7pY/FB5dO2L9Y4tU6ZUSNB9rFJ
AqInuu43M/A9vc6/wAXPJIJfaduSE2Q5hrz2vfsjDBymFHmG9zh3dYjtvwo2Op92
iYPJP/2GZgHJGaAvPIckzzi2ICqbi/Ny1ODoyngWB5oLdFST1IerglNr88mUNxSO
F7TUPJYChQzOuvnxEPKUbWMgBw15SI+8MwSPeZl9twztbQ1Zzc/nXZlh2SB52PmA
LAWvUzbpAk1Khf57Mm4aiP7xrQMxIqWUTnkqXsDAGniH/QEX5koBBfNlc45cUjo=
=tFsf
-END PGP SIGNATURE-

Re: stats module

[Thomas Leuxner]

* Jeff Abrahamson  2017.11.03 17:45:

> > >>     -rw-r--r-- 1 root root  1856 Nov  3 16:11 91-stats

Please take note of the include scheme:
!include conf.d/*.conf

Regards
Thomas


signature.asc
Description: PGP signature

mail_crypt plugin, few questions

[Zbyszek Żółkiewski]

Hi,

I have few questions regarding mail_crypt:

1) Is mail_crypt_global_private_key file read upon dovecot start/restart only 
or it is/can be read in any other time? I have made few tests by starting 
dovecot and removing master key for decryption - therefore it is not available 
on the platform - it only reside in memory, removing one of attack vectors

2) Is there planned any “rollout” for key rotation in the feature ?

3) is there any better way to encrypt mails that were sent before enabling 
mail_crypt ? I have made some simple script to automate and in-place 
encryption: https://gist.github.com/kolargol/d551d132949068ce6efce7bc85a317cb 
but maybe there is better way? If someone want to use it, please read code 
before as it require updating magic(5) local database

thanks!

_
Zbyszek Żółkiewski

Re: dovecot-lda without starting dovecot?

[Sami Ketola]

> On 4 Nov 2017, at 10.31, Stephan von Krawczynski  wrote:
> 
> On Sat, 4 Nov 2017 01:57:31 +0200
> Sami Ketola  wrote:
>> Again that does not answer my question why? Why do you want all the locking
>> problems and multi-access problems that come with setup like that? What is
>> the actual problem that you are trying to solve?
>> 
>> Sami
> 
> Really, I can hardly believe you don't now large loadbalancing ISP setups with
> multiple nodes per single service ...?
> The simple problem: massive numbers of emails

Nope. Has never been done. Has never been recommended way. You will get more 
problems
with that setup that you are seeking to solve.

Use multiple dovecot backends with director ring in front and switch to lmtp 
delivery via the director
ring if you have scalability problems. Then you can just increase number of 
backends in case
they are overloaded.

Sami