Re: suse.de dovecot issues
> On 12/09/2021 14:54 William Edwards wrote: > > > Bernhard M. Wiedemann schreef op 2021-09-12 10:34: > > On 10/09/2021 11.33, William Edwards wrote: > >>> > >>> 2. > >>> We have 2 backends so that we can do maintenance on one of them while > >>> users can still access their emails through the other backend. > >>> However, we found that stopping dovecot on one backend left users > >>> unable > >>> to access their mails. > >> > >> Director doesn't do health checks. > > > > Is there are recommendation on how to have a director with multiple > > backends with automatic fail-over? > > Are there scripts that do health-checks and call doveadm director > > commands? > > https://doc.dovecot.org/configuration_manual/dovemon/ > > -- > With kind regards, > > William Edwards There is also poolmon. https://github.com/brandond/poolmon Aki
Re: suse.de dovecot issues
Bernhard M. Wiedemann schreef op 2021-09-12 10:34: On 10/09/2021 11.33, William Edwards wrote: 2. We have 2 backends so that we can do maintenance on one of them while users can still access their emails through the other backend. However, we found that stopping dovecot on one backend left users unable to access their mails. Director doesn't do health checks. Is there are recommendation on how to have a director with multiple backends with automatic fail-over? Are there scripts that do health-checks and call doveadm director commands? https://doc.dovecot.org/configuration_manual/dovemon/ -- With kind regards, William Edwards
Re: suse.de dovecot issues
On 10/09/2021 11.33, William Edwards wrote: >> >> 2. >> We have 2 backends so that we can do maintenance on one of them while >> users can still access their emails through the other backend. >> However, we found that stopping dovecot on one backend left users unable >> to access their mails. > > Director doesn't do health checks. Is there are recommendation on how to have a director with multiple backends with automatic fail-over? Are there scripts that do health-checks and call doveadm director commands?
Re: suse.de dovecot issues
On 9/10/21 12:19, Bernhard M. Wiedemann wrote: > I added it to > /etc/dovecot/dovecot-ldap.conf.ext > but it did not help. > I guess, because we map usernames "bwiedemann@foo" that are in sql, but > ldap only has the plain usernames. Why do you have such a mixed SQL-LDAP-setup? Why not just use one or the other? Ciao, Michael.
Re: suse.de dovecot issues
> On 10/09/2021 13:19 Bernhard M. Wiedemann wrote: > > > On 10/09/2021 11.43, Aki Tuomi wrote: > >> https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > > > It seems you are normalizing SQL usernames by doing SELECT username, but > > you are not doing it for LDAP. > > > > See if the problem goes away with > > > > pass_attrs = uid=username > > I added it to > /etc/dovecot/dovecot-ldap.conf.ext > but it did not help. > I guess, because we map usernames "bwiedemann@foo" that are in sql, but > ldap only has the plain usernames. For director to work correctly, your usernames *must* map to same. The redirection is calculated by the full user, so if you bwiedemann and bwiedemann@foo, these two may end up in different backends. Aki
Re: suse.de dovecot issues
On 10/09/2021 11.43, Aki Tuomi wrote: >> https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > It seems you are normalizing SQL usernames by doing SELECT username, but you > are not doing it for LDAP. > > See if the problem goes away with > > pass_attrs = uid=username I added it to /etc/dovecot/dovecot-ldap.conf.ext but it did not help. I guess, because we map usernames "bwiedemann@foo" that are in sql, but ldap only has the plain usernames.
Re: suse.de dovecot issues
On 10/09/2021 11.32, Aki Tuomi wrote:
> Can you post your `doveconf -n`? LMTP should not end up in different backend.
# 2.3.15 (0503334ab1): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.15 (e6a84e31)
# OS: Linux 5.3.18-59.19-default x86_64
# Hostname: dovecot-director2.suse-dmz.suse.de
auth_mechanisms = plain login
auth_verbose = yes
default_client_limit = 8192
default_process_limit = 16384
default_vsz_limit = 2 G
director_mail_servers = imap1.suse-dmz.suse.de@nuernberg
imap2.suse-dmz.suse.de@nuernberg
director_servers = dovecot-director1.suse-dmz.suse.de
dovecot-director2.suse-dmz.suse.de
doveadm_api_key = # hidden, use -P to show it
doveadm_password = # hidden, use -P to show it
doveadm_port = 12345
haproxy_trusted_networks = 192.168.254.0/24
hostname = dovecot-director2.suse.de
lmtp_proxy = yes
login_log_format_elements = user=<%n> application=<%d> method=%m rip=%r
lip=%l mpid=%e %c %k
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
override_fields = proxy=yes director_tag=nuernberg starttls=yes
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
override_fields = proxy=yes director_tag=nuernberg starttls=yes
}
plugin {
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap pop3 lmtp submission sieve
service anvil {
client_limit = 69636
}
service auth-worker {
process_limit = 4096
}
service auth {
client_limit = 98304
}
service director {
fifo_listener login/proxy-notify {
mode = 0666
}
inet_listener {
address = 192.168.254.65,127.0.0.1,::1
port = 9090
}
unix_listener director-userdb {
mode = 0600
}
unix_listener login/director {
mode = 0666
}
}
service doveadm {
inet_listener {
address = 192.168.254.65,127.0.0.1,::1
port = 12345
}
inet_listener http {
address = 192.168.254.65,127.0.0.1,::1
port = 8080
}
}
service imap-login {
executable = imap-login director
process_limit = 4096
}
service lmtp {
inet_listener lmtp {
address = 192.168.254.65, 192.168.254.85, 127.0.0.1, ::1
port = 24
}
inet_listener lmtp_haproxy {
address = 192.168.254.65, 192.168.254.85, 127.0.0.1, ::1
haproxy = yes
port = 23
}
process_limit = 4096
unix_listener lmtp {
mode = 0666
}
}
service managesieve-login {
executable = managesieve-login director
inet_listener sieve {
address = 192.168.254.65, 192.168.254.85, 127.0.0.1, ::1
}
process_limit = 4096
}
service pop3-login {
executable = pop3-login director
}
service submission-login {
executable = submission-login director
inet_listener submission {
port = 587
ssl = no
}
process_limit = 4096
}
ssl = required
ssl_ca = at
imap1.suse-dmz.suse.de:24: 250 2.0.0 IOniGJDEOWHuCwAAGKfGzw
Saved (1/1 at 3618 ms)
Sep 09 08:24:07 dovecot-director2 dovecot[1424]: lmtp(2196):
lmtp-server: conn 149.44.160.134:50360 [4]: rcpt msucha...@imap.suse.de:
KNOyFaTEOWGUCAAApTUePA: Sent message to at
imap1.suse-dmz.suse.de:24: 250 2.0.0 yHSOJaTEOWFMDwAAGKfGzw
Saved (1/1 at 2921 ms)
Sep 09 08:27:13 dovecot-director2 dovecot[1424]: imap-login:
proxy(msuchanek@offlineimap,192.168.254.74:143): Started proxying to
imap2.suse-dmz.suse.de (0.076 secs): user=,
application=, method=PLAIN, rip=192.168.254.67,
lip=192.168.254.85, TLS, TLSv1.3 with cipher TLS_AES_256_GCM_SHA384
(256/256 bits)
Re: suse.de dovecot issues
You can setup the nfs users with NIS over idmapd. Then setup the dovecot server with NIS logins, so the the user login from any server over nis will get uid, gid, userhome etc, It mean it can read the nis users passwd and shadow. On Fri, 10 Sep, 2021, 2:49 pm Bernhard M. Wiedemann, wrote: > Hi, > > I am one of the people taking over our new suse.de email setup > (consisting of dovecot+rspamd+postfix) > and wanted to report some issues we experience: > > > 1. > we use dovecot-director to distribute users between 2 backend servers > that share an NFS mount. > We found that it proxies lmtp to a different backend than imap of the > same user and that caused NFS stale-filehandle errors on the > dovecot-uidlist. > It then proceeds to re-generate the dovecot-uidlist with new UIDs that > creates trouble for users. > > a) shouldn't dovecot use locks (fcntl or flock) to protect such files > from concurrent updates? > > b) could it generate uidlist in a way that re-generating it, assigns the > same UIDs again? E.g. via hash over file content > > c) how to get dovecot-director to send all traffic for a user to one > backend? > > > 2. > We have 2 backends so that we can do maintenance on one of them while > users can still access their emails through the other backend. > However, we found that stopping dovecot on one backend left users unable > to access their mails. > Maybe this is related to how user auth works? > How to get this HA setup right, so that we don't have a single point of > failure? > > > grep PRETTY /etc/os-release > PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" > > rpm -q dovecot23 > dovecot23-2.3.15 > > https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > Ciao > Bernhard M. >
Re: suse.de dovecot issues
> On 10/09/2021 12:19 Bernhard M. Wiedemann wrote: > > > Hi, > > I am one of the people taking over our new suse.de email setup > (consisting of dovecot+rspamd+postfix) > and wanted to report some issues we experience: > > > 1. > we use dovecot-director to distribute users between 2 backend servers > that share an NFS mount. > We found that it proxies lmtp to a different backend than imap of the > same user and that caused NFS stale-filehandle errors on the > dovecot-uidlist. > It then proceeds to re-generate the dovecot-uidlist with new UIDs that > creates trouble for users. > > a) shouldn't dovecot use locks (fcntl or flock) to protect such files > from concurrent updates? > > b) could it generate uidlist in a way that re-generating it, assigns the > same UIDs again? E.g. via hash over file content > > c) how to get dovecot-director to send all traffic for a user to one > backend? > > > 2. > We have 2 backends so that we can do maintenance on one of them while > users can still access their emails through the other backend. > However, we found that stopping dovecot on one backend left users unable > to access their mails. > Maybe this is related to how user auth works? > How to get this HA setup right, so that we don't have a single point of > failure? > > > grep PRETTY /etc/os-release > PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" > > rpm -q dovecot23 > dovecot23-2.3.15 > > https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > Ciao > Bernhard M. It seems you are normalizing SQL usernames by doing SELECT username, but you are not doing it for LDAP. See if the problem goes away with pass_attrs = uid=username Aki
Re: suse.de dovecot issues
> On 10/09/2021 12:32 Aki Tuomi wrote: > > > > On 10/09/2021 12:19 Bernhard M. Wiedemann wrote: > > > > > > Hi, > > > > I am one of the people taking over our new suse.de email setup > > (consisting of dovecot+rspamd+postfix) > > and wanted to report some issues we experience: > > > > > > 1. > > we use dovecot-director to distribute users between 2 backend servers > > that share an NFS mount. > > We found that it proxies lmtp to a different backend than imap of the > > same user and that caused NFS stale-filehandle errors on the > > dovecot-uidlist. > > It then proceeds to re-generate the dovecot-uidlist with new UIDs that > > creates trouble for users. > > > > a) shouldn't dovecot use locks (fcntl or flock) to protect such files > > from concurrent updates? > > > > b) could it generate uidlist in a way that re-generating it, assigns the > > same UIDs again? E.g. via hash over file content > > > > c) how to get dovecot-director to send all traffic for a user to one > > backend? > > > > > > 2. > > We have 2 backends so that we can do maintenance on one of them while > > users can still access their emails through the other backend. > > However, we found that stopping dovecot on one backend left users unable > > to access their mails. > > Maybe this is related to how user auth works? > > How to get this HA setup right, so that we don't have a single point of > > failure? > > > > > > grep PRETTY /etc/os-release > > PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" > > > > rpm -q dovecot23 > > dovecot23-2.3.15 > > > > https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > > > Ciao > > Bernhard M. > > Can you post your `doveconf -n`? LMTP should not end up in different backend. > > Aki And now I noticed you had those sysreports there... Nvm, I'll look at those... Aki
Re: suse.de dovecot issues
Bernhard M. Wiedemann schreef op 2021-09-10 11:19: Hi, I am one of the people taking over our new suse.de email setup (consisting of dovecot+rspamd+postfix) and wanted to report some issues we experience: 1. we use dovecot-director to distribute users between 2 backend servers that share an NFS mount. We found that it proxies lmtp to a different backend than imap of the same user and that caused NFS stale-filehandle errors on the dovecot-uidlist. It then proceeds to re-generate the dovecot-uidlist with new UIDs that creates trouble for users. a) shouldn't dovecot use locks (fcntl or flock) to protect such files from concurrent updates? b) could it generate uidlist in a way that re-generating it, assigns the same UIDs again? E.g. via hash over file content c) how to get dovecot-director to send all traffic for a user to one backend? 2. We have 2 backends so that we can do maintenance on one of them while users can still access their emails through the other backend. However, we found that stopping dovecot on one backend left users unable to access their mails. Director doesn't do health checks. Maybe this is related to how user auth works? How to get this HA setup right, so that we don't have a single point of failure? grep PRETTY /etc/os-release PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" rpm -q dovecot23 dovecot23-2.3.15 https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. Ciao Bernhard M. -- With kind regards, William Edwards
Re: suse.de dovecot issues
> On 10/09/2021 12:19 Bernhard M. Wiedemann wrote: > > > Hi, > > I am one of the people taking over our new suse.de email setup > (consisting of dovecot+rspamd+postfix) > and wanted to report some issues we experience: > > > 1. > we use dovecot-director to distribute users between 2 backend servers > that share an NFS mount. > We found that it proxies lmtp to a different backend than imap of the > same user and that caused NFS stale-filehandle errors on the > dovecot-uidlist. > It then proceeds to re-generate the dovecot-uidlist with new UIDs that > creates trouble for users. > > a) shouldn't dovecot use locks (fcntl or flock) to protect such files > from concurrent updates? > > b) could it generate uidlist in a way that re-generating it, assigns the > same UIDs again? E.g. via hash over file content > > c) how to get dovecot-director to send all traffic for a user to one > backend? > > > 2. > We have 2 backends so that we can do maintenance on one of them while > users can still access their emails through the other backend. > However, we found that stopping dovecot on one backend left users unable > to access their mails. > Maybe this is related to how user auth works? > How to get this HA setup right, so that we don't have a single point of > failure? > > > grep PRETTY /etc/os-release > PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" > > rpm -q dovecot23 > dovecot23-2.3.15 > > https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. > > Ciao > Bernhard M. Can you post your `doveconf -n`? LMTP should not end up in different backend. Aki
suse.de dovecot issues
Hi, I am one of the people taking over our new suse.de email setup (consisting of dovecot+rspamd+postfix) and wanted to report some issues we experience: 1. we use dovecot-director to distribute users between 2 backend servers that share an NFS mount. We found that it proxies lmtp to a different backend than imap of the same user and that caused NFS stale-filehandle errors on the dovecot-uidlist. It then proceeds to re-generate the dovecot-uidlist with new UIDs that creates trouble for users. a) shouldn't dovecot use locks (fcntl or flock) to protect such files from concurrent updates? b) could it generate uidlist in a way that re-generating it, assigns the same UIDs again? E.g. via hash over file content c) how to get dovecot-director to send all traffic for a user to one backend? 2. We have 2 backends so that we can do maintenance on one of them while users can still access their emails through the other backend. However, we found that stopping dovecot on one backend left users unable to access their mails. Maybe this is related to how user auth works? How to get this HA setup right, so that we don't have a single point of failure? grep PRETTY /etc/os-release PRETTY_NAME="SUSE Linux Enterprise Server 15 SP3" rpm -q dovecot23 dovecot23-2.3.15 https://www.zq1.de/~bernhard/temp/dovecot/ has some sysreports. Ciao Bernhard M.
