Re: Issue with LMTP proxying and port number
Op 06/01/2019 om 19:35 schreef Stephan Bosch: Op 06/01/2019 om 17:02 schreef Steven Craig: On 1/6/2019 8:00 AM, Stephan Bosch wrote: Op 06/01/2019 om 11:50 schreef Stephan Bosch: Op 31/12/2018 om 23:59 schreef Steven Craig: Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. Yes, output from `dovecot -n` could be helpful. Also a literal dump of those errors in the log could help. Never mind, I can reproduce it here, even with master. We'll get back to you. Actually, this is equally problematic for Dovecot v2.2.10. I could reproduce this there too by my method (after some considerable difficulty to get that compiled on a modern system). So, first of all, I am wondering why this worked at your end in the first place and why it finally failed upon upgrade. The problem is that services like imap, pop3, managesieve, and submission define a literal default port for proxying, while lmtp really doesn't. Instead, LMTP uses the local port of the server as the default for the outgoing proxy connection. With unix services, there is no local port and hence no default, which instead results in using an (erroneous) port 0. Arguably, it would be useful to define a lmtp_proxy_port setting to have a default for situations when none other can be obtained. LMTP has no properly registered standard port, so hard-coding the default to 24 could be a bad idea. Tracking internally as DOP-868. Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 12/01/2019 om 23:08 schreef Stephan Bosch: Op 06/01/2019 om 17:05 schreef Stephan Bosch: Op 31/12/2018 om 06:32 schreef Laz C. Peterson: Hello Sami, yes, see below. We run Dovecot at a different versions, mainly 2.2.10 (CentOS), 2.2.22 (Ubuntu) and now 2.2.36 (CentOS). The issue is weird, because it only happened after the update from 2.2.10->36. Just to understand it would be great. I'm actually checking out the configs now ... Our SQL userdb does not specify port. So I'm guessing this may be to blame? (This was by design, though -- we don't want to specify one port for different client protocols. Though, I do recall seeing some hack online using CASE in SQL query ...) These servers run LMTP as a unix socket as well as a TCP port 24 serving all IP sources. The internal servers are running LMTP on TCP port 24 (as well as unix socket, but that's irrelevant), but no LMTP comm happens between directors and backend mail servers after the 2.2.10->36 update on the directors with our config. I do apologize that I can't get more specific than those versions ... The backend mail servers function the same in our environment on both versions 2.2.10 and 2.2.36. We are good now, as we changed the config to go to the TCP port instead of unix socket. But we had a good jolt of fun this morning. :-) Would love to understand what we have done wrong, or how we misunderstood the configuration directives -- in either version. I can reproduce it here, even with master. We'll get back to you. BTW, similar thread here: https://www.dovecot.org/pipermail/dovecot/2019-January/114071.html Hmm, did you try returning a protocol=lmtp field from passdb? This is ignored by services other than lmtp and the code tells me it will then default to port 24. That should be a workaround. Oh, right, this is v2.2. There, this apparently doesn't apply :/ Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 06/01/2019 om 17:05 schreef Stephan Bosch: Op 31/12/2018 om 06:32 schreef Laz C. Peterson: Hello Sami, yes, see below. We run Dovecot at a different versions, mainly 2.2.10 (CentOS), 2.2.22 (Ubuntu) and now 2.2.36 (CentOS). The issue is weird, because it only happened after the update from 2.2.10->36. Just to understand it would be great. I'm actually checking out the configs now ... Our SQL userdb does not specify port. So I'm guessing this may be to blame? (This was by design, though -- we don't want to specify one port for different client protocols. Though, I do recall seeing some hack online using CASE in SQL query ...) These servers run LMTP as a unix socket as well as a TCP port 24 serving all IP sources. The internal servers are running LMTP on TCP port 24 (as well as unix socket, but that's irrelevant), but no LMTP comm happens between directors and backend mail servers after the 2.2.10->36 update on the directors with our config. I do apologize that I can't get more specific than those versions ... The backend mail servers function the same in our environment on both versions 2.2.10 and 2.2.36. We are good now, as we changed the config to go to the TCP port instead of unix socket. But we had a good jolt of fun this morning. :-) Would love to understand what we have done wrong, or how we misunderstood the configuration directives -- in either version. I can reproduce it here, even with master. We'll get back to you. BTW, similar thread here: https://www.dovecot.org/pipermail/dovecot/2019-January/114071.html Hmm, did you try returning a protocol=lmtp field from passdb? This is ignored by services other than lmtp and the code tells me it will then default to port 24. That should be a workaround. Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 06/01/2019 om 17:02 schreef Steven Craig: Thanks, its a weird one. On 1/6/2019 8:00 AM, Stephan Bosch wrote: Op 06/01/2019 om 11:50 schreef Stephan Bosch: Op 31/12/2018 om 23:59 schreef Steven Craig: Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. Yes, output from `dovecot -n` could be helpful. Also a literal dump of those errors in the log could help. Never mind, I can reproduce it here, even with master. We'll get back to you. Actually, this is equally problematic for Dovecot v2.2.10. I could reproduce this there too by my method (after some considerable difficulty to get that compiled on a modern system). So, first of all, I am wondering why this worked at your end in the first place and why it finally failed upon upgrade. The problem is that services like imap, pop3, managesieve, and submission define a literal default port for proxying, while lmtp really doesn't. Instead, LMTP uses the local port of the server as the default for the outgoing proxy connection. With unix services, there is no local port and hence no default, which instead results in using an (erroneous) port 0. Arguably, it would be useful to define a lmtp_proxy_port setting to have a default for situations when none other can be obtained. LMTP has no properly registered standard port, so hard-coding the default to 24 could be a bad idea. Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 31/12/2018 om 06:32 schreef Laz C. Peterson: Hello Sami, yes, see below. We run Dovecot at a different versions, mainly 2.2.10 (CentOS), 2.2.22 (Ubuntu) and now 2.2.36 (CentOS). The issue is weird, because it only happened after the update from 2.2.10->36. Just to understand it would be great. I'm actually checking out the configs now ... Our SQL userdb does not specify port. So I'm guessing this may be to blame? (This was by design, though -- we don't want to specify one port for different client protocols. Though, I do recall seeing some hack online using CASE in SQL query ...) These servers run LMTP as a unix socket as well as a TCP port 24 serving all IP sources. The internal servers are running LMTP on TCP port 24 (as well as unix socket, but that's irrelevant), but no LMTP comm happens between directors and backend mail servers after the 2.2.10->36 update on the directors with our config. I do apologize that I can't get more specific than those versions ... The backend mail servers function the same in our environment on both versions 2.2.10 and 2.2.36. We are good now, as we changed the config to go to the TCP port instead of unix socket. But we had a good jolt of fun this morning. :-) Would love to understand what we have done wrong, or how we misunderstood the configuration directives -- in either version. I can reproduce it here, even with master. We'll get back to you. BTW, similar thread here: https://www.dovecot.org/pipermail/dovecot/2019-January/114071.html Regards, Stephan.
Re: Issue with LMTP proxying and port number
Thanks, its a weird one. On 1/6/2019 8:00 AM, Stephan Bosch wrote: Op 06/01/2019 om 11:50 schreef Stephan Bosch: Op 31/12/2018 om 23:59 schreef Steven Craig: Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. Yes, output from `dovecot -n` could be helpful. Also a literal dump of those errors in the log could help. Never mind, I can reproduce it here, even with master. We'll get back to you. Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 06/01/2019 om 11:50 schreef Stephan Bosch: Op 31/12/2018 om 23:59 schreef Steven Craig: Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. Yes, output from `dovecot -n` could be helpful. Also a literal dump of those errors in the log could help. Never mind, I can reproduce it here, even with master. We'll get back to you. Regards, Stephan.
Re: Issue with LMTP proxying and port number
Op 31/12/2018 om 23:59 schreef Steven Craig: Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. Yes, output from `dovecot -n` could be helpful. Also a literal dump of those errors in the log could help. Regards, Stephan.
Issue with LMTP proxying and port number
Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary.
Re: Issue with LMTP proxying and port number
Hello Sami, yes, see below.
We run Dovecot at a different versions, mainly 2.2.10 (CentOS), 2.2.22 (Ubuntu)
and now 2.2.36 (CentOS). The issue is weird, because it only happened after
the update from 2.2.10->36. Just to understand it would be great.
I'm actually checking out the configs now ... Our SQL userdb does not specify
port. So I'm guessing this may be to blame?
(This was by design, though -- we don't want to specify one port for different
client protocols. Though, I do recall seeing some hack online using CASE in
SQL query ...)
These servers run LMTP as a unix socket as well as a TCP port 24 serving all IP
sources. The internal servers are running LMTP on TCP port 24 (as well as unix
socket, but that's irrelevant), but no LMTP comm happens between directors and
backend mail servers after the 2.2.10->36 update on the directors with our
config. I do apologize that I can't get more specific than those versions ...
The backend mail servers function the same in our environment on both versions
2.2.10 and 2.2.36.
We are good now, as we changed the config to go to the TCP port instead of unix
socket. But we had a good jolt of fun this morning. :-)
Would love to understand what we have done wrong, or how we misunderstood the
configuration directives -- in either version.
Thank you Sami.
Here is the SQL:
driver = mysql
connect = host=10.1.1.110 dbname=someoneelsesmail user=notouruser
password=chicken
default_pass_scheme = PLAIN
password_query = SELECT CONCAT(a.username,'@',d.domain) AS
user,a.password,'captainmaster' AS master,'kentuckyfriedchicken' AS pass,'Y' AS
proxy FROM accounts a INNER JOIN domains d ON a.domain_id=d.id WHERE
a.username='%n' AND d.domain='%d' AND a.active=1 AND d.active=1
Here is the doveconf:
# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.24 (124e06aa)
doveconf: Warning: service auth { client_limit=5200 } is lower than required
under max. load (6144)
# OS: Linux 3.10.0-693.17.1.el7.centos.plus.x86_64 x86_64 CentOS Linux release
7.6.1810 (Core)
# Hostname: imapdir1.randomdomain.net
auth_master_user_separator = *
auth_mechanisms = plain login cram-md5
default_client_limit = 5200
default_process_limit = 1024
director_mail_servers = 10.9.5.131 10.9.5.132
director_servers = 10.9.5.151 10.9.5.152
disable_plaintext_auth = no
doveadm_port = 24245
first_valid_uid = 1000
lmtp_proxy = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext
mbox_write_locks = fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
quota = maildir:User quota
quota_grace = 10%%
quota_status_nouser = DUNNO
quota_status_overquota = 522 5.2.2 Mailbox is full
quota_status_success = DUNNO
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap pop3 lmtp sieve
service auth {
unix_listener auth-client {
mode = 0660
user = exim
}
}
service director {
fifo_listener login/proxy-notify {
mode = 0600
user = $default_login_user
}
inet_listener {
port = 9090
}
unix_listener director-userdb {
mode = 0600
}
unix_listener login/director {
mode = 0666
}
}
service doveadm {
inet_listener {
port = 24245
}
}
service imap-login {
executable = imap-login director
}
service ipc {
unix_listener ipc {
user = dovecot
}
}
service lmtp {
inet_listener lmtp {
port = 24
}
process_min_avail = 5
}
service managesieve-login {
executable = managesieve-login director
}
service pop3-login {
executable = pop3-login director
}
service quota-warning {
executable = script /etc/dovecot/quota-warning.sh
user = dovecot
}
ssl_cert =
wrote:
On 31 Dec 2018, at 0.43, Laz C. Peterson wrote: Hello
there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after
updating to version 2.2.36, our director servers are not able to proxy LMTP. We
are sending mail from exim to the local Dovecot LMTP socket, which then used to
send it to our internal mail servers on port 24. Besides the update, nothing
has changed in the config files. The configuration has been working great for
months … Then we had a great idea of doing holiday maintenance. (Currently, we
are working around this issue by having exim send mail to the IP and port,
instead of the socket.) Doing a tcpdump capture shows no TCP traffic either.
Re: Issue with LMTP proxying and port number
> On 31 Dec 2018, at 0.43, Laz C. Peterson wrote: > > Hello there, > > Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after > updating to version 2.2.36, our director servers are not able to proxy LMTP. > > We are sending mail from exim to the local Dovecot LMTP socket, which then > used to send it to our internal mail servers on port 24. > > Besides the update, nothing has changed in the config files. The > configuration has been working great for months … Then we had a great idea of > doing holiday maintenance. > > (Currently, we are working around this issue by having exim send mail to the > IP and port, instead of the socket.) > > Doing a tcpdump capture shows no TCP traffic either. The logs seem to say > that it is trying to go to port 0, so my guess is that it’s not getting the > port directive from either static or sql — since the docs state that if these > settings aren't specified, it will always use the same connection type for > the proxy that it received the connection on. > > Any thoughts? I can share config if necessary. > > ~ Laz Peterson > Paravis, LLC > can you please post doveconf -n and your sql userdb setup? Sami
Issue with LMTP proxying and port number
Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. ~ Laz Peterson Paravis, LLC
