Re: Timo - is the v2.3.15 GCC limitation really necessarily or it's just a bug?

[Adi Pircalabu]

On 29-07-2021 21:33, Mart Pirita wrote:

Hi,

This is very bad news.

I don't think that disto is old, if I can compile almost every software 
with it.


If that CentOS distro is EOL and/or you can't even find the source rpms 
to compile/rebuild them to retrofit patches addressing security 
vulnerabilities, with or without devtoolset, *THIS* is very, very bad 
news.

Just saying.

--
Adi Pircalabu

Re: Traffic accounting

[Adi Pircalabu]

On 20-07-2021 8:13, Jesús Ángel del Pozo Domínguez wrote:

Hello,

Could you please tell me whether is it possible to do traffic
accounting
using Dovecot 2.3.4?

What I'd like to do is to collect network traffic (both in & out) for
each user (both POP and IMAP traffic).

Regards,


It's possible, you need to adjust the IMAP & POP3 logging configuration, 
then parse the mail log and collect in/out values. E.g.:


doveconf -a | egrep '(imap|pop3).*logout_format'
imap_logout_format = rcvd=%i, sent=%o
imap_urlauth_logout_format = in=%i out=%o
pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, 
size=%s


egrep 'dovecot: service=imap, user=u...@domain.com.au.*Logged out' 
/var/log/maillog | tail -n 1
Jul 20 15:59:38 server dovecot: service=imap, user=u...@domain.com.au, 
ip=[127.0.0.1]. Logged out rcvd=38, sent=593


YMMV

--
Adi Pircalabu

Re: High Availability Dovecot / Roundcube / PostfixAdmin ?

[Adi Pircalabu]

On 09-07-2021 19:15, White, Daniel E. (GSFC-770.0)[NICS] wrote:

This is a new setup, running on RHEL 8 with the latest everything.

Has anyone out there set up a high availability pair of Dovecot
servers - with Roundcube and PostfixAdmin - successfully ?


Yes. NFS or GlusterFS for shared storage, Keepalived, Percona 
XtraDB+ProxySQL as database backend, Postfix, Dovecot, nginx as reverse 
proxy for Apache w. PHP-FPM. Suggest you start from the bottom up and 
*please* add monitoring for all services, look at Nagios/NRPE w. Percona 
monitoring plugins, more often than not a life saver. If things can go 
pear-shaped they will :)

Cheers,

--
Adi Pircalabu

Re: What imap ssl/auth settings work best with MS Outlook?

[Adi Pircalabu]

On 29-04-2021 23:08, @lbutlr wrote:

On 29 Apr 2021, at 03:22, Steve Dondley wrote:
I am totally unfamiliar with Exchange servers. What do they offer, 
exactly, that dovecot/postfix does not (besides a revenue stream for 
MS)?


A monthly stipend to Microsoft?

(I think they actuallyy do offer some useful tools for things like
meetings and calendars and such, including the 'feature' of being able
to automatically add people to your itinerary.)




Fact: Exchange (especially hosted) is 2010-ish, Office365 is the 
buzzword these days. Microsoft have been trying their best for quite 
some time now to cripple the IMAP support in Outlook as much as they can 
so that the email users will move their email business with o365 which - 
surprise surprise! - is s easy to autodiscover, autoconfigure, 
autothis, autothat. It's all about integrated services run by few well 
known powerful monopolies and it's only gonna get worse.




--
Adi Pircalabu

Re: Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

On 08-12-2020 13:18, John Fawcett wrote:

On 08/12/2020 01:01, Adi Pircalabu wrote:

On 08-12-2020 10:33, Adi Pircalabu wrote:

On 08-12-2020 9:41, John Fawcett wrote:

On 07/12/2020 23:22, John Fawcett wrote:

On 07/12/2020 23:09, Adi Pircalabu wrote:

On 08-12-2020 3:13, John Fawcett wrote:

On 07/12/2020 06:02, Adi Pircalabu wrote:

Hi,

I have a Dovecot proxy setup with several proxy machines 
(currently
running 2.3.11.3) in front of the real Dovecot servers 
(2.3.10.1)

storing the mailboxes. "doveconf -a | egrep lip" returns:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l
mpid=%e
%c session=<%{session}>

In the real server maillog I'm expecting to have "lip" replaced
with
the IP address of the proxy. It works as expected for imap-login
processes, however for pop3-login processes I still see the real
server IP instead of the proxy IP. Ideas?

Regards,


Hi Adi

in general people want to get the original ip not the proxied ip.
The
proxying of the original ip is done by a different method for 
imap

and pop3

https://wiki.dovecot.org/Design/ParameterForwarding

However, unless I'm reading this wrongly, both methods are
affected by
trusted_networks settings. I guess for people to help further, 
you'd

need to give more info your configuration settings.

Thanks John. login_trusted_networks, if this is the setting you're
referring to, lists the proxy IPs. I'd have thought, by having 
this
setting on the real servers, the proxy IP will be logged by both 
IMAP
and POP3 login processes, but it appears it isn't the case. It 
works

for IMAP, not for POP3.
The reason I need the proxy IP in the "lip" instead of the local
IP in
the real server mail log is that I need to filter certain
connections,
both IMAP and POP3, that are coming directly into the real server 
IP.
By capturing the IMAP & POP3 traffic on the real servers and 
matching
the results to the mail log entries I *should* be able to tell 
what

mail accounts from which remote IP addresses are coming in via the
proxies and which ones are coming into the real servers directly.
Hope
that makes sense.
Cheers,


The way I read it is that by specifing login_trusted_networks the
proxy
ip can be overwritten by the real ip. I think that's the opposite 
of

what you need.

I can't throw any light on why that is not working for imap but is
working for pop3. But as you don't want the overwriting, maybe you
should try without login_trusted_networks.

John

You're probably not getting the real ip logged for imap despite 
having

login_trusted_networks due to the default for imap_id_retain on the
proxies.

John


(Aki cc-ed)
Thanks. I actually need login_trusted_networks on the real servers so
that the real server has access to the client IP address, aka "rip" 
in

the log entry. What I need is consistent values for "lip" field for
both IMAP and POP3 login processes. Looking at
https://doc.dovecot.org/configuration_manual/proxy_settings/ there's
no mention the setting is working for IMAP only, not for POP3. What I
need for my use case is to get consistent logging for both protocols.
More precisely, considering:
- REALSERVER.IP as the real server IP address
- CLIENT.IP as the client IP address
- "login_trusted_networks = PROXY.IP" set in the real server config
I'm expecting to see the following information in the mail log of 
real

server for both IMAP and POP3 login processes:
user=, method=, rip=CLIENT.IP, lip=PROXY.IP, mpid=MPID,
TLS, session=
What I'm seeing instead is:
1. imap-login: user=, method=, rip=CLIENT.IP,
lip=PROXY.IP, mpid=MPID, TLS, session=
2. pop3-login: user=, method=, rip=CLIENT.IP,
lip=REALSERVER.IP, mpid=MPID, TLS, session=
If I didn't have "login_trusted_networks = PROXY.IP" I'd get
"rip=PROXY.IP" instead of "rip=CLIENT.IP" and this isn't what I want.
login_trusted_networks does its job just fine for the purpose, but I
was expecting it to effect "lip=%l" field for both IMAP and POP3
services in the same way.


Making some inroads here. Following
https://doc.dovecot.org/settings/core/#setting-login-log-format-elements
I'm now using:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e
%c session=<%{session}> real_rip=%{real_rip} real_lip=%{real_lip}

And these are the results in the real server mail log:

I. For connections coming via the proxy:
1. imap-login: user=, method=, rip=CLIENT.IP,
lip=PROXY.IP, mpid=MPID, TLS, session=, real_rip=PROXY.IP,
real_lip=REALSERVER.IP
2. pop3-login: user=, method=, rip=CLIENT.IP,
lip=REALSERVER.IP, mpid=MPID, TLS, session=,
real_rip=PROXY.IP, real_lip=REALSERVER.IP

II. For connections coming into the real server directly:
1. imap-login: user=, method=, rip=CLIENT.IP,
lip=REALSERVER.IP, mpid=MPID, TLS, session=,
real_rip=CLIENT.IP, real_lip=REALSERVER.IP
2. pop3-

Re: Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

On 08-12-2020 10:33, Adi Pircalabu wrote:

On 08-12-2020 9:41, John Fawcett wrote:

On 07/12/2020 23:22, John Fawcett wrote:

On 07/12/2020 23:09, Adi Pircalabu wrote:

On 08-12-2020 3:13, John Fawcett wrote:

On 07/12/2020 06:02, Adi Pircalabu wrote:

Hi,

I have a Dovecot proxy setup with several proxy machines 
(currently

running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1)
storing the mailboxes. "doveconf -a | egrep lip" returns:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l 
mpid=%e

%c session=<%{session}>

In the real server maillog I'm expecting to have "lip" replaced 
with

the IP address of the proxy. It works as expected for imap-login
processes, however for pop3-login processes I still see the real
server IP instead of the proxy IP. Ideas?

Regards,


Hi Adi

in general people want to get the original ip not the proxied ip. 
The

proxying of the original ip is done by a different method for imap
and pop3

https://wiki.dovecot.org/Design/ParameterForwarding

However, unless I'm reading this wrongly, both methods are affected 
by
trusted_networks settings. I guess for people to help further, 
you'd

need to give more info your configuration settings.

Thanks John. login_trusted_networks, if this is the setting you're
referring to, lists the proxy IPs. I'd have thought, by having this
setting on the real servers, the proxy IP will be logged by both 
IMAP

and POP3 login processes, but it appears it isn't the case. It works
for IMAP, not for POP3.
The reason I need the proxy IP in the "lip" instead of the local IP 
in
the real server mail log is that I need to filter certain 
connections,
both IMAP and POP3, that are coming directly into the real server 
IP.
By capturing the IMAP & POP3 traffic on the real servers and 
matching

the results to the mail log entries I *should* be able to tell what
mail accounts from which remote IP addresses are coming in via the
proxies and which ones are coming into the real servers directly. 
Hope

that makes sense.
Cheers,

The way I read it is that by specifing login_trusted_networks the 
proxy

ip can be overwritten by the real ip. I think that's the opposite of
what you need.

I can't throw any light on why that is not working for imap but is
working for pop3. But as you don't want the overwriting, maybe you
should try without login_trusted_networks.

John


You're probably not getting the real ip logged for imap despite having
login_trusted_networks due to the default for imap_id_retain on the 
proxies.


John


(Aki cc-ed)
Thanks. I actually need login_trusted_networks on the real servers so
that the real server has access to the client IP address, aka "rip" in
the log entry. What I need is consistent values for "lip" field for
both IMAP and POP3 login processes. Looking at
https://doc.dovecot.org/configuration_manual/proxy_settings/ there's
no mention the setting is working for IMAP only, not for POP3. What I
need for my use case is to get consistent logging for both protocols.
More precisely, considering:
- REALSERVER.IP as the real server IP address
- CLIENT.IP as the client IP address
- "login_trusted_networks = PROXY.IP" set in the real server config
I'm expecting to see the following information in the mail log of real
server for both IMAP and POP3 login processes:
user=, method=, rip=CLIENT.IP, lip=PROXY.IP, mpid=MPID,
TLS, session=
What I'm seeing instead is:
1. imap-login: user=, method=, rip=CLIENT.IP,
lip=PROXY.IP, mpid=MPID, TLS, session=
2. pop3-login: user=, method=, rip=CLIENT.IP,
lip=REALSERVER.IP, mpid=MPID, TLS, session=
If I didn't have "login_trusted_networks = PROXY.IP" I'd get
"rip=PROXY.IP" instead of "rip=CLIENT.IP" and this isn't what I want.
login_trusted_networks does its job just fine for the purpose, but I
was expecting it to effect "lip=%l" field for both IMAP and POP3
services in the same way.


Making some inroads here. Following 
https://doc.dovecot.org/settings/core/#setting-login-log-format-elements 
I'm now using:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c 
session=<%{session}> real_rip=%{real_rip} real_lip=%{real_lip}


And these are the results in the real server mail log:

I. For connections coming via the proxy:
1. imap-login: user=, method=, rip=CLIENT.IP, 
lip=PROXY.IP, mpid=MPID, TLS, session=, real_rip=PROXY.IP, 
real_lip=REALSERVER.IP
2. pop3-login: user=, method=, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=, real_rip=PROXY.IP, 
real_lip=REALSERVER.IP


II. For connections coming into the real server directly:
1. imap-login: user=, method=, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=, 
real_rip=CLIENT.IP, real_lip=REALSERVER.IP
2. pop3-login: user=, method=, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, se

Re: Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

On 08-12-2020 9:41, John Fawcett wrote:

On 07/12/2020 23:22, John Fawcett wrote:

On 07/12/2020 23:09, Adi Pircalabu wrote:

On 08-12-2020 3:13, John Fawcett wrote:

On 07/12/2020 06:02, Adi Pircalabu wrote:

Hi,

I have a Dovecot proxy setup with several proxy machines (currently
running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1)
storing the mailboxes. "doveconf -a | egrep lip" returns:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l 
mpid=%e

%c session=<%{session}>

In the real server maillog I'm expecting to have "lip" replaced 
with

the IP address of the proxy. It works as expected for imap-login
processes, however for pop3-login processes I still see the real
server IP instead of the proxy IP. Ideas?

Regards,


Hi Adi

in general people want to get the original ip not the proxied ip. 
The

proxying of the original ip is done by a different method for imap
and pop3

https://wiki.dovecot.org/Design/ParameterForwarding

However, unless I'm reading this wrongly, both methods are affected 
by

trusted_networks settings. I guess for people to help further, you'd
need to give more info your configuration settings.

Thanks John. login_trusted_networks, if this is the setting you're
referring to, lists the proxy IPs. I'd have thought, by having this
setting on the real servers, the proxy IP will be logged by both IMAP
and POP3 login processes, but it appears it isn't the case. It works
for IMAP, not for POP3.
The reason I need the proxy IP in the "lip" instead of the local IP 
in
the real server mail log is that I need to filter certain 
connections,

both IMAP and POP3, that are coming directly into the real server IP.
By capturing the IMAP & POP3 traffic on the real servers and matching
the results to the mail log entries I *should* be able to tell what
mail accounts from which remote IP addresses are coming in via the
proxies and which ones are coming into the real servers directly. 
Hope

that makes sense.
Cheers,

The way I read it is that by specifing login_trusted_networks the 
proxy

ip can be overwritten by the real ip. I think that's the opposite of
what you need.

I can't throw any light on why that is not working for imap but is
working for pop3. But as you don't want the overwriting, maybe you
should try without login_trusted_networks.

John


You're probably not getting the real ip logged for imap despite having
login_trusted_networks due to the default for imap_id_retain on the 
proxies.


John


(Aki cc-ed)
Thanks. I actually need login_trusted_networks on the real servers so 
that the real server has access to the client IP address, aka "rip" in 
the log entry. What I need is consistent values for "lip" field for both 
IMAP and POP3 login processes. Looking at 
https://doc.dovecot.org/configuration_manual/proxy_settings/ there's no 
mention the setting is working for IMAP only, not for POP3. What I need 
for my use case is to get consistent logging for both protocols. More 
precisely, considering:

- REALSERVER.IP as the real server IP address
- CLIENT.IP as the client IP address
- "login_trusted_networks = PROXY.IP" set in the real server config
I'm expecting to see the following information in the mail log of real 
server for both IMAP and POP3 login processes:
user=, method=, rip=CLIENT.IP, lip=PROXY.IP, mpid=MPID, 
TLS, session=

What I'm seeing instead is:
1. imap-login: user=, method=, rip=CLIENT.IP, 
lip=PROXY.IP, mpid=MPID, TLS, session=
2. pop3-login: user=, method=, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=
If I didn't have "login_trusted_networks = PROXY.IP" I'd get 
"rip=PROXY.IP" instead of "rip=CLIENT.IP" and this isn't what I want. 
login_trusted_networks does its job just fine for the purpose, but I was 
expecting it to effect "lip=%l" field for both IMAP and POP3 services in 
the same way.


Cheers,

--
Adi Pircalabu

Re: Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

On 08-12-2020 3:13, John Fawcett wrote:

On 07/12/2020 06:02, Adi Pircalabu wrote:

Hi,

I have a Dovecot proxy setup with several proxy machines (currently
running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1)
storing the mailboxes. "doveconf -a | egrep lip" returns:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e
%c session=<%{session}>

In the real server maillog I'm expecting to have "lip" replaced with
the IP address of the proxy. It works as expected for imap-login
processes, however for pop3-login processes I still see the real
server IP instead of the proxy IP. Ideas?

Regards,


Hi Adi

in general people want to get the original ip not the proxied ip. The
proxying of the original ip is done by a different method for imap and 
pop3


https://wiki.dovecot.org/Design/ParameterForwarding

However, unless I'm reading this wrongly, both methods are affected by
trusted_networks settings. I guess for people to help further, you'd
need to give more info your configuration settings.


Thanks John. login_trusted_networks, if this is the setting you're 
referring to, lists the proxy IPs. I'd have thought, by having this 
setting on the real servers, the proxy IP will be logged by both IMAP 
and POP3 login processes, but it appears it isn't the case. It works for 
IMAP, not for POP3.
The reason I need the proxy IP in the "lip" instead of the local IP in 
the real server mail log is that I need to filter certain connections, 
both IMAP and POP3, that are coming directly into the real server IP. By 
capturing the IMAP & POP3 traffic on the real servers and matching the 
results to the mail log entries I *should* be able to tell what mail 
accounts from which remote IP addresses are coming in via the proxies 
and which ones are coming into the real servers directly. Hope that 
makes sense.

Cheers,

--
Adi Pircalabu

Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

Hi,

I have a Dovecot proxy setup with several proxy machines (currently 
running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1) 
storing the mailboxes. "doveconf -a | egrep lip" returns:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c 
session=<%{session}>


In the real server maillog I'm expecting to have "lip" replaced with the 
IP address of the proxy. It works as expected for imap-login processes, 
however for pop3-login processes I still see the real server IP instead 
of the proxy IP. Ideas?


Regards,

--
Adi Pircalabu

Re: dovecot Digest, Vol 210, Issue 27

[Adi Pircalabu]

On 14-10-2020 14:54, webad...@exalt.com.au wrote:
I am investigating whether dovecot(https://github.com/dovecot/core/) 
handles

case insensitive Message-ID headers as per RFC.


Again, Dovecot has nothing to do with this in the context. Fix your SMTP 
client and the problem will go away.


And you seem to have more than one issue to deal with, your email has 
failed the SPF check at my end and went straight into Junk because my 
server has received it from 103.27.34.234, which isn't listed in the TXT 
record for exalt.com.au:


---CUT HERE---
Authentication-Results: mx1.quick.net.au; spf=softfail (mailfrom) 
smtp.mailfrom=exalt.com.au (client-ip=103.27.34.234; 
helo=se6.syd.hostingplatform.net.au; 
envelope-from=webad...@exalt.com.au; receiver=)
Received: from se6.syd.hostingplatform.net.au 
(se6.syd.hostingplatform.net.au [103.27.34.234])

(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
---CUT HERE---

dig exalt.com.au txt +short
"v=spf1 ip4:103.27.34.46 +a +mx +ip4:43.241.54.89 ~all"
"MS=ms59508959"

Cheers,

--
Adi Pircalabu

Re: DKIM fail if WHM adds Message-ID, should be Message-Id

[Adi Pircalabu]

On 12-10-2020 19:16, Robert Martin wrote:

I created a client library to send emails for a webapp.

After connecting to the SMTP server with credential setup in CPANEL,
and then do NOT add Message-Id header, the DKIM signature 'h' record
created by dovecot/WHM is wrong, and a Message-ID (with a capital D)
header is added, invalidating the generated DKIM signature value.

This causes outlook, yahoo, gmail and other email recipients to add
'dkim:fail' to the message, and thus relegate it to junk or spam.

The work around is to add to the message a Message-Id with a  little
'd' header.  Then the SMTP server processes the email with the correct
generated DKIM, correct DKIM 'h' record and does not add a Message-ID
header.

My SMTP hosting providers that run the WHM/dovecot/CPANEL software are
refusing to raise this as a bug and have requested that I do it.


Hardly a bug imo. Best to add the Message-Id header from your library 
and this will become a non-issue. There are certain filters who don't 
quite like emails without message-id header, or incorrectly formatting 
ones because this is an usual fingerprint of a broken client / spam bot, 
this is why your provider is adding that header.


--
Adi Pircalabu

Re: handling spam from gmail.

[Adi Pircalabu]

First thing first, this isn't necessary a Dovecot related thread and 
using a challenge-response system like the one suggested by the 
initiator ("click here if you're not yet another bloody SEO guru") is 
plain wrong for several reasons, having said that:


On 12-06-2020 11:56, Andreas Born wrote:

Am 12.06.2020 um 02:03 schrieb Ralph Seichter:

* Andreas Born:

[...]
For example: Postfix supports both before-queue filters and 
after-queue filters. Milter-regex[1] supports both multi-header and 
body checks.


Of course, and there is nothing wrong with it. It just runs into the
issue I tried to describe: incomplete SMTP implementations from MTAs.

Pre-queue filtering happens, before the mail was accepted to be
queued. So a before-queue milter can trigger an 5xx status code to
reject the mail. This code can be sent in response to steps 2, 3 or 4.
According to the smtp specs. But for many years it was code of
practice to send error/rejection codes latest after the RCPT TO
command, and at this time the milter, independent of what software you
use, has no information about email header or content. Rejecting a
mail AFTER the DATA command (when the content becomes available) was
discouraged because of incorrect behaving MTAs. (e.g. generating
backscatter, or even treating the mail as successfully sent)


$ telnet server 25
Trying x.x.x.x...
Connected to server
Escape character is '^]'.
220-server ESMTP Postfix <=== Postscreen trap here ;)
220 server ESMTP Postfix
HELO client.domain.com
250 server
MAIL FROM:<>
250 2.1.0 Ok
RCPT TO:
250 2.1.5 Ok
DATA
354 End data with .
From: Me
To: You
Subject: Test

SA GTube string here
.
550 5.7.1 Blocked, see you later.
QUIT
221 2.0.0 Bye
Connection closed by foreign host.

In this case the rejection comes after DATA, a content filter should be 
able to return either 4xx or 5xx *after* swallowing the entire email.



Maybe, and I really hope so, this problem no longer exists. I will
immediately reconfigure my mail system, if rejecting mails after DATA
will be safe and reliable nowadays.


Rejecting or deferring after DATA is perfectly fine these days. If the 
sending MTA, acting as a client in the SMTP conversation, doesn't behave 
properly to 5xx after DATA, it's not the recipient's MTA problem, the 
sender is broken and there's nothing the receiving MTA can do about it. 
Make it their problem, not yours.


--
Adi Pircalabu

Re: fail2ban setup centos 7 not picking auth fail?

[Adi Pircalabu]

On 22-05-2020 15:45, Voytek Eymont wrote:

On Fri, May 22, 2020 2:05 pm, Adi Pircalabu wrote:

On 22-05-2020 10:38, Voytek Eymont wrote:




Hardly a Dovecot issue. Can you please post the output of this 
command?

/usr/bin/fail2ban-regex /var/log/dovecot.log
/etc/fail2ban/filter.d/dovecot.conf



Adi,

thanks, what I get is:


[...]


Results
===

Failregex: 5149 total

[...]


Lines: 338975 lines, 0 ignored, 5149 matched, 333826 missed
[processed in 87.44 sec]


Right, so it's not a regex problem then, you're getting some matches 
there, although you might want to revisit it it the result is not 
consistent with your own searches. It might be that Dovecot isn't 
logging to systemd' journal, or the regex doesn't match the journal 
entries. Try to comment out "journalmatch = 
_SYSTEMD_UNIT=dovecot.service" entry in your filter file, restart f2b 
and see if there's any change.

P.S. Let's try and keep the replies to the list :)

--
Adi Pircalabu

Re: fail2ban setup centos 7 not picking auth fail?

[Adi Pircalabu]

On 22-05-2020 10:38, Voytek Eymont wrote:

I'm trying to set up fail2ban with dovecot, I have it working on 'old'
server Centos 6, but, not getting anywhere with 'new' server on Centos 
7

using standard filters

I've copied same 'filter' to new server, still get nothing
any idea how to figure this out ?

on old server, it logs to syslog/messages
CentOS release 6.10 (Final) dovecot 2.3.10.1 (a3d0e1171)

old # fail2ban-client status dovecot
Status for the jail: dovecot
|- Filter
|  |- Currently failed: 2
|  |- Total failed: 168
|  `- File list:/var/log/dovecot.log
`- Actions
   |- Currently banned: 0
   |- Total banned: 32
   `- Banned IP list:

on new server CentOS Linux release 7.8.2003 dovecot 2.3.10.1 
(a3d0e1171)

nothing shows up in fail2ban log (ssh, postfix does, only no dovecot)
I've copied the actual /etc/fail2ban/filter.d/dovecot.conf from old
server, still nothing
not sure where/how to look
is there a standard/approved doveot filter..?


Hardly a Dovecot issue. Can you please post the output of this command?
/usr/bin/fail2ban-regex /var/log/dovecot.log 
/etc/fail2ban/filter.d/dovecot.conf


--
Adi Pircalabu

Re: Can't connect to Managesieve Server - what´s wrong?

[Adi Pircalabu]

On 14-05-2020 8:28, Anton Blau wrote:

Am 14.05.2020 um 00:22 schrieb Benny Pedersen:

On 2020-05-14 00:14, Anton Blau wrote:


What can I do to get connect from Roundcube to dovecot-managemsieve?


if roundcube is installed on same server as dovecot then disable ssl
in roundcube managesive plugin as same way for imap and 
submission/smtps




Thank you for your very fast answer.

There are two virtual machines:

1. dovocot (postfix, rspamd)

2. roundcube


Doesn't appear to be a Dovecot issue. By default Roundcube's managesieve 
plugin connects to localhost, you may have to tinker with its 
configuration to specify the dovecot host *and* enable tls. See 
https://github.com/roundcube/roundcubemail/blob/master/plugins/managesieve/config.inc.php.dist


Cheers,

--
Adi Pircalabu

Re: Ms Exchange vs dovecot

[Adi Pircalabu]

On 13-05-2020 4:24, Sami Ketola wrote:

On 12. May 2020, at 19.18, Benny Pedersen  wrote:

On 2020-05-12 17:54, Robert Schetterer wrote:


At the end the subject question makes no sense...


lets play football then :)

i just wish that dovecot could be next generation exchange server, no 
kidding


Our parent company Open-Xchange offers one. It's called App Suite. 
Actually Dovecot Oy no longer exists as we are part of Open-Xchange 
now.



at the current state i get more on using cyrus-*

why was dovecot-oy even created ?


To provide paid support and consulting.


... and that's pretty end of thread ladies and gentlemen. There's no 
such thing as a free lunch, people still need to pay their bills at the 
end of the day and, every so often, some *really* great software such as 
Dovecot / Sieve blossoms out as a result of that :)


--
Adi Pircalabu

Re: Using dovecot Replication in a medium to large enterprise.

[Adi Pircalabu via dovecot]

On 2019-11-28 18:35, Brent Clark via dovecot wrote:

Good day Guys

Just wanted to pick the communities brain and experience(s) for a 
second.


At $CORP where I work. My team has inherited a single server mail 
solution.


We need to look to building a standby / replicated solution.

One of the things we were looking at is 
https://wiki.dovecot.org/Replication


You may want to look into a block device replication solution, like 
DRBD, integrated in a Pacemaker cluster. Can offer you a reliable, 
tested and resource friendly solution even in the simplest approaches 
consisting of a dual node, active-passive cluster.


--
Adi Pircalabu

Re: NFS Locking and Submission Service Authentication

[Adi Pircalabu via dovecot]

On 2019-09-26 03:44, Asai via dovecot wrote:

Greetings,

We're in the process of upgrading our Dovecot server to new hardware
and new expanded storage.  We planned on using an NFS share for the
mail storage, as we're running Postfix / Dovecot on a VM and wanted to
separate out the mail storage from the VM for backup reasons.

I read as much as I could find on line regarding configuring Dovecot
to use NFS, and set it up as best I could, but I'm still running into
lock errors e.g.:

Sep 25 10:30:35 triata4 dovecot:
imap(user@triata.globalchange.media)<75580>: Error:
fcntl(/vmail/triata.globalchange.media/user/dovecot.index.log,
write-lock, F_SETLKW) locking failed: No locks available
Sep 25 10:30:35 triata4 dovecot:
imap(user@triata.globalchange.media)<75580>: Error:
mail_index_wait_lock_fd() failed with file
/vmail/triata.globalchange.media/user/dovecot.index.log: No locks
available


How is your NFS export mounted on the client? Can you post the output of 
"egrep nfs /proc/mounts"?


--
Adi Pircalabu

Re: maildir very dirty sync option

[Adi Pircalabu via dovecot]

On 2019-08-20 17:05, Yousif Alkhateeb via dovecot wrote:

Hello ,

I have an active passive dovecot setup with glusterfs as a mail
storage and using maildirs, we used to have a problem when users with
large mailboxes sync their folders. This caused the load average to
increase in the server , after a while we have enabled the
very_dirty_sync option in dovecot , things got better and the problem
disappeared but we need to know if there is any thing else that we
need to do or know about the very_dirty_sync option that may cause
future problems .


Last time I tested Glusterfs as mail storage (stock RHEL 6 kernel, think 
6-7 years ago?) the performance on large maildirs was abysmal. We've 
ended up with DRBD & NFS (TCP, with UDP it'd freeze in less than half an 
hour under stress testing) and haven't looked back since. I know this 
isn't the answer you're looking for and I don't know how your deployment 
looks like, nor the scale, but I'm just chipping in. Tuning various 
Dovecot may be just kicking the can down the road, looking into 
alternate storage backends could be an option you should perhaps 
consider.

Cheers,

--
Adi Pircalabu

Re: index problems after update

[Adi Pircalabu via dovecot]

On 2019-02-21 22:18, Sami Ketola via dovecot wrote:
On 21 Feb 2019, at 12.23, Hajo Locke via dovecot  
wrote:
I think mbox+procmail is a classic setup and wide used and good 
solution for many usecases. Same setup we use many years.
We run ~2 mio mailboxes. our automated systems depends on this setup. 
creating mailboxes, managing mailboxes, creating automated 
filterrules, backupsystem to tell something of them. we can not switch 
our whole mailsetup to work around this bug.
How to get a dump if dovecot not crashing but has wrong behaviour? I 
would like to help and provide useful info, but it depends on kind of 
problem.
I think if a classic setup is not working in dovecot any more, this is 
a serious problem.


In you first email to this thread it says:

Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Fatal: master: 
service(imap): child 14135 killed with signal 6 (core dumped)


So imap is crashing and even dumping a core.

Also I must disagree with your mbox+procmail statement. mbox has
always been very unoptimised mailbox format and everyone should be
emphasised not to use it.
Also that combination has always had problems with indexing and file
locking. I would not use it on high volume mailservers. Or even medium
volume mailservers.


Not directly affected by this issue since I'm not using mbox for any 
production system nor have I for many years. And it'd take a lot of 
effort to convince me to use mbox for anything someone would even dare 
to classify, even remotely, as "production". But if I understand OP's 
point of view correctly, he's not arguing necessarily for or against a 
specific mailbox format. Instead, he's flagging a regression and people 
will be very reluctant to upgrade or even adopt a certain feature in a 
new release of a product if regressions are seen as acceptable. 
Something that previously worked in an otherwise unchanged environment 
stopped working after an upgrade and this is a regression. Trying to 
convince people to move away from mbox is a very sensible approach, I'm 
all for it, but in cases like this not practical.


--
Adi Pircalabu

Re: Password expiration: how to trigger it?

[Adi Pircalabu via dovecot]

On 2018-12-21 05:56, Cédric Jeanneret wrote:

Dear Dovecot Team,

I'm in the (long) process of migrating my whole email infrastructure. 
Of

course, dovecot is in the place, and is working just fine.

Still, I have an issue: password expiration.

I'm now using FreeIPA backend for the user authentication, and it
includes the capacity to expire passwords. Basically, it's an LDAP with
fancy things, among them a field named krbPasswordExpiration (yes,
that's kerberos).

In order to make things simple, I'd rather NOT force my users to set up
a kerberos/gssapi/whatever on their personal computer (most of them 
will

just have blank gaze if I start talking about that).

Is there a way to make Dovecot use that field? It's apparently a simple
date in %Y%M%D%H%m%sZ format, so a pretty neat thing to test. If 
there's

some support for that in Dovecot, that is.


One option would be the post login script, see:
https://wiki.dovecot.org/PostLoginScripting

Can also hook a password expiry check in dovecot-lda to send periodic 
reminders, although that's a bit unorthodox.


--
Adi Pircalabu

Re: Apple mail fails with Submission

[Adi Pircalabu via dovecot]

On 2018-12-19 03:17, Ruud Voorjans wrote:

Postfix debug peer logging

Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: >
server.example.org [4][XX.XX.XX.XX]: 250 2.1.5 Ok
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: watchdog_pat:
0x55ef4ec020180
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]:
vstream_fflush_some: fd 10 flush 28
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]:
vstream_buf_get_ready: fd 10 got 15
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: <
server.example.org [4]  [ XX.XX.XX.XX]: BDAT 326 LAST
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: match_string:
smtpd_forbidden_commands: bdat ~? connect
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: match_string:
smtpd_forbidden_commands: bdat ~? get
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: match_string:
smtpd_forbidden_commands: bdat ~? post
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]:
match_list_match: BDAT: no match
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: >
server.example.org [4]  [ XX.XX.XX.XX]  : 502 5.5.2 Error: command not
recognized
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: watchdog_pat:
0x55ef4ec020180
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]:
vstream_fflush_some: fd 10 flush 41
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]:
vstream_buf_get_ready: fd 10 got 326
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: <
server.example.org [4]  [ XX.XX.XX.XX]  : Content-Type: text/plain;
charset=us-ascii
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: warning:
non-SMTP command from  server.example.org [4]  [ XX.XX.XX.XX]  :
Content-Type: text/plain; charset=us-ascii
Dec 18 17:08:11 mail postfix/submission/smtpd[10626]: >
server.example.org [4]  [ XX.XX.XX.XX]  ]: 221 2.7.0 Error: I can
break rules, too. Goodbye.


Do you have the submission logs for the same timestamp? You server 
doesn't support BDAT command. However, looking at the logs below I have 
a suspicion your submission is advertising CHUNKING incorrectly. 
Misconfiguration or bug?

https://tools.ietf.org/html/rfc1830

--
Adi Pircalabu



Op di 18 dec. 2018 om 17:01 schreef Ruud Voorjans


doveconf -n output:
# 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-12-generic x86_64 Ubuntu 18.10
# Hostname: mail.example.org [1]
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
auth_verbose = yes
director_mail_servers = XX.XX.XX.XX
hostname = mail.example.org [1]
log_path = /var/log/dovecot.log
login_trusted_networks = XX.XX.XX.XX
mail_debug = yes
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = proxy=y host=XX.XX.XX nopassword=y
driver = static
}
protocols = imap submission
service director {
fifo_listener login/proxy-notify {
mode = 0600
user = $default_login_user
}
inet_listener {
port = 9090
}
unix_listener director-userdb {
mode = 0600
}
unix_listener login/director {
mode = 0666
}
}
service imap-login {
executable = imap-login director
}
service submission-login {
executable = submission-login
}
ssl = required
ssl_cert = 
AES256+EECDH:AES256+EDH:ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5

ssl_dh =  # hidden, use -P to show it
ssl_key =  # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes
submission_relay_host = XX.XX.XX.XX
submission_relay_rawlog_dir = /var/log/dovecot.log
submission_relay_trusted = yes
verbose_ssl = yes

Logging:
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
Connection created
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
Received new command: EHLO [10.225.11.41]
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO; 250 reply: Submitted
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO: Ready to reply
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
Trigger output
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
Sending replies
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO: Completed
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
Connection state reset
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO; 250 reply: Sent: 250-mail.example.org [3] 8BITMIME
BURL imap CHUNKING ENHANCEDSTATUSCODES SIZE STARTTLS PIPELINING
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO: Destroy
Dec 18 16:36:39 submission-login: Debug: smtp-server: conn   [0]:
command EHLO; 250 reply: Destroy
Dec 18 16:36:39 submission-login:

Re: Apple mail fails with Submission

[Adi Pircalabu via dovecot]

On 2018-12-18 07:33, Ruud Voorjans wrote:

Dear all,

I'm running dovecot # 2.3.2.1 - Pigeonhole version 0.5.2 () - OS:
Linux 4.18.0-12-generic x86_64 Ubuntu 18.10 with Submission.
It works great except with apple mail (Iphone).

I get an error with the MTA  (postfix):
""postfix/submission/smtpd[32552]: warning: non-SMTP command from
mail.example.org [1][xx.xx.xx.xx]: Content-Transfer-Encoding: 7bit""

with other mail-client(s) (Outlook (Desktop and Iphone app)) i have no
problem and it proxy-sends the e-mail beautiful out to the recipient.


Hardly anything to do with Dovecot. When it comes to email clients Apple 
Mail has been and is still one of the worst flops (no offence intended, 
just my opinion based on personal experience). If you can reliably 
reproduce it, try and log the raw SMTP conversation between Postfix and 
the client by enabling per IP debugging in Postfix:

postconf -e "debug_peer_level = 20"
postconf -e "debug_peer_list = xx.xx.xx.xx"
postfix reload
where xx.xx.xx.xx is the unlucky client IP address.

Possibly some crappy SMTP PIPELINING implementation at the Apple end, 
who knows.


--
Adi Pircalabu

Re: huge increase in storage activity afther dovecot upgrade

[Adi Pircalabu]

On 2018-11-16 07:24, Adrian Minta wrote:

Yes, multiple imap servers using one shared nfs storage. With the same
config on 2.2.13 the public interface traffic was similar to the
storage interface, around 100 mbps.

After we switch to 2.2.27 the storage interface traffic jumped 10
times while the public interface stayed the same. This make us
thinking that something is wrong and each time a user logs in the
whole Inbox content is read by dovecot.

What you are suggesting goes against the documentations and it may not
be save, but I will give a  thought.


I was expecting you have multiple IMAP servers using the same shared NFS 
storage, however my question was: are the *individual mailboxes* on that 
share accessed *at the same time* from more than one IMAP server?


--
Adi Pircalabu


On 11/15/18 6:23 AM, Adi Pircalabu wrote:
Are you connecting to the same mailbox over NFS from multiple IMAP 
servers? If not and, at any given time, any mailbox will be accessed 
from a single NFS client, try to "dupe" Dovecot into thinking it's not 
using NFS. We're running quite successfully such setup with NFSv3 over 
TCP, which turned out to be the fastest and most reliable throughout 
the years. Here are the mount options:
rw,noatime,nodiratime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountvers=3,mountport=1892,mountproto=tcp,local_lock=none 
On the Dovecot side we're running with:

lock_method = dotlock
mail_fsync = never
mail_nfs_index = no
mail_nfs_storage = no
maildir_very_dirty_syncs = yes
mmap_disable = yes
protocol lda {
  mail_fsync = optimized
}
protocol lmtp {
  mail_fsync = optimized
}

Note: we're using Maildir and the usual "works for me(c), may not work 
for everyone" applies.

Re: huge increase in storage activity afther dovecot upgrade

[Adi Pircalabu]

Are you connecting to the same mailbox over NFS from multiple IMAP 
servers? If not and, at any given time, any mailbox will be accessed 
from a single NFS client, try to "dupe" Dovecot into thinking it's not 
using NFS. We're running quite successfully such setup with NFSv3 over 
TCP, which turned out to be the fastest and most reliable throughout the 
years. Here are the mount options:

rw,noatime,nodiratime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountvers=3,mountport=1892,mountproto=tcp,local_lock=none
On the Dovecot side we're running with:
lock_method = dotlock
mail_fsync = never
mail_nfs_index = no
mail_nfs_storage = no
maildir_very_dirty_syncs = yes
mmap_disable = yes
protocol lda {
  mail_fsync = optimized
}
protocol lmtp {
  mail_fsync = optimized
}

Note: we're using Maildir and the usual "works for me(c), may not work 
for everyone" applies.


--
Adi Pircalabu

On 2018-11-14 21:47, Adrian M wrote:

Thanks, they are as in example, except for "mailbox_list_index = yes"
witch is from https://wiki.dovecot.org/PerformanceTuning

On Wed, Nov 14, 2018 at 12:18 PM Aki Tuomi
 wrote:


You should review https://wiki2.dovecot.org/NFS to see that the
settings make sense.

Aki
On 14.11.2018 12.00, Adrian M wrote:

Thank you !
I was little concerned that the following settings are not in line
with the new version:

mail_nfs_index = yes
mail_nfs_storage = yes
mail_fsync = always
mailbox_list_index = yes
maildir_stat_dirs = yes
mmap_disable = yes

On Wed, Nov 14, 2018 at 10:19 AM Aki Tuomi
 wrote:

It should eventually wind down once all the problems are fixed. Of
course if it does not happen, you can always run force-resync for
the problem users.

Aki
On 14.11.2018 10.08, Adrian M wrote:

Hi,
we upgraded our servers from version 2.2.13 to 2.2.27. After the
upgrade we notice a 10x increase in traffic with the nfs storage an
errors like this in the logfile:

Nov 12 09:48:16 mail dovecot: imap(...): Error: Corrupted index
cache file /.../dovecot.index.cache: invalid record size
Nov 12 09:48:16 mail dovecot: imap(...): Error:
unlink(/.../dovecot.index.cache) failed: No such file or directory
(in mail-cache.c:29)
Nov 12 09:48:16 mail dovecot: imap(...): Error: Corrupted index
cache file /.../dovecot.index.cache: invalid record size
Nov 12 09:48:16 mail dovecot: imap(...): Error: Broken file
/.../dovecot-uidlist line 8: Invalid data:

Is this normal ?

Will the activity wind down ?
Can we do something, like deleting the old dovecot.index* or
dovecot-uidlist files from maildirs, or doing an doveadm
force-refresh for all inboxes ?

Thank you !

Here's my configuration:

# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 nfs
auth_failure_delay = 15 secs
auth_mechanisms = plain login
auth_verbose = yes
auth_worker_max_count = 256
default_client_limit = 4
default_process_limit = 512
dict {
lastlogin = mysql:/etc/dovecot/mysql/dovecot-dict-lastlogin.conf
quotadict = mysql:/etc/dovecot/mysql/dovecot-dict-quota.conf
}
disable_plaintext_auth = no
first_valid_uid = 100
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pid=%p
%c
mail_fsync = always
mail_location = maildir:_/home/virtual/_%d/%u
mail_max_userip_connections = 16
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = zlib quota mail_log notify
mail_privileged_group = mail
mailbox_list_index = yes
maildir_stat_dirs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
mmap_disable = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Spam {
auto = subscribe
special_use = \Junk
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
mailbox virtual/All {
special_use = \All
}
prefix =
}
passdb {
args = /etc/dovecot/mysql/dovecot-sql.conf
driver = sql
}
plugin {
last_login_dict = proxy::lastlogin
last_login_key = last-login/%u
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename
mail_log_fields = uid box msgid size
quota = dict:user::proxy::quotadict
quota_rule2 = Trash:ignore
quota_rule3 = Spam:ignore
quota_status_nouser = DUNNO
quota_status_overquota = 552 5.2.2 The email account that you
tried to reach is over quota
quota_status_success = DUNNO
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
sieve = ~/.dovecot.sieve
sieve_before = /etc/dovecot/sieve/default.sieve
sieve_dir = ~/sieve
stats_refresh = 30 secs
stats_track_cmds = yes
trash

Re: Dovecot proxy: per user/domain 'namespace/inbox/prefix' from MySQL

[Adi Pircalabu]

Forgot to add "doveconf -n" for the proxy server:

# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.24 (124e06aa)
# OS: Linux 4.14.81-6.el7xen.x86_64 x86_64 CentOS Linux release 7.5.1804 
(Core)

# Hostname: proxy1.0aditest.local
auth_cache_negative_ttl = 5 mins
auth_cache_size = 16 M
auth_cache_ttl = 18 hours
auth_debug = yes
auth_verbose = yes
mail_debug = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapflags notify

mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_extensions = +notify +imapflags
}
protocols = imap pop3 lmtp sieve
service imap-login {
  inet_listener imap {
port = 1143
  }
  inet_listener imaps {
port = 1993
ssl = yes
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 0
  vsz_limit = 128 M
}
service managesieve {
  process_limit = 1024
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
port = 995
ssl = yes
  }
}
ssl = required
ssl_cert = 
As a way to try and avoid using "prefix = INBOX." ad infinitum for the
inbox namespace, I'm looking for ways to move on to "prefix =" for new
mail accounts, and grandfather the existing ones. Previously running
Courier-IMAP, now Dovecot, I looked at
https://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_Courier_IMAP
and decided it's too risky to go down that path and use namespace
compat, with so many IMAP clients out there the scope of testing is
huge and the outcome is uncertain and not worth it.
After reading
https://wiki.dovecot.org/Namespaces#Per-user_Namespace_Location_From_SQL
I thought I might be able to overwrite the server configuration per
user returning 'namespace/inbox/prefix' value from SQL. Here's the
setup I attempted, briefly:

1. Client connects to the Dovecot proxy, which authenticates the user
and proxies to the backend using a query like this in
/etc/dovecot/conf.d/dovecot-sql.conf.ext:
driver = mysql
connect = 
password_query = SELECT NULL AS password, 'Y' as nopassword, host,
'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE email = '%u'
AND disabled_smtpauth=0
Works a treat.

2. Next, I'm trying to add the prefix lookup in the picture. In the
same file I've added:
user_query = SELECT ns_inbox_prefix AS 'namespace/inbox/prefix' FROM
mailbox WHERE email = '%u' AND disabled_smtpauth=0

3. The mailbox table schema reads:
CREATE TABLE `mailbox` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `email` varchar(255) NOT NULL DEFAULT '',
  `password` varchar(255) NOT NULL DEFAULT '',
  `clear_password` varchar(255) NOT NULL DEFAULT '',
  `name` varchar(255) NOT NULL DEFAULT '',
  `host` varchar(32) DEFAULT NULL,
  `port` varchar(32) DEFAULT NULL,
  `ns_inbox_prefix` varchar(255) NOT NULL DEFAULT '',
  `lastlog_remote_ips` bigint(20) unsigned NOT NULL DEFAULT 0,
  `curlog_remote_ips` bigint(20) unsigned NOT NULL DEFAULT 0,
  `disabled_smtpauth` tinyint(1) NOT NULL DEFAULT 0,
  `last_modified` timestamp NOT NULL DEFAULT current_timestamp() ON
UPDATE current_timestamp(),
  PRIMARY KEY (`id`),
  UNIQUE KEY `email` (`email`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
The 2 queries above return:
MariaDB [postfix]> SELECT NULL AS password, 'Y' as nopassword, host,
'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE email =
'adi2@0aditest.local' AND disabled_smtpauth=0;
+--+++--+---+
| password | nopassword | host   | starttls | proxy |
+--+++--+---+
| NULL | Y  | 192.168.123.24 | any-cert | Y |
+--+++--+---+
1 row in set (0.00 sec)
MariaDB [postfix]> SELECT ns_inbox_prefix AS 'namespace/inbox/prefix'
FROM mailbox WHERE email = 'adi2@0aditest.local' AND
disabled_smtpauth=0;
++
| namespace/inbox/prefix |
++
||
++
1 row in set (0.00 sec)

After reloading dovecot service with auth_debug = yes are the maillog
for an IMAP session:
Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Loading modules from
directory: /usr/lib64/dovecot/auth
Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Module loaded:
/usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so
Nov 15 12:43:48 proxy1 dov

Dovecot proxy: per user/domain 'namespace/inbox/prefix' from MySQL

[Adi Pircalabu]

ox WHERE email = 
'adi2@0aditest.local' AND disabled_smtpauth=0
Nov 15 12:43:53 proxy1 dovecot: auth: Debug: client passdb out: 
OK#0111#011user=adi2@0aditest.local#011host=192.168.123.24#011starttls=any-cert#011proxy#011pass=

Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...]
Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...]
Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...]
Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...]
Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...]
Nov 15 12:43:53 proxy1 dovecot: imap-login: proxy(adi2@0aditest.local): 
started proxying to 192.168.123.24:143: user=, 
method=PLAIN, rip=::1, lip=::1, secured, 
session=


Looks like user_query isn't executed, why? And here's the corresponding 
IMAP session:


Trying ::1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE 
IDLE STARTTLS AUTH=PLAIN] Dovecot ready.

. LOGIN adi2@0aditest.local 
. OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE 
IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS 
THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN 
NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH 
ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY 
SPECIAL-USE QUOTA] Logged in

. NAMESPACE
* NAMESPACE (("INBOX." ".")) NIL NIL
. OK Namespace completed (0.000 + 0.000 secs).
. LIST "" *
* LIST (\HasChildren) "." INBOX
* LIST (\HasNoChildren \Trash) "." INBOX.Trash
* LIST (\HasNoChildren) "." INBOX.Templates
* LIST (\HasNoChildren \Sent) "." INBOX.Sent
* LIST (\HasNoChildren \Drafts) "." INBOX.Drafts
* LIST (\HasNoChildren \Archive) "." INBOX.Archives
* LIST (\HasNoChildren \UnMarked \Junk) "." INBOX.Spam
. OK List completed (0.000 + 0.000 secs).
. LSUB "" *
* LSUB (\Archive) "." INBOX.Archives
* LSUB (\Drafts) "." INBOX.Drafts
* LSUB (\Sent) "." INBOX.Sent
* LSUB (\Junk) "." INBOX.Spam
* LSUB () "." INBOX.Templates
* LSUB (\Trash) "." INBOX.Trash
. OK Lsub completed (0.000 + 0.000 secs).
. LOGOUT
* BYE Logging out
. OK Logout completed (0.000 + 0.000 secs).
Connection closed by foreign host.

How do I overwrite 'namespace/inbox/prefix' for an user on the Dovecot 
proxy? Is user_query working in this context?


--
Adi Pircalabu

Re: Trying to do antispam with Sieve

[Adi Pircalabu]

For the archives: after reading 
https://www.dovecot.org/list/dovecot/2017-February/107039.html I found & 
fixed the issue, it appears I *must* use the inbox prefix, hence the 
configuration should be:


plugin {
  sieve_plugins = sieve_imapsieve sieve_extprograms
  imapsieve_mailbox1_name = INBOX.Spam
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_before = 
file:/usr/lib64/dovecot/sieve/report-spam.sieve

  imapsieve_mailbox2_name = *
  imapsieve_mailbox2_from = INBOX.Spam
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_before = 
file:/usr/lib64/dovecot/sieve/report-ham.sieve

  sieve_pipe_bin_dir = /usr/lib64/dovecot/sieve
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
}

Perhaps worth adding a note to 
https://wiki.dovecot.org/HowTo/AntispamWithSieve for this case?


--
Adi Pircalabu

On 2018-11-14 15:08, Adi Pircalabu wrote:

On 2018-11-14 14:25, Adi Pircalabu wrote:

On 2018-11-14 13:51, Adi Pircalabu wrote:

On 2018-11-14 13:11, Adi Pircalabu wrote:

Hi,

Using https://wiki.dovecot.org/HowTo/AntispamWithSieve I'm trying to
execute scripts when moving to/from Spam folder, however nothing's
happening. The actions are:
1. Move to Spam: redirect :copy "spamcop_spam@domain.local";
2. Move from Spam: redirect :copy "spamcop_ham@domain.local";

[...]

/usr/lib64/dovecot/sieve/report-spam.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_spam@domain.local";
/usr/lib64/dovecot/sieve/report-ham.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_ham@domain.local";


More information after enabling mail_debug, in maillog I see:
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Mail set keywords
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX.Spam: FLAG event (changed
flags: Junk)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX: MOVE event
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)

So imapsieve "sees" the configuration, then I went and enabl

Re: Trying to do antispam with Sieve

[Adi Pircalabu]

On 2018-11-14 14:25, Adi Pircalabu wrote:

On 2018-11-14 13:51, Adi Pircalabu wrote:

On 2018-11-14 13:11, Adi Pircalabu wrote:

Hi,

Using https://wiki.dovecot.org/HowTo/AntispamWithSieve I'm trying to
execute scripts when moving to/from Spam folder, however nothing's
happening. The actions are:
1. Move to Spam: redirect :copy "spamcop_spam@domain.local";
2. Move from Spam: redirect :copy "spamcop_ham@domain.local";

[...]

/usr/lib64/dovecot/sieve/report-spam.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_spam@domain.local";
/usr/lib64/dovecot/sieve/report-ham.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_ham@domain.local";


More information after enabling mail_debug, in maillog I see:
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Mail set keywords
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX.Spam: FLAG event (changed
flags: Junk)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX: MOVE event
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)

So imapsieve "sees" the configuration, then I went and enabled
debugging in the 2 sieve scripts which now read:
1. /usr/lib64/dovecot/sieve/report-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", 
"vnd.dovecot.debug"];

debug_log "/var/tmp/report-spam.sieve.debug";
redirect :copy "spamcop_spam@domain.local";
2. /usr/lib64/dovecot/sieve/report-ham.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", 
"vnd.dovecot.debug"];

debug_log "/var/tmp/report-ham.sieve.debug";
redirect :copy "spamcop_ham@domain.local";

Should I expect to see debugging in /var/tmp/report-ham.sieve.debug
and /var/tmp/report-spam.sieve.debug, respectively? The 2 files aren't
created, nothing in that directory.
BTW, getenforce=Disabled.


Tried with another set of ham/spam scripts that are supposed to log to
syslog. 

Re: Trying to do antispam with Sieve

[Adi Pircalabu]

On 2018-11-14 13:51, Adi Pircalabu wrote:

On 2018-11-14 13:11, Adi Pircalabu wrote:

Hi,

Using https://wiki.dovecot.org/HowTo/AntispamWithSieve I'm trying to
execute scripts when moving to/from Spam folder, however nothing's
happening. The actions are:
1. Move to Spam: redirect :copy "spamcop_spam@domain.local";
2. Move from Spam: redirect :copy "spamcop_ham@domain.local";

[...]

/usr/lib64/dovecot/sieve/report-spam.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_spam@domain.local";
/usr/lib64/dovecot/sieve/report-ham.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_ham@domain.local";


More information after enabling mail_debug, in maillog I see:
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Mail set keywords
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX.Spam: FLAG event (changed
flags: Junk)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: Module loaded:
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: mailbox INBOX: MOVE event
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554)
initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole
version 0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam'
from=`*' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local,
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*'
from=`Spam' causes=(COPY) =>
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)

So imapsieve "sees" the configuration, then I went and enabled
debugging in the 2 sieve scripts which now read:
1. /usr/lib64/dovecot/sieve/report-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "vnd.dovecot.debug"];
debug_log "/var/tmp/report-spam.sieve.debug";
redirect :copy "spamcop_spam@domain.local";
2. /usr/lib64/dovecot/sieve/report-ham.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "vnd.dovecot.debug"];
debug_log "/var/tmp/report-ham.sieve.debug";
redirect :copy "spamcop_ham@domain.local";

Should I expect to see debugging in /var/tmp/report-ham.sieve.debug
and /var/tmp/report-spam.sieve.debug, respectively? The 2 files aren't
created, nothing in that directory.
BTW, getenforce=Disabled.


Tried with another set of ham/spam scripts that are supposed to log to 
syslog. Now using:

1. log-ham.sieve which contains:
require [&

Re: Trying to do antispam with Sieve

[Adi Pircalabu]

On 2018-11-14 13:11, Adi Pircalabu wrote:

Hi,

Using https://wiki.dovecot.org/HowTo/AntispamWithSieve I'm trying to
execute scripts when moving to/from Spam folder, however nothing's
happening. The actions are:
1. Move to Spam: redirect :copy "spamcop_spam@domain.local";
2. Move from Spam: redirect :copy "spamcop_ham@domain.local";

[...]

/usr/lib64/dovecot/sieve/report-spam.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_spam@domain.local";
/usr/lib64/dovecot/sieve/report-ham.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_ham@domain.local";


More information after enabling mail_debug, in maillog I see:
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: Module loaded: 
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: Mail set keywords
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: mailbox INBOX.Spam: FLAG event (changed 
flags: Junk)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554) initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is 
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' 
from=`*' causes=(COPY) => 
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' 
from=`Spam' causes=(COPY) => 
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: Module loaded: 
/usr/libexec/dovecot/modules/lib95_imap_sieve_plugin.so
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: mailbox INBOX: MOVE event
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Pigeonhole version 0.4.16 (fed8554) initializing
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: include: sieve_global is not set; it is 
currently not possible to include `:global' scripts.
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 
0.4.16 (fed8554) loaded
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' 
from=`*' causes=(COPY) => 
before=`file:/usr/lib64/dovecot/sieve/report-spam.sieve' after=(none)
Nov 14 13:41:52 plesk12 dovecot: service=imap, user=adi1@adit1.local, 
ip=[::1]. Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' 
from=`Spam' causes=(COPY) => 
before=`file:/usr/lib64/dovecot/sieve/report-ham.sieve' after=(none)


So imapsieve "sees" the configuration, then I went and enabled debugging 
in the 2 sieve scripts which now read:

1. /usr/lib64/dovecot/sieve/report-spam.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "vnd.dovecot.debug"];
debug_log "/var/tmp/report-spam.sieve.debug";
redirect :copy "spamcop_spam@domain.local";
2. /usr/lib64/dovecot/sieve/report-ham.sieve
require ["vnd.dovecot.pipe", "copy", "imapsieve", "vnd.dovecot.debug"];
debug_log "/var/tmp/report-ham.sieve.debug";
redirect :copy "spamcop_ham@domain.local";

Should I expect to see debugging in /var/tmp/report-ham.sieve.debug and 
/var/tmp/report-spam.sieve.debug, respectively? The 2 files aren't 
created, nothing in that directory.

BTW, getenforce=Disabled.

--
Adi Pircalabu

Trying to do antispam with Sieve

[Adi Pircalabu]

Hi,

Using https://wiki.dovecot.org/HowTo/AntispamWithSieve I'm trying to 
execute scripts when moving to/from Spam folder, however nothing's 
happening. The actions are:

1. Move to Spam: redirect :copy "spamcop_spam@domain.local";
2. Move from Spam: redirect :copy "spamcop_ham@domain.local";

Here's the configuration I'm working with:
doveconf: Warning: service anvil { client_limit=1000 } is lower than 
required under max. load (1153)

# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.14.80-6.el7xen.x86_64 x86_64 CentOS Linux release 7.5.1804 
(Core)  ext4

auth_mechanisms = plain login digest-md5 cram-md5 apop
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@'

disable_plaintext_auth = no
first_valid_uid = 30
imap_client_workarounds = delay-newmail
imap_logout_format = rcvd=%i, sent=%o
mail_attribute_dict = file:/var/qmail/mailnames/%Ld/dovecot-attributes
mail_fsync = never
mail_home = /var/qmail/mailnames/%Ld/%Ln
mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir
mail_log_prefix = "service=%s, user=%u, ip=[%r]. "
mail_max_userip_connections = 100
mail_plugins = " quota"
mailbox_list_index = yes
maildir_very_dirty_syncs = yes
managesieve_logout_format = rcvd=%i, sent=%o
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapflags notify imapsieve

mmap_disable = yes
namespace {
  hidden = no
  list = children
  location = 
maildir:/var/qmail/mailnames/%Ld/%%Ln/Maildir:INDEXPVT=/var/qmail/mailnames/%Ld/%Ln/user/%%u/Maildir

  prefix = Other Users.%%n.
  separator = .
  subscriptions = no
  type = shared
}
namespace {
  list = children
  location = 
maildir:/var/qmail/mailnames/%Ld/public/Maildir:INDEXPVT=/var/qmail/mailnames/%Ld/%Ln/public/Maildir

  prefix = Public.
  separator = .
  subscriptions = no
  type = public
}
namespace inbox {
  inbox = yes
  location =
  mailbox Archives {
auto = subscribe
special_use = \Archive
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Spam {
auto = subscribe
autoexpunge = 90 days
special_use = \Junk
  }
  mailbox Templates {
auto = subscribe
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix = INBOX.
  separator = .
  type = private
}
passdb {
  driver = plesk
}
plugin {
  acl = vfile
  acl_shared_dict = file:/var/qmail/mailnames/%Ld/shared-mailboxes
  imapsieve_mailbox1_before = 
file:/usr/lib64/dovecot/sieve/report-spam.sieve

  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Spam
  imapsieve_mailbox2_before = 
file:/usr/lib64/dovecot/sieve/report-ham.sieve

  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Spam
  imapsieve_mailbox2_name = *
  quota = maildir:User quota
  quota_grace = 0
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=99%% quota-warning 99 %u
  sieve = ~/.dovecot.sieve
  sieve_after = /etc/dovecot/sieve/after
  sieve_dir = ~/sieve
  sieve_extensions = +notify +imapflags
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
  sieve_pipe_bin_dir = /usr/lib64/dovecot/sieve
  sieve_plugins = sieve_imapsieve sieve_extprograms
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, 
size=%s

protocols = imap pop3 sieve
service auth-worker {
  group =
  user =
}
service auth {
  group =
  unix_listener auth-userdb {
group = popuser
mode = 0600
user = popuser
  }
  user =
}
service imap-login {
  process_limit = 850
  service_count = 1
}
service imap {
  process_limit = 700
  service_count = 1
}
service pop3 {
  process_limit = 700
  service_count = 1
}
service quota-warning {
  executable = script /usr/local/bin/mail-quota-warning.sh
  group = popuser
  unix_listener quota-warning {
group = popuser
user = popuser
  }
  user = popuser
}
ssl_cert = require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_spam@domain.local";
/usr/lib64/dovecot/sieve/report-ham.sieve contains:
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", 
"variables"];

redirect :copy "spamcop_ham@domain.local";

I must be missing something obvious. Thanks,

--
Adi Pircalabu

Re: online conversion using replication?

[Adi Pircalabu]

On 9/4/18 4:49 AM, B. Reino wrote:

On Mon, 3 Sep 2018, Sami Ketola wrote:


On 3 Sep 2018, at 4.18, Daniel Miller wrote:

That works for a one-time migration, or perhaps via a cron-job, but 
what I want is basically a constant one-way backup and it seems 
replication could do it more elegantly & efficiently.




So you want real-time archiving? What we have done with couple of 
customers is that we just configure MTA to replicate all incoming 
mails to secondary site.


Would you mind showing how you're doing it?
(hopefully with postfix, otherwise it may not be so interesting to me..)

Thanks.


See postfix always_bcc[1] parameter, as well as sender_bcc_maps and 
recipient_bcc_maps for fine grained adjustments.


[1] http://www.postfix.org/postconf.5.html

--
Adi Pircalabu

Re: Best practices for backing up small mailserver to remote location

[Adi Pircalabu]

On 09-08-2018 10:05, Kenneth Porter wrote:

On 8/7/2018 5:08 PM, Adi Pircalabu wrote:
- Since you're on dynamic IP at home, set up a VPN tunnel using the 
mailserver as server and HTPC as client. OpenVPN is ubiquitous and 
widely supported.

- rsync your mailboxes using the tunnel connection.
This way you can back up your entire server, not only the mailboxes.


Instead of openvpn, I use openssh. Use compression in the ssh tunnel,
not the rsync connection, as rsync compression tends to be buggy and
interrupts the download. I run sshd on a non-standard port to keep my
logs relatively free of script kiddy noise from people looking for an
ssh connection to crack. Run fail2ban to lock out the remaining script
kiddies. Use a client certificate to log in with ssh unprompted,
making it easy to download in a cron job.


There's more than one way to skin a cat :) Moving the ssh port and 
adding fail2ban in the mix is another option. Personally tend to use VPN 
tunnels for dynamic IP clients for various reasons, such as being able 
to lock clients out by revoking keys.


--
Adi Pircalabu

Re: Best practices for backing up small mailserver to remote location

[Adi Pircalabu]

On 08-08-2018 7:48, Ian Evans wrote:

My webserver also houses our mailserver. There's about six users on
that mail system and I'm thinking it would be good to back up the
mailboxes to my always on HTPC computer at home, which is reachable
via a dynamic IP service.

I know (or think) I need to use doveadm-backup for this but rather
than reinvent the wheel (or use the wrong wheel altogether) I'm
wondering if anyone can recommend a good tutorial or wiki entry that
shows the best way to loop through the users and send their backups to
a remote server.


Assuming you're running *nix on your HTPC and can install your own 
software on it a safe, secure and reliable way of doing it is:
- Since you're on dynamic IP at home, set up a VPN tunnel using the 
mailserver as server and HTPC as client. OpenVPN is ubiquitous and 
widely supported.

- rsync your mailboxes using the tunnel connection.
This way you can back up your entire server, not only the mailboxes.
You can add doveadm in the mix if you want, or use imapsync and so on 
and so forth.


YMMV

--
Adi Pircalabu

Re: [sieve][regex] Matching multiple strings in the "Received" header

[Adi Pircalabu]

On 08-05-2018 16:20, Gerald Galster wrote:

Hello Adi,

did you try:

" 
from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com) 
"


If you need to specify the posix character class:

[[:blank:]] means space and tab. With pcre it would be like [ \t]
[[:space:]] includes space, tab, newline, linefeed, formfeed, vertical
tab (in pcre like [ \t\n\r\f\v])

"[[:blank:]]from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)[[:blank:]]"


Thanks Gerald, none of your solutions worked, but I've just figured it 
out now. In the expression the space should only be added at the end, 
*not* at the beginning! In the Received header the first character isn't 
[[:blank:]], but "f", so I've been chasing the wild goose all this time 
because I started with the wrong assumption :)

Sorry for the noise, all good now.

Cheers,

---
Adi Pircalabu

Re: [sieve][regex] Matching multiple strings in the "Received" header

[Adi Pircalabu]

On 08-05-2018 2:43, Benny Pedersen wrote:

Adi Pircalabu skrev den 2018-05-07 05:10:


How should I write it to also match the space character at both the
beginning and end of the expression?


use \ before space char


Tks. Just tried these two, unsuccessfully:
"\.from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)\."
"\ 
from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)\ 
"


However, this expression always matches:
"from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)"

What am I missing?

---
Adi Pircalabu

Re: [sieve][regex] Matching multiple strings in the "Received" header

[Adi Pircalabu]

On 07-05-2018 12:13, Adi Pircalabu wrote:


I'm trying to use this expression in Sieve, but for some reason the
filter doesn't work:

require ["fileinto","regex"];

# rule:[gmail-outlook-yahoo-aol-friends]
if header :regex "received"
".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)."
{
fileinto "INBOX.gmail-hotmail-yahoo-aol-friends";
stop;
}


Update: this works:

if header :regex "received" 
"from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)"

{
fileinto "INBOX.gmail-hotmail-yahoo-aol-friends";
stop;
}

How should I write it to also match the space character at both the 
beginning and end of the expression?


---
Adi Pircalabu

[sieve][regex] Matching multiple strings in the "Received" header

[Adi Pircalabu]

Hi,

I'm trying to use this expression in Sieve, but for some reason the 
filter doesn't work:


require ["fileinto","regex"];

# rule:[gmail-outlook-yahoo-aol-friends]
if header :regex "received" 
".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)."

{
fileinto "INBOX.gmail-hotmail-yahoo-aol-friends";
stop;
}

However, it's working fine with egrep:
egrep 
".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)." 
*
1525657297.M401428P1459.host01.quick.net.au,S=10073,W=10275:2,S:Received: 
from APC01-PU1-obe.outbound.protection.outlook.com 
(mail-oln040092254061.outbound.protection.outlook.com [40.92.254.61])


Am I using Sieve correctly here? Is 
".from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)." 
expression valid for Sieve? Or do I have to split it in an array as per 
https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Flagging_or_Highlighting_your_mail 
example?


Thanks,

--
Adi Pircalabu

Re: Migrating maildirs - Courier to Dovecot

[Adi Pircalabu]

On 22-09-2017 4:34, Stroller wrote:
[...]


I think my main question is whether there's any reason I shouldn't
just rsync the maildirs across from the old mail server to the new
one?

There aren't many clients using this server, so I don't care if
clients have to redownload all their messages (in fact, I expect
they'll probably end up doing so anyway).

I'd like to preserve read/unread status of each message, but can't
think of anything else important.

[...]

Using rsync should be fine, I've done it myself recently several times. 
What you need to consider:

1. The downtime required during the final incremental transfer.
2. If you're using the same uid/gid on the destination server make sure 
you preserve them when transferring the data across.
3. To avoid duplicate messages in the destination you *must* use 
--delete rsync switch for the incremental transfers.


Important: I'm assuming you're using virtual mailboxes under the same 
uid/gid.


Suggested mandatory steps, ymmv:
1. Configure Dovecot in the destination to use Maildir and test 
everything: logging, SSL, authentication, mail delivery and so on. If 
you have Courier-IMAP specific configuration, e.g. folders that are 
being automatically created/subscribed upon the first login, replicate 
it and test it on the Dovecot server as well.
2. Do the initial data transfer using "-avz --numeric-ids" and see if 
you're happy with the result in the destination.
3. Run several incrementals adding "--delete" switch, followed by 
courier-dovecot-migrate.pl *executed as the mail user* to get a ballpark 
figure for the estimated outage window.
4. Test few mailboxes post-migration and compare the results with the 
source server.
5. On Day D, stop Courier-IMAP and Dovecot services on both servers to 
prevent any mailbox changes and run the last incremental, sanity checks, 
IP reconfiguration if Dovecot is the drop-in replacement, start Dovecot, 
another round of sanity checks, check the logs and so on. Here you're 
already at the point of no return :)


---
Adi Pircalabu

Re: Dovecot and Letsencrypt certs

[Adi Pircalabu]

On 13/09/2017 05:31, Joseph Tam wrote:

On Tue, 12 Sep 2017, dovecot-request wrote:


What's wrong with using a certbot "post-hook" script such as:

#!/bin/bash
echo "Letsencrypt renewal hook running..."
echo "RENEWED_DOMAINS=$RENEWED_DOMAINS"
echo "RENEWED_LINEAGE=$RENEWED_LINEAGE"

if grep --quiet "your.email.domain" <<< "$RENEWED_DOMAINS"; then
??? /usr/local/sbin/dovecot reload
?? /usr/sbin/postfix reload
fi


Nothing, if you let your certbot run as root.  (I'm assuming that's
how these hooks work -- it's called after cert renewal using the same
credentials as the certbot.)

If you use privilege separation, and run the certbot as a regular user
process, this won't work.  You might have this scenario if, for example
using the context of web serving, you serve many virtual sites with
different owners, and you don't want give each owner administrative
access.


There are options when running certbot as non-privileged user, such as 
sudo, inotifywait -s -e modify /path/to/bundle.pem && doveadm reload and 
so on.


--
Adi Pircalabu

Re: under some kind of attack

[Adi Pircalabu]

On 21/07/2017 04:03, mj wrote:

Hi Robert,


i dont understand why you focused on that ldap strings
fail2ban should trigger on some "Authentication failure" regex in the
related syslog

perhaps this will help to make it more clear

http://www.stefan-seelmann.de/wiki/fail2ban#postfix-and-dovecot


Yes, but I have that as well. :-)

I wanted two kinds of blockings:

#1: Everybody trying the well-known passwords (password, 123321, 1q2w3e, 
etc, etc) to become blocked *immediately* and for *always*.


This can be very tricky at times and you may actually hit quite a few 
legit users who are using weak passwords and have forgotten / mistyped 
them by accident. Seen this enough times and the amount of support 
required to make a sloppy & lazy customer happy again isn't always 
trivial. If they're few and far apart you can live with it, otherwise 
you'll have to reevaluate it :)


Adi Pircalabu

Re: Dovecot 2.2.27 proxy - enforcing per client IP connection limits

[Adi Pircalabu]

On 21/03/17 07:03, Joseph Tam wrote:

Sami Ketola writes:


Can anyone with Solr installed confirm/refute this:  does installing
Solr keep iOS clients from roofing the connection count?


I doubt it, but since IMAP SEARCH goes all the way down to the backends
mail_max_userip_connections can be used to limit the number of
connections.


Understood -- that's the current situation I'm in now.  Our iOS users
would launch a search resulting in a connection burst, hit the connection
cap, log out all IMAP sessions out, then start the cycle again.  This
sometimes lasts for 10's of minutes.  I'm not sure what the users sees.

[...]

Of course, the real fix is for iOS mail-app developers to stop assuming
the IMAP server is owned exclusively by the user by configuring some
reasonable connection throttles.


Thing is, one should never rely on the intentions or abilities of a 3rd 
party to fix their buggy code, especially when that 3rd party is Apple. 
Their IMAP implementation is shambolic at best and, by far and large, 
the clients using Apple mail clients are causing the most grief. Oh, did 
I mention that wonderful feature named iOS Profile which has so much 
potential if designed & implemented properly, but in A.D. 2017 it's 
still incomplete?
It's been more than obvious for years Apple can't be relied on for 
interoperability, the only way to improve the services offered to the 
clients is to look at the server side, whenever possible. And one of the 
options for limiting the IMAP client hammering is to enforce the limits 
on the proxies directly. Especially in an environment where the backend 
IMAP server isn't Dovecot and mail_max_userip_connections isn't an 
option. Even if the proxies don't exchange IMAP login information 
between them, being able to enforce the limit on the proxy can be a 
significant improvement to the current situation when the Courier-IMAP 
servers are open to IMAP abuse because they always see the proxy IP for 
the incoming connection.


Just my .02AUD

--
Adi Pircalabu

Re: Dovecot 2.2.27 proxy - enforcing per client IP connection limits

[Adi Pircalabu]

On 16/03/17 11:03, Timo Sirainen wrote:


No plans to support enforcing at proxy level. One problem here is that there 
are no guarantees that the connections even end up in the same proxies, 
although I guess if your load balancer does IP stickiness that could work well 
enough.



With or without a load balancer in front of the proxies, it's still very 
manageable. Even without a load balancer, if you have say 
proxy_mail_max_userip_connections=n and m proxies, the maximum number of 
connections that can hit the backend at any time for an user is n*m.

Would this help me to better manage the resources? Think it would.
Is there a business case for the feature? For us it is, we're 
periodically getting hammered by iOS devices that try to open 300+ 
simultaneous IMAP connections for a single user from the same IP, while 
the average hovers usually below 50 for the busier mailboxes with many 
folders.


Thanks,
Adi Pircalabu, System Administrator

Re: Dovecot 2.2.27 proxy - enforcing per client IP connection limits

[Adi Pircalabu]

Thanks,

I thought this might be the case. Is there any solution to enforce this 
on the proxy? If not, will a feature request be considered anytime soon? 
I see the proxies as the first line of defense against IMAP "abuse" and 
I think it's consistent having the same configurable option available on 
both backends and the proxies.


---
Adi Pircalabu

On 14-03-2017 20:17, Sami Ketola wrote:

Hi,

mail_max_userip_connections is only enforced at the backend level. The
setting has no effect on proxy. If you want to force the limit then
you can only do it in the backend.

Sami


On 9 Mar 2017, at 12.05, Adi Pircalabu  wrote:

Quick follow-up: updated the proxies to 2.2.28, but I still couldn't 
find a way to limit the inbound IMAP connections per IP & username. I 
know "mail_max_userip_connections" limit works for the mail stores, 
but it doesn't seem to have any effect on the proxies. I'm using a mix 
of Dovecot & Courier-IMAP servers as backends.
Basically I need to find a way to enforce the maximum limit for the 
username<>remoteip so that, if I have:
ESTCONNS=`doveadm -f flow proxy list | grep 
"username=us...@domain.com.proto=imap" | wc -l`

$ESTCONNS is lower or equal than the configured limit.
The proxies are configured as per 
https://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy to 
forward the password to the remote server using MySQL. In 
dovecot-sql.conf.ext I have:
password_query = SELECT NULL AS password, 'Y' as nopassword, host, 
email as email, 'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox 
WHERE email = '%u' AND disabled_smtpauth=0


At the moment the only way I can limit the number of established 
connections per source IP address on the Dovecot proxies is using 
iptables, which isn't what I want.

Where else can I look?

Adi Pircalabu, System Administrator
DDNS, a Total Internet Company
159 Barkly Avenue, Burnley, Vic 3121, T +61 3 9815 6868

On 08/03/17 12:32, Adi Pircalabu wrote:

Hi,
Trying to keep abusive/buggy IMAP clients at bay on a number of 
Dovecot proxy servers, I've reconfigured them to use 
"mail_max_userip_connections = 50" in the "protocol imap" section, 
followed by restarting Dovecot. Yet, I'm still seeing 160+ 
established connections from a single IP address for the same email 
account. Am I missing anything?

# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 2.6.32-642.4.2.el6.x86_64 x86_64 CentOS release 6.8 
(Final)

auth_cache_negative_ttl = 5 mins
auth_cache_size = 16 M
auth_cache_ttl = 18 hours
default_client_limit = 6120
default_process_limit = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapflags notify

mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_extensions = +notify +imapflags
}
protocols = imap pop3 lmtp sieve
service auth {
  client_limit = 6120
}
service imap-login {
  process_limit = 2048
  process_min_avail = 20
  service_count = 0
  vsz_limit = 256 M
}
service imap {
  process_limit = 2048
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 0
  vsz_limit = 128 M
}
service managesieve {
  process_limit = 1024
}
service pop3 {
  process_limit = 1024
}
[...]
protocol imap {
  imap_capability = IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
  mail_max_userip_connections = 50
}

Re: Dovecot 2.2.27 proxy - enforcing per client IP connection limits

[Adi Pircalabu]

Quick follow-up: updated the proxies to 2.2.28, but I still couldn't 
find a way to limit the inbound IMAP connections per IP & username. I 
know "mail_max_userip_connections" limit works for the mail stores, but 
it doesn't seem to have any effect on the proxies. I'm using a mix of 
Dovecot & Courier-IMAP servers as backends.
Basically I need to find a way to enforce the maximum limit for the 
username<>remoteip so that, if I have:
ESTCONNS=`doveadm -f flow proxy list | grep 
"username=us...@domain.com.proto=imap" | wc -l`

$ESTCONNS is lower or equal than the configured limit.
The proxies are configured as per 
https://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy to forward 
the password to the remote server using MySQL. In dovecot-sql.conf.ext I 
have:
password_query = SELECT NULL AS password, 'Y' as nopassword, host, email 
as email, 'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE 
email = '%u' AND disabled_smtpauth=0


At the moment the only way I can limit the number of established 
connections per source IP address on the Dovecot proxies is using 
iptables, which isn't what I want.

Where else can I look?

Adi Pircalabu, System Administrator
DDNS, a Total Internet Company
159 Barkly Avenue, Burnley, Vic 3121, T +61 3 9815 6868

On 08/03/17 12:32, Adi Pircalabu wrote:

Hi,

Trying to keep abusive/buggy IMAP clients at bay on a number of Dovecot 
proxy servers, I've reconfigured them to use 
"mail_max_userip_connections = 50" in the "protocol imap" section, 
followed by restarting Dovecot. Yet, I'm still seeing 160+ established 
connections from a single IP address for the same email account. Am I 
missing anything?


# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 2.6.32-642.4.2.el6.x86_64 x86_64 CentOS release 6.8 (Final)
auth_cache_negative_ttl = 5 mins
auth_cache_size = 16 M
auth_cache_ttl = 18 hours
default_client_limit = 6120
default_process_limit = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapflags notify

mbox_write_locks = fcntl
namespace inbox {
   inbox = yes
   location =
   mailbox Drafts {
 special_use = \Drafts
   }
   mailbox Junk {
 special_use = \Junk
   }
   mailbox Sent {
 special_use = \Sent
   }
   mailbox "Sent Messages" {
 special_use = \Sent
   }
   mailbox Trash {
 special_use = \Trash
   }
   prefix =
}
passdb {
   args = /etc/dovecot/dovecot-sql.conf.ext
   driver = sql
}
plugin {
   sieve = file:~/sieve;active=~/.dovecot.sieve
   sieve_extensions = +notify +imapflags
}
protocols = imap pop3 lmtp sieve
service auth {
   client_limit = 6120
}
service imap-login {
   process_limit = 2048
   process_min_avail = 20
   service_count = 0
   vsz_limit = 256 M
}
service imap {
   process_limit = 2048
}
service managesieve-login {
   inet_listener sieve {
 port = 4190
   }
   service_count = 0
   vsz_limit = 128 M
}
service managesieve {
   process_limit = 1024
}
service pop3 {
   process_limit = 1024
}
[...]
protocol imap {
   imap_capability = IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
   mail_max_userip_connections = 50
}

Dovecot 2.2.27 proxy - enforcing per client IP connection limits

[Adi Pircalabu]

Hi,

Trying to keep abusive/buggy IMAP clients at bay on a number of Dovecot 
proxy servers, I've reconfigured them to use 
"mail_max_userip_connections = 50" in the "protocol imap" section, 
followed by restarting Dovecot. Yet, I'm still seeing 160+ established 
connections from a single IP address for the same email account. Am I 
missing anything?


# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 2.6.32-642.4.2.el6.x86_64 x86_64 CentOS release 6.8 (Final)
auth_cache_negative_ttl = 5 mins
auth_cache_size = 16 M
auth_cache_ttl = 18 hours
default_client_limit = 6120
default_process_limit = 500
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapflags notify

mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_extensions = +notify +imapflags
}
protocols = imap pop3 lmtp sieve
service auth {
  client_limit = 6120
}
service imap-login {
  process_limit = 2048
  process_min_avail = 20
  service_count = 0
  vsz_limit = 256 M
}
service imap {
  process_limit = 2048
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 0
  vsz_limit = 128 M
}
service managesieve {
  process_limit = 1024
}
service pop3 {
  process_limit = 1024
}
[...]
protocol imap {
  imap_capability = IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
  mail_max_userip_connections = 50
}


--
Adi Pircalabu

Re: v2.2.26 release candidate released

[Adi Pircalabu]

Reading the summary below I can't see any remote mention of a possible 
fix for the crashes from:

http://dovecot.org/pipermail/dovecot/2016-October/105567.html
Just confirming this the case.

Adi Pircalabu

On 20/10/16 08:01, Timo Sirainen wrote:

http://dovecot.org/releases/2.2/rc/dovecot-2.2.26.rc1.tar.gz
http://dovecot.org/releases/2.2/rc/dovecot-2.2.26.rc1.tar.gz.sig

There are quite a lot of changes since v2.2.25. Please try out this RC so we 
can get a good and stable v2.2.26 out.

* master: Removed hardcoded 511 backlog limit for listen(). The kernel
  should limit this as needed.
* doveadm import: Source user is now initialized the same as target
  user. Added -U parameter to override the source user.
* Mailbox names are no longer limited to 16 hierarchy levels. We'll
  check another way to make sure mailbox names can't grow larger than
  4096 bytes.

+ Added a concept of "alternative usernames" by returning user_* extra
  field(s) in passdb. doveadm proxy list shows these alt usernames in
  "doveadm proxy list" output. "doveadm director&proxy kick" adds
  -f  parameter. The alt usernames don't have to be
  unique, so this allows creation of user groups and kicking them in
  one command.
+ auth: passdb/userdb dict allows now %variables in key settings.
+ auth: If passdb returns noauthenticate=yes extra field, assume that
  it only set extra fields and authentication wasn't actually performed.
+ auth: passdb static now supports password={scheme} prefix.
+ imapc: Added imapc_max_line_length to limit maximum memory usage.
+ imap, pop3: Added rawlog_dir setting to store IMAP/POP3 traffic logs.
  This replaces at least partially the rawlog plugin.
+ dsync: Added dsync_features=empty-header-workaround setting. This
  makes incremental dsyncs work better for servers that randomly return
  empty headers for mails. When an empty header is seen for an existing
  mail, dsync assumes that it matches the local mail.
+ doveadm sync/backup: Added -I  parameter to skip too
  large mails.
+ doveadm sync/backup: Fixed -t parameter and added -e for "end date".
+ doveadm mailbox metadata: Added -s parameter to allow accessing
  server metadata by using empty mailbox name.

- master process's listener socket was leaked to all child processes.
  This might have allowed untrusted processes to capture and prevent
  "doveadm service stop" comands from working.
- auth: userdb fields weren't passed to auth-workers, so %{userdb:*}
  from previous userdbs didn't work there.
- auth: Each userdb lookup from cache reset its TTL.
- auth: Fixed auth_bind=yes + sasl_bind=yes to work together
- auth: Blocking userdb lookups reset extra fields set by previous
  userdbs.
- auth: Cache keys didn't include %{passdb:*} and %{userdb:*}
- auth-policy: Fixed crash due to using already-freed memory if policy
  lookup takes longer than auth request exists.
- lib-auth: Unescape passdb/userdb extra fields. Mainly affected
  returning extra fields with LFs or TABs.
- lmtp_user_concurrency_limit>0 setting was logging unnecessary
  anvil errors.
- lmtp_user_concurrency_limit is now checked before quota check with
  lmtp_rcpt_check_quota=yes to avoid unnecessary quota work.
- lmtp: %{userdb:*} variables didn't work in mail_log_prefix
- autoexpunge settings for mailboxes with wildcards didn't work when
  namespace prefix was non-empty.
- Fixed writing >2GB to iostream-temp files (used by fs-compress,
  fs-metawrap, doveadm-http)
- director: Ignore duplicates in director_servers setting.
- zlib, IMAP BINARY: Fixed internal caching when accessing multiple
  newly created mails. They all had UID=0 and the next mail could have
  wrongly used the previously cached mail.
- doveadm stats reset wasn't reseting all the stats.
- auth_stats=yes: Don't update num_logins, since it doubles them when
  using with mail stats.
- quota count: Fixed deadlocks when updating vsize header.
- dict-quota: Fixed crashes happening due to memory corruption.
- dict proxy: Fixed various timeout-related bugs.
- doveadm proxying: Fixed -A and -u wildcard handling.
- doveadm proxying: Fixed hangs and bugs related to printing.
- imap: Fixed wrongly triggering assert-crash in
  client_check_command_hangs.
- imap proxy: Don't send ID command pipelined with nopipelining=yes
- imap-hibernate: Don't execute quota_

Re: [imap-login] SSL related crashes using the latest 2.2.25

[Adi Pircalabu]

Thanks. See the "sanitized" doveconf -n output below. Unfortunately I 
can't post log entries.
Looking at the various data I'm collecting, the crashes are always 
occurring during busy periods, when the maximum numbers of connections 
configured on the backend IMAP servers is reached. As a side note, all 
the backend servers are running using valid SSL certificates. Perhaps 
under load, or when the per IP connections limit is reached, one of them 
is disconnecting unexpectedly, or doesn't send the certificate?


# 2.2.25 (7be1766): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-642.4.2.el6.x86_64 x86_64 CentOS release 6.8 (Final)
auth_cache_negative_ttl = 5 mins
auth_cache_size = 16 M
auth_cache_ttl = 18 hours
default_client_limit = 6120
default_process_limit = 500
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
service auth {
  client_limit = 6120
}
service imap-login {
  process_limit = 2048
  process_min_avail = 20
  service_count = 0
  vsz_limit = 256 M
}
service imap {
  process_limit = 2048
}
service pop3 {
  process_limit = 1024
}
ssl_cert = 
It seems to error on ssl certificate not received.

Can you post doveconf -n and logs?

doveconf -a is usually not wanted.

Aki


On October 6, 2016 at 7:27 AM Adi Pircalabu  wrote:


I'm running Dovecot as proxy in front of some IMAP/POP3 Dovecot &
Courier-IMAP servers and in the last couple of days I've been seeing a
lot of imap-login crashes (signal 11) on both 2.2.18 and 2.2.25, all 
SSL

related. The following backtraces are taken running 2.2.25, built from
source on a test system similar to the live proxy servers.
OS: CentOS 6.8 64bit
Packages: openssl-1.0.1e-48.el6_8.3.x86_64, 
dovecot-2.2.25-2.el6.x86_64

built from source RPM.

Can post "doveconf -a" if required.

Core was generated by `dovecot/imap-login -D'.
Program terminated with signal 11, Segmentation fault.
#0  ssl_proxy_has_broken_client_cert (proxy=0x0) at
ssl-proxy-openssl.c:677
677 {
(gdb) bt full
#0  ssl_proxy_has_broken_client_cert (proxy=0x0) at
ssl-proxy-openssl.c:677
No locals.
#1  0x7fdec4e6b489 in login_proxy_ssl_handshaked 
(context=0x14b4170)

at login-proxy.c:759
 proxy = 0x14b4170
#2  0x7fdec4e70e4b in ssl_handshake (proxy=0x169d7b0) at
ssl-proxy-openssl.c:468
 ret = 
#3  ssl_step (proxy=0x169d7b0) at ssl-proxy-openssl.c:519
No locals.
#4  0x7fdec4beee0b in io_loop_call_io (io=0x13fdab0) at 
ioloop.c:564

 ioloop = 0x12a07b0
 t_id = 2
 __FUNCTION__ = "io_loop_call_io"
#5  0x7fdec4bf0407 in io_loop_handler_run_internal (ioloop=) at ioloop-epoll.c:220
 ctx = 0x12fb8d0
 events = 
 event = 0x171fb20
 list = 0x15f8c50
 io = 
 tv = {tv_sec = 46, tv_usec = 134490}
 events_count = 
 msecs = 
 ret = 1
 i = 
 call = 
 __FUNCTION__ = "io_loop_handler_run_internal"
#6  0x7fdec4beeeb5 in io_loop_handler_run (ioloop=0x12a07b0) at
ioloop.c:612
No locals.
#7  0x7fdec4bef058 in io_loop_run (ioloop=0x12a07b0) at 
ioloop.c:588

 __FUNCTION__ = "io_loop_run"
#8  0x7fdec4b81b23 in master_service_run (service=0x12a0650,
callback=) at master-service.c:640
No locals.
#9  0x7fdec4e6e593 in login_binary_run (binary=, argc=2, argv=0x12a0390) at main.c:486
 set_pool = 0x12a0b80
 login_socket = 
 c = 
#10 0x7fdec47dad1d in __libc_start_main (main=0x402ac0 ,
argc=2, ubp_av=0x7ffc53ee5688, init=, fini=, rtld_fini=,
stack_end=0x7ffc53ee5678) at libc-start.c:226
 result = 
 unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0,
5496455093114277129, 4204960, 140721716614784, 0, 0,
-5494405746439844599, -5477823887334535927}, mask_was_saved = 0}}, 
priv

= {pad = {0x0, 0x0, 0x404f70, 0x7ffc53ee5688}, data = {
   prev = 0x0, cleanup = 0x0, canceltype = 4214640}}}
 not_first_call = 
#11 0x004029c9 in _start ()
No symbol table info available.

Core was generated by `dovecot/imap-login -D'.
Program terminated with signal 11, Segmentation fault.
#0  0x7f1a58620dec in _IO_vfprintf_internal (s=, format=, ap=) at
vfprintf.c:1641
1641  process_string_arg (((struct printf_spec *) NULL));
(gdb) bt full
#0  0x7f1a58620dec in _IO_vfprintf_internal (s=, format=, ap=) at
vfprintf.c:1641
 len = 
 string_malloced = 
 step0_jumps = {0, -1285, -1198, 3818, 3910, 3206, 3307, 4086,
1925, 2133, 2249, 3731, 4474, -4059, -1109, -1062, 868, 956, 968, 980,
-1505, -495, 665, 755, 827, -3962, 395, 4392, -4059, 3997}
  

[imap-login] SSL related crashes using the latest 2.2.25

[Adi Pircalabu]

on.c:644
static_tab = {{key = 115 's', value = 0x0, long_key = 0x0}, {key 
= 36 '$', value = 0x0, long_key = 0x0}, {key = 0 '\000', value = 0x0, 
long_key = 0x0}}
func_table = {{key = 0x7f029b3e2d0c "passdb", func = 
0x7f029b3d7c70 }, {key = 0x0, func = 0}}

tab = 
e = 
str = 
str2 = 
pos = 
#4  0x7f029b3d847a in client_log_err (client=0x221ee70,
msg=0x187fe38 "proxy: SSL certificate not received from 
\314-A\235q\210\021\b\354\062Lzح)\367.\002 \031\233 
\362w⊓\224\356K7\343\224 
\002\037\364!+\266\371\277O`K\021\bͰ\a\202\001:6") at 
client-common.c:692

_data_stack_cur_id = 3
#5  0x7f029b3db51e in login_proxy_ssl_handshaked (context=0x19b2530) 
at login-proxy.c:765

proxy = 0x19b2530
#6  0x7f029b3e0e4b in ssl_handshake (proxy=0x195df70) at 
ssl-proxy-openssl.c:468

ret = 
#7  ssl_step (proxy=0x195df70) at ssl-proxy-openssl.c:519
No locals.
#8  0x7f029b15ee0b in io_loop_call_io (io=0x216d790) at ioloop.c:564
ioloop = 0x18207b0
t_id = 2
__FUNCTION__ = "io_loop_call_io"
#9  0x7f029b160407 in io_loop_handler_run_internal (ioloop=optimized out>) at ioloop-epoll.c:220

ctx = 0x187b8d0
events = 
event = 0x1df4668
list = 0x2025710
io = 
tv = {tv_sec = 11, tv_usec = 323409}
events_count = 
msecs = 
ret = 3
i = 
call = 
__FUNCTION__ = "io_loop_handler_run_internal"
#10 0x7f029b15eeb5 in io_loop_handler_run (ioloop=0x18207b0) at 
ioloop.c:612

No locals.
#11 0x7f029b15f058 in io_loop_run (ioloop=0x18207b0) at ioloop.c:588
__FUNCTION__ = "io_loop_run"
#12 0x7f029b0f1b23 in master_service_run (service=0x1820650, 
callback=) at master-service.c:640

No locals.
#13 0x7f029b3de593 in login_binary_run (binary=out>, argc=2, argv=0x1820390) at main.c:486

set_pool = 0x1820b80
login_socket = 
c = 
#14 0x7f029ad4ad1d in __libc_start_main (main=0x402ac0 , 
argc=2, ubp_av=0x7ffd637fd608, init=, fini=optimized out>, rtld_fini=, 
stack_end=0x7ffd637fd5f8) at libc-start.c:226

result = 
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 
-4141182239951058275, 4204960, 140726272775680, 0, 0, 
4142562126330825373, 4071998539020864157}, mask_was_saved = 0}}, priv = 
{pad = {0x0, 0x0, 0x404f70, 0x7ffd637fd608}, data = {

  prev = 0x0, cleanup = 0x0, canceltype = 4214640}}}
not_first_call = 
#15 0x004029c9 in _start ()
No symbol table info available.


--
Adi Pircalabu

Re: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration

[Adi Pircalabu]

On Mon, 03 Sep 2012 01:55:21 +0200
Benny Pedersen  wrote:

> Den 2012-09-03 01:47, Timo Sirainen skrev:
> 
> > Maybe use IMAP authentication as the backend? pam_imap at least can
> > do that. Or you can already also use Dovecot v2.1's passdb imap to
> > do this, pretty much equivalent to pam_imap.

I had a look at pam_imap and had to fix the spec file to get it built
for CentOS 6 64bit. For the record, I had to add the following in the
%build section:
export CFLAGS="%{optflags} -fPIC"
export CXXFLAGS="%{optflags} -fPIC"
However:

> drop pam, and use saslauthd with remote imap, or setup cyrus sasl
> with sql/ldap/whatever one needs

Yep, saslauthd with rimap appears to be exactly what I need.

> just in case one more ask why i did not use dovecot :)

The password forwarding feature is one of the reasons I started looking
into Dovecot.

Thanks to you both for your help.

-- 
Adi Pircalabu, System Administrator

Re: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration

[Adi Pircalabu]

On Fri, 31 Aug 2012 17:11:07 +0300
Timo Sirainen  wrote:

> > The POP/IMAP part is working fine. What I'm trying to do is to use
> > Dovecot SASL implementation in Postfix to do SMTP authentication in
> > a similar manner. The problem I have with my current configuration
> > is that SMTP authentication succeeds if only the username matches,
> > because password forwarding works if the authentication succeeds
> > with any given password, as documented at
> > http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy  
> 
> Dovecot has no SMTP proxy (currently). And anyway Postfix doesn't use
> SMTP to do authentication, Postfix authenticates using Dovecot's
> internal protocol, which replies that Postfix should do the proxying,
> which it of course doesn't do.

Yes, I know and that's exactly what I was trying to do: use Dovecot
authentication method in Postfix to authenticate the user. Because of
the fact that Dovecot doesn't do SMTP authentication, I was thinking of
a way of using its authentication service by getting the SMTP login
credentials from the backend POP/IMAP server. The request may sound a
bit unusual, but in our case it makes sense. For our setup we currently
run:
- a farm of backend SMTP/POP/IMAP servers that are hosting the
  mailboxes and where the user credentials are managed. They are running
  Courier IMAP.
- a group of SMTP/POP/IMAP proxies. These proxies are currently
  replicating the login credentials from the backend servers and the
  routing to the backends using a local database. Perdition is currently
  the POP/IMAP proxy, but having it replaced with Dovecot would help us
  in getting the password forwarding to the backends running, which
  means we wouldn't need to store the credentials on the proxy, only
  the user->host routing entries.
Are there any plans to have Dovecot authentication service to do SMTP
authentication against IMAP or POP3 proxy provided information? This,
of course, means we'd have the authentication result tied to the
response of the backend IMAP/pop3 server.

> > My question is, given the above: is there a way to get SMTP
> > authentication properly in this scenario?  
> 
> Make Postfix authenticate against the backend Dovecot server. You'll
> need to setup service auth { inet_listener } to some port for it.

The POP/IMAP backends are running Courier IMAP, as I've just mentioned
and due to the existing hosting environment it's very unlikely to
replace it with something else.

-- 
Adi Pircalabu, System Administrator
Discount Domain Name Services Pty Ltd, a Total Internet Company
PO Box 887, Hawthorn Vic 3122, Australia, T +61 3 9815 6868
Ask me about cloud hosting services

[Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration

[Adi Pircalabu]

Hi,

I'm relatively new to Dovecot and I did a bit of search but couldn't
find a possible solution for the particular setup I'm working on.
Basically I have an SMTP/POP/IMAP proxy setup running Postfix &
Dovecot. IMAP/POP authentication is done using the password proxy
feature, where the login credentials are passed to the backend server
after a db lookup, which does the actual authentication.
The POP/IMAP part is working fine. What I'm trying to do is to use
Dovecot SASL implementation in Postfix to do SMTP authentication in a
similar manner. The problem I have with my current configuration is that
SMTP authentication succeeds if only the username matches, because
password forwarding works if the authentication succeeds with any given
password, as documented at
http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy
My question is, given the above: is there a way to get SMTP
authentication properly in this scenario? The way I see it now, Dovecot
SASL accepting the login if only the user matches isn't quite
"complete", the auth process should go further and authenticate against
the backend server, same as for POP/IMAP connections. Does this
requirement make any sense?
Maybe I'm missing something in Dovecot configuration to get the SMTP
authentication work in password forwarding mode, few pointers will be
highly appreciated. PopBSMTP is not an sensible alternative in my case.
Here's my configuration, plus some dovecot auth_debug log entries. As
you can see, SMTP authentication succeeds with any given password.

---Dovecot---

 dovecot -n
# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.2.1.el6.centos.plus.x86_64 x86_64 CentOS release 6.3 
(Final) auth_cache_size = 4 k
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
auth_verbose_passwords = plain
mbox_write_locks = fcntl
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
}
ssl_ca = , method=PLAIN, 
rip=192.168.1.56, lip=192.168.1.222, TLS
[...]

(SMTP connection)
Aug 31 11:36:14 centos6 postfix/smtpd[11213]: connect from 
unknown[192.168.1.200]
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Loading modules from directory: 
/usr/lib64/dovecot/auth
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libauthdb_ldap.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libdriver_mysql.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libdriver_sqlite.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libmech_gssapi.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: auth client connected (pid=11213)
Aug 31 11:36:14 centos6 dovecot: auth: Debug: client in: 
AUTH#0111#011PLAIN#011service=smtp#011nologin#011lip=192.168.1.222#011rip=192.168.1.200#011resp=AGFAMGFkaXRlc3QubmV0AGFzZA==
Aug 31 11:36:14 centos6 dovecot: auth: Debug: 
cache(a...@0aditest.net,192.168.1.200): miss
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Loading modules from directory: 
/usr/lib64/dovecot/auth
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libauthdb_ldap.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libdriver_mysql.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libdriver_sqlite.so
Aug 31 11:36:14 centos6 dovecot: auth: Debug: Module loaded: 
/usr/lib64/dovecot/auth/libmech_gssapi.so
Aug 31 11:36:14 centos6 dovecot: auth: mysql: Connected to 
/var/lib/mysql/mysql.sock (postfix)
Aug 31 11:36:14 centos6 dovecot: auth: Debug: 
sql(a...@0aditest.net,192.168.1.200): query: SELECT NULL AS password, 'Y' as 
nopassword, host, email, 'Y' AS proxy FROM mailbox WHERE email = 
'a...@0aditest.net'
Aug 31 11:36:14 centos6 dovecot: auth: Debug: client out: 
OK#0111#011user=a...@0aditest.net#011host=203.63.79.87#011email=a...@0aditest.net#011proxy#011pass=anygivenpassword
Aug 31 11:36:14 centos6 postfix/smtpd[11213]: C9620600A9: 
client=unknown[192.168.1.200], sasl_method=PLAIN, 
sasl_username=a...@0aditest.net
Aug 31 11:36:14 centos6 postfix/cleanup[11219]: C9620600A9: 
message-id=<20120831113614.72ed3...@adi.ddns.local>
[...]

---/var/log/maillog---

-- 
Adi Pircalabu