Re: replication and spam removal ("doveadm expunge")

2020-04-06 Thread Olaf Hopp

OK, further findings:
when user logs in on the node where the "doveadm expunge" was run
this has no effect on the spambox the other side.

But if the user logs in on the other side (after a manual failover of the 
"cluster" IP)
and opens his spambox then all of a sudden those expunged mails got deleted.

So if this behaviour is the intended then you may regard this issue as closed.

Thanks, Olaf



On 4/6/20 12:40 PM, Olaf Hopp wrote:


Hi Aki,

On 4/4/20 8:12 PM, Aki Tuomi wrote:

Can you provide doveconf -n and try turning on mail_debug=yes on both ends and 
try doveadm -Dv expunge 


mail_debug=yes
is on on both ends and dovecot was restarted but anyway nothing is logged when I issue 
"doveadm -Dv expunge "
In the shell where I issue the "expunge" I see the following:

# /usr/bin/doveadm -Dv  expunge  -u test4  mailbox INBOX.spambox  BEFORE 13m
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: 
acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: 
expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: 
quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined 
symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore 
this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: 
fts_user_get_language_list (this is usually intentional, so just ignore this 
message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: 
mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore 
this message)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Started 
userdb lookup
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Client connected (fd=10)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): auth USER 
input: test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Finished 
userdb lookup (username=test4 system_groups_user=test4 uid=1805 gid=2300 
home=/home/irams1-test/test4)
doveadm(test4): Debug: Effective uid=1805, gid=2300, 
home=/home/irams1-test/test4
doveadm(test4): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., 
inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(test4): Debug: maildir++: root=/home/irams1-test/test4/Maildir, index=, 
indexpvt=, control=, inbox=/home/irams1-test/test4/Maildir, alt=
doveadm(test4): Debug: Namespace : type=private, prefix=, sep=, inbox=no, 
hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
doveadm(test4): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
doveadm(test4): Debug: Mailbox INBOX.spambox: Mailbox opened because: expunge
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=38
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=39
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=40
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=41
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=42
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=43
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=44
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=45
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=46
doveadm(test4): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb 
(pid=15116,uid=0): Disconnected: Connection closed (fd=10)

A few mails are deleted from the spambox but the deletion is not synced to the 
other side
Even if I do a "doveadm  force-resync  -u test4 '*'" on both sides the 
deletions are not replicated
but anyway

#doveadm replicator status test4
username   

Re: replication and spam removal ("doveadm expunge")

2020-04-06 Thread Olaf Hopp


Hi Aki,

On 4/4/20 8:12 PM, Aki Tuomi wrote:

Can you provide doveconf -n and try turning on mail_debug=yes on both ends and 
try doveadm -Dv expunge 


mail_debug=yes
is on on both ends and dovecot was restarted but anyway nothing is logged when I issue 
"doveadm -Dv expunge "
In the shell where I issue the "expunge" I see the following:

# /usr/bin/doveadm -Dv  expunge  -u test4  mailbox INBOX.spambox  BEFORE 13m
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: 
acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: 
expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: 
quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined 
symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore 
this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: 
fts_user_get_language_list (this is usually intentional, so just ignore this 
message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: 
mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore 
this message)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Started 
userdb lookup
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Client connected (fd=10)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): auth USER 
input: test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Finished 
userdb lookup (username=test4 system_groups_user=test4 uid=1805 gid=2300 
home=/home/irams1-test/test4)
doveadm(test4): Debug: Effective uid=1805, gid=2300, 
home=/home/irams1-test/test4
doveadm(test4): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., 
inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(test4): Debug: maildir++: root=/home/irams1-test/test4/Maildir, index=, 
indexpvt=, control=, inbox=/home/irams1-test/test4/Maildir, alt=
doveadm(test4): Debug: Namespace : type=private, prefix=, sep=, inbox=no, 
hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
doveadm(test4): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
doveadm(test4): Debug: Mailbox INBOX.spambox: Mailbox opened because: expunge
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=38
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=39
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=40
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=41
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=42
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=43
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=44
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=45
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=46
doveadm(test4): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb 
(pid=15116,uid=0): Disconnected: Connection closed (fd=10)

A few mails are deleted from the spambox but the deletion is not synced to the 
other side
Even if I do a "doveadm  force-resync  -u test4 '*'" on both sides the 
deletions are not replicated
but anyway

#doveadm replicator status test4
username  
priority fast sync full sync success sync failed
test4 
none 00:26:47  18:19:46  00:26:47 -

on both sides.
If new spam arrives it got well replicated to the other side as expected

"doveconf -n" ist attached below.

Thanks,
Olaf




Aki

On 04/04/2020 20:03 Olaf Hopp < olaf.h...@kit.edu <mailto:olaf.h...@kit.edu>> 
wrote:


Nobody ? :-(

On 3/3

Re: replication and spam removal ("doveadm expunge")

2020-04-04 Thread Olaf Hopp


Nobody ? :-(

On 3/30/20 5:26 PM, Olaf Hopp wrote:

Hello everybody,
since now I did no replication and spam is delivered into users folder "spambox"
Every night there is a cronjob which deletes spam older than 30 days via 
something like
 "find  -ctime +30 -delete"
Now I'm going to set up replication (two way) and I thought that
doing "rm" is not a good idea.
So I modified the job to something like
 /usr/bin/doveadm expunge  -u test1 mailbox INBOX.spambox  BEFORE  30d
which works like intended, but I see that on the replication
partner the spam isn't deleted. Even if I do a
 doveadm  force-resync  -u test1 '*'
or log in via imap to the replication partner I still see the old spam.

So my question is:
is this the intended behaviour and I have to run the "doveadm expunge" on both 
replication partners
or should the deletion via "expunge" being replicated to the partner and I have 
a misconfiguration
or maybe I hit a bug ?

Regards, Olaf


/etc/dovecot/conf.d/12-replication.conf:


mail_plugins = $mail_plugins notify replication
service aggregator {
     fifo_listener replication-notify-fifo {
     mode = 0666
     }
     unix_listener replication-notify {
     mode = 0666
     }
}
service replicator {
     process_min_avail = 1
     unix_listener replicator-doveadm {
     mode = 0666
     }
}
service doveadm {
     inet_listener {
     port = 1109
     }
}
doveadm_password = X
plugin {
     mail_replica = tcp:X.Y.Z.X:1109
}





--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


replication and spam removal ("doveadm expunge")

2020-03-30 Thread Olaf Hopp

Hello everybody,
since now I did no replication and spam is delivered into users folder "spambox"
Every night there is a cronjob which deletes spam older than 30 days via 
something like
"find  -ctime +30 -delete"
Now I'm going to set up replication (two way) and I thought that
doing "rm" is not a good idea.
So I modified the job to something like
/usr/bin/doveadm expunge  -u test1 mailbox INBOX.spambox  BEFORE  30d
which works like intended, but I see that on the replication
partner the spam isn't deleted. Even if I do a
doveadm  force-resync  -u test1 '*'
or log in via imap to the replication partner I still see the old spam.

So my question is:
is this the intended behaviour and I have to run the "doveadm expunge" on both 
replication partners
or should the deletion via "expunge" being replicated to the partner and I have 
a misconfiguration
or maybe I hit a bug ?

Regards, Olaf


/etc/dovecot/conf.d/12-replication.conf:


mail_plugins = $mail_plugins notify replication
service aggregator {
fifo_listener replication-notify-fifo {
mode = 0666
}
unix_listener replication-notify {
mode = 0666
}
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0666
}
}
service doveadm {
inet_listener {
port = 1109
}
}
doveadm_password = X
plugin {
mail_replica = tcp:X.Y.Z.X:1109
}


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: Not able to start dovecot

2020-03-26 Thread Olaf Hopp

On 3/20/20 4:43 PM, Alexander Dalloz wrote:


Use a proper systemd unit file instead of the initd script.

Alexander



Use e.g. the following for  getting started:

# This file is part of Dovecot
#
# DO NOT CUSTOMIZE THIS FILE, INSTEAD
# create the file:
#   `/etc/systemd/system/dovecot.service.d/service.conf'.
# or copy this as
#   `/etc/systemd/system/dovecot.service` and edit then
# and put your changes there

[Unit]
Description=Dovecot IMAP/POP3 email server
Documentation=man:dovecot(1)
Documentation=http://wiki2.dovecot.org/
After=local-fs.target network-online.target dovecot-init.service
Requires=dovecot-init.service

[Service]
Type=simple
ExecStartPre=/usr/libexec/dovecot/prestartscript
ExecStart=/usr/sbin/dovecot -F
PIDFile=/var/run/dovecot/master.pid
ExecReload=/usr/bin/doveadm reload
ExecStop=/usr/bin/doveadm stop
PrivateTmp=true
NonBlocking=yes
# this will make /usr /boot /etc read only for dovecot
ProtectSystem=full
ProtectHome=no
PrivateDevices=true

# You can add environment variables with e.g.:
#Environment='CORE_OUTOFMEM=1'
# If you have trouble with `Too many open files', increase
LimitNOFILE=65535
# If you want to allow the Dovecot services to produce core dumps, use:
#LimitCORE=infinity

[Install]
WantedBy=multi-user.target


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: lmtp panic with many recipients

2018-05-09 Thread Olaf Hopp

On 05/09/2018 11:10 AM, Stephan Bosch wrote:



Op 09/05/2018 om 10:17 schreef Ralf Hildebrandt:

* Stephan Bosch <step...@rename-it.nl>:


Op 08/05/2018 om 10:34 schreef Olaf Hopp:

Hi,

I had an email with 58 recipients in the "To" and 13 in the "CC"
Delivering it from exim to dovecot lmtp panics (see below)
Panic: file smtp-address.c: line 533 (smtp_address_write): assertion
failed: (smtp_char_is_qpair(*p))

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Do you have an example e-mail that triggers the problem and the sieve
scripts that are involved for the recipient that causes the crash?

That looks a bit like
https://www.mail-archive.com/dovecot@dovecot.org/msg72690.html


Indeed. But I'd like to make sure this is the same problem.

Regards,

Stephan.


Seems to be. There is a
=?iso-8859-1?Q?s=2Ev=F6gele=40X=2Ede?= <s.vög...@x.de>
within the "To"-header, together with the line
not address :all :comparator "i;octet" :contains "To" "robocup"
in my global sieve_before script seems to trigger it

Fucking german umlaut.


Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: lmtp panic with many recipients

2018-05-09 Thread Olaf Hopp

On 05/09/2018 10:04 AM, Stephan Bosch wrote:



Op 08/05/2018 om 10:34 schreef Olaf Hopp:

Hi,

I had an email with 58 recipients in the "To" and 13 in the "CC"
Delivering it from exim to dovecot lmtp panics (see below)
Panic: file smtp-address.c: line 533 (smtp_address_write): assertion failed: 
(smtp_char_is_qpair(*p))

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)



Do you have an example e-mail that triggers the problem and the sieve scripts 
that are involved for the recipient that causes the crash?



Hi Stephan,

the original mail got stuck within exim on my production server and was from "somebody" 
to many "somebodys" but not me.
So with exim I added as envelope recipient a test user "ms2test" on my test system 
"irams2.ira.uka.de"
The test user "ms2test" has just an empty sieve script (all comments).
There is a global sieve_before-Script doing spam delivery into spambox:

#
require ["fileinto", "regex", "envelope"];
#
#
if allof (
not header :comparator "i;ascii-casemap" :regex "Subject" 
"fail.*deliver",
not header :comparator "i;octet" :contains "Subject" "DBWORLD",
not header :comparator "i;octet" :contains "List-Id" "ieft.org",
not header :comparator "i;octet" :contains "Subject" "Google Alert",
not header :comparator "i;octet" :contains "Subject" "Google Gaga",
not header :comparator "i;octet" :contains "Subject" "foo",
not header :comparator "i;octet" :contains "Subject" "Woechentliche 
Spam-Benachrichtigung",
not address :all :comparator "i;octet" :contains "To" "robocup",
not header :comparator "i;octet" :is "Envelope-to" "ms2s...@ira.uka.de",

exists [ "X-ATIS-Spam-Flag" ] ) {

fileinto "INBOX.spambox";
stop;
}

The mail in question contains third party adresses and content. So I can't post 
it here.
I will try to reproduce it by myself with just dummy addresses.

Ok, another finding:
if I strip down the global sieve_before just to

require ["fileinto", "regex", "envelope"];
if allof (
not address :all :comparator "i;octet" :contains "To" "robocup"
) {

fileinto "INBOX.spambox";
stop;
}
the mail got stuck.
If I reenable all other original lines, but remove the "rococup" line the mail 
get's delivered.


"doveconf -n" see below

Olaf


# doveconf -n
# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
doveconf: Warning: service auth { client_limit=2000 } is lower than required 
under max. load (20192)
doveconf: Warning: service anvil { client_limit=2000 } is lower than required 
under max. load (17195)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)
# Hostname: irams1.ira.uka.de
auth_failure_delay = 3 secs
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = yes
auth_verbose_passwords = plain
auth_worker_max_count = 60
default_client_limit = 2000
default_process_limit = 3000
first_valid_uid = 1000
last_valid_uid = 65533
mail_location = maildir:~/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox "Deleted Items" {
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox "Deleted Messages" {
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox "Gelöschte Objekte" {
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox "Gel Objekte" {
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox Papierkorb {
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
autoexpunge = 30 days
special_use = \Trash
  }
  mailbox spambox {
auto = create
special_use = \Junk
  }
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/master-users
  driver = passwd-file
  master = yes
}
passdb {
  args = dovecot
  driver = pam
}
plugin {
  siev

lmtp panic with many recipients

2018-05-08 Thread Olaf Hopp
 
Debug: sieve: Executing script from `/etc/dovecot/sieve-master.svbin'
May  8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: 
Mailbox : Opened mail UID=1 because: header List-Id (Cache file is 
unusable)
May  8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: 
Panic: file smtp-address.c: line 533 (smtp_address_write): assertion failed: 
(smtp_char_is_qpair(*p))
May  8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xca92a) 
[0x7fd7b6d0a92a] -> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler+0x33) [0x7fd7b6d0b003] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3ce61) 
[0x7fd7b6c7ce61] -> /usr/lib64/dovecot/libdovecot.so.0(+0x403fb) [0x7fd7b6c803fb] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_address_encode+0x29) [0x7fd7b6c80429] 
-> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x46892) [0x7fd7b4146892] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_match+0xbf) [0x7fd7b41475ff] -> 
/usr/lib64/dovecot/libdovecot-sieve.so.0(+0x4d9ec) [0x7fd7b414d9ec] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0x71) [0x7fd7b413c4b1] 
-> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd7b413d07b] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x522fb) [0x7fd7b41522fb] 
-> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_multiscript_run+0x5a) [0x7fd7b415252a] -> /usr/lib64/dovecot/lib90_sieve_plugin.so(+0x36ad) [0x7fd7b43bb6ad] 
-> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0xd2) [0x7fd7b7314942] -> dovecot/lmtp [local DATA](+0x6ffe) [0x7fd7b7756ffe] -> dovecot/lmtp [local 
DATA](lmtp_local_data+0x156) [0x7fd7b7757466] -> dovecot/lmtp [local DATA](cmd_data_continue+0x218) [0x7fd7b7756088] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x4ee3d) [0x7fd7b6c8ee3d] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4f242) [0x7fd7b6c8f242] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x4f837) [0x7fd7b6c8f837] -> /usr/lib64/dovecot/libdovecot.so.0(smtp_server_command_next_to_reply+0x3b) [0x7fd7b6c9212b] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x55a30) [0x7fd7b6c95a30] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7fd7b6d21785] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x95) [0x7fd7b6d218b5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd7b6d21a98] -> 
/usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd7b6c9f493] -> dovecot/lmtp [loc
May  8 10:01:52 irams2 dovecot: lmtp: Fatal: master: service(lmtp): child 17557 
killed with signal 6 (core dumps disabled - 
https://dovecot.org/bugreport.html#coredumps)

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-24 Thread Olaf Hopp

On 04/23/2018 03:46 PM, Olaf Hopp wrote:

On 04/23/2018 03:22 PM, Stephan Bosch wrote:



Op 20-4-2018 om 14:01 schreef Olaf Hopp:

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Probably same as issue in this thread:

https://www.dovecot.org/pipermail/dovecot/2018-April/111482.html



Yes maybe.
But I didn't see any sieve errors in the logs.
In my case there is exim sitting in front of dovecot lmtp and as said
 trusted_users = exim:dovecot
in thge exim.conf resolved this issue for me.

Regards, Olaf


I digged deeper: in 
https://www.dovecot.org/pipermail/dovecot/2018-April/111485.html Stephan wrote

| Yeah, this is likely due to the fact that sendmail is now invoked using
| the program-client (same as Sieve extprograms), which takes great care
| to drop any unwanted (seteuid) root privileges.

and thats the reason why my exim now needs the dovecot user as trusted user so 
that
those redirects can retain the original envelope sender.

Thanks, Olaf


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-23 Thread Olaf Hopp

On 04/23/2018 03:22 PM, Stephan Bosch wrote:



Op 20-4-2018 om 14:01 schreef Olaf Hopp:

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Probably same as issue in this thread:

https://www.dovecot.org/pipermail/dovecot/2018-April/111482.html



Yes maybe.
But I didn't see any sieve errors in the logs.
In my case there is exim sitting in front of dovecot lmtp and as said
trusted_users = exim:dovecot
in thge exim.conf resolved this issue for me.

Regards, Olaf



--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-23 Thread Olaf Hopp

On 04/23/2018 07:28 AM, Steffen Kaiser wrote:



Envelope *senders* should never ever be modified.


If the domain of sender A has SPF records installed and B redirects to C, but 
keeps the envelope sender A, the SPF check will fail on C.



That's the reason why I say SPF is broken by design.
People using it, should hopefully know what they are doing.
But that's a little bit OT for this list.
Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-22 Thread Olaf Hopp

On 04/21/2018 03:25 PM, Bill Shirley wrote:

On 4/20/2018 8:53 AM, Olaf Hopp wrote:

On 04/20/2018 02:01 PM, Olaf Hopp wrote:

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Regards,
Olaf



I moved one version back, same config except those changes in 10-ssl.conf 
necessary for the 2.2->2.3 upgrade

# 2.2.35 (b1cb664): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.23 (b2e41927)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)

and this version keeps the envelope sender untouched.
So this a regression with 2.3 / 0.5
Envelope *senders* should never ever be modified.

Regards,
Olaf




My father is subscribed to a mailing list that instead of using l...@xyz.org in 
the envelope
it actually modifies the envelope to the poster's email address. When they try 
to send
the email to my server and the envelope says "Hi, I'm coming from 
b...@example.com",
I know they are lying because *my mail server is the mail handler* for 
example.com. REJECT

If you accept mail that's obviously forging the envelope sender, any spammer 
can just
send email saying I am you and get passed by a whitelist statement in 
Spamassassin
because... u...@example.com "oh, he's a good guy.  Let him through."

Bill



Of course, mailing lists are an exeption to this.
It's usual to put listname-bounces@... into the envelope sender,
so that bounce processing might be done by the mailing list software.
Olaf


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-20 Thread Olaf Hopp

OK, I found a solution:

trusted_users = exim:dovecot

in my exim.conf fixed it.

Anyway this is an important change of behavour between 2.2 und 2.3
In 2.2 the "dovecot" under exims "trusted_users" was not necessary.

Olaf


On 04/20/2018 02:53 PM, Olaf Hopp wrote:

On 04/20/2018 02:01 PM, Olaf Hopp wrote:

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Regards,
Olaf



I moved one version back, same config except those changes in 10-ssl.conf 
necessary for the 2.2->2.3 upgrade

# 2.2.35 (b1cb664): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.23 (b2e41927)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)

and this version keeps the envelope sender untouched.
So this a regression with 2.3 / 0.5
Envelope *senders* should never ever be modified.

Regards,
Olaf





--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-20 Thread Olaf Hopp

On 04/20/2018 02:01 PM, Olaf Hopp wrote:

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Regards,
Olaf



I moved one version back, same config except those changes in 10-ssl.conf 
necessary for the 2.2->2.3 upgrade

# 2.2.35 (b1cb664): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.23 (b2e41927)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)

and this version keeps the envelope sender untouched.
So this a regression with 2.3 / 0.5
Envelope *senders* should never ever be modified.

Regards,
Olaf


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5

2018-04-20 Thread Olaf Hopp

Hi (Stephan?),
is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" 
changes the envelope sender of
a redirected mail or simply a bug ?

A sends mail to B, B redirects to C
C sees B (not A!) as envelope sender.
It is not a problem if C gets the mail but if that mail bounces
for various reasons it goes back to B and A will never know about this.

I thick this is came with 2.3 / pigeonhole 0.5 ?

# 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.devel (61b47828)
# OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)


Regards,
Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Dovecot 2.3.0 TLS

2018-01-22 Thread Olaf Hopp

On 01/11/2018 12:22 PM, Aki Tuomi wrote:



On 11.01.2018 13:20, Hauke Fath wrote:

On Thu, 11 Jan 2018 12:20:45 +0200, Aki Tuomi wrote:

Was the certificate path bundled in the server certificate?

No, as a separate file, provided from the local (intermediate) CA:

ssl_cert = 

Seems we might've made a unexpected change here when we revamped the ssl
code. Can you try if it works if you concatenate the cert and cert-chain
to single file? We'll start looking if this is misunderstanding or bug.

Aki



Hello,
let me confirm this issue.
I have a setup similar to Hauke Fath. Doing the workaround suggested by Aki

 cat /etc/openssl/certs/ca-cert-chain.pem >> /etc/openssl/certs/server.cert

and removing "ssl_ca" from the config file presents the correct CA-Chain.
Whereas the original config presented my three time my own server cert as chain.


Since server certs tend to change more frequent than the CA chains
I really want to keep them in separate files.

So this is really a show stopper for me.

CU, Olaf




--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: sieve filter move wrong email to Junk folder

2017-12-15 Thread Olaf Hopp

On 12/15/2017 02:36 PM, Alex JOST wrote:

Am 14.12.2017 um 18:47 schrieb Gao:

I use a sieve filter to move spam email to user's Junk folder:
# cat spam_to_junk.sieve
require "fileinto";
   if exists "X-Spam-Status" {
   if header :contains "X-Spam-Status" "YES" {
   fileinto "Junk";
   stop;
   } else {
   }
   }
   if header :contains "subject" ["SPAM?"] {
 fileinto "Junk";
 stop;
   }

Most time this filter works fine but occasionally it move non-spam in to Junk 
folder. Here is an example, this email is from dovecot mailling list and it end 
up in my Junk folder. Mailllog and header here. Would someone help me to figure 
out what went wrong here?



X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00
  autolearn=ham version=3.3.2, No


AFAIK, header matching is case-insensitive. That's why 'YES' matches 'BAYES' 
and triggers the action.


So any spammer might simply add a Header "X-Spam-Status: No"
and the Mail gets into the INBOX ?

Thats why my exim / spamasssassin combination adds the
header "X-Spam-Status:" with all the various checks and
if and only if the score is above e.g. 5 points it additionally adds
the header "X-Spam-Flag: YES" .
Ham mail is not affected with this "X-Spam-Flag".

My global sieve filter only checks for the existance of
the header line - not the value. I think this can't be
circumvented by the spammers.

Regards, Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: password reset

2017-09-08 Thread Olaf Hopp

On 09/08/2017 11:20 AM, Steffen Kaiser wrote:




When I try to change the password with sudo, the timestamp on /etc/passwd gets 
updated but there is nothing logged to anything in /var/log/


/etc/shadow should get updated but not /etc/passwd !
make a copy of /etc/passwd before the change and look at the diff

Olaf



--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: under another kind of attack

2017-07-30 Thread Olaf Hopp

On 07/29/2017 01:34 PM, Davide Marchi wrote:

Hi to all,

@Olaf Hopp I've this filter enabled for fail2ban, my question is: could my 
filters overlap or interfere with those suggested by you?

this is my filter:



Davide,
yours is all postfix and thus has got no overlap with dovecot.
So no interference.
Olaf



Contents of /etc/fail2ban/jail.conf:

[postfix]
# Ban for 10 minutes if it fails 6 times within 10 minutes
enabled  = true
port = smtp,ssmtp
filter   = postfix
logpath  = /var/log/mail.log
maxretry = 6
bantime  = 600
findtime = 600

Contents of /etc/fail2ban/filter.d/postfix.conf:

# Fail2Ban configuration file
# Author: Cyril Jaquier
# $Revision$

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failures messages in the logfile. The
#  host must be matched by a group named "host". The tag "" can
#  be used for standard IP/hostname matching and is only an alias for
#  (?:::f{4,6}:)?(?P[\w\-.^_]+)
# Values:  TEXT
#

# Jul 11 02:35:08 mail postfix/smtpd[16299]: lost connection after AUTH from 
unknown[196.12.178.73]

failregex = lost connection after AUTH from unknown\[\]

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =


Many thanks!



--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: under another kind of attack

2017-07-27 Thread Olaf Hopp

On 07/26/2017 10:01 PM, Joseph Tam wrote:

Olaf Hopp <olaf.h...@kit.edu> wrote:


And I have a new one just for "unknown user" and here my bantime and findtime
are much bigger and the retries are just '2'. So here I'm much harsher.
I'll keep an eye on my logs and maybe some more twaeking is necessary.


Just be careful about typos (like twaeking!): users could simply misspell
their username, or get mixed up with some another account or alias.
This is why I favour targetting known bad accounts, not merely accounts
that don't exist.


Joseph,
but how often do you have to type your username ?
Only on the initial config of your mailer. After that
you are done. Exception is my webmail server.
But that IP is of course on the "ignoreip" list of fail2ban.
Otherwise it would be very easy to trigger a DOS without
much effort.
So this is why I decided to use two distinct jails with
different policies. It seems to work reasonable well.

Regards, Olaf


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: under another kind of attack

2017-07-27 Thread Olaf Hopp

On 07/27/2017 05:19 AM, James Brown wrote:



On 26 Jul 2017, at 7:57 pm, Olaf Hopp <olaf.h...@kit.edu> wrote:

Dear collegues,

many thanks for your valuable input.

Since we are an university GEO-IP blocking is not an option for us.
Somestimes I think it should ;-)

My "mistake" was that I had just *one* fail2ban filter for both cases:
"wrong password" and "unknown user".

Now I have two distinct jails:
The first one just for "wrong password" and here the findtime, bantime, retries
are tolerant to typos.

And I have a new one just for "unknown user" and here my bantime and findtime
are much bigger and the retries are just '2'. So here I'm much harsher.
I'll keep an eye on my logs and maybe some more twaeking is necessary.

Another interesting observation:
I activated
auth_verbose_passwords = plain
to log the plain password when (and only when) there is "unknown user".
It reveals that all different IPs trying one unknown account always try with the
same stupid password scheme 1234. So this doesn't look very well
coordinated between the bots ;-)


Olaf, how do you do this only for the unknown user?

Can you share the Dovecot settings?

I’m under the same sort of slow distributed attack.

Also the two fail2ban jails would be helpful.



Nothing special in the dovecot config


/etc/fail2ban/jail.local


[dovecot]

enabled = true
filter  = dovecot
action  = iptables-multiport[name=dovecot, 
port="pop3,pop3s,imap,imaps,submission,465,sieve", protocol=tcp]
logpath = /var/log/dovecot
bantime = 600
findtime= 600
maxretry= 5
backend = auto


[dovecot_unknown]

ignoreip = X.X.X.0/24
enabled = true
filter  = dovecot_unknown
action  = iptables-multiport[name=dovecot_unknown, 
port="pop3,pop3s,imap,imaps,submission,465,sieve", protocol=tcp]
logpath = /var/log/dovecot
bantime = 14400
findtime= 14400
maxretry= 2
backend = auto


/etc/fail2ban/filter.d/dovecot.local
=

[INCLUDES]
before = common.conf

[Definition]
failregex =  dovecot: auth-worker\(\d+\): pam\(.*,,\<.*\>\): 
pam_authenticate\(\) failed: Authentication failure \(password mismatch\?\)
ignoreregex =

/etc/fail2ban/filter.d/dovecot_unknown.local


[INCLUDES]
before = common.conf

[Definition]
failregex =  dovecot: auth-worker\(\d+\): pam\(.*,,\<.*\>\): unknown 
user.*
ignoreregex =


The failregex lines may need adaption to your log format.
"fail2ban-regex" is your friend.

On my Dovecot 2.2.31 unknows user log lines are
 Jul 26 14:58:56 irams1 dovecot: auth-worker(2822): 
pam(inikul,112.54.93.34,): unknown user (given password: 
inikul2017)

and "wrong password" lines look like this
 Jul 26 15:01:41 irams1 dovecot: auth-worker(3530): 
pam(johndoe,120.209.164.118,<r+xPDDhVGJh40aR2>): pam_authenticate() failed: 
Authentication failure (password mismatch?)


Regards, Olaf
--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: under another kind of attack

2017-07-26 Thread Olaf Hopp

Dear collegues,

many thanks for your valuable input.

Since we are an university GEO-IP blocking is not an option for us.
Somestimes I think it should ;-)

My "mistake" was that I had just *one* fail2ban filter for both cases:
"wrong password" and "unknown user".

Now I have two distinct jails:
The first one just for "wrong password" and here the findtime, bantime, retries
are tolerant to typos.

And I have a new one just for "unknown user" and here my bantime and findtime
are much bigger and the retries are just '2'. So here I'm much harsher.
I'll keep an eye on my logs and maybe some more twaeking is necessary.

Another interesting observation:
I activated
auth_verbose_passwords = plain
to log the plain password when (and only when) there is "unknown user".
It reveals that all different IPs trying one unknown account always try with the
same stupid password scheme 1234. So this doesn't look very well
coordinated between the bots ;-)

Regards,
Olaf


On 07/25/2017 04:37 PM, Olaf Hopp wrote:

Hi folks,

"somehow" similar to the thread "under some kind oof attack" started by "MJ":

I have dovecot shielded by fail2ban which works fine.
But since a few days I see many many IPs per day knocking on
my doors with wron password and/or users. But the rate at which they are 
knocking
is very very low. So fail2ban will never catch them.

For example one IP:

Jul 25 14:03:17 irams1 dovecot: auth-worker(2212): 
pam(eurodisc,101.231.247.210,): unknown user
Jul 25 15:16:36 irams1 dovecot: auth-worker(11047): 
pam(gergei,101.231.247.210,): pam_authenticate() failed: 
Authentication failure (password mismatch?)
Jul 25 16:08:51 irams1 dovecot: auth-worker(3379): 
pam(icpe,101.231.247.210,): unknown user
Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): 
pam(endsulei,101.231.247.210,): unknown user

Note the timestamps.
If I look the other way round (tries to one account) I'll get

Jul 25 01:30:48 irams1 dovecot: auth-worker(11276): 
pam(endsulei,60.166.12.117,): unknown user
Jul 25 01:31:26 irams1 dovecot: auth-worker(11276): 
pam(endsulei,222.243.211.200,<s0+6nBhVabHe89PI>): unknown user
Jul 25 13:29:22 irams1 dovecot: auth-worker(4745): 
pam(endsulei,60.2.50.114,<4elhpCJVtcw8AjJy>): unknown user
Jul 25 13:30:27 irams1 dovecot: auth-worker(4747): 
pam(endsulei,222.84.118.83,): unknown user
Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): 
pam(endsulei,101.231.247.210,): unknown user
Jul 25 16:11:45 irams1 dovecot: auth-worker(5933): 
pam(endsulei,206.214.0.120,): unknown user

Also note the timestamps!

And I see many many distinct IPs per day (a few hundred) trying many many 
existing and non-existings accounts.
As you see in the timestamps in my examples, this can not be handled by 
fail2ban without affecting
regular users with typos.
Is anybody observing something similar ?
Anybody an idea against this ?
Many of these observed IPs are chinese mobile IPs, if this matters. But we have 
also chinese students and
researchers all abroad.


Regards,
Olaf




--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


under another kind of attack

2017-07-25 Thread Olaf Hopp

Hi folks,

"somehow" similar to the thread "under some kind oof attack" started by "MJ":

I have dovecot shielded by fail2ban which works fine.
But since a few days I see many many IPs per day knocking on
my doors with wron password and/or users. But the rate at which they are 
knocking
is very very low. So fail2ban will never catch them.

For example one IP:

Jul 25 14:03:17 irams1 dovecot: auth-worker(2212): 
pam(eurodisc,101.231.247.210,): unknown user
Jul 25 15:16:36 irams1 dovecot: auth-worker(11047): 
pam(gergei,101.231.247.210,): pam_authenticate() failed: 
Authentication failure (password mismatch?)
Jul 25 16:08:51 irams1 dovecot: auth-worker(3379): 
pam(icpe,101.231.247.210,): unknown user
Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): 
pam(endsulei,101.231.247.210,): unknown user

Note the timestamps.
If I look the other way round (tries to one account) I'll get

Jul 25 01:30:48 irams1 dovecot: auth-worker(11276): 
pam(endsulei,60.166.12.117,): unknown user
Jul 25 01:31:26 irams1 dovecot: auth-worker(11276): 
pam(endsulei,222.243.211.200,<s0+6nBhVabHe89PI>): unknown user
Jul 25 13:29:22 irams1 dovecot: auth-worker(4745): 
pam(endsulei,60.2.50.114,<4elhpCJVtcw8AjJy>): unknown user
Jul 25 13:30:27 irams1 dovecot: auth-worker(4747): 
pam(endsulei,222.84.118.83,): unknown user
Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): 
pam(endsulei,101.231.247.210,): unknown user
Jul 25 16:11:45 irams1 dovecot: auth-worker(5933): 
pam(endsulei,206.214.0.120,): unknown user

Also note the timestamps!

And I see many many distinct IPs per day (a few hundred) trying many many 
existing and non-existings accounts.
As you see in the timestamps in my examples, this can not be handled by 
fail2ban without affecting
regular users with typos.
Is anybody observing something similar ?
Anybody an idea against this ?
Many of these observed IPs are chinese mobile IPs, if this matters. But we have 
also chinese students and
researchers all abroad.


Regards,
Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: v2.2.30.1 released

2017-06-07 Thread Olaf Hopp

On 06/06/2017 01:14 PM, Aki Tuomi wrote:



On 06.06.2017 14:11, Olaf Hopp wrote:

On 06/05/2017 11:05 AM, Angel L. Mateo wrote:

  I have updated my dovecot proxy servers from 2.2.28 to 2.2.30.
Since the upgrade I'm having the error:

Jun  5 10:54:51 musio12 dovecot: auth: Fatal: master: service(auth):
child 63632 killed with signal 11 (core not dumped)





Me too, with

# 2.2.30.1 (eebd877): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.18 (29cc74d)
# OS: Linux 2.6.32-696.3.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)

OS ist up2date.
Please fix this ASAP.

Olaf


Hi!

We have identified a bug in auth process, and are working with a fix.

Aki



Great. Working clean with 2.2.30.2
Thanks, Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: v2.2.30.1 released

2017-06-06 Thread Olaf Hopp

On 06/05/2017 11:05 AM, Angel L. Mateo wrote:

 I have updated my dovecot proxy servers from 2.2.28 to 2.2.30. Since the 
upgrade I'm having the error:

Jun  5 10:54:51 musio12 dovecot: auth: Fatal: master: service(auth): child 
63632 killed with signal 11 (core not dumped)





Me too, with

# 2.2.30.1 (eebd877): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.18 (29cc74d)
# OS: Linux 2.6.32-696.3.1.el6.x86_64 x86_64 CentOS release 6.9 (Final)

OS ist up2date.
Please fix this ASAP.

Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: SSL problem - no banner

2017-05-30 Thread Olaf Hopp

On 05/29/2017 08:35 PM, Aki Tuomi wrote:



On May 29, 2017 at 9:27 PM Marcio Merlone <marcio.merl...@a1.ind.br> wrote:





--
*Marcio Merlone*


It is not exactly obvious what you are expecting to happen. You won't get plain 
text banner out of port 993, if you want to use STARTTLS, use port 143.

Aki



To see the Dovecot-IMAP-banner via SSL-port 993 use openssl:

openssl s_client -connect localhost:993

Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: several misc questions, public folders and sharing, quota, ssl

2017-04-15 Thread Olaf Hopp

On 04/14/2017 02:04 AM, David Mehler wrote:


First ssl, is my cipher list good? I'm trying for pfs and wanting to
ensure these cipherlist is appropriate:

ssl_cipher_list = EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH


You can check the quality of your SSL/TLS setup via
https://www.htbridge.com/ssl/

Regards, Olaf


--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: SSL connection reset by peer

2016-08-02 Thread Olaf Hopp

On 07/27/2016 11:55 PM, Vince42 wrote:

Hi,

[Steffen Kaiser] - [2016-07-26 09:05]

I am running a dovecot server and have set up an external
monitoring, where every five minutes a login with SSL on port
993 is done. I usually get once a day an error "connection
reset by peer - SSL connect", which goes away until the next
monitor is executed.



that looks like a basic networking issue to me. Do you have logs
how many users try to connect at this time? Is it always the same
time range? Is the server load very high?



My server has nice specs (in fact a 30 times lower scaled server
never had this kind of problems), I also don't host many domains
and users, therefore I doubt that some kind of limit might be
touched. I also suspected some internal system load, but
unfortunately the error occurs arbitrarily, which makes me think
that no scheduled process is responsible for this. I also ran 'top'
during such an event without any obvious load tasks. The system
statistics also show no weird peaks. I read about the "running out
of random" phenomenon, but during such an event there were still
enough resources random-wise.



what about the network itself? Does the monitor crosses a firewall?


I do not know all the details about my provider's data center, but the
monitor is an internal one running on one of their machines in their
infrastructure. I therefore doubt that this error could be related to
some network issue. The monitor just makes a normal IMAP login and fails
with the SSL error - and a few minutes later everything is fine again.


Could it be that I need to offer more login processes or that I
should raise some of my configuration values? The
mail_max_userip_connections does not seem to solve the problem.



usually you get some warning in the logs, if such limit is reached.


I desperately searched all kinds of logs - but nothing indicates a
problem that would explain these arbitrary logon errors. I always
thought that I should be more generous with login processes or other
system resources in order to overcome this - but it seems that I am on
the wrong track, if my doveconf -n does not show any oddities.

I fear I will have to accept this error as being "normal" - which is
really odd as my former server ran for years with the same config
without any warning at all. Maybe the next will do it again ... :)))



Hi Vince,
just a shot into the dark:
if you are running out of entropy, you might get SSL errors.
If this is a virtual machine, there are not many entropy sources.
Consider installing alternative entropy sources like haveged(*),
available in many distro repos.

Regards, Olaf

(*) http://www.issihosts.com/haveged/

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: autoexpunge problems

2016-01-12 Thread Olaf Hopp

On 01/04/2016 08:25 PM, Timo Sirainen wrote:

On 18 Dec 2015, at 05:26, Olaf Hopp <olaf.h...@kit.edu> wrote:


Hello,
I tried to use the new autoexpunge for my Trash folders
I had in 15-mailboxes.conf

mailbox Trash {
special_use = \Trash
auto = subscribe
}

(dovecot -n of the original config is below)
and added just the line
  autoexpunge = 1h
Just a short period on my test system.
But it failed. In the log I see

Dec 18 10:54:07 irams2 dovecot: imap(ms2test): Error: Failed to autoexpunge 
mailbox 'Trash': Invalid mailbox name 'Trash': Missing namespace prefix 'INBOX.'


Oops. This should fix it: 
https://github.com/dovecot/core/commit/76e5f0fe5e9e8bdee24d0e047378a665e01b808d



Hi Timo,
now it looks good (and also works fine).
Thanks, Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


autoexpunge problems

2015-12-18 Thread Olaf Hopp

Hello,
I tried to use the new autoexpunge for my Trash folders
I had in 15-mailboxes.conf

 mailbox Trash {
 special_use = \Trash
 auto = subscribe
 }

(dovecot -n of the original config is below)
and added just the line
  autoexpunge = 1h
Just a short period on my test system.
But it failed. In the log I see

Dec 18 10:54:07 irams2 dovecot: imap(ms2test): Error: Failed to autoexpunge 
mailbox 'Trash': Invalid mailbox name 'Trash': Missing namespace prefix 'INBOX.'

But in 10-mail.conf I have

namespace inbox {
  prefix = INBOX.
  separator = .
  inbox = yes
}

So I changed the line "mailbox Trash" to "mailbox  INBOX.Trash". I now have

 mailbox INBOX.Trash {
 special_use = \Trash
 auto = subscribe
 autoexpunge = 1h
 }

Now it seems to work (Trash is emptied, log file is fine)
but in the mail client I see two Trash folders.
Once the normal one below INBOX and another one also called
Trash which is under a new created folder named INBOX below the original INBOX
Thus I have

INBOX
INBOX.Trash
INBOX.INBOX.Trash

autoexpunge seems to work, but this is not what I like to offer to my users.

Any glues ?
Olaf



# 2.2.21 (5345f22): /opt/dovecot/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.10 (d61ff8a5af9e+)
doveconf: Warning: service auth { client_limit=2000 } is lower than required 
under max. load (23192)
doveconf: Warning: service anvil { client_limit=2000 } is lower than required 
under max. load (17195)
# OS: Linux 2.6.32-573.12.1.el6.x86_64 x86_64 CentOS release 6.7 (Final)
auth_failure_delay = 3 secs
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = yes
auth_worker_max_count = 60
default_client_limit = 2000
default_process_limit = 3000
mail_location = maildir:~/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  mailbox spambox {
auto = create
special_use = \Junk
  }
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/master-users
  driver = passwd-file
  master = yes
}
passdb {
  args = dovecot
  driver = pam
}
plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_before = /etc/dovecot/sieve-master
  sieve_max_redirects = 20
}
protocols = imap pop3 lmtp sieve sieve
quota_full_tempfail = yes
service imap-login {
  process_limit = 8192
  process_min_avail = 16
  service_count = 0
}
service imap {
  process_limit = 8192
}
service lmtp {
  executable = lmtp -L
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
service pop3 {
  process_limit = 8192
}
ssl = required
ssl_ca = 

smime.p7s
Description: S/MIME Cryptographic Signature


Re: Why is Sieve trying to re-compile global scripts?

2015-03-16 Thread Olaf Hopp

On 03/15/2015 12:37 AM, Stephan Bosch wrote:

On 3/12/2015 11:53 PM, Stephan Bosch wrote:

On 3/12/2015 11:56 AM, Olaf Hopp wrote:

On 03/12/2015 12:02 AM, Stephan Bosch wrote:


Please do. I cannot reproduce this so far.

Since E.B. still got an obscure debug message about metadata not being
up to date, I added debug lines to the remaining places where this could
emerge (currently only available from hg).

Regards,

Stephan.


Hi,
I'm still trying but currently I can not reproduce the bug.
But I will keep on hammering on it.

Looks like I found the bug. Will need some time to fix this properly.


I released rc2. Please check whether this resolves the issues.



With RC2 everything looks good !

And finally I could reproduce the bug:
with 0.4.5 and 0.4.7 RC1 you can trigger it when you compile
the master sieve script with a *relative* path:

cd /etc/dovecot
/usr/bin/sievec -D ./sieve-master

will trigger it. Whereas
 /usr/bin/sievec -D /etc/dovecot/sieve-master
even with 0.4.5 will run fine.

With 0.4.7 RC2 it makes no difference, wether you use an absolute
or a relative path to the sieve-master script.

Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Universität des Landes Baden-Württemberg und nationales Forschungszentrum 
in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature


Re: Why is Sieve trying to re-compile global scripts?

2015-03-12 Thread Olaf Hopp

On 03/12/2015 12:02 AM, Stephan Bosch wrote:

On 3/11/2015 11:10 AM, Olaf Hopp wrote:

Please see the thread with subject
Sieve permissions issue following update
I tested sucessfully a developper issue last month
on the hint of Stephan. Yesterday I started to test the currenr RCs.

First I was disappointed, because the error seems to persist.
So I double checked everything, recreated / recompiled everything
an the error went away. So I thought it was mistake on my side.
I gave Spephan postive feedback. And I'm waiting for the final release
for my production server.

But when I read your mails, I'm not feeling happy.
I think it's a kink of luck/voodoo/whatever.

What you must do, I think, is to compile the sieve script with the
exact version running afterwards.
And I think you should the remove the compiled .svbin files
before recreating them again. Don't overwrite them with the compiler.

I think I'll also dig into this any further today.


Please do. I cannot reproduce this so far.

Since E.B. still got an obscure debug message about metadata not being
up to date, I added debug lines to the remaining places where this could
emerge (currently only available from hg).

Regards,

Stephan.



Hi,
I'm still trying but currently I can not reproduce the bug.
But I will keep on hammering on it.

Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT - Universität des Landes Baden-Württemberg und nationales Forschungszentrum 
in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Why is Sieve trying to re-compile global scripts?

2015-03-12 Thread Olaf Hopp

On 03/11/2015 07:17 AM, E.B. wrote:


Might be unpredictable caching.  Might be the error didn't go away
last time I recreated due to different methods of creating the files.
Who knows, I think I should give up and stop spamming the list
with uneducated guesswork.


No - no spam at least for me.

Please see the thread with subject
Sieve permissions issue following update
I tested sucessfully a developper issue last month
on the hint of Stephan. Yesterday I started to test the currenr RCs.

First I was disappointed, because the error seems to persist.
So I double checked everything, recreated / recompiled everything
an the error went away. So I thought it was mistake on my side.
I gave Spephan postive feedback. And I'm waiting for the final release
for my production server.

But when I read your mails, I'm not feeling happy.
I think it's a kink of luck/voodoo/whatever.

What you must do, I think, is to compile the sieve script with the
exact version running afterwards.
And I think you should the remove the compiled .svbin files
before recreating them again. Don't overwrite them with the compiler.

I think I'll also dig into this any further today.

Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT - Universität des Landes Baden-Württemberg und nationales Forschungszentrum 
in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: Sieve permissions issue following update

2015-01-26 Thread Olaf Hopp

On 01/01/2015 05:22 PM, Stephan Bosch wrote:

On 1/1/2015 4:17 PM, Robert Blayzor wrote:

On Jan 1, 2015, at 9:58 AM, Robert Blayzor rblayzor.b...@inoc.net wrote:

Hmm. This smells like a bug. I notice that your modification times of
the .sieve and .svbin file are exactly the same (that is somewhat
unusual). I'm looking at a potential bug that would explain your problem.

To confirm, could you try running sievec again, so that the .svbin is
actually newer than the .sieve?


If it makes any difference at all...  I only see this using dovecot-lda.  If 
I change my Exim transport to use Dovecot's LMTP, I do not see this problem.


That is odd.



Hi Stephan and Robert,
the same issue here and I'm using Exim with dovecot-lmtp and
not with dovecot-lda.
So it doesn't seem to be a problem of LDA vs. lmtp

Pigeonhole 0.4.5
Dovecot2.2.15
CentOS 6.6

Regards,
Olaf

--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
atis.informatik.kit.edu

www.kit.edu

KIT - Universität des Landes Baden-Württemberg und nationales Forschungszentrum 
in der Helmholtz-Gemeinschaft




smime.p7s
Description: S/MIME Cryptographic Signature