subject:"Re\: replication and spam removal \(\"doveadm expunge\"\)"

Re: replication and spam removal ("doveadm expunge")

2020-04-06 Thread Olaf Hopp


OK, further findings:
when user logs in on the node where the "doveadm expunge" was run
this has no effect on the spambox the other side.

But if the user logs in on the other side (after a manual failover of the 
"cluster" IP)
and opens his spambox then all of a sudden those expunged mails got deleted.

So if this behaviour is the intended then you may regard this issue as closed.

Thanks, Olaf



On 4/6/20 12:40 PM, Olaf Hopp wrote:


Hi Aki,

On 4/4/20 8:12 PM, Aki Tuomi wrote:

Can you provide doveconf -n and try turning on mail_debug=yes on both ends and 
try doveadm -Dv expunge 


mail_debug=yes
is on on both ends and dovecot was restarted but anyway nothing is logged when I issue 
"doveadm -Dv expunge "
In the shell where I issue the "expunge" I see the following:

# /usr/bin/doveadm -Dv  expunge  -u test4  mailbox INBOX.spambox  BEFORE 13m
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: 
acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: 
expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: 
quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined 
symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore 
this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: 
fts_user_get_language_list (this is usually intentional, so just ignore this 
message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: 
mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore 
this message)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Started 
userdb lookup
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Client connected (fd=10)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): auth USER 
input: test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Finished 
userdb lookup (username=test4 system_groups_user=test4 uid=1805 gid=2300 
home=/home/irams1-test/test4)
doveadm(test4): Debug: Effective uid=1805, gid=2300, 
home=/home/irams1-test/test4
doveadm(test4): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., 
inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(test4): Debug: maildir++: root=/home/irams1-test/test4/Maildir, index=, 
indexpvt=, control=, inbox=/home/irams1-test/test4/Maildir, alt=
doveadm(test4): Debug: Namespace : type=private, prefix=, sep=, inbox=no, 
hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
doveadm(test4): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
doveadm(test4): Debug: Mailbox INBOX.spambox: Mailbox opened because: expunge
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=38
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=39
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=40
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=41
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=42
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=43
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=44
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=45
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=46
doveadm(test4): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb 
(pid=15116,uid=0): Disconnected: Connection closed (fd=10)

A few mails are deleted from the spambox but the deletion is not synced to the 
other side
Even if I do a "doveadm  force-resync  -u test4 '*'" on both sides the 
deletions are not replicated
but anyway

#doveadm replicator status test4
username  
priority fast sync full sync success sync failed
test4

Re: replication and spam removal ("doveadm expunge")

2020-04-06 Thread Olaf Hopp



Hi Aki,

On 4/4/20 8:12 PM, Aki Tuomi wrote:

Can you provide doveconf -n and try turning on mail_debug=yes on both ends and 
try doveadm -Dv expunge 


mail_debug=yes
is on on both ends and dovecot was restarted but anyway nothing is logged when I issue 
"doveadm -Dv expunge "
In the shell where I issue the "expunge" I see the following:

# /usr/bin/doveadm -Dv  expunge  -u test4  mailbox INBOX.spambox  BEFORE 13m
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: 
acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: 
expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: 
quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined 
symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore 
this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: 
fts_user_get_language_list (this is usually intentional, so just ignore this 
message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: 
/usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: 
mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore 
this message)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Started 
userdb lookup
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(test4)<19830><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb (pid=15116,uid=0): Client connected (fd=10)
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): auth USER 
input: test4 system_groups_user=test4 uid=1805 gid=2300 home=/home/irams1-test/test4
doveadm(test4)<19830><>: Debug: auth-master: userdb lookup(test4): Finished 
userdb lookup (username=test4 system_groups_user=test4 uid=1805 gid=2300 
home=/home/irams1-test/test4)
doveadm(test4): Debug: Effective uid=1805, gid=2300, 
home=/home/irams1-test/test4
doveadm(test4): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., 
inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(test4): Debug: maildir++: root=/home/irams1-test/test4/Maildir, index=, 
indexpvt=, control=, inbox=/home/irams1-test/test4/Maildir, alt=
doveadm(test4): Debug: Namespace : type=private, prefix=, sep=, inbox=no, 
hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
doveadm(test4): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
doveadm(test4): Debug: Mailbox INBOX.spambox: Mailbox opened because: expunge
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=38
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=39
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=40
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=41
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=42
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=43
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=44
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=45
doveadm(test4): Debug: expunge: box=INBOX.spambox uid=46
doveadm(test4): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb 
(pid=15116,uid=0): Disconnected: Connection closed (fd=10)

A few mails are deleted from the spambox but the deletion is not synced to the 
other side
Even if I do a "doveadm  force-resync  -u test4 '*'" on both sides the 
deletions are not replicated
but anyway

#doveadm replicator status test4
username  
priority fast sync full sync success sync failed
test4 
none 00:26:47  18:19:46  00:26:47 -

on both sides.
If new spam arrives it got well replicated to the other side as expected

"doveconf -n" ist attached below.

Thanks,
Olaf




Aki

On 04/04/2020 20:03 Olaf Hopp < olaf.h...@kit.edu > 
wrote:


Nobody ? :-(

On 3/30/20 5:26 PM, Olaf Hopp wrote:

Hello everybody,
since now I did no replication and spam is delivered into users fol

Re: replication and spam removal ("doveadm expunge")

2020-04-04 Thread Aki Tuomi



 
 
  
   Can you provide doveconf -n and try turning on mail_debug=yes on both ends and try doveadm -Dv expunge 
  
  
   
  
  
   Aki
  
  
   
On 04/04/2020 20:03 Olaf Hopp <
olaf.h...@kit.edu> wrote:
   
   

   
   

   
   
Nobody ? :-(
   
   

   
   
On 3/30/20 5:26 PM, Olaf Hopp wrote:
   
   

 Hello everybody,


 since now I did no replication and spam is delivered into users folder "spambox"


 Every night there is a cronjob which deletes spam older than 30 days via something like


 "find  -ctime +30 -delete"


 Now I'm going to set up replication (two way) and I thought that


 doing "rm" is not a good idea.


 So I modified the job to something like


 /usr/bin/doveadm expunge  -u test1 mailbox INBOX.spambox  BEFORE  30d


 which works like intended, but I see that on the replication


 partner the spam isn't deleted. Even if I do a


 doveadm  force-resync  -u test1 '*'


 or log in via imap to the replication partner I still see the old spam.


 


 So my question is:


 is this the intended behaviour and I have to run the "doveadm expunge" on both replication partners


 or should the deletion via "expunge" being replicated to the partner and I have a misconfiguration


 or maybe I hit a bug ?


 


 Regards, Olaf


 


 


 /etc/dovecot/conf.d/12-replication.conf:


 


 


 mail_plugins = $mail_plugins notify replication


 service aggregator {


     fifo_listener replication-notify-fifo {


     mode = 0666


     }


     unix_listener replication-notify {


     mode = 0666


     }


 }


 service replicator {


     process_min_avail = 1


     unix_listener replicator-doveadm {


     mode = 0666


     }


 }


 service doveadm {


     inet_listener {


     port = 1109


     }


 }


 doveadm_password = X


 plugin {


     mail_replica = tcp:X.Y.Z.X:1109


 }


 


 

   
   

   
   
--
   
   
Karlsruher Institut für Technologie (KIT)
   
   
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik
   
   

   
   
Dipl.-Geophys. Olaf Hopp
   
   
- Leitung IT-Dienste -
   
   

   
   
Am Fasanengarten 5, Gebäude 50.34, Raum 009
   
   
76131 Karlsruhe
   
   
Telefon: +49 721 608-43973
   
   
Fax: +49 721 608-46699
   
   
E-Mail: 
olaf.h...@kit.edu
   
   
www.atis.informatik.kit.edu
   
   

   
   
www.kit.edu
   
   

   
   
KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft
   
   

   
   
Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.
   
  
  
   
  
  
   ---
Aki Tuomi

Re: replication and spam removal ("doveadm expunge")

2020-04-04 Thread Olaf Hopp



Nobody ? :-(

On 3/30/20 5:26 PM, Olaf Hopp wrote:

Hello everybody,
since now I did no replication and spam is delivered into users folder "spambox"
Every night there is a cronjob which deletes spam older than 30 days via 
something like
 "find  -ctime +30 -delete"
Now I'm going to set up replication (two way) and I thought that
doing "rm" is not a good idea.
So I modified the job to something like
 /usr/bin/doveadm expunge  -u test1 mailbox INBOX.spambox  BEFORE  30d
which works like intended, but I see that on the replication
partner the spam isn't deleted. Even if I do a
 doveadm  force-resync  -u test1 '*'
or log in via imap to the replication partner I still see the old spam.

So my question is:
is this the intended behaviour and I have to run the "doveadm expunge" on both 
replication partners
or should the deletion via "expunge" being replicated to the partner and I have 
a misconfiguration
or maybe I hit a bug ?

Regards, Olaf


/etc/dovecot/conf.d/12-replication.conf:


mail_plugins = $mail_plugins notify replication
service aggregator {
     fifo_listener replication-notify-fifo {
     mode = 0666
     }
     unix_listener replication-notify {
     mode = 0666
     }
}
service replicator {
     process_min_avail = 1
     unix_listener replicator-doveadm {
     mode = 0666
     }
}
service doveadm {
     inet_listener {
     port = 1109
     }
}
doveadm_password = X
plugin {
     mail_replica = tcp:X.Y.Z.X:1109
}





--
Karlsruher Institut für Technologie (KIT)
ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik

Dipl.-Geophys. Olaf Hopp
- Leitung IT-Dienste -

Am Fasanengarten 5, Gebäude 50.34, Raum 009
76131 Karlsruhe
Telefon: +49 721 608-43973
Fax: +49 721 608-46699
E-Mail: olaf.h...@kit.edu
www.atis.informatik.kit.edu

www.kit.edu

KIT - Die Forschungsuniversität in der Helmholtz-Gemeinschaft

Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.



smime.p7s
Description: S/MIME Cryptographic Signature

Re: replication and spam removal ("doveadm expunge")

Re: replication and spam removal ("doveadm expunge")

Re: replication and spam removal ("doveadm expunge")

Re: replication and spam removal ("doveadm expunge")

4 matches

Site Navigation

Mail list logo

Footer information