Re: [Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
On Fri, 2010-02-19 at 17:17 +0100, Baptiste Malguy wrote: - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. Uh oh. This is completely the reverse of how they were supposed to be working. I'm surprised that there haven't been more problems. But it's probably a bit too dangerous to change it anymore now in v1.x. It should be fixed already with v2.0.
Re: [Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello,
In case my nasty changes are right, here comes the patch (including
Makefile.in Makefile.am ... I'm not really good with automake/autoconf)
I've applied to dovecot 1.2.10 (from Debian backport repository for Lenny,
source package version 1:1.2.10-1~bpo50+1, expecting my changes won't be
different from the vanilla version).
Regards,
--
Baptiste MALGUY
PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2
diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/lazy-expunge-plugin.c dovecot-1.2.10-patched/src/plugins/lazy-expunge/lazy-expunge-plugin.c
--- dovecot-1.2.10/src/plugins/lazy-expunge/lazy-expunge-plugin.c 2010-01-25 00:14:17.0 +0100
+++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/lazy-expunge-plugin.c 2010-02-19 18:29:27.0 +0100
@@ -630,24 +630,11 @@
struct lazy_expunge_mailbox_list *llist =
LAZY_EXPUNGE_LIST_CONTEXT(storage-list);
struct lazy_expunge_mail_storage *lstorage;
- const char *const *p;
- unsigned int i;
-
if (storage-ns-type != NAMESPACE_PRIVATE) {
/* this works only for private namespaces. */
return;
}
- /* if this is one of our internal storages, mark it as such before
- quota plugin sees it */
- p = t_strsplit_spaces(getenv(LAZY_EXPUNGE), );
- for (i = 0; i LAZY_NAMESPACE_COUNT *p != NULL; i++, p++) {
- if (strcmp(storage-ns-prefix, *p) == 0) {
- storage-ns-flags |= NAMESPACE_FLAG_NOQUOTA;
- break;
- }
- }
-
llist-storage = storage;
lstorage = p_new(storage-pool, struct lazy_expunge_mail_storage, 1);
@@ -670,6 +657,18 @@
static void lazy_expunge_mailbox_list_created(struct mailbox_list *list)
{
struct lazy_expunge_mailbox_list *llist;
+ const char *const *p;
+ unsigned int i;
+
+ /* if this is one of our internal storages, mark it as such before
+ quota plugin sees it */
+ p = t_strsplit_spaces(getenv(LAZY_EXPUNGE), );
+ for (i = 0; i LAZY_NAMESPACE_COUNT *p != NULL; i++, p++) {
+ if (strcmp(list-ns-prefix, *p) == 0) {
+ list-ns-flags |= NAMESPACE_FLAG_NOQUOTA;
+ break;
+ }
+ }
if (lazy_expunge_next_hook_mailbox_list_created != NULL)
lazy_expunge_next_hook_mailbox_list_created(list);
diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.am dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.am
--- dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.am 2008-06-17 03:58:38.0 +0200
+++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.am 2010-02-19 19:28:15.0 +0100
@@ -8,12 +8,12 @@
-I$(top_srcdir)/src/lib-imap \
-I$(top_srcdir)/src/plugins/quota
-lib02_lazy_expunge_plugin_la_LDFLAGS = -module -avoid-version
+lib09_lazy_expunge_plugin_la_LDFLAGS = -module -avoid-version
module_LTLIBRARIES = \
- lib02_lazy_expunge_plugin.la
+ lib09_lazy_expunge_plugin.la
-lib02_lazy_expunge_plugin_la_SOURCES = \
+lib09_lazy_expunge_plugin_la_SOURCES = \
lazy-expunge-plugin.c
noinst_HEADERS = \
@@ -22,7 +22,7 @@
install-exec-local:
for d in imap pop3; do \
$(mkdir_p) $(DESTDIR)$(moduledir)/$$d; \
- rm -f $(DESTDIR)$(moduledir)/$$d/lib02_lazy_expunge_plugin$(MODULE_SUFFIX); \
- $(LN_S) ../lib02_lazy_expunge_plugin$(MODULE_SUFFIX) $(DESTDIR)$(moduledir)/$$d; \
+ rm -f $(DESTDIR)$(moduledir)/$$d/lib09_lazy_expunge_plugin$(MODULE_SUFFIX); \
+ $(LN_S) ../lib09_lazy_expunge_plugin$(MODULE_SUFFIX) $(DESTDIR)$(moduledir)/$$d; \
done
diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.in dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.in
--- dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.in 2010-01-25 00:32:10.0 +0100
+++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.in 2010-02-19 21:00:42.0 +0100
@@ -69,13 +69,13 @@
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
am__installdirs = $(DESTDIR)$(moduledir)
LTLIBRARIES = $(module_LTLIBRARIES)
-lib02_lazy_expunge_plugin_la_LIBADD =
-am_lib02_lazy_expunge_plugin_la_OBJECTS = lazy-expunge-plugin.lo
-lib02_lazy_expunge_plugin_la_OBJECTS = \
- $(am_lib02_lazy_expunge_plugin_la_OBJECTS)
-lib02_lazy_expunge_plugin_la_LINK = $(LIBTOOL) --tag=CC \
+lib09_lazy_expunge_plugin_la_LIBADD =
+am_lib09_lazy_expunge_plugin_la_OBJECTS = lazy-expunge-plugin.lo
+lib09_lazy_expunge_plugin_la_OBJECTS = \
+ $(am_lib09_lazy_expunge_plugin_la_OBJECTS)
+lib09_lazy_expunge_plugin_la_LINK = $(LIBTOOL) --tag=CC \
$(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \
- $(AM_CFLAGS) $(CFLAGS) $(lib02_lazy_expunge_plugin_la_LDFLAGS) \
+ $(AM_CFLAGS) $(CFLAGS) $(lib09_lazy_expunge_plugin_la_LDFLAGS) \
$(LDFLAGS) -o $@
DEFAULT_INCLUDES = -...@am__isrc@ -I$(top_builddir)
depcomp = $(SHELL) $(top_srcdir)/depcomp
@@ -90,8 +90,8 @@
LINK = $(LIBTOOL) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) \
--mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) \
$(LDFLAGS) -o $@
-SOURCES = $(lib02_lazy_expunge_plugin_la_SOURCES)
-DIST_SOURCES = $(lib02_lazy_expunge_plugin_la_SOURCES)
+SOURCES = $(lib09_lazy_expunge_plugin_la_SOURCES)
+DIST_SOURCES =
[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello, I may have missed a point, but I could have found an issue between plugins quota and lazy_expunge. I have noticed that the lazy_expunged mails are counted as part of the quota, while the documentation says it should not be. I have first tried to find out if this was a configuration mistake somewhere. Notably, you can notice that the expunged mails are in directory ~/expunged while the usual mail is in ~/Maildir : expunged mails are not in a subdirectory of ~/Maildir (this was just to make sure there were no side effect with this). I have deleted some expunged mails to be sure : at its next cycle, the file ~/Maildir/maildirsize contained expected value. So yes, I do know expunged mails in the expunged directory are counted (size, number of mails). So I've taken a look at the source code, adding some i_info(...) at different places of the quota and lazy_expunge plugins. From the source code reading session, I understand that : 1. In src/lazy-expunge/lazy-expunge-plugin.c 1.1. lazy_expunge_mail_storage_init() sets the expunged mails namespace flag with NAMESPACE_FLAG_NOQUOTA 1.2. lazy_expunge_mail_storage_init() is called by lazy_expunge_mail_storage_created() 1.3. lazy_expunge_mail_storage_created() is part of the callback list hook_mail_storage_create 2. In src/plugins/quota/quota-storage.c: 2.1. The only place in the whole dovecot source code where NAMESPACE_FLAG_NOQUOTA is checked is in function quota_mailbox_list_created() 3. In src/plugins/quota/quota-plugin.c: 3.1 quota_plugin_init() add quota_mailbox_list_created() to the callback list hook_mailbox_list_created. From my observations : - Callback functions in quota_mailbox_list_created are called _before_ callback functions in hook_mail_storage_create. - This observation led me to move the piece of code that sets NAMESPACE_FLAG_NOQUOTA from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created(). - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. - There I have also learnt that callback functions of the same list are called in the reverse order compared to the library filenames. So see if my observations were right, I have renamed lib10_quota_plugin.so to lib01_quota_plugin.so, then it worked. But I doubt this is the right way to fix it. This also reverses the order other callbacks between the two plugins are called, which migh not be expected at all. I'm not event sure moving some code from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created() was right : I've improvized the reading the source code and do not understand the whole logic, so I suppose this could break something else. Timo, what do you propose ? Thanks for your great software. -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 # 1.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-bpo.1-amd64 x86_64 Debian 5.0.4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl: required ssl_ca_file: /etc/ssl/certs/ ssl_cert_file: /etc/ssl/certs/x ssl_key_file: /etc/ssl/private/x ssl_parameters_regenerate: 24 verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_greeting: X login_max_processes_count: 256 max_mail_processes: 1024 first_valid_uid: 500 last_valid_uid: 500 first_valid_gid: 501 last_valid_gid: 501 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(imap): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u list: yes namespace: type: private separator: / prefix: .ARCHIVES/ location: maildir:~/Maildir/archives list: yes subscriptions: yes namespace: type: private separator: / prefix: .EXPUNGED/ location: maildir:~/expunged list: yes subscriptions: yes lda: postmaster_address: postmas...@xx mail_plugins: sieve expire acl quota_full_tempfail: yes auth default: debug: yes passdb: driver: pam
