Re: [Dovecot] IMAP Folders Don't Make Sense
Hello, Le 27/02/10 09:45, Stan Hoeppner a écrit : > Brian Hayden put forth on 2/25/2010 3:10 PM: > > To add insult to injury, mailbox migration from Exch 5.5 IMAP to Dovecot > IMAP was done using TB as the conduit, simply copying folders from one > "account" to another. Took forever, but it was the only reliable way I > could think of at the time to do it. It actually went very smoothly. > imapsync is a _very_ useful tool to accomplish this. I don't know Exchange, but if it allows you to have a kind of "admin" account having access to all accounts or at least one at a time (and automatically select which one), you can even automate the migration. On Dovecot, I set a non-real user account, with the mailbox path being the same as the user under migration. A few scripting and you're done. I've used it to move accounts from Gmail to Dovecot and from Cyrus IMAP to Dovecot. Only pay attention with Gmail special IMAP folders: "All Messages" and so forth. Regards, -- Baptiste MALGUY - http://www.malguy.net PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 signature.asc Description: OpenPGP digital signature
Re: [Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello, In case my nasty changes are right, here comes the patch (including Makefile.in & Makefile.am ... I'm not really good with automake/autoconf) I've applied to dovecot 1.2.10 (from Debian backport repository for Lenny, source package version 1:1.2.10-1~bpo50+1, expecting my changes won't be different from the vanilla version). Regards, -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/lazy-expunge-plugin.c dovecot-1.2.10-patched/src/plugins/lazy-expunge/lazy-expunge-plugin.c --- dovecot-1.2.10/src/plugins/lazy-expunge/lazy-expunge-plugin.c 2010-01-25 00:14:17.0 +0100 +++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/lazy-expunge-plugin.c 2010-02-19 18:29:27.0 +0100 @@ -630,24 +630,11 @@ struct lazy_expunge_mailbox_list *llist = LAZY_EXPUNGE_LIST_CONTEXT(storage->list); struct lazy_expunge_mail_storage *lstorage; - const char *const *p; - unsigned int i; - if (storage->ns->type != NAMESPACE_PRIVATE) { /* this works only for private namespaces. */ return; } - /* if this is one of our internal storages, mark it as such before - quota plugin sees it */ - p = t_strsplit_spaces(getenv("LAZY_EXPUNGE"), " "); - for (i = 0; i < LAZY_NAMESPACE_COUNT && *p != NULL; i++, p++) { - if (strcmp(storage->ns->prefix, *p) == 0) { - storage->ns->flags |= NAMESPACE_FLAG_NOQUOTA; - break; - } - } - llist->storage = storage; lstorage = p_new(storage->pool, struct lazy_expunge_mail_storage, 1); @@ -670,6 +657,18 @@ static void lazy_expunge_mailbox_list_created(struct mailbox_list *list) { struct lazy_expunge_mailbox_list *llist; + const char *const *p; + unsigned int i; + + /* if this is one of our internal storages, mark it as such before + quota plugin sees it */ + p = t_strsplit_spaces(getenv("LAZY_EXPUNGE"), " "); + for (i = 0; i < LAZY_NAMESPACE_COUNT && *p != NULL; i++, p++) { + if (strcmp(list->ns->prefix, *p) == 0) { + list->ns->flags |= NAMESPACE_FLAG_NOQUOTA; + break; + } + } if (lazy_expunge_next_hook_mailbox_list_created != NULL) lazy_expunge_next_hook_mailbox_list_created(list); diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.am dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.am --- dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.am 2008-06-17 03:58:38.0 +0200 +++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.am 2010-02-19 19:28:15.0 +0100 @@ -8,12 +8,12 @@ -I$(top_srcdir)/src/lib-imap \ -I$(top_srcdir)/src/plugins/quota -lib02_lazy_expunge_plugin_la_LDFLAGS = -module -avoid-version +lib09_lazy_expunge_plugin_la_LDFLAGS = -module -avoid-version module_LTLIBRARIES = \ - lib02_lazy_expunge_plugin.la + lib09_lazy_expunge_plugin.la -lib02_lazy_expunge_plugin_la_SOURCES = \ +lib09_lazy_expunge_plugin_la_SOURCES = \ lazy-expunge-plugin.c noinst_HEADERS = \ @@ -22,7 +22,7 @@ install-exec-local: for d in imap pop3; do \ $(mkdir_p) $(DESTDIR)$(moduledir)/$$d; \ - rm -f $(DESTDIR)$(moduledir)/$$d/lib02_lazy_expunge_plugin$(MODULE_SUFFIX); \ - $(LN_S) ../lib02_lazy_expunge_plugin$(MODULE_SUFFIX) $(DESTDIR)$(moduledir)/$$d; \ + rm -f $(DESTDIR)$(moduledir)/$$d/lib09_lazy_expunge_plugin$(MODULE_SUFFIX); \ + $(LN_S) ../lib09_lazy_expunge_plugin$(MODULE_SUFFIX) $(DESTDIR)$(moduledir)/$$d; \ done diff -ru --exclude=debian dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.in dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.in --- dovecot-1.2.10/src/plugins/lazy-expunge/Makefile.in 2010-01-25 00:32:10.0 +0100 +++ dovecot-1.2.10-patched/src/plugins/lazy-expunge/Makefile.in 2010-02-19 21:00:42.0 +0100 @@ -69,13 +69,13 @@ sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' am__installdirs = "$(DESTDIR)$(moduledir)" LTLIBRARIES = $(module_LTLIBRARIES) -lib02_lazy_expunge_plugin_la_LIBADD = -am_lib02_lazy_expunge_plugin_la_OBJECTS = lazy-expunge-plugin.lo -lib02_lazy_expunge_plugin_la_OBJECTS = \ - $(am_lib02_lazy_expunge_plugin_la_OBJECTS) -lib02_lazy_expunge_plugin_la_LINK = $(LIBTOOL) --tag=CC \ +lib09_lazy_expunge_plugin_la_LIBADD = +am_lib09_lazy_expunge_plugin_la_OBJECTS = lazy-expunge-plugin.lo +lib09_lazy_expunge_plugin_la_OBJECTS = \ + $(am_lib09_lazy_expunge_plugin_la_OBJECTS) +lib09_lazy_expunge_plugin_la_LINK = $(LIBTOOL) --tag=CC \ $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \ - $(AM_CFLAGS) $(CFLAGS) $(lib02_lazy_expunge_plugin_la_LDFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) $(lib09_lazy_expunge_plugin_la_LDFLAGS) \ $(LDFLAGS) -o $@ DEFAULT_INCLUDES = -...@am__isrc@ -I$(top_builddir) depcomp = $(SHELL) $(top_srcdir)/depcomp @@ -90,8 +90,8 @@ LINK = $(LIBTOOL) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) \ --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLA
[Dovecot] quota and lazy_expunge plugins both used: quotas go wrong with lazy_expunge'd mails
Hello, I may have missed a point, but I could have found an issue between plugins quota and lazy_expunge. I have noticed that the "lazy_expunge"d mails are counted as part of the quota, while the documentation says it should not be. I have first tried to find out if this was a configuration mistake somewhere. Notably, you can notice that the expunged mails are in directory ~/expunged while the usual mail is in ~/Maildir : expunged mails are not in a subdirectory of ~/Maildir (this was just to make sure there were no side effect with this). I have deleted some expunged mails to be sure : at its next cycle, the file ~/Maildir/maildirsize contained expected value. So yes, I do know expunged mails in the expunged directory are counted (size, number of mails). So I've taken a look at the source code, adding some i_info(...) at different places of the quota and lazy_expunge plugins. >From the source code reading session, I understand that : 1. In src/lazy-expunge/lazy-expunge-plugin.c 1.1. lazy_expunge_mail_storage_init() sets the expunged mails namespace flag with NAMESPACE_FLAG_NOQUOTA 1.2. lazy_expunge_mail_storage_init() is called by lazy_expunge_mail_storage_created() 1.3. lazy_expunge_mail_storage_created() is part of the callback list hook_mail_storage_create 2. In src/plugins/quota/quota-storage.c: 2.1. The only place in the whole dovecot source code where NAMESPACE_FLAG_NOQUOTA is checked is in function quota_mailbox_list_created() 3. In src/plugins/quota/quota-plugin.c: 3.1 quota_plugin_init() add quota_mailbox_list_created() to the callback list hook_mailbox_list_created. >From my observations : - Callback functions in quota_mailbox_list_created are called _before_ callback functions in hook_mail_storage_create. - This observation led me to move the piece of code that sets NAMESPACE_FLAG_NOQUOTA from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created(). - But there, lazy_expunge_mail_storage_created() is called after quota_mail_storage_created() because names of the library files are lib10_quota_plugin.so and lib02_lazy_expunge_plugin.so. - There I have also learnt that callback functions of the same list are called in the reverse order compared to the library filenames. So see if my observations were right, I have renamed lib10_quota_plugin.so to lib01_quota_plugin.so, then it worked. But I doubt this is the right way to fix it. This also reverses the order other callbacks between the two plugins are called, which migh not be expected at all. I'm not event sure moving some code from lazy_expunge_mail_storage_init() to lazy_expunge_mail_storage_created() was right : I've improvized the reading the source code and do not understand the whole logic, so I suppose this could break something else. Timo, what do you propose ? Thanks for your great software. -- Baptiste MALGUY PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2 # 1.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-bpo.1-amd64 x86_64 Debian 5.0.4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve ssl: required ssl_ca_file: /etc/ssl/certs/ ssl_cert_file: /etc/ssl/certs/x ssl_key_file: /etc/ssl/private/x ssl_parameters_regenerate: 24 verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_greeting: X login_max_processes_count: 256 max_mail_processes: 1024 first_valid_uid: 500 last_valid_uid: 500 first_valid_gid: 501 last_valid_gid: 501 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(imap): autocreate expire fts fts_squat lazy_expunge quota imap_quota antispam acl imap_acl mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: shared separator: / prefix: shared/%%u/ location: maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u list: yes namespace: type: private separator: / prefix: .ARCHIVES/ location: maildir:~/Maildir/archives list: yes subscriptions: yes namespace: type: private separator: / prefix: .EXPUNGED/ location: maildir:~/expunged list: yes subscriptions: yes lda: postmaster_address: postmas...@xx mail_plugins: sieve expire acl quota_full_tempfail: yes auth default:
Re: [Dovecot] mail_uid and mail_gid are not set in environment (for expire-tool and so)
I haven't been into details about my userdb. It's a LDAP backend. Before mail_uid and mail_gid parameters arrive, I'd have to play the trick with the value returned by the LDAP backend, adding some uid/gid values. Now, with these new configuration parameters, I understand I'm not supposed to workaround this way, but use mail_uid and mail_gid. By the way, in the meantime, I've added : export MAIL_UID=vmail export MAIL_GID=vmail in my my-expire-tool script, waiting for a nicer solution. Regards, -- Baptiste MALGUY - http://www.malguy.net PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2
[Dovecot] mail_uid and mail_gid are not set in environment (for expire-tool and so)
Hello, I've just subscribed to the list, as a recent user of Dovecot (1.2.4 by now). I eventually missed a point, so I tried to find other doc or comments about my issue, but failed to. If the following has already been discussed, please simply reply with a URL to the corresponding thread or doc ;-) Now that I have covered my ass, here is the problem I meet. In dovecot.conf, I have the following: mail_uid = vmail mail_guid = vmail which is confirmed with 'dovecot -n' I have a script my-expire-tool-test containing: #!/bin/sh set which basically only display environment variables. When I run : /usr/sbin/dovecot --exec-mail ext my-expire-tool-test I do not see any variable MAIL_UID nor MAIL_GID. As a consquence, when I run /usr/sbin/dovecot --exec-mail ext my-expire-tool with my-expire-tool containing: #!/bin/sh # This script is a workaround for Dovecot 1.2 having issue to load modules when running expire-tool. # More information: http://wiki.dovecot.org/Plugins/Expire, search Dovecot v1.2 MAIL_PLUGINS=${MAIL_PLUGINS//antispam/} MAIL_PLUGINS=${MAIL_PLUGINS//imap_acl/} MAIL_PLUGINS=${MAIL_PLUGINS//imap_quota/} MAIL_PLUGINS=${MAIL_PLUGINS//mail_log/} exec /usr/lib/dovecot/expire-tool "$@" It politely tells me: Error: userdb(u...@domain) didn't return uid and mail_uid not set Actually, this is this message error that lead me to check the environment variables. Then I also checked the source code, and run: grep -r env_put . | grep -i mail_ No trace of mail_uid nor mail_gid. Did I find a bug or did I miss a point ? Regards, -- Baptiste MALGUY - http://www.malguy.net PGP fingerprint: 49B0 4F6E 4AA8 B149 B2DF 9267 0F65 6C1C C473 6EC2