Re: [Dovecot] problems when using dsync between servers

[Yann Dupont]

Le 08/12/2011 06:15, Timo Sirainen a écrit :

On Wed, 2011-12-07 at 12:02 +0100, Yann Dupont wrote:


Much faster (using all the cores), but dsync still hangs on some
accounts. After a certain time dsync exit with an error like that :

   Panic: file dsync-worker-local.c: line 1688
(local_worker_save_msg_continue): assertion failed: (ret == -1)


Unfortunately there are still some bugs in dsync.. But you could work
around this by checking if dsync returned failure, and if it did just
retry the user.


ok will try this.


This is on a RUNNNING server.

My aim was to make some pass on the running server, then stop it and
make a last pass to finish the process.


That should work.


Is dsync supposed to run only on a stopped server ?


No.


OK, thanks for your answer,

--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : yann.dup...@univ-nantes.fr

Re: [Dovecot] using dsync to convert mailboxes looses caching options

[Timo Sirainen]

On Thu, 2011-12-08 at 07:53 +0200, Timo Sirainen wrote:

> But yes, it is a problem that dsync doesn't update caching decisions..
> Hmm. I guess I'll have to fix that for v2.1.

Could you try if the attached patch fixes your problems when patching
against latest v2.1 hg? It's annoyingly large, and it makes v2.1 dsync
incompatible with v2.0, but maybe it's better to do it sooner than
later..

diff -r ddfe3a0f75e6 src/doveadm/doveadm-dump-index.c
--- a/src/doveadm/doveadm-dump-index.c	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/doveadm/doveadm-dump-index.c	Thu Dec 08 09:32:03 2011 +0200
@@ -9,7 +9,6 @@
 #include "message-part-serialize.h"
 #include "mail-index-private.h"
 #include "mail-cache-private.h"
-#include "mail-cache-private.h"
 #include "mail-index-modseq.h"
 #include "doveadm-dump.h"
 
@@ -344,7 +343,7 @@
 			printf("   - ");
 		printf("%-4s %.16s\n",
 		   cache_decision2str(field->decision),
-		   unixdate2str(cache->fields[cache_idx].last_used));
+		   unixdate2str(field->last_used));
 	}
 }
 
diff -r ddfe3a0f75e6 src/dsync/dsync-data.c
--- a/src/dsync/dsync-data.c	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/dsync/dsync-data.c	Thu Dec 08 09:32:03 2011 +0200
@@ -10,7 +10,8 @@
 dsync_mailbox_dup(pool_t pool, const struct dsync_mailbox *box)
 {
 	struct dsync_mailbox *dest;
-	const char *const *cache_fields = NULL, *dup;
+	const struct mailbox_cache_field *cache_fields = NULL;
+	struct mailbox_cache_field *dup;
 	unsigned int i, count = 0;
 
 	dest = p_new(pool, struct dsync_mailbox, 1);
@@ -24,8 +25,9 @@
 	else {
 		p_array_init(&dest->cache_fields, pool, count);
 		for (i = 0; i < count; i++) {
-			dup = p_strdup(pool, cache_fields[i]);
-			array_append(&dest->cache_fields, &dup, 1);
+			dup = array_append_space(&dest->cache_fields);
+			*dup = cache_fields[i];
+			dup->name = p_strdup(pool, dup->name);
 		}
 	}
 	return dest;
diff -r ddfe3a0f75e6 src/dsync/dsync-data.h
--- a/src/dsync/dsync-data.h	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/dsync/dsync-data.h	Thu Dec 08 09:32:03 2011 +0200
@@ -28,7 +28,7 @@
 	   otherwise it's the last rename timestamp. */
 	time_t last_change;
 	enum dsync_mailbox_flags flags;
-	ARRAY_TYPE(const_string) cache_fields;
+	ARRAY_TYPE(mailbox_cache_field) cache_fields;
 };
 ARRAY_DEFINE_TYPE(dsync_mailbox, struct dsync_mailbox *);
 #define dsync_mailbox_is_noselect(dsync_box) \
diff -r ddfe3a0f75e6 src/dsync/dsync-proxy-client.c
--- a/src/dsync/dsync-proxy-client.c	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/dsync/dsync-proxy-client.c	Thu Dec 08 09:32:03 2011 +0200
@@ -893,7 +893,7 @@
 static void
 proxy_client_worker_select_mailbox(struct dsync_worker *_worker,
    const mailbox_guid_t *mailbox,
-   const ARRAY_TYPE(const_string) *cache_fields)
+   const ARRAY_TYPE(mailbox_cache_field) *cache_fields)
 {
 	struct proxy_client_dsync_worker *worker =
 		(struct proxy_client_dsync_worker *)_worker;
@@ -908,7 +908,7 @@
 		str_append(str, "BOX-SELECT\t");
 		dsync_proxy_mailbox_guid_export(str, mailbox);
 		if (cache_fields != NULL)
-			dsync_proxy_strings_export(str, cache_fields);
+			dsync_proxy_cache_fields_export(str, cache_fields);
 		str_append_c(str, '\n');
 		proxy_client_worker_cmd(worker, str);
 	} T_END;
diff -r ddfe3a0f75e6 src/dsync/dsync-proxy-server-cmd.c
--- a/src/dsync/dsync-proxy-server-cmd.c	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/dsync/dsync-proxy-server-cmd.c	Thu Dec 08 09:32:03 2011 +0200
@@ -315,7 +315,7 @@
 cmd_box_select(struct dsync_proxy_server *server, const char *const *args)
 {
 	struct dsync_mailbox box;
-	unsigned int i, count;
+	const char *error;
 
 	memset(&box, 0, sizeof(box));
 	if (args[0] == NULL ||
@@ -325,10 +325,11 @@
 	}
 	args++;
 
-	count = str_array_length(args);
-	t_array_init(&box.cache_fields, count + 1);
-	for (i = 0; i < count; i++)
-		array_append(&box.cache_fields, &args[i], 1);
+	if (dsync_proxy_cache_fields_import(args, pool_datastack_create(),
+	&box.cache_fields, &error) < 0) {
+		i_error("box-select: %s", error);
+		return -1;
+	}
 	dsync_worker_select_mailbox(server->worker, &box);
 	return 1;
 }
diff -r ddfe3a0f75e6 src/dsync/dsync-proxy.c
--- a/src/dsync/dsync-proxy.c	Thu Dec 08 09:30:14 2011 +0200
+++ b/src/dsync/dsync-proxy.c	Thu Dec 08 09:32:03 2011 +0200
@@ -8,27 +8,104 @@
 #include "hex-binary.h"
 #include "mail-types.h"
 #include "imap-util.h"
+#include "mail-cache.h"
 #include "dsync-data.h"
 #include "dsync-proxy.h"
 
 #include 
 
-void dsync_proxy_strings_export(string_t *str,
-const ARRAY_TYPE(const_string) *strings)
+#define DSYNC_CACHE_DECISION_NO 'n'
+#define DSYNC_CACHE_DECISION_YES 'y'
+#define DSYNC_CACHE_DECISION_TEMP 't'
+#define DSYNC_CACHE_DECISION_FORCED 'f'
+
+void dsync_proxy_cache_fields_export(string_t *str,
+ const ARRAY_TYPE(mailbox_cache_field) *_fields)
 {
-	const char *const *fields;
+	const struct mailbox_cache_field *fields;
 	unsigned int i, count;
 
-	if (!array_is_created(strings))
+	if (!array_is_created(_fields))
 		return;
 
-	fields = arra

Re: [Dovecot] sql dict: commit failed: MySQL server has gone away

[Mark Zealey]

Correct we're not enforcing the quotas at present just want an easy way to 
monitor everyone's mailbox usage.

Thanks for the patch; will it work ok against 2.0.16? if so I'll recompile and 
give it a test today.

Mark

From: Timo Sirainen [t...@iki.fi]
Sent: 08 December 2011 05:03
To: Mark Zealey
Cc: dovecot@dovecot.org
Subject: Re: [Dovecot] sql dict: commit failed: MySQL server has gone away

On Wed, 2011-12-07 at 20:38 +0200, Mark Zealey wrote:
> Following on from this thread:
> http://www.dovecot.org/list/dovecot/2010-September/052704.html
> 
> in 2.0.16 I am seeing the same issues. I believe this would be related
> to some timeouts set on our mysql servers which are pretty low (30 or 60
> seconds or so to drop idle connections).

Yes, most likely. But it only happens if the quota isn't enforced (e.g.
unlimited quota), because otherwise the previous "current quota usage"
lookup would have automatically reconnected.

> This is happening for imap, pop
> and lmtp processes when they try to update quotas stored in a mysql
> database; presumably it is the dict process that actually has these
> issues - the lmtp delivery isn't lasting longer than 1 second so
> shouldn't be seeing any timeouts there. This is always followed by the
> "Error: dict quota: Quota update failed, it's now desynced" line;
> presumably it is telling the truth but I'm not sure how I could tell.

Yes. All sql dict updates go through dict process, which keeps a long
running sql connection open.

> We are also seeing this "MySQL server has gone away" issue in login
> processes but it isn't causing an issue there as presumably dovecot
> transparently retries rather than just giving up as it seems to in the
> quota updates. Any thoughts or advice would be appreciated.

You mean auth processes. In case of auth the lookup is retried though.

Anyway, v2.1 now retries the commit:
http://hg.dovecot.org/dovecot-2.1/rev/782f09d13ece

Re: [Dovecot] using dsync to convert mailboxes looses caching options

[Mark Zealey]

Apologies for top-posting but I can't figure out how to make this client do 
inline... I am seeing on the first run (we are using 'backup') we don't get any 
of the cache copied just the index files created. On the second run (ie when 
dest exists); a cache file is created and populated with the bits that are 
required for the sync presumably - guid. As you say the yes/tmp caching 
decisions are copied over (and visible in the cache file) but because the last 
used date is not copied; these fields are not activated for any of the messages 
so none of their data actually gets cached. I'm not seeing a compression at the 
end as the tmp etc fields are still there (mostly don't have any yes fields in 
our source caches) but as I say, because they don't have a last used date then 
the none of them are ever actually used until the client requests them via 
pop/imap.

Mark

From: Timo Sirainen [t...@iki.fi]
Sent: 08 December 2011 05:53
To: Mark Zealey
Cc: Dovecot Mailing List
Subject: Re: [Dovecot] using dsync to convert mailboxes looses caching options

On Sat, 2011-11-26 at 18:33 +0200, Mark Zealey wrote:

> We're trying to convert users from Maildir to sdbox at present; I'm
> using dsync to achieve this (2.0.16) however when the user's have been
> converted we only get minimal information in the caching files. Is there
> some way to preserve all the caching decisions that were previously made
> so that when the user logs in to the new mailbox we don't have to cause
> an io storm rebuilding the cache that we know was good? Dovecot seems to
> be partially doing this - if i remove the logs/cache from the source
> mailbox no cache files are built in the conversion; if i put them back
> then we get a cache file built but it only contains a few bits of
> information (guid, date.save). Looking into this a bit further i find
> that when the caches are present at source the fields are preserved but
> the 'last used' date and caching decisions are not which I suspect means
> dsync doesn't bother caching on import - only fields with a yes decision
> in the source are copied (but their decision is only copied as a tmp
> with the date of import). For example:

How are you calling dsync? Does the destination already exist? I tried
with:

rm -rf /tmp/foo; dsync -u tss -m INBOX mirror sdbox:/tmp/foo

It sets all of the cache fields with "yes" or "tmp" decision, as it
should. But yes, the "last used" field should probably be copied as
well.

Perhaps the problem with you is that dsync actually writes all of the
cache fields, but then it does a "cache compression" at the end, which
sees that the "last used" fields are so old, so it deletes them.

But yes, it is a problem that dsync doesn't update caching decisions..
Hmm. I guess I'll have to fix that for v2.1.

Re: [Dovecot] using dsync to convert mailboxes looses caching options

[Timo Sirainen]

On Sat, 2011-11-26 at 18:33 +0200, Mark Zealey wrote:

> We're trying to convert users from Maildir to sdbox at present; I'm 
> using dsync to achieve this (2.0.16) however when the user's have been 
> converted we only get minimal information in the caching files. Is there 
> some way to preserve all the caching decisions that were previously made 
> so that when the user logs in to the new mailbox we don't have to cause 
> an io storm rebuilding the cache that we know was good? Dovecot seems to 
> be partially doing this - if i remove the logs/cache from the source 
> mailbox no cache files are built in the conversion; if i put them back 
> then we get a cache file built but it only contains a few bits of 
> information (guid, date.save). Looking into this a bit further i find 
> that when the caches are present at source the fields are preserved but 
> the 'last used' date and caching decisions are not which I suspect means 
> dsync doesn't bother caching on import - only fields with a yes decision 
> in the source are copied (but their decision is only copied as a tmp 
> with the date of import). For example:

How are you calling dsync? Does the destination already exist? I tried
with:

rm -rf /tmp/foo; dsync -u tss -m INBOX mirror sdbox:/tmp/foo

It sets all of the cache fields with "yes" or "tmp" decision, as it
should. But yes, the "last used" field should probably be copied as
well.

Perhaps the problem with you is that dsync actually writes all of the
cache fields, but then it does a "cache compression" at the end, which
sees that the "last used" fields are so old, so it deletes them.

But yes, it is a problem that dsync doesn't update caching decisions..
Hmm. I guess I'll have to fix that for v2.1.

Re: [Dovecot] Assertion failure in 2.0.16 - SEARCH ()

[Timo Sirainen]

On Fri, 2011-11-25 at 22:36 -0800, Daniel Colascione wrote:
> Nov 25 22:25:59 dancol dovecot: imap(dancol): Panic: file mail-search-build.c:
> line 59 (mail_search_build_key_int): assertion failed: (sarg->value.subargs !=
> NULL)

Thanks, fixed: http://hg.dovecot.org/dovecot-2.0/rev/ba68f9be07c4

Re: [Dovecot] POP3/IMAP crash signal 10

[Timo Sirainen]

On Mon, 2011-11-28 at 03:24 +, Patrick Wood wrote:
> Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.error] master: Error: 
> service(pop3-login): child 28277 killed with signal 10 (core not dumped - set 
> service pop3-log
> in { drop_priv_before_exec=yes })

You can easily reproduce this? gdb backtrace is needed to get anything
useful out of this crash. Try with:

service pop3-login {
  executable = pop3-login -D
  drop_priv_before_exec = yes
}

Now when it crashes, it hopefully says that it's dumping a core
(to /usr/local/var/run/dovecot/login/core)? If not, try attaching gdb
directly to the process:

service pop3-login {
  process_min_avail = 1
  service_count = 0
}

gdb /usr/local/libexec/dovecot/pop3-login 
cont

bt full

Re: [Dovecot] Questions about Proxy/Director

[Timo Sirainen]

On Tue, 2011-11-29 at 15:24 +0100, Patrick Westenberg wrote:

> I have a working test environment not but I don't feel very comfortable 
> with a non encrypted connection between the proxy/director and the 
> backend servers.
> 
> If I set ssl = required on my backend server I get this error:
> imap-login: Disconnected (tried to use disabled plaintext auth)
> 
> Is there a chance to encrypt this connection?

See the ssl stuff in
http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy

Re: [Dovecot] v2.1.rc1 missing SnowballAnalyzer.h

[Timo Sirainen]

On Thu, 2011-12-01 at 18:15 +0200, Eray Aslan wrote:
> Sure enough, there is no SnowballAnalyzer.h in the tarball.
> Looks like a missing a backslash in src/plugins/fts-lucene/Makefile.am:
> [...]
> noinst_HEADERS = \
>   fts-lucene-plugin.h \
>   lucene-wrapper.h
>   SnowballAnalyzer.h \
>   SnowballFilter.h

Yes, thanks, will be fixed in rc2.

Re: [Dovecot] Panic when attempting index with Apache Solr : fts_solr plugin

[Timo Sirainen]

On Fri, 2011-12-02 at 12:47 -0600, A Dovecot User wrote:
> [root@mymailserver ~]# doveadm -D index -u username mailbox inbox

The "mailbox" isn't necessary above.

> doveadm(username): Info: INBOX: Caching mails seq=1..466
> doveadm(username): Panic: file solr-connection.c: line 545
> (solr_connection_post_more): assertion failed: (maxfd >= 0)

This happens every time? It can't index any mails at all? Have you tried
with Dovecot v2.0?

I can't reproduce it. Anyway I guess either I'm using libcurl wrong
somehow, or there's a bug in libcurl..

Re: [Dovecot] problems when using dsync between servers

[Timo Sirainen]

On Wed, 2011-12-07 at 12:02 +0100, Yann Dupont wrote:

> Much faster (using all the cores), but dsync still hangs on some 
> accounts. After a certain time dsync exit with an error like that :
> 
>   Panic: file dsync-worker-local.c: line 1688 
> (local_worker_save_msg_continue): assertion failed: (ret == -1)

Unfortunately there are still some bugs in dsync.. But you could work
around this by checking if dsync returned failure, and if it did just
retry the user.

> This is on a RUNNNING server.
> 
> My aim was to make some pass on the running server, then stop it and 
> make a last pass to finish the process.

That should work.

> Is dsync supposed to run only on a stopped server ?

No.

Re: [Dovecot] lazy_expunge plugin and folder subscription trouble

[Timo Sirainen]

On Wed, 2011-12-07 at 17:12 +0100, Jürgen Obermann wrote:
> Hello,
> I wanted to test the lazy expunge plugin together with mdbox. It seems  
> to me that the folder subscriptions file in the mdbox directory was  
> overwritten with only a few folder names every time I added or removed  
> something in the list of subscribed folders, or even if I turned the  
> imap folder subscription feature off and on. Here is my setup:
..
> namespace {
>location = mdbox:~/mdbox:MAILBOXDIR=expunged
>prefix = .EXPUNGED/
>separator = /
> }

Right, with this it thinks that it owns the ~/mdbox/subscriptions file,
but that is the same path as for the inbox namespace. I think with
subscriptions=no it should have worked ok, but if not, use a different
subscriptions filename:

location = 
mdbox:~/mdbox:MAILBOXDIR=expunged:SUBSCRIPTIONS=subscriptions-expunged

Re: [Dovecot] postlogin script

[Timo Sirainen]

On Wed, 2011-12-07 at 21:57 +0200, Selcuk Yazar wrote:
> Hi,
> 
> I have Postfix + OpenLdap + DoveCot configuration, and it's running
> succesfuly,
> 
> i wantto convert users pop3 password NTPassword and LMPassword, so i ne
> plain passwor dof users,
> 
> how can i do that. (Normaly using perl's ntlmgen function i convert
> password , but in plain)

1) You need to enable userdb prefetch and make it work:
http://wiki2.dovecot.org/UserDatabase/Prefetch

2) Once you can successfully log in with userdb prefetch enabled, add
another field to pass_attrs:

pass_attrs = ..., =userdb_plaintext_pass=%w

3) Now in your post-login script you should have PLAINTEXT_PASS
environment containing the password.

Re: [Dovecot] sql dict: commit failed: MySQL server has gone away

[Timo Sirainen]

On Wed, 2011-12-07 at 20:38 +0200, Mark Zealey wrote:
> Following on from this thread: 
> http://www.dovecot.org/list/dovecot/2010-September/052704.html 
> 
>  
> in 2.0.16 I am seeing the same issues. I believe this would be related 
> to some timeouts set on our mysql servers which are pretty low (30 or 60 
> seconds or so to drop idle connections). 

Yes, most likely. But it only happens if the quota isn't enforced (e.g.
unlimited quota), because otherwise the previous "current quota usage"
lookup would have automatically reconnected.

> This is happening for imap, pop 
> and lmtp processes when they try to update quotas stored in a mysql 
> database; presumably it is the dict process that actually has these 
> issues - the lmtp delivery isn't lasting longer than 1 second so 
> shouldn't be seeing any timeouts there. This is always followed by the 
> "Error: dict quota: Quota update failed, it's now desynced" line; 
> presumably it is telling the truth but I'm not sure how I could tell.

Yes. All sql dict updates go through dict process, which keeps a long
running sql connection open.

> We are also seeing this "MySQL server has gone away" issue in login 
> processes but it isn't causing an issue there as presumably dovecot 
> transparently retries rather than just giving up as it seems to in the 
> quota updates. Any thoughts or advice would be appreciated.

You mean auth processes. In case of auth the lookup is retried though.

Anyway, v2.1 now retries the commit:
http://hg.dovecot.org/dovecot-2.1/rev/782f09d13ece

Re: [Dovecot] overriding userdb connection host for doveadm-user

[Timo Sirainen]

On Wed, 2011-12-07 at 13:02 -0500, Micah Anderson wrote:
> I've got my users in a replicated database setup and dovecot configured
> with two connect lines in dovecot-sql.conf:
> 
> connect = host=127.0.0.1 port=3306 dbname=users user=user password=pass
> connect = host=127.0.0.2 port=3306 dbname=users user=user password=pass
> 
> this works really well to help balance the load. 

If you actually have it like that, it should only be using 127.0.0.2, as
the second connect setting overrides the first one.. The right way would
have been:

connect = host=127.0.0.1 host=127.0.0.2 port=...

> I would like a way to see if the new user's information has been
> replicated yet before sending the email. I was thinking I could use
> 'doveadm user' to lookup the user, but I am not sure that I can specify
> one of the specific 'connect' lines in my configuration to check. Is
> there a way to override that configuration variable to do this test? I
> tried various incarnations of passing '-o connect=host='... but doveadm
> user would just ignore them.

doveadm does the lookup via auth-userdb socket. You have two
possibilities:

a) Have doveadm use another auth-userdb socket that points to another
dovecot installation with different configuration.

b) Use a different userdb configuration for doveadm:

protocol !doveadm {
  userdb {
driver = sql
args = /.../dovecot-sql.conf.ext
  }
}
protocol doveadm {
  userdb {
driver = sql
args = /.../dovecot-sql-slave.conf.ext
  }
}

Re: [Dovecot] dovecot 2.1.rc1 and folders with umlauts

[Timo Sirainen]

On Wed, 2011-12-07 at 18:05 +0100, Mike Constabel wrote:

> if I try to crate a folder in Thunderbird 8 which contains 
> umlauts this happens:
> 
> - Create folder "Tür"
> - Thunderbird show T&APw-r in folder list

I can't reproduce. Is it possible this is a TB bug? Try talking IMAP
protocol directly:

/usr/local/libexec/dovecot/imap -u user@domain
a list "" T*

If it lists:

* LIST (\HasNoChildren) "." "T&APw-r"

Then Dovecot is working correctly.

> drwx-- 3 vmail vmail 4,0K  7. Dez 17:45 T&APw-r

This is correct.

> On another server with
> 
> # 2.0.13 (537d4b6d9a7a): /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-5-686 i686 Debian 6.0.2 
> 
> an an server with
> 
> # 1.2.16: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 5.0.9 
> 
> umlauts are working.

It wouldn't be impossible for TB to be simply confused with this setup
for some reason :)

Re: [Dovecot] doveadm fetch vs search+fetch

[Timo Sirainen]

On Wed, 2011-12-07 at 19:56 -0800, Joseph Tam wrote:
> > If you want to fetch message headers/bodies separately for multiple
> > messages, then doveadm search is useful. With "doveadm fetch" you can't
> > really know where one message ends and another one begins (although the
> > ^L character between messages is a good indicator, it's not 100%
> > reliable). And in any case it's more difficult to use for separating
> > messages in a shell script.
> 
> You could use, as I suggested,
> 
>   doveadm -f tab fetch -u bob 'mailbox-guid uid' ... | \

Oh, you meant this.

>   while read guid uid; do
>   ...
>   done
> 
> which pretty much does the same thing (but you have to dump the first
> line, ugh!).  It doesn't matter all that much except some extra man
> page reading, but it will save you the bother of having to maintain a
> separate binary, man page, etc. for what is essentially a duplicate.

Yeah, but it's more effort to remove it than to keep it as it is :) Also
a "search" is somewhat simpler and prettier to use.

Re: [Dovecot] 2.1.rc1 (1e118913348b): Panic: file mail-user.c: line 128 (mail_user_ref): assertion failed: (user->refcount > 0)

[Timo Sirainen]

On Wed, 2011-12-07 at 17:17 +0100, Pascal Volk wrote:

> right after delivering the message to the INBOX the lmtp crashes.
> I wonder about the "Sent" in the attached backtrace.

http://hg.dovecot.org/dovecot-2.1/rev/3b70254e0596 should fix it.

Re: [Dovecot] Questions about single intance storage

[Timo Sirainen]

On Wed, 2011-12-07 at 17:02 +0100, Yann Dupont wrote:
> > before doing rm -rf for the user's mails. And in the archiving step you 
> > should do it with dsync with mail_attachment_dir disabled in the 
> > destination storage, so the the attachments get written to the archive 
> > directly instead of only referencing SIS.
> >
> Yes, I understand, it will work. But, if case of any error (even our 
> fault : premature end of script, for example) you can still end up with 
> attachement forever lost on the filesystem.
> 
> Right, it SHOULD not happen, and it probably won't represent a big 
> volume. But Still, it could happen under specific circonstances. In that 
> case, I don't see any simple way to detect that kind of files ?
> 
> Do you see how a script could detect some orphaned links ??

It wouldn't be simple. The only safe way would be to:

1. Scan through all the attachment HASH-GUID names and save them. This
scanning step could already detect some orphaned attachments, where the
hashes/HASH file exists with nlink=1 (i.e. HASH-GUID* files have been
deleted, but the HASH itself hasn't been for some reason).

2. Read through all users' all dboxes contents and get a list of all
referenced attachment HASH-GUIDs.

3. Delete all attachments that exist in list 1, but not in list 2.

I guess there should be a "doveadm sis rescan" command that does this.

Re: [Dovecot] doveadm fetch vs search+fetch

[Joseph Tam]

On Thu, 8 Dec 2011, Timo Sirainen wrote:


It seems "doveadm fetch" functionally supercedes "doveadm search"
making the latter obsolete.  It can even be closely simulated
with "doveadm -f tab fetch ... 'mailbox-guid uid' search-query ...".


If you want to fetch message headers/bodies separately for multiple
messages, then doveadm search is useful. With "doveadm fetch" you can't
really know where one message ends and another one begins (although the
^L character between messages is a good indicator, it's not 100%
reliable). And in any case it's more difficult to use for separating
messages in a shell script.


You could use, as I suggested,

doveadm -f tab fetch -u bob 'mailbox-guid uid' ... | \
while read guid uid; do
...
done

which pretty much does the same thing (but you have to dump the first
line, ugh!).  It doesn't matter all that much except some extra man
page reading, but it will save you the bother of having to maintain a
separate binary, man page, etc. for what is essentially a duplicate.

Joseph Tam 

Re: [Dovecot] doveadm fetch vs search+fetch

[Timo Sirainen]

On Wed, 2011-12-07 at 18:00 -0800, Joseph Tam wrote:
> What's the pactical difference between using "doveadm fetch" rather
> than using "doveadm search" in conjunction with "doveadm fetch", as the
> man pages suggest?
> 
> It seems "doveadm fetch" functionally supercedes "doveadm search"
> making the latter obsolete.  It can even be closely simulated
> with "doveadm -f tab fetch ... 'mailbox-guid uid' search-query ...".

If you want to fetch message headers/bodies separately for multiple
messages, then doveadm search is useful. With "doveadm fetch" you can't
really know where one message ends and another one begins (although the
^L character between messages is a good indicator, it's not 100%
reliable). And in any case it's more difficult to use for separating
messages in a shell script.

[Dovecot] doveadm fetch vs search+fetch

[Joseph Tam]

What's the pactical difference between using "doveadm fetch" rather
than using "doveadm search" in conjunction with "doveadm fetch", as the
man pages suggest?

It seems "doveadm fetch" functionally supercedes "doveadm search"
making the latter obsolete.  It can even be closely simulated
with "doveadm -f tab fetch ... 'mailbox-guid uid' search-query ...".

Joseph Tam 

[Dovecot] postlogin script

[Selcuk Yazar]

Hi,

I have Postfix + OpenLdap + DoveCot configuration, and it's running
succesfuly,

i wantto convert users pop3 password NTPassword and LMPassword, so i ne
plain passwor dof users,

how can i do that. (Normaly using perl's ntlmgen function i convert
password , but in plain)

thanks in advance

[Dovecot] sql dict: commit failed: MySQL server has gone away

[Mark Zealey]

Following on from this thread: 
http://www.dovecot.org/list/dovecot/2010-September/052704.html 
 
in 2.0.16 I am seeing the same issues. I believe this would be related 
to some timeouts set on our mysql servers which are pretty low (30 or 60 
seconds or so to drop idle connections). This is happening for imap, pop 
and lmtp processes when they try to update quotas stored in a mysql 
database; presumably it is the dict process that actually has these 
issues - the lmtp delivery isn't lasting longer than 1 second so 
shouldn't be seeing any timeouts there. This is always followed by the 
"Error: dict quota: Quota update failed, it's now desynced" line; 
presumably it is telling the truth but I'm not sure how I could tell.


We are also seeing this "MySQL server has gone away" issue in login 
processes but it isn't causing an issue there as presumably dovecot 
transparently retries rather than just giving up as it seems to in the 
quota updates. Any thoughts or advice would be appreciated.


Mark

[Dovecot] overriding userdb connection host for doveadm-user

[Micah Anderson]

I've got my users in a replicated database setup and dovecot configured
with two connect lines in dovecot-sql.conf:

connect = host=127.0.0.1 port=3306 dbname=users user=user password=pass
connect = host=127.0.0.2 port=3306 dbname=users user=user password=pass

this works really well to help balance the load. 

However, when a new user is created, the replication information
sometimes has not made it to the slave database before my user creation
process tries to send that new user an email about their new email
account. If it hasn't and the email gets sent, then it bounces because
postfix thinks the user doesn't exist yet. 

Replication usually happens within seconds, but could be delayed due to
problems.

I would like a way to see if the new user's information has been
replicated yet before sending the email. I was thinking I could use
'doveadm user' to lookup the user, but I am not sure that I can specify
one of the specific 'connect' lines in my configuration to check. Is
there a way to override that configuration variable to do this test? I
tried various incarnations of passing '-o connect=host='... but doveadm
user would just ignore them.

thanks for any suggestions!
micah

ps - yes, I can make my user creation process just 'sleep' for an
arbitrary amount of time and by that time the replication should have
occured, but that is a bit of a shot in the dark, and doesn't handle
longer replication delays that could potentially happen.

-- 



pgp5kLjzEjaju.pgp
Description: PGP signature

[Dovecot] dovecot 2.1.rc1 and folders with umlauts

[Mike Constabel]

Hi,

if I try to crate a folder in Thunderbird 8 which contains 
umlauts this happens:

- Create folder "Tür"
- Thunderbird show T&APw-r in folder list

In log:

Dec 07 17:45:39 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/T&APw-r doesn't exist yet, using 
default permissions
Dec 07 17:45:39 imap(te...@in-preetz.de): Debug: Namespace : Using permissions 
from /mailspool1/in-preetz.de/test1/mdbox: mode=0700 gid=-1

# ll /mailspool1/in-preetz.de/test1/mdbox/mailboxes/
insgesamt 32K
drwx-- 8 vmail vmail 4,0K  7. Dez 17:45 .
drwx-- 4 vmail vmail 4,0K  7. Dez 17:45 ..
drwx-- 3 vmail vmail 4,0K 29. Nov 11:11 Drafts
drwx-- 3 vmail vmail 4,0K 29. Nov 11:04 INBOX
drwx-- 3 vmail vmail 4,0K 29. Nov 11:11 Junk
drwx-- 3 vmail vmail 4,0K 29. Nov 11:11 Sent
drwx-- 3 vmail vmail 4,0K  7. Dez 17:45 T&APw-r
drwx-- 4 vmail vmail 4,0K  7. Dez 17:44 Trash

If I click in TB on the folder, TB gives an empty message, the 
log shows:

Dec 07 17:42:53 imap(te...@in-preetz.de): Debug: Effective uid=1000, gid=1000, 
home=/mailspool1/in-preetz.de/test1
Dec 07 17:42:53 imap(te...@in-preetz.de): Debug: quota: No quota setting - 
plugin disabled
Dec 07 17:42:53 imap(te...@in-preetz.de): Debug: Namespace inbox: type=private, 
prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes 
location=mdbox:~/mdbox
Dec 07 17:42:53 imap(te...@in-preetz.de): Debug: fs: 
root=/mailspool1/in-preetz.de/test1/mdbox, index=, control=, inbox=, alt=
Dec 07 17:42:53 imap(te...@in-preetz.de): Debug: Namespace : Using permissions 
from /mailspool1/in-preetz.de/test1/mdbox: mode=0700 gid=-1
Dec 07 17:43:02 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/&ANY-de doesn't exist yet, using 
default permissions
Dec 07 17:43:02 imap(te...@in-preetz.de): Debug: Namespace : Using permissions 
from /mailspool1/in-preetz.de/test1/mdbox: mode=0700 gid=-1
Dec 07 17:44:19 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/&-ANY-de doesn't exist yet, 
using default permissions
Dec 07 17:44:19 imap(te...@in-preetz.de): Debug: Namespace : Using permissions 
from /mailspool1/in-preetz.de/test1/mdbox: mode=0700 gid=-1
Dec 07 17:44:19 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/Trash/&-ANY-de doesn't exist 
yet, using default permissions
Dec 07 17:44:24 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/Trash/T&APw-r doesn't exist yet, 
using default permissions
Dec 07 17:45:39 imap(te...@in-preetz.de): Debug: Namespace : 
/mailspool1/in-preetz.de/test1/mdbox/mailboxes/T&APw-r doesn't exist yet, using 
default permissions
Dec 07 17:45:39 imap(te...@in-preetz.de): Debug: Namespace : Using permissions 
from /mailspool1/in-preetz.de/test1/mdbox: mode=0700 gid=-1


On another server with

# 2.0.13 (537d4b6d9a7a): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686 i686 Debian 6.0.2 

an an server with

# 1.2.16: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 5.0.9 

umlauts are working.

Is there an problem with my config or with the rc1?


thanks,
Mike


dovecot -n for not working 2.1.rc1:

# 2.1.rc1: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.3 
auth_cache_size = 1 M
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login cram-md5
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/dovecot/dovecot-debug.log
default_client_limit = 2600
disable_plaintext_auth = no
hostname = mx4.constabel-it.de
info_log_path = /var/log/dovecot/dovecot-info.log
log_path = /var/log/dovecot/dovecot.log
mail_debug = yes
mail_location = mdbox:~/mdbox
mail_plugins = quota notify mail_log
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location = 
  prefix = 
  separator = /
  type = private
}
passdb {
  args = scheme=PLAIN username_format=%u /etc/imap.userdb
  driver = passwd-file
}
plugin {
  autocreate = Trash
  autocreate2 = Junk
  autocreate3 = Sent
  autocreate4 = Drafts
  autosubscribe = Trash
  autosubscribe2 = Junk
  autosubscribe3 = Sent
  autosubscribe4 = Drafts
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
postmaster_address = postmaster@localhost
protocols = " imap lmtp sieve pop3"
service auth {
  inet_listener {
address = 192.168.10.117
port = 12345
  }
}
service imap-login {
  inet_listener imap {
address = 192.168.10.117
port = 143
  }
  inet_listener imaps {
address = 192.168.10.117
port = 993
ssl = yes
  }
}
service lmtp {
  inet_listener lmtp {
address = 192.168.10.117
port = 24
  }
}
service managesieve-login {
  inet_list

Re: [Dovecot] IMAP SPECIAL-USE extension

[Willie Gillespie]

On 12/07/2011 08:56 AM, Timo Sirainen wrote:

On 7.12.2011, at 17.45, Willie Gillespie wrote:


Outlook 2010 lets you select a mailbox for Trash and Sent (version 2007 only 
lets you select a mailbox for Sent messages).
Outlook default names for Trash is "Deleted Items" (as someone pointed out) and for junk 
is "Junk E-mail".


And the default name for Sent is "Sent Items"?


Yes

[Dovecot] 2.1.rc1 (1e118913348b): Panic: file mail-user.c: line 128 (mail_user_ref): assertion failed: (user->refcount > 0)

[Pascal Volk]

Hi Timo,

right after delivering the message to the INBOX the lmtp crashes.
I wonder about the "Sent" in the attached backtrace.


Regards,
Pascal
-- 
The trapper recommends today: c01dcofe.1134...@localdomain.org
# 2.1.rc1 (1e118913348b): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.1.0-1-amd64 x86_64 Debian wheezy/sid 
auth_mechanisms = plain login
debug_log_path = /var/log/dovecot_dbg.log
dict {
  quota = pgsql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext
}
first_valid_gid = 7
first_valid_uid = 7
hostname = orange.example.com
listen = 203.0.113.168, 2001:db8:436d:eaff:587:995:0:
lmtp_save_to_detail_mailbox = yes
login_access_sockets = tcpwrap
mail_attachment_dir = /srv/mail/.SiS
mail_attachment_hash = %{sha256}
mail_location = mdbox:~/mdbox
mail_plugins = quota zlib
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
  separator = /
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  quota = dict:user:%{uid}:noenforcing:proxy::quota
  quota_rule = *:storage=2G:messages=0
  quota_rule2 = Trash:storage=+100M
  recipient_delimiter = +
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  zlib_save = bz2
  zlib_save_level = 6
}
postmaster_address = postmas...@example.com
protocols = imap lmtp sieve
service auth-worker {
  unix_listener auth-worker {
group = $default_internal_user
mode = 0660
user = doveauth
  }
  user = doveauth
}
service auth {
  client_limit = 1425
  unix_listener /var/spool/postfix-orange/private/dovecot-auth {
group = postfix
mode = 0600
user = postfix
  }
  user = doveauth
}
service dict {
  unix_listener dict {
mode = 0666
  }
}
service lmtp {
  executable = lmtp -D
  unix_listener /var/spool/postfix-blau/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
  unix_listener /var/spool/postfix-orange/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service tcpwrap {
  unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
  }
}
ssl = required
ssl_cert = #0  0x7f4a53e03405 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x7f4a53e06680 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x7f4a541a4aa8 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) 
at failures.c:187
backtrace = 0x1d9f5f8 "/usr/local/lib/dovecot/libdovecot.so.0(+0x4fa81) 
[0x7f4a541a4a81] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x50d67) 
[0x7f4a541a5d67] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7f4a541a4"...
#3  0x7f4a541a5d67 in i_internal_fatal_handler (ctx=0x7fff2e5cfc50, 
format=0x7f4a5470a310 "file %s: line %d (%s): assertion failed: (%s)", 
args=0x7fff2e5cfc38) at failures.c:645
status = 0
#4  0x7f4a541a4d91 in i_panic (format=0x7f4a5470a310 "file %s: line %d 
(%s): assertion failed: (%s)") at failures.c:259
ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0}
args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 
0x7fff2e5cfd20, reg_save_area = 0x7fff2e5cfc60}}
#5  0x7f4a5464de71 in mail_user_ref (user=0x1de7130) at mail-user.c:128
__FUNCTION__ = "mail_user_ref"
#6  0x7f4a54648720 in mail_storage_obj_ref (storage=0x1dec780) at 
mail-storage.c:439
__FUNCTION__ = "mail_storage_obj_ref"
#7  0x7f4a54649074 in mailbox_alloc (list=0x1deb140, vname=0x1df5b50 
"Sent", flags=MAILBOX_FLAG_READONLY) at mail-storage.c:664
new_list = 0x1deb140
storage = 0x1dec780
box = 0x1deeae0
__FUNCTION__ = "mailbox_alloc"
#8  0x7f4a52d6ef16 in quota_count_mailbox (root=0x1ddfe10, ns=0x1ddfed0, 
vname=0x1df5b50 "Sent", bytes_r=0x7fff2e5cff20, count_r=0x7fff2e5cff18) at 
quota-count.c:30
rule = 0x0
box = 0x7fff2e5cfe40
trans = 0x0
ctx = 0x1dfca00
mail = 0x1dfa500
search_args = 0x1dfa3b0
error = 32586
size = 1421264208
ret = 0
#9  0x7f4a52d6f0ce in quota_count_namespace (root=0x1ddfe10, ns=0x1ddfed0, 
bytes=0x7fff2e5cff20, count=0x7fff2e5cff18) at quota-count.c:78
ctx = 0x1dfa330
info = 0x1dfa508
ret = 0
#10 0x7f4a52d6f1df in quota_count (root=0x1ddfe10, bytes_r=0x7fff2e5cff20, 
count_r=0x7fff2e5cff18) at quota-count.c:106
namespaces = 0x1de0e80
i = 0
count = 1
ret = 0
#11 0x7f4a52d70e76 in dict_quota_count (root=0x1ddfe10, want_bytes=true

[Dovecot] lazy_expunge plugin and folder subscription trouble

[Jürgen Obermann]

Hello,
I wanted to test the lazy expunge plugin together with mdbox. It seems  
to me that the folder subscriptions file in the mdbox directory was  
overwritten with only a few folder names every time I added or removed  
something in the list of subscribed folders, or even if I turned the  
imap folder subscription feature off and on. Here is my setup:


# 2.0.16: /opt/local/etc/dovecot/dovecot.conf
# OS: SunOS 5.10 sun4v

mail_location = mdbox:~/mdbox
mail_plugins = mail_log notify quota lazy_expunge zlib

mdbox_rotate_interval = 1 days
mdbox_rotate_size = 16 M
namespace {
  inbox = yes
  location =
  prefix =
  separator = /
  type = private
}
namespace {
  hidden = yes
  list = no
  location =
  prefix = Mail/
  separator = /
  subscriptions = yes
  type = private
}
namespace {
  location =  
maildir:/var/mail/public:CONTROL=~/Maildir/public:INDEX=~/Maildir/public

  prefix = Public/
  separator = /
  subscriptions = no
  type = public
}
namespace {
  location = mdbox:~/mdbox:MAILBOXDIR=expunged
  prefix = .EXPUNGED/
  separator = /
}
plugin {
  autocreate = Trash
  autocreate2 = caughtspam
  autocreate3 = Sent
  autocreate4 = Drafts
  autosubscribe = Trash
  autosubscribe2 = caughtspam
  autosubscribe3 = Sent
  autosubscribe4 = Drafts
  fts = squat
  lazy_expunge = .EXPUNGED/
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
  mail_log_fields = uid box msgid size
  quota = fs:Home-Verzeichnis:noenforcing
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  zlib_save = gz
  zlib_save_level = 3
}
protocols = imap pop3 lmtp sieve
quota_full_tempfail = yes
sendmail_path = /usr/lib/sendmail
service auth {
  client_limit = 11120
}
service imap-login {
  process_min_avail = 16
  service_count = 0
  vsz_limit = 640 M
}
service imap {
  process_limit = 4096
  vsz_limit = 1 G
}
ssl_cert =   mail_plugins = mail_log notify quota lazy_expunge zlib autocreate  
fts fts_squat imap_quota imap_zlib

}
protocol lmtp {
  mail_plugins = mail_log notify quota lazy_expunge zlib sieve
}
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %08Xv%08Xu
}

--
Hochschulrechenzentrum der | Mail: juergen.oberm...@hrz.uni-giessen.de
Justus-Liebig-Universitaet | WWW:  http://www.uni-giessen.de/obermann/
Heinrich-Buff-Ring 44  | Tel:  0641-99-13054 (0641-99-13001)
D-35392 Giessen, Germany   | Fax:  0641-99-13009

Re: [Dovecot] Questions about single intance storage

[Yann Dupont]

Le 07/12/2011 15:15, Timo Sirainen a écrit :

On 7.12.2011, at 12.25, Yann Dupont wrote:



doveadm expunge -u user mailbox '*' all

before doing rm -rf for the user's mails. And in the archiving step you should 
do it with dsync with mail_attachment_dir disabled in the destination storage, 
so the the attachments get written to the archive directly instead of only 
referencing SIS.


Yes, I understand, it will work. But, if case of any error (even our 
fault : premature end of script, for example) you can still end up with 
attachement forever lost on the filesystem.


Right, it SHOULD not happen, and it probably won't represent a big 
volume. But Still, it could happen under specific circonstances. In that 
case, I don't see any simple way to detect that kind of files ?


Do you see how a script could detect some orphaned links ??

For the archiving, good idea to use dsync, thanks for your answer.

--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : yann.dup...@univ-nantes.fr

Re: [Dovecot] IMAP SPECIAL-USE extension

[Timo Sirainen]

On 7.12.2011, at 17.45, Willie Gillespie wrote:

> Outlook 2010 lets you select a mailbox for Trash and Sent (version 2007 only 
> lets you select a mailbox for Sent messages).
> Outlook default names for Trash is "Deleted Items" (as someone pointed out) 
> and for junk is "Junk E-mail".

And the default name for Sent is "Sent Items"?

Wonder if I should add those also to the example config..

Re: [Dovecot] IMAP SPECIAL-USE extension

[Willie Gillespie]

On 12/06/2011 03:13 PM, Timo Sirainen wrote:

On Fri, 2011-12-02 at 17:12 +0200, Timo Sirainen wrote:


The mailbox names require a namespace prefix, so if you use e.g. "INBOX." 
prefix you'll need to use:


I changed these so that they are inside namespace {} section and don't
require namespace prefix in the name. That way I can include common
mailbox definitions in example-config and still allow the namespace
prefix to be easily changed.

I'm anyway now wondering what the defaults should be? Could someone
check what these defaults are for Outlook and any other clients you
have:

  * "Drafts" is used by all clients
  * "Trash" is used by all clients
  * "Junk" is used by all clients? Or is "Spam" used by some?
  * "Sent" is used by Thunderbird, Evolution
  * "Sent Messages" is used by Apple Mail

So I'm mainly wondering about "Sent" vs. "Sent Messages".


Outlook 2010 lets you select a mailbox for Trash and Sent (version 2007 
only lets you select a mailbox for Sent messages).
Outlook default names for Trash is "Deleted Items" (as someone pointed 
out) and for junk is "Junk E-mail".


Willie

[Dovecot] dovecot 2.0 sieve if working, managesieve not.

[Sacha Mura]

Hi,
this is my configuration

# OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Red Hat Enterprise Linux Server 
release 6.1 (Santiago) ext4
auth_debug = yes
auth_master_user_separator = *
auth_username_format = %Lu
auth_verbose = yes
disable_plaintext_auth = no
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_greeting = Dovecot ready
mail_debug = yes
mail_gid = 999
mail_home = /MAIL/%d/%n/
mail_location = maildir:/MAIL/%d/%n/
mail_log_prefix = "%Us(%u): "
mail_plugins = " quota"
mail_privileged_group = mail
mail_uid = 999
passdb {
  args = /etc/dovecot/dovecot-ldap.conf
  driver = ldap
}
passdb {
  args = /etc/dovecot/passwd.masterusers
  driver = passwd-file
  master = yes
  pass = yes
}
plugin {
  quota = maildir:User quota
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/sieve/dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = imap pop3 sieve
service auth {
  unix_listener auth-userdb {
group = mail
mode = 0660
user = postfix
  }
}
service managesieve-login {
  executable = /usr/libexec/dovecot/managesieve-login
  inet_listener sieve {
port = 4190
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
group = mail
mode = 0666
user = mail
  }
  user = mail
}
ssl = no
userdb {
  args = /etc/dovecot/dovecot-ldap.user
  driver = ldap
}
doveconf: Error: protocols: Unknown protocol: sieve
doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: 
protocols: Unknown protocol: sieve

I get the error Unknown protocol: sieve

The installed rpm package is this:
dovecot-2.0.9-2.el6.x86_64

The sieve filtering is working:

2011-12-07 12:12:29 LDA(utente01@***): Debug: sieve: opening script 
/MAIL/***/user//sieve/dovecot.sieve
2011-12-07 12:12:29 LDA(utente01@***): Debug: sieve: script 
/MAIL/***/user//sieve/dovecot.sieve successfully compiled
2011-12-07 12:12:29 LDA(utente01@***): Debug: sieve: executing script from 
/MAIL/***/utente01//sieve/dovecot.sieve
2011-12-07 12:12:29 LDA(utente01@***): Info: sieve: 
msgid=<4EDF4A1C.9010509@***>: stored mail into mailbox 'INBOX'

But the manage sieve won't work.
It seems that pigeonhole is installed because 
/usr/libexec/dovecot/managesieve-login exist and is not empty.

Can you help me?

Thank You

Sushi.

Re: [Dovecot] IMAP SPECIAL-USE extension

[Charles Marcus]

On 2011-12-07 8:55 AM, Pascal Volk 
 wrote:

On 12/07/2011 02:28 PM Arne K. Haaje wrote:

>  Thunderbird also use a Templates folder.

Yes, I also noticed that. But
http://tools.ietf.org/html/rfc6154#section-2  doesn't mention a mailbox
for templates.


Nevertheless, it is very handy functionality... it isn't just a folder.

--

Best regards,

Charles

Re: [Dovecot] IMAP SPECIAL-USE extension

[Robert Schetterer]

Am 07.12.2011 15:35, schrieb Patrick Ben Koetter:
> * Pascal Volk :
>> On 12/07/2011 02:28 PM Arne K. Haaje wrote:
>>> Thunderbird also use a Templates folder.
>>
>> Yes, I also noticed that. But
>> http://tools.ietf.org/html/rfc6154#section-2 doesn't mention a mailbox
>> for templates.
> 
> IIRC IETF allows to register new SPECIAL-USE mailboxes. 'templates' might be a
> good candidate.
> 
> p@rick
> 

TB also has a archives folder as default

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: [Dovecot] IMAP SPECIAL-USE extension

[Patrick Ben Koetter]

* Pascal Volk :
> On 12/07/2011 02:28 PM Arne K. Haaje wrote:
> > Thunderbird also use a Templates folder.
> 
> Yes, I also noticed that. But
> http://tools.ietf.org/html/rfc6154#section-2 doesn't mention a mailbox
> for templates.

IIRC IETF allows to register new SPECIAL-USE mailboxes. 'templates' might be a
good candidate.

p@rick

-- 
state of mind ()
Digitale Kommunikation

http://www.state-of-mind.de

Franziskanerstraße 15  Telefon +49 89 3090 4664
81669 München  Telefax +49 89 3090 4666

Amtsgericht MünchenPartnerschaftsregister PR 563

[Dovecot] v2.x services documentation

[Timo Sirainen]

If you've ever wanted to know everything about the service {} blocks, this 
should be quite helpful: http://wiki2.dovecot.org/Services

Re: [Dovecot] Questions about single intance storage

[Timo Sirainen]

On 7.12.2011, at 12.25, Yann Dupont wrote:

> Other problem I'm thinking of, because I'd like to use SIS on our production 
> servers ; but right now I think I can't .
> 
> We have lots of users (+5000 teachers/engineers) on our first setup, +7 
> students on our 2nd setup.
> 
> The user base is on LDAP, and move on a daily basis. When a user leave the 
> university, he has right to use mailbox for a certain time and then we close 
> the account.
> 
> Right now, we archive and then delete the mailbox directories (we don't use 
> special dovecot mecanism : we migrated from another system not long time ago 
> and we had special scripts for that).
> 
> If we use SIS, what happen to the attachements ? The usage count will never 
> go to 0, and the attachements will stay there forever.
> 
> In that situation, I think we have no means to correct the attachement usage 
> count ?

You'll need to change the deletion script then. Run:

doveadm expunge -u user mailbox '*' all

before doing rm -rf for the user's mails. And in the archiving step you should 
do it with dsync with mail_attachment_dir disabled in the destination storage, 
so the the attachments get written to the archive directly instead of only 
referencing SIS.

Re: [Dovecot] IMAP SPECIAL-USE extension

[Pascal Volk]

On 12/07/2011 02:28 PM Arne K. Haaje wrote:
> Thunderbird also use a Templates folder.

Yes, I also noticed that. But
http://tools.ietf.org/html/rfc6154#section-2 doesn't mention a mailbox
for templates.


Regards,
Pascal
-- 
The trapper recommends today: face1e55.1134...@localdomain.org

Re: [Dovecot] IMAP SPECIAL-USE extension

[Arne K. Haaje]

Tirsdag 6. desember 2011 14.58.04 skrev WJCarpenter:
> > On Fri, 2011-12-02 at 17:12 +0200, Timo Sirainen wrote:
> > 
> > I'm anyway now wondering what the defaults should be? Could someone
> > check what these defaults are for Outlook and any other clients you
> > 
> > have:
> >   * "Drafts" is used by all clients
> >   * "Trash" is used by all clients
> >   * "Junk" is used by all clients? Or is "Spam" used by some?
> >   * "Sent" is used by Thunderbird, Evolution
> >   * "Sent Messages" is used by Apple Mail
> 
> SquirrelMail (http://www.squirrelmail.org) and Hastymail2
> (http://www.hastymail.org) webmail clients let you customize
> draft/trash/sent, but the defaults for both are Drafts, Trash, and Sent.

Thunderbird also use a Templates folder.

Arne
-- 
Arne K. Haaje - Dr Linux
http://www.drlinux.no/  
LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb

Re: [Dovecot] Questions about single intance storage

[Charles Marcus]

On 2011-12-07 5:25 AM, Yann Dupont  wrote:


Right now, we archive and then delete the mailbox directories (we don't
use special dovecot mecanism : we migrated from another system not long
time ago and we had special scripts for that).

If we use SIS, what happen to the attachements ? The usage count will
never go to 0, and the attachements will stay there forever.


?

Attachment count for any messages that were *only* in those deleted 
mailbox directories would go to zero after you delete them, and then the 
attachments would be deleted. Dovecot wouldn't know about any that were 
archived outside of dovecots knowledge.


--

Best regards,

Charles

Re: [Dovecot] master user only for sieve

[Edgar Fuß]

I have the following for 1.2. You may search the list archive, I asked that 
question about halv a year ago and Timo answererd it for both 1.2 and 2.0.

auth default {
[...]
  # Master-Benutzer für Sieve.
  # Wirkt nur für Sieve, weil es master.imap und master.pop nicht gibt.
  passdb passwd-file {
master = yes
args = DOVECOT_CONFDIR/master.%s
# kein pass = yes, weil das bei LDAP-Authentication-Binds nicht geht.
  }
[...]
}

And a master.sieve file containing
sievemaster:{SHA}..

Re: [Dovecot] IMAP SPECIAL-USE extension

[Charles Marcus]

On 2011-12-06 5:13 PM, Timo Sirainen  wrote:

  * "Trash" is used by all clients


Outlook uses 'Deleted Items', not 'Trash'...

--

Best regards,

Charles

Re: [Dovecot] master user only for sieve

[Jan-Frode Myklebust]

On Wed, Dec 07, 2011 at 12:32:21PM +0100, Stephan Bosch wrote:
> 
> However, I do have a suggestion for a different approach. You can
> define a vacation response action with an expiry date, as follows:
> 
> require ["date", "relational", "vacation"];
> 
> if currentdate :value "le" "date" "2011-12-09" {
>vacation "I'm out of office this week. I'll be back on
> Saturday 10th to reply to your e-mails.";
> }

Thanks! I was looking for something like this, but don't know sieve
enough to find it myself. This is a much nicer approach. I'll suggest
it to the sogo developers.



  -jf

Re: [Dovecot] master user only for sieve

[Stephan Bosch]

Op 7-12-2011 11:11, Jan-Frode Myklebust schreef:

Changes in SOGo is forcing me to define a master user, so that SOGo can
disable sieve out-of-office processing on a given date. I'm not too
happy about having master users that are allowed to read the users
messages, so is it possible to define a master user that can only log in
trough manage-sieve?


I must say I don't really know about that.

However, I do have a suggestion for a different approach. You can define 
a vacation response action with an expiry date, as follows:


require ["date", "relational", "vacation"];

if currentdate :value "le" "date" "2011-12-09" {
   vacation "I'm out of office this week. I'll be back on Saturday 
10th to reply to your e-mails.";

}

This means that there is no need to remove the vacation rule explicitly. 
It is inactive and can be removed at a (much) later time. You can also 
plan vacation responses in the future using the same method.


Of course, the question is whether the SoGo GUI supports something like 
this, but I think it should.


Regards,

Stephan.

[Dovecot] problems when using dsync between servers

[Yann Dupont]

Hello,
I'm trying to migrate from maildir++ to mdbox. The maildir++ is a 
leftover of our previous setup (was using courier)


We have 3 virtuals servers for our +7 accounts.

Now, I'd like to use mdbox, so I'm trying using dsync for that. I setup 
a new virtual server, configured for mdbox.


1st, I tried a basic setup: I established a file with all my accounts 
(taken from LDAP) and then I use this script :


FILENAME=$1

while read LINE
do

dsync -u $LINE  backup $LINE@my_new_server

done < $FILENAME


But it's quite slow and dsync hangs.

So I tried another approach using multi threaded python.

Much faster (using all the cores), but dsync still hangs on some 
accounts. After a certain time dsync exit with an error like that :


 Panic: file dsync-worker-local.c: line 1688 
(local_worker_save_msg_continue): assertion failed: (ret == -1)
dsync-remote(E072184L): Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7f34fc577d8a] -> 
/usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) 
[0x7f34fc577e72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) 
[0x7f34fc55119f] -> dsync() [0x40ff0b] -> dsync() [0x410142] -> 
dsync(dsync_worker_msg_save+0x99) [0x40f639] -> dsync() [0x40e24a] -> 
dsync() [0x40d8ef] -> dsync() [0x40dbfc] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x7f34fc5830d8] 
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f34fc58403f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f34fc583068] -> 
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f34fc5713a3] -> dsync(main+0x9e1) [0x4078b1] -> 
/lib/libc.so.6(__libc_start_main+0xfd) [0x7f34fbde9c4d] -> dsync() 
[0x406cf9]




This is on a RUNNNING server.

My aim was to make some pass on the running server, then stop it and 
make a last pass to finish the process.


Is dsync supposed to run only on a stopped server ?

Is my approach totally wrong ?



--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : yann.dup...@univ-nantes.fr

Re: [Dovecot] Questions about single intance storage

[Yann Dupont]

Le 05/12/2011 02:45, Timo Sirainen a écrit :

On 5.12.2011, at 3.03, Stan Hoeppner wrote:


To cope with catastrophic failure, create a "special" Dovecot
administrator only mailbox (real/virtual/whatever) that "contains" all
of the SiS files, a special Dovecot index.


I'm not thinking about a catastrophe. For that a regular full filesystem 
backup+restore would work mostly okay (a snapshot would be perfect, without 
snapshot some extra work would be needed). The problem is that people want to 
recover only one specific user's mails from some older backup, because they 
accidentally deleted the mails.. This needs to be somewhat easy to implement 
with SIS, but it isn't.




Other problem I'm thinking of, because I'd like to use SIS on our 
production servers ; but right now I think I can't .


We have lots of users (+5000 teachers/engineers) on our first setup, 
+7 students on our 2nd setup.


The user base is on LDAP, and move on a daily basis. When a user leave 
the university, he has right to use mailbox for a certain time and then 
we close the account.


Right now, we archive and then delete the mailbox directories (we don't 
use special dovecot mecanism : we migrated from another system not long 
time ago and we had special scripts for that).


If we use SIS, what happen to the attachements ? The usage count will 
never go to 0, and the attachements will stay there forever.


In that situation, I think we have no means to correct the attachement 
usage count ?


--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : yann.dup...@univ-nantes.fr

[Dovecot] master user only for sieve

[Jan-Frode Myklebust]

Changes in SOGo is forcing me to define a master user, so that SOGo can
disable sieve out-of-office processing on a given date. I'm not too
happy about having master users that are allowed to read the users
messages, so is it possible to define a master user that can only log in
trough manage-sieve?


  -jf