Debugging rules with Pigeonhole Sieve
I have a lot of rules in my Sieve script. Sometimes an email message ends up in the wrong folder, due to a problem with one of the rules. However, this is very difficult to debug, because I have no idea which rule matched the email. A log file might be helpful, but that wouldn't be readable without shell access to the mail server. This is not a solution that would work for everyday users, who configure their rules via a web page UI. What would be great would be for the Sieve interpreter to append a custom header to each email message after it gets processed. This header would record which Sieve rules matched the email. Does this feature exist already? Or is there a way to accomplish it via the Dovecot server configuration? Thanks!
Re: Duplicate mails on pop3 expunge with dsync replication on 2.2.35 (2.2.33.2 works)
Hi Tim,
> Do you have any new insights on the problem with reappearing mail using
> dovecot replication + pop3?
>
> It's driving me mad. I'm running dovecot 2.2.34 (874deae) on OpenBSD and it
> looks like I have the same problem as you have.
unfortunately there has been no response, I'm stuck with 2.2.33.2 for the time
being.
I can only suspect it has something to do with mailbox locking which was
introduced in 2.2.34 (int dsync_mailbox_lock)
or maybe with some cached values (drop_older_timestamp) as it just happens when
the mail is deleted on the node that
it has been synced to but not if it's deleted on the node where it has
originally been received.
Best regards,
Gerald
diff -Nru dovecot-2.2.33.2/src/doveadm/dsync/dsync-brain-mailbox.c
dovecot-2.2.34/src/doveadm/dsync/dsync-brain-mailbox.c
--- dovecot-2.2.33.2/src/doveadm/dsync/dsync-brain-mailbox.c2017-10-05
19:10:44.0 +0200
+++ dovecot-2.2.34/src/doveadm/dsync/dsync-brain-mailbox.c 2018-02-28
15:45:34.0 +0100
@@ -522,25 +529,33 @@
}
/* mailbox appears to have changed. do a full sync here and get
the
- state again */
+ state again. Lock before syncing. */
+ if (dsync_mailbox_lock(brain, box, ) < 0) {
+ brain->failed = TRUE;
+ mailbox_free();
+ return -1;
+ }
if (mailbox_sync(box, MAILBOX_SYNC_FLAG_FULL_READ) < 0) {
i_error("Can't sync mailbox %s: %s",
mailbox_get_vname(box),
mailbox_get_last_internal_error(box,
>mail_error));
...
@@ -599,6 +615,7 @@
static void
dsync_cache_fields_update(const struct dsync_mailbox *local_box,
const struct dsync_mailbox *remote_box,
+ struct mailbox *box,
struct mailbox_update *update)
{
ARRAY_TYPE(mailbox_cache_field) local_sorted, remote_sorted, changes;
@@ -630,7 +647,8 @@
local_fields = array_get(_sorted, _count);
remote_fields = array_get(_sorted, _count);
t_array_init(, local_count + remote_count);
- drop_older_timestamp = ioloop_time - MAIL_CACHE_FIELD_DROP_SECS;
+ drop_older_timestamp = ioloop_time -
+ box->index->optimization_set.cache.unaccessed_field_drop_secs;
diff -Nru dovecot-2.2.33.2/src/doveadm/dsync/dsync-mailbox.c
dovecot-2.2.34/src/doveadm/dsync/dsync-mailbox.c
--- dovecot-2.2.33.2/src/doveadm/dsync/dsync-mailbox.c 2017-06-23
13:18:28.0 +0200
+++ dovecot-2.2.34/src/doveadm/dsync/dsync-mailbox.c2018-02-28
15:45:34.0 +0100
@@ -1,7 +1,9 @@
-/* Copyright (c) 2013-2017 Dovecot authors, see the included COPYING file */
+/* Copyright (c) 2013-2018 Dovecot authors, see the included COPYING file */
#include "lib.h"
#include "istream.h"
+#include "mail-storage-private.h"
+#include "dsync-brain-private.h"
#include "dsync-mailbox.h"
void dsync_mailbox_attribute_dup(pool_t pool,
@@ -20,3 +22,40 @@
dest_r->last_change = src->last_change;
dest_r->modseq = src->modseq;
}
+
+int dsync_mailbox_lock(struct dsync_brain *brain, struct mailbox *box,
+ struct file_lock **lock_r)
+{
+ const char *path, *error;
+ int ret;
+
+ /* Make sure the mailbox is open - locking requires it */
+ if (mailbox_open(box) < 0) {
+ i_error("Can't open mailbox %s: %s", mailbox_get_vname(box),
+ mailbox_get_last_internal_error(box,
>mail_error));
+ return -1;
+ }
+
+ ret = mailbox_get_path_to(box, MAILBOX_LIST_PATH_TYPE_INDEX, );
+ if (ret < 0) {
+ i_error("Can't get mailbox %s path: %s", mailbox_get_vname(box),
+ mailbox_get_last_internal_error(box,
>mail_error));
+ return -1;
+ }
+ if (ret == 0) {
+ /* No index files - don't do any locking. In theory we still
+ could, but this lock is mainly meant to prevent replication
+ problems, and replication wouldn't work without indexes. */
+ *lock_r = NULL;
+ return 0;
+ }
+
+ if (mailbox_lock_file_create(box, DSYNC_MAILBOX_LOCK_FILENAME,
+brain->mailbox_lock_timeout_secs,
+lock_r, ) <= 0) {
+ i_error("Failed to lock mailbox %s for dsyncing: %s",
+ box->vname, error);
+ return -1;
+ }
+ return 0;
+}
Re: Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
On 02-08-2018 15:29, Aki Tuomi wrote: On 02 August 2018 at 16:22 Henrik Larsson wrote: On 02-08-2018 15:16, Aki Tuomi wrote: >> On 02 August 2018 at 16:08 Henrik Larsson >> wrote: >> >> >> On 15-07-2018 11:42, Henrik Larsson wrote: >> > After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, >> > managesieve stopped working. >> > >> > I'm using FreeBSD ports tree, to build these. Only domains have been >> > modified in below output. >> > >> > A FreeBSD bug report have also been created. But I'm not sure if this >> > is FreeBSD ports related or Dovecot/Pigeonhole related: >> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 >> > >> >> [Removed output from "uname", "pkg version", "dmesg" and "dovecot -n" >> from my original post] >> >> Anyone able to give me some input how to solve this? >> >> Best regards >> Henrik Larsson >> > > Hi! > > i_stream_read_memarea was added in v2.3.0. > > Can you try uninstalling /usr/local/libexec/dovecot/managesieve, > recompiling pigeonhole 0.5.2 and reinstalling it? > > Aki Re-compiled today, same error. /usr/local/libexec/dovecot # ll managesieve -rwxr-xr-x 1 root wheel 64664 2 aug 14:50 managesieve* From Dovecot log: Aug 02 15:09:10 master: Info: Dovecot v2.3.2.1 (0719df592) starting up for imap, pop3, lmtp, sieve Aug 02 15:09:10 config: Error: /usr/local/libexec/dovecot/managesieve: Undefined symbol "i_stream_read_memarea" Aug 02 15:09:10 config: Error: managesieve-login: dump-capability process returned 1 /Henrik Can you do ldd /usr/local/libexec/dovecot/managesieve and make sure it points to correct libdovecot.so? Aki ldd showed that managesieve was linked to "libdovecot.so" in "/usr/local/lib/compat/pkg". /usr/local/lib/compat/pkg # ll -rwxr-xr-x 1 root wheel 164080 28 mar 14:50 libdovecot-fts.so.0* -rwxr-xr-x 1 root wheel46016 28 mar 14:50 libdovecot-lda.so.0* -rwxr-xr-x 1 root wheel 122808 28 mar 14:50 libdovecot-login.so.0* -rwxr-xr-x 1 root wheel45728 28 mar 14:50 libdovecot-sql.so.0* -rwxr-xr-x 1 root wheel 1282080 28 mar 14:50 libdovecot-storage.so.0* -rwxr-xr-x 1 root wheel 1203968 28 mar 14:50 libdovecot.so.0* I removed these and recompiled. Now it is indeed working again. I'm not sure how I ended up in this situation. Thank you for the help. /Henrik
Re: Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
> On 02 August 2018 at 16:22 Henrik Larsson wrote: > > > On 02-08-2018 15:16, Aki Tuomi wrote: > >> On 02 August 2018 at 16:08 Henrik Larsson > >> wrote: > >> > >> > >> On 15-07-2018 11:42, Henrik Larsson wrote: > >> > After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, > >> > managesieve stopped working. > >> > > >> > I'm using FreeBSD ports tree, to build these. Only domains have been > >> > modified in below output. > >> > > >> > A FreeBSD bug report have also been created. But I'm not sure if this > >> > is FreeBSD ports related or Dovecot/Pigeonhole related: > >> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 > >> > > >> > >> [Removed output from "uname", "pkg version", "dmesg" and "dovecot -n" > >> from my original post] > >> > >> Anyone able to give me some input how to solve this? > >> > >> Best regards > >> Henrik Larsson > >> > > > > Hi! > > > > i_stream_read_memarea was added in v2.3.0. > > > > Can you try uninstalling /usr/local/libexec/dovecot/managesieve, > > recompiling pigeonhole 0.5.2 and reinstalling it? > > > > Aki > > Re-compiled today, same error. > > /usr/local/libexec/dovecot # ll managesieve > -rwxr-xr-x 1 root wheel 64664 2 aug 14:50 managesieve* > > From Dovecot log: > Aug 02 15:09:10 master: Info: Dovecot v2.3.2.1 (0719df592) starting up > for imap, pop3, lmtp, sieve > Aug 02 15:09:10 config: Error: /usr/local/libexec/dovecot/managesieve: > Undefined symbol "i_stream_read_memarea" > Aug 02 15:09:10 config: Error: managesieve-login: dump-capability > process returned 1 > > /Henrik Can you do ldd /usr/local/libexec/dovecot/managesieve and make sure it points to correct libdovecot.so? Aki
Re: Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
On 02-08-2018 15:16, Aki Tuomi wrote: On 02 August 2018 at 16:08 Henrik Larsson wrote: On 15-07-2018 11:42, Henrik Larsson wrote: > After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, > managesieve stopped working. > > I'm using FreeBSD ports tree, to build these. Only domains have been > modified in below output. > > A FreeBSD bug report have also been created. But I'm not sure if this > is FreeBSD ports related or Dovecot/Pigeonhole related: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 > [Removed output from "uname", "pkg version", "dmesg" and "dovecot -n" from my original post] Anyone able to give me some input how to solve this? Best regards Henrik Larsson Hi! i_stream_read_memarea was added in v2.3.0. Can you try uninstalling /usr/local/libexec/dovecot/managesieve, recompiling pigeonhole 0.5.2 and reinstalling it? Aki Re-compiled today, same error. /usr/local/libexec/dovecot # ll managesieve -rwxr-xr-x 1 root wheel 64664 2 aug 14:50 managesieve* From Dovecot log: Aug 02 15:09:10 master: Info: Dovecot v2.3.2.1 (0719df592) starting up for imap, pop3, lmtp, sieve Aug 02 15:09:10 config: Error: /usr/local/libexec/dovecot/managesieve: Undefined symbol "i_stream_read_memarea" Aug 02 15:09:10 config: Error: managesieve-login: dump-capability process returned 1 /Henrik
Re: Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
> On 02 August 2018 at 16:08 Henrik Larsson wrote: > > > On 15-07-2018 11:42, Henrik Larsson wrote: > > After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, > > managesieve stopped working. > > > > I'm using FreeBSD ports tree, to build these. Only domains have been > > modified in below output. > > > > A FreeBSD bug report have also been created. But I'm not sure if this > > is FreeBSD ports related or Dovecot/Pigeonhole related: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 > > > > [Removed output from "uname", "pkg version", "dmesg" and "dovecot -n" > from my original post] > > Anyone able to give me some input how to solve this? > > Best regards > Henrik Larsson > Hi! i_stream_read_memarea was added in v2.3.0. Can you try uninstalling /usr/local/libexec/dovecot/managesieve, recompiling pigeonhole 0.5.2 and reinstalling it? Aki
Re: Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
On 15-07-2018 11:42, Henrik Larsson wrote: After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, managesieve stopped working. I'm using FreeBSD ports tree, to build these. Only domains have been modified in below output. A FreeBSD bug report have also been created. But I'm not sure if this is FreeBSD ports related or Dovecot/Pigeonhole related: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 [Removed output from "uname", "pkg version", "dmesg" and "dovecot -n" from my original post] Anyone able to give me some input how to solve this? Best regards Henrik Larsson
