Re: Pigeonhole / Bug with "duplicate" ?
On 28/04/2017 00:39, Stephan Bosch wrote: Looks like a bug that appeared in 0.4.17? Am I correct? This bug is much older than that. It was exposed by recent changes though. This should fix it: https://github.com/dovecot/pigeonhole/commit/3e1a17a286ab0e084577fc267a442cb12aed1cbc Hi Stephan, Thanks for taking the time to work on this issue. I applied your patch and the problem seems to be gone now. Regards, Gilles. smime.p7s Description: S/MIME Cryptographic Signature
Pigeonhole / Bug with "duplicate" ?
Hello,
Previously, while running Dovecot 2.2.27/Pigeonhole 0.4.16 the following
code snippet was working as one would expect:
if duplicate {
addflag ["\\seen", "Duplicate"];
fileinto "Duplicate";
stop;
}
if address :contains "to" "user.em...@example.com" {
addflag "$label1";
}
After upgrading to Dovecot 2.2.29.1/Pigeonhole 0.4.18 it's not working
anymore. When an e-mail is sent to the user the "$label1" isn't applied
at all to the message (in the case it's NOT a duplicate).
Downgrading to Dovecot 2.2.29.1/Pigeonhole 0.4.17 exposes the same
incorrect behaviour.
Downgrading to Dovecot 2.2.29.1/Pigeonhole 0.4.16 is OK.
When running pigeonhole 0.4.17 & 0.4.18, commenting the "if duplicate
{}" block restores the expected behaviour.
Looks like a bug that appeared in 0.4.17? Am I correct?
Kind regards,
Gilles.
smime.p7s
Description: S/MIME Cryptographic Signature
Re: Reappearing emails
On 17/01/2017 16:15, Ron Cleven wrote: The dovecot version (2.2.10) we were running had a timing window problem in the context of replication (we use maildir on CentOS 7). After doing elaborate tests to isolate the problem, we wound up upgrading to 2.2.23. That fixed it, and that version has been very reliable. Hi, We are running Dovecot v2.2.27 here on CentOS 7 (replication and sdbox on both sides) and I have at least one user that reported me a similar issue. This seems to happen randomly. Looking at the mail logs, it looks like the mail is reappearing from nowhere although the mail was copied from INBOX to another mailbox and was expunged by her MUA (Thunderbird) right after. She is not using SIEVE filtering and prefer having local filters. Each time this issue pops in, her MUA is running. The pattern is the following: 1) lmtp delivers a new mail to her INBOX 2) Thunderbird, almost instantly, copies and expunges this mail 3) The email is reappearing (with a new UID) in her INBOX 4) GOTO 2 and repeat several times (she received one mail duplicated 6 times today) Looks like a bug but I wasn't able to reproduce it by myself. Regards, Gilles smime.p7s Description: S/MIME Cryptographic Signature
Panic: file dsync-mailbox-tree-sync.c: line 576 (node_mailbox_trees_cmp): assertion failed: (ret != 0)
Hi,
Here is a crash that's happening using the latest Dovecot version
(v2.2.27 on CentOS7 x86_64):
We are using replication. Judging by the second server's logs, I believe
this has something to do with the fact that we're using the lazy_expunge
plugin.
Every night after midnight, we purge the lazy_expunge namespace by
running a command similar to following one on the main server (server01
in the logs below):
# doveadm expunge -A MAILBOX 'EXPUNGED/*' SAVEDBEFORE "$(date +'%F')"
The lazy_expunge namespace is configured as follows:
namespace expunged {
disabled = no
hidden = yes
ignore_on_failure = no
inbox = no
list = no
location =
sdbox:/srv/mail/expunged/%1n/%n:SUBSCRIPTIONS=subscriptions-expunged
order = 0
prefix = EXPUNGED/
separator = /
subscriptions = no
type = private
}
plugin {
lazy_expunge = EXPUNGED/
}
First server's logs:
2016-12-06T13:09:31.212620+01:00 server01 dovecot: dsync-server(user01):
Panic: file dsync-mailbox-tree-sync.c: line 576
(node_mailbox_trees_cmp): assertion failed: (ret != 0)
2016-12-06T13:09:31.213473+01:00 server01 dovecot: dsync-server(user01):
Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x92d70)
[0x7f57cc18ad70] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x92e4e)
[0x7f57cc18ae4e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f57cc1264e0] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes]() [0x447584] -> dovecot/doveadm-server
[10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x447a57] ->
dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]()
[0x447849] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes]() [0x447849] -> dovecot/doveadm-server
[10.0.0.2 user01
recv_mailbox_tree_deletes](dsync_mailbox_trees_sync_init+0x1f9)
[0x447cc9] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes](dsync_brain_recv_mailbox_tree_deletes+0x207)
[0x43a477] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes](dsync_brain_run+0x5fe) [0x436e4e] ->
dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]()
[0x437280] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes]() [0x44c5af] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
[0x7f57cc19e4d2] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7)
[0x7f57cc19fa07] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c)
[0x7f57cc19e56c] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f57cc19e728] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes]() [0x41faae] -> dovecot/doveadm-server
[10.0.0.2 user01 recv_mailbox_tree_deletes]() [0x421316] ->
dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]()
[0x433b64] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
[0x7f57cc19e4d2] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7)
[0x7f57cc19fa07] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c)
[0x7f57cc19e56c] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f57cc19e728] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f57cc12c513] -> dovecot/doveadm-server [10.0.0.2 user01
recv_mailbox_tree_deletes](main+0x197) [0x413b57] ->
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f57cbd57b15] ->
dovecot/doveadm-server [10.0.0.2 user01 recv_mailbox_tree_deletes]()
[0x413bf5]
2016-12-06T13:09:31.214076+01:00 server01 dovecot: dsync-server(user01):
Fatal: master: service(doveadm): child 6837 killed with signal 6 (core
dumps disabled)
2016-12-06T13:11:42.880516+01:00 server01 dovecot: dsync-local(user01):
Error: read(server02.localdomain) failed: EOF (last sent=mailbox_delete,
last recv=mailbox_tree_node)
Second server's logs:
2016-12-06T12:09:36.220591+01:00 server02 dovecot: doveadm(user01):
Error: Duplicate mailbox GUID eab35824e49b4658840001001776124f for
mailboxes EXPUNGED/Trash/C2i -
Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i and
EXPUNGED/Trash/C2i - Coordination/Demissions C2i - giving a new GUID
c4ffaf08709c465813aea26dfa6e to EXPUNGED/Trash/C2i -
Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/Demissions C2i
2016-12-06T12:09:36.220859+01:00 server02 dovecot: doveadm(user01):
Error: Duplicate mailbox GUID 2ab10a2bee9b4658840001001776124f for
mailboxes EXPUNGED/Trash/C2i -
Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015 and
EXPUNGED/Trash/C2i - Coordination/2014-2015 - giving a new GUID
c6ffaf08709c465813aea26dfa6e to EXPUNGED/Trash/C2i -
Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2014-2015
2016-12-06T12:09:36.221091+01:00 server02 dovecot: doveadm(user01):
Error: Duplicate mailbox GUID eeb35824e49b4658840001001776124f for
mailboxes EXPUNGED/Trash/C2i -
Coordination-temp-ac1fea3af7e1f2c29ce1c62442a06272/2012-2013 and
EXPUNGED/Trash/C2i - Coordination/2012-2013 - g
Re: Panic: file dsync-brain-mailbox.c: line 358 (dsync_brain_sync_mailbox_deinit): assertion failed: (brain->failed || brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_CHANGED)
Hi Timo, On 28/10/2016 16:28, Timo Sirainen wrote: This code hasn't changed for quite a long time. So I don't think this is a new bug in 2.2.26. Can you try reproduce it easily? If yes, could you try if the attached patch fixes it? The last time we played with Dovecot's replication was during the v2.1 era and we ended avoiding its use due to numerous bugs and serious issues. Now, we are planning on migrating our Dovecot 2.2.18 VM to two physical servers running the latest release and we thought it would be a good idea to run some new tests, 4 years later, to see how it goes now! We started our new tests some days ago with v2.2.25. This explains why, if this problem isn't new, I wasn't able to report it sooner. Back on topic: While typing the same commands as before, the problem doesn't seem to reproduce after your patch was applied. I'll let you know if it shows up again. Thanks, Regards, Gilles. smime.p7s Description: S/MIME Cryptographic Signature
Panic: file dsync-brain-mailbox.c: line 358 (dsync_brain_sync_mailbox_deinit): assertion failed: (brain->failed || brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_CHANGED)
Hello, Here is a Panic that happened while doing some testing with two servers both running Dovecot v2.2.26 on CentOS 7. These are test servers owning 32 accounts whose data were copied from our production server. What I've done is: server01# doveadm force-resync -A '*' server01# doveadm replicator replicate -f '*' For 5 accounts I obtained the following crash: 2016-10-28T14:09:43.236946+02:00 server01 dovecot: dsync-server(someuser): Panic: file dsync-brain-mailbox.c: line 358 (dsync_brain_sync_mailbox_deinit): assertion failed: (brain->failed || brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_CHANGED) 2016-10-28T14:09:43.237441+02:00 server01 dovecot: dsync-server(someuser): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x8f7e0) [0x7f3d9318d7e0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x8f8be) [0x7f3d9318d8be] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3d9312b9be] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_sync_mailbox_deinit+0x163) [0x438243] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_slave_recv_mailbox+0x277) [0x438da7] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_run+0x5fe) [0x4368be] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x436c71] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x44becf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f3d931a0c3c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f3d931a1fd7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f3d931a0cc5] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d931a0e78] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x41fc7e] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x421256] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x433654] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f3d931a0c3c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f3d931a1fd7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f3d931a0cc5] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d931a0e78] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3d93131a23] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](main+0x197) [0x413c87] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f3d92d5db15] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x413d25] 2016-10-28T14:09:43.238013+02:00 server01 dovecot: dsync-server(someuser): Fatal: master: service(doveadm): child 96390 killed with signal 6 (core dumps disabled) 2016-10-28T14:09:43.505098+02:00 server01 dovecot: dsync-server(someuser): Error: read(server02.localdomain) failed: read(size=5807) failed: Connection reset by peer (last sent=mailbox_state, last recv=mailbox_state) Regards, Gilles. smime.p7s Description: S/MIME Cryptographic Signature
Re: Correct Way to Run expunge for All?
Hello, On 29/01/2016 12:53, Haravikk wrote: Fatal: expunge: To avoid accidents, search query must contain MAILBOX in all search branches While I can see the logic in this, it isn’t an accidental use of this command, so who do I force it to proceed? What about this command ? → doveadm expunge -A DELETED MAILBOX '*' Greetings, Gilles. smime.p7s Description: S/MIME Cryptographic Signature
Re: mail_log plugin and uid=error in log file
On 01/12/2015 17:14, Marcus Rueckert wrote:
mail_log_prefix = "%s(%u): %{session} "
only downside is some lines will then end up with the session ID twice.
When i asked Timo said he doesnt want to change that behavior in 2.2.
but in 2.3 it might be an option to unify that.
Hi Marcus,
I will try this soon.
Thanks for the tip.
Regards,
Gilles
smime.p7s
Description: S/MIME Cryptographic Signature
Re: mail_log plugin and uid=error in log file
On 01/12/2015 16:31, Timo Sirainen wrote: Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a Hi Timo, Sorry about the thread hijacking but, speaking about the mail_log plugin, what do you think about the ability to add the session number to the log lines produced by this plugin? This could be a useful information to have too, especially on a large traffic mailhost. Regards, Gilles smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] Mail logger plugin, improvement request
Hi Timo, Would it be possible for the mail logger plugin to also log the imap MOVE events (RFC6851) ? Thanks. Regards, Gilles.
Re: [Dovecot] replication + attachment sis + zlib bug ? (HEAD version from xi.rename-it.nl)
On 10/04/2014 13:38, Pavel Stano wrote: > Hi, > > i have setup with mail_attachment single instance store + replication + > zlib and got this bug when i try to replicate one test mailbox: > > On master1 in mail.log: > Apr 10 13:25:22 master1 dovecot: > dsync-local(z...@blabla666.sk): Error: > read(/nfsmnt/mailnfs1/attachments1/6b/57/6b57ad34cf6c414662233d833a7801fde4e1cdcb-92b5052558774653a72813e2b982[base64:18 > b/l]) failed: Stream is larger than expected (97824 > 97823, eof=1) Apr > 10 13:25:22 master1 dovecot: dsync-local(z...@blabla666.sk): > Error: dsync(master2): > read(attachments-connector(zlib(/nfsmnt/mailnfs1/b/l/blabla666.sk/z...@blabla666.sk/mdbox/storage/m.9))) > failed: > read(/nfsmnt/mailnfs1/attachments1/6b/57/6b57ad34cf6c414662233d833a7801fde4e1cdcb-92b5052558774653a72813e2b982[base64:18 > b/l]) failed: Stream is larger than expected (97824 > 97823, eof=1) > > > This is on master2 in mail.log > Apr 10 13:32:21 master2 dovecot: dsync-server(z...@blabla666.sk): Error: > dsync(master1): read() failed: read(10.10.30.2) failed: > dot-input stream ends without '.' line > Hi, Your problem looks quite similar to the one I reported 2 months ago. → http://markmail.org/message/tt4jpjnpsa6lmlz2 Regards, Gilles
Re: [Dovecot] Dsync crash (v2.2.10, sdbox+sis → mbox)
Hi Timo, I've made some further research on this issue (Dovecot was upgraded to the latest release in the meantime but, unsurprisingly, to no avail) and here's what I've found so far. On 09/02/2014 10:42, Gilles Chauvin wrote: > dsync(user2): Error: > read(/zfspool/clone_srv_attachments/ad/0c/ad0cef35cc6f0b2dae2197c4ff2b61a2bd58070d-9e8345192ccbf352c21044c1c7e7-6efa5f2e522db350ed3d94b229f9-15470[base64:18 > b/l]) failed: Stream is larger than expected (194476 > 194475, eof=1) > dsync(user2): Error: copy: i_stream_read() failed: Invalid argument > dsync(user2): Panic: file mail-index-transaction-update.c: line 19 > (mail_index_transaction_lookup): assertion failed: (seq >= > t->first_new_seq && seq <= t->last_new_seq) The original mail got an attachment which is base64 encoded on 72 cols. The last 3 lines are: MAAxADMAIAAyADAAOgAwADEAOgA1ADQADQAKAGwAJwB1AHQAaQBsAGkAcwBhAHQAZQB1AHIA IABkAGUAIABsAG8AZwBpAG4AOgAgAGsAZQBsAGUAbQBhAHIAaQAgAGEAIADpAHQA6QAgAGMA cgDpAOkAIABsAGUAIAAyADEALwAwADMALwAyADAAMQAzACAAMgAwADoAMAAyADoAMAA0AA0ACgA= For no good reason, the last line lacks a CR before the final "CgA=" part. I guess this is where Dovecot yells about the "stream larger than expected" because when it reencodes the attachment, it does it correctly by adding a proper CR before "CgA=" hence the one byte difference (tested using the "base64" command line tool). During my tests, each time dsync failed with this particular error, the same pattern applied (malformed base64 last line). Looks like a pretty hard problem to solve but, for now, it prevents us from restoring a mailbox. Regards, Gilles
Re: [Dovecot] Dsync Panic
Hi, Here is another dsync Panic while using: $ dsync -Dvf -u user -R backup ssh r...@server.domain.tld dsync -u user Dovecot 2.2.11 is running on both sides: dsync-local(user): Debug: brain M: in state=master_recv_handshake dsync-local(user): Debug: brain M: out state=master_recv_handshake changed=0 dsync-local(user): Debug: brain M: in state=master_recv_handshake dsync-local(user): Debug: brain M: out state=send_mailbox_tree changed=1 dsync-local(user): Debug: brain M: in state=send_mailbox_tree dsync-local(user): Debug: brain M: out state=send_mailbox_tree_deletes changed=1 dsync-local(user): Debug: brain M: in state=send_mailbox_tree_deletes dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=1 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=1 dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-remote(user): Panic: file dsync-mailbox-tree-sync.c: line 401 (sync_rename_node_to_temp): assertion failed: (ctx->sync_type != DSYNC_MAILBOX_TREES_SYNC_TYPE_PRESERVE_LOCAL) dsync-remote(user): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x68aea) [0x7f616d58aaea] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f616d58abf2] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f616d54423f] -> dsyn() [0x437c06] -> dsyn() [0x438122] -> dsyn() [0x438494] -> dsyn() [0x43821c] -> dsyn(dsync_mailbox_trees_sync_init+0xe6) [0x439766] -> dsyn(dsync_brain_recv_mailbox_tree_deletes+0x102) [0x42d602] -> dsyn(dsync_brain_run+0x2e6) [0x42afb6] -> dsyn() [0x42b910] -> dsyn() [0x43db50] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f616d59a666] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f616d59b6d7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f616d59a5d8] -> dsyn() [0x4282f4] -> dsyn() [0x411ca7] -> dsyn(doveadm_mail_try_run+0x238) [0x4120b8] -> dsyn(main+0x3d1) [0x41aaf1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f616d1acd1d] -> dsyn() [0x411429] dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(user): Error: read(server.domain.tld) failed: EOF dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-local(user): Error: Remote command returned error 25 Regards, Gilles.
[Dovecot] Dsync crash (v2.2.10, sdbox+sis → mbox)
Hi, I'm trying to use dsync to convert sdbox + sis mailboxes to mbox (mbox is chosen here to "re-attach" the attachments to their original place) # dsync -Dv -u $LOGIN -o "mail_location=sdbox:/zfspool/clone_srv_mail/$LOGIN" -o "mail_attachment_dir=/zfspool/clone_srv_attachments" backup "mbox:/zfspool/restore/$LOGIN/mbox:DIRNAME=mBoX-MeSsAgEs:INDEX=/zfspool/restore/$LOGIN/indexes:CONTROL=/zfspool/restore/$LOGIN/control" For 5 users out of a sample of 24, here is what's happening: dsync(user1): Error: read(/zfspool/clone_srv_attachments/cb/0a/cb0aad465a4ff95bf6fa6ece0fba94b43e8892cf-19dc51309fc2f3527e3144c1c7e7-b55eb9176ca1b350e56594b229f9-30810[base64:19 b/l]) failed: Stream is larger than expected (163244 > 163243, eof=1) dsync(user1): Error: copy: i_stream_read() failed: Invalid argument dsync(user1): Panic: file mail-index-transaction-update.c: line 19 (mail_index_transaction_lookup): assertion failed: (seq >= t->first_new_seq && seq <= t->last_new_seq) dsync(user1): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x6889a) [0x7f58a95a189a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f58a95a19a2] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f58a955b1cf] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc0287) [0x7f58a98ca287] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc3145) [0x7f58a98cd145] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_decision_state_update+0xb6) [0x7f58a98bcb06] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_lookup_headers+0x91) [0x7f58a98be5e1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xa0ac3) [0x7f58a98aaac3] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0x4a) [0x7f58a98ab04a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c021) [0x7f58a98a6021] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c151) [0x7f58a98a6151] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_close+0xf5) [0x7f58a98a6295] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x48) [0x7f58a98867c8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x92) [0x7f58a9880e32] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f) [0x7f58a9886c2f] -> dsync() [0x42f750] -> dsync(dsync_brain_sync_mails+0x459) [0x42e9c9] -> dsync(dsync_brain_run+0x2a1) [0x42ac51] -> dsync() [0x42876f] -> dsync() [0x411c97] -> dsync(doveadm_mail_try_run+0x238) [0x4120a8] -> dsync(main+0x3d1) [0x41aaa1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f58a91c3d1d] -> dsync() [0x411419] dsync(user2): Error: read(/zfspool/clone_srv_attachments/ad/0c/ad0cef35cc6f0b2dae2197c4ff2b61a2bd58070d-9e8345192ccbf352c21044c1c7e7-6efa5f2e522db350ed3d94b229f9-15470[base64:18 b/l]) failed: Stream is larger than expected (194476 > 194475, eof=1) dsync(user2): Error: copy: i_stream_read() failed: Invalid argument dsync(user2): Panic: file mail-index-transaction-update.c: line 19 (mail_index_transaction_lookup): assertion failed: (seq >= t->first_new_seq && seq <= t->last_new_seq) dsync(user2): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x6889a) [0x7f2e2248d89a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f2e2248d9a2] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2e224471cf] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc0287) [0x7f2e227b6287] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc3145) [0x7f2e227b9145] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_decision_state_update+0xb6) [0x7f2e227a8b06] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_lookup_headers+0x91) [0x7f2e227aa5e1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xa0ac3) [0x7f2e22796ac3] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0x4a) [0x7f2e2279704a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c021) [0x7f2e22792021] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c151) [0x7f2e22792151] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_close+0xf5) [0x7f2e22792295] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x48) [0x7f2e227727c8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x92) [0x7f2e2276ce32] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f) [0x7f2e22772c2f] -> dsync() [0x42f750] -> dsync(dsync_brain_sync_mails+0x459) [0x42e9c9] -> dsync(dsync_brain_run+0x2a1) [0x42ac51] -> dsync() [0x42876f] -> dsync() [0x411c97] -> dsync(doveadm_mail_try_run+0x238) [0x4120a8] -> dsync(main+0x3d1) [0x41aaa1] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f2e220afd1d] -> dsync() [0x411419] dsync(user3): Error: read(/zfspool/clone_srv_attachments/23/8a/238a781b53bb4d1b1bee989a5ff38636b616d5c5-41ba47152912f4522c6f44c1c7e7-f3b06c2f5aa1b350d56594b229f9-38650[base64:19 b/l]) failed: Stream is larger than expected (33191 > 33190, eof=1)
Re: [Dovecot] Question regarding quotas (is this a bug or intended behavior) ?
On 09/12/2013 14:42, Timo Sirainen wrote: > On 9.12.2013, at 14.08, Gilles Chauvin wrote: > >> On 08/12/2013 22:05, Timo Sirainen wrote: >>> On 4.12.2013, at 19.02, Gilles Chauvin wrote: >>> >>>> I tried to add the following quota_rule: >>>> quota_rule3 = Trash/*:ignore >>>> >>>> But this doesn't fix anything. >>>> >>>> Do you have any idea about why this is not working ? >>> >>> It wasn’t implemented. But I guess it’s useful and there’s no harm in >>> adding it, so: http://hg.dovecot.org/dovecot-2.2/rev/78b34eb7c6c5 >> >> Looks like something is broken now :-/ > > Right, it actually came to my mind when I went to sleep that I probably > messed up this: http://hg.dovecot.org/dovecot-2.2/rev/feb656fd212e > That's better, thanks. Although there is still a problem. # doveadm mailbox status -u myuser "messages vsize" '*' TEST messages=765 vsize=151182915 Junk messages=0 vsize=0 Trash messages=0 vsize=0 Drafts messages=0 vsize=0 Sent messages=0 vsize=0 INBOX messages=0 vsize=0 # doveadm -f flow quota get -u myuser Quota name=Userquota Type=STORAGE Value=145564 Limit=1048576 %=13 Quota name=Userquota Type=MESSAGE Value=765 Limit=- %=0 If now I just drag n' drop the "TEST" mailbox to the Trash, the log says "Mailbox renamed: TEST -> Trash/TEST" But... This is only taken into account if I issue a "quota recalc" command. # doveadm -f flow quota get -u myuser Quota name=Userquota Type=STORAGE Value=145564 Limit=1048576 %=13 Quota name=Userquota Type=MESSAGE Value=765 Limit=- %=0 # doveadm quota recalc -u myuser # doveadm -f flow quota get -u myuser Quota name=Userquota Type=STORAGE Value=0 Limit=1048576 %=0 Quota name=Userquota Type=MESSAGE Value=0 Limit=- %=0 If this is not easy to fix for you, I guess I'll have to recalc the quotas in a cron job! Thanks, Gilles.
Re: [Dovecot] Question regarding quotas (is this a bug or intended behavior) ?
On 08/12/2013 22:05, Timo Sirainen wrote:
> On 4.12.2013, at 19.02, Gilles Chauvin wrote:
>
>> I tried to add the following quota_rule:
>> quota_rule3 = Trash/*:ignore
>>
>> But this doesn't fix anything.
>>
>> Do you have any idea about why this is not working ?
>
> It wasn’t implemented. But I guess it’s useful and there’s no harm in adding
> it, so: http://hg.dovecot.org/dovecot-2.2/rev/78b34eb7c6c5
>
Looks like something is broken now :-/
Config:
plugin {
quota = dict:Userquota::file:%h/dovecot-quota
quota_rule = *:storage=1G
}
# doveadm mailbox status -u myuser "messages vsize" INBOX
INBOX messages=765 vsize=151182915
# doveadm -f flow quota get -u myuser
Quota name=Userquota Type=STORAGE Value=145564 Limit=1048576 %=13
Quota name=Userquota Type=MESSAGE Value=765 Limit=- %=0
Config change only:
plugin {
quota = dict:Userquota::file:%h/dovecot-quota
quota_rule = *:storage=1G
quota_rule2 = Trash:ignore
}
# doveadm mailbox status -u myuser "messages vsize" INBOX
INBOX messages=765 vsize=151182915
# doveadm -f flow quota get -u myuser
Quota name=Userquota Type=STORAGE Value=145564 Limit=- %=0
Quota name=Userquota Type=MESSAGE Value=765 Limit=- %=0
Regards,
Gilles.
[Dovecot] Question regarding quotas (is this a bug or intended behavior) ?
Hi,
I was wondering if this is a normal behavior (test was made using
Dovecot v2.2.9).
In my config, quotas are configured as follows:
plugin {
quota = dict:Userquota::file:%h/dovecot-quota
quota_rule = *:storage=1G
quota_rule2 = Trash:ignore
}
# doveadm mailbox status -u my_user "messages vsize" '*'
Trash messages=4997 vsize=229535631
Drafts messages=0 vsize=0
Sent messages=0 vsize=0
Junk messages=0 vsize=0
INBOX messages=0 vsize=0
# doveadm -f flow quota get -u my_user
Quota name=Userquota Type=STORAGE Value=0 Limit=1048576 %=0
Quota name=Userquota Type=MESSAGE Value=0 Limit=- %=0
The 4997 mails in the Trash mailbox are ignored as desired, but now, if
I have the following case:
# doveadm mailbox status -u my_user "messages vsize" '*'
Trash messages=0 vsize=0
Trash/TEST messages=4997 vsize=229535631
Drafts messages=0 vsize=0
Sent messages=0 vsize=0
Junk messages=0 vsize=0
INBOX messages=0 vsize=0
(As you can see mails were moved to a Trash/TEST mailbox)
# doveadm -f flow quota get -u my_user
Quota name=Userquota Type=STORAGE Value=220918 Limit=1048576 %=21
Quota name=Userquota Type=MESSAGE Value=4997 Limit=- %=0
I tried to add the following quota_rule:
quota_rule3 = Trash/*:ignore
But this doesn't fix anything.
Do you have any idea about why this is not working ?
Thanks,
Regards,
Gilles.
Re: [Dovecot] v2.2.7 released
On 03/11/2013 21:08, Timo Sirainen wrote: > + mdbox: Added "mdbox_deleted" storage, which can be used to access > messages with refcount=0. For example: doveadm import > mdbox_deleted:~/mdbox "" mailbox inbox subject oops > Hi Timo, We're currently running Dovecot 2.1.16. To ease the recovery process, in case of accidental mail deletion, we're using the lazy_expunge plugin to keep deleted mail in a user hidden namespace during a couple of days before they really get deleted. Could this be replaced by this new feature? I guess the mdbox_deleted storage get emptied after a purge (which is what we're doing every night)? Regards, Gilles.
Re: [Dovecot] Any way to let dovecot block pop3 attempts?
On Friday 10 May 2013 09:17:28 Steve Campbell wrote: > But I believe fail2ban uses iptables, and I don't run a local firewall > on the server. I'd prefer not to use a separate server to inject > firewall rules on the border firewall. I might be wrong about fail2ban, > though. > > I was hoping there was a file for pop and imap in dovecot similar to the > smtp "access" file in sendmail (which is what I use, BTW) Yes, Fail2Ban uses iptables. I don't think there is another way (using Dovecot itself) to block a remote host since Fail2Ban is documented on Dovecot' wiki: http://wiki2.dovecot.org/HowTo/Fail2Ban (it looks like one of the best way to achieve this). Gilles. -- = Gilles CHAUVIN Administrateur systèmes Pôle Systèmes Direction de l'informatique & des systèmes d'information Université de ROUEN Bat.16-IRESE-B-Place Émile Blondel 76821 MONT-SAINT-AIGNAN CÉDEX Accès: http://goo.gl/cYgtX Tél: 02.35.14.82.92 Fax: 02.35.14.64.64 Accueil DSI: 02.35.14.61.00 Mail fonc: syst...@univ-rouen.fr Mail pers: gilles.chau...@univ-rouen.fr =
Re: [Dovecot] Any way to let dovecot block pop3 attempts?
On Friday 10 May 2013 08:17:50 Steve Campbell wrote: > Is there a way using dovecot facilities to block an IP from attempting > POP3 connections (similar to the sendmail access file for smtp > connections)? I usually do this at my border firewall, but if there's a > quick and dirty way in dovecot to do this, it'd make life a little simpler. > Hi Steve, We've been using Fail2Ban on our mail proxies for a while without any problem. It may be what you're looking for. Regards, Gilles.
[Dovecot] lazy_expunge plugin with virtual mailboxes
Hi, When using both lazy_expunge and virtual mailboxes (with, for instance, a "All Messages" virtual mailbox) as soon as a user deletes/expunges a mail in one of its mailbox, Dovecot is saving two mails to the lazy_expunge namespace (the one in the user mailbox and the one in the virtual mailbox). Is there some kind of trick to make Dovecot ignore any mail deleted from a virtual mailbox namespace ? Thanks, Gilles.
Re: [Dovecot] v2.1.16 released
On Friday 05 April 2013 15:13:58 Timo Sirainen wrote: > Probably too much trouble to fix this in v2.1. v2.2 should have it working > though. So we'll wait till the first 2.2 stable releases are published :). Thanks, Gilles
Re: [Dovecot] v2.1.16 released
On Friday 05 April 2013 15:03:19 Timo Sirainen wrote: > On 5.4.2013, at 15.01, Gilles CHAUVIN wrote: > > On Friday 05 April 2013 14:36:18 Timo Sirainen wrote: > > > Try with mailbox_list_index=yes. It should make the status command > > > faster. > > > And if you're using Maildir, set maildir_very_dirty_syncs=yes. > > > > We're using mdbox so I tried mailbox_list_index = yes (wasn't aware of > > this command) and once activated got a huge amount of these in the logs > > :-/ > > > > 2013-04-05T13:54:55.505235+02:00 host dovecot: imap(xx): Error: Raw > > backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4636a) > > [0x7f2dfaab936a] -> /usr/local/lib/dovecot/libdovecot.s > You're missing the error message before this one, that actually shows where > it crashed. Anyway, I was somehow thinking you were using v2.2. > mailbox_list_index=yes is somewhat buggy in v2.1, but should work great in > v2.2. Oops... The 2 lines before says: 2013-04-05T13:54:55.503668+02:00 host dovecot: master: Error: service(imap): child 3825 killed with signal 6 (core dumps disabled) 2013-04-05T13:54:55.504496+02:00 host dovecot: imap(xx): Panic: file mail- index-sync.c: line 440 (mail_index_sync_begin_to2): assertion failed: (!index- >syncing) Gilles
Re: [Dovecot] v2.1.16 released
On Friday 05 April 2013 14:36:18 Timo Sirainen wrote: > Try with mailbox_list_index=yes. It should make the status command faster. > And if you're using Maildir, set maildir_very_dirty_syncs=yes. We're using mdbox so I tried mailbox_list_index = yes (wasn't aware of this command) and once activated got a huge amount of these in the logs :-/ 2013-04-05T13:54:55.505235+02:00 host dovecot: imap(xx): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4636a) [0x7f2dfaab936a] - > /usr/local/lib/dovecot/libdovecot.s o.0(+0x463b6) [0x7f2dfaab93b6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2dfaa8cc0f] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xbc3e0) [0x7f2dfadc13e0] -> /usr/loca l/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f) [0x7f2dfadc145f] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mail_index_sync_begin+0x1e) [0x7f2dfadc14de] -> /usr/l ocal/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_sync+0xd4) [0x7f2dfad920e4] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mailbox_list_index_refresh+0x9a) [0x7f2dfad90a7a] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mailbox_list_index_iter_init+0xf6) [0x7f2dfad90f96] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mailbox_list_iter_init_multiple+0xf3) [0 x7f2dfad884f3] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mailbox_list_iter_init+0x19) [0x7f2dfad88c39] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0xbd24) [0x7f2df850fd24] -> /usr /local/lib/dovecot/lib01_acl_plugin.so(+0xc65a) [0x7f2df851065a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_list_index_sync+0x1cd) [0x7f2dfad921dd] -> /usr/local/lib/dovecot/l ibdovecot-storage.so.0(mailbox_list_index_refresh+0x9a) [0x7f2dfad90a7a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x8c32b) [0x7f2dfad9132b] -> /usr/local/lib/dovecot/libdovecot-sto rage.so.0(+0x8c960) [0x7f2dfad91960] -> /usr/local/lib/dovecot/libdovecot- storage.so.0(mailbox_sync_deinit+0x2a) [0x7f2dfad7fc6a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sy nc+0x3b) [0x7f2dfad7fd2b] -> dovecot/imap [xx 10.193.97.50 SELECT] (cmd_select_full+0x16e) [0x40d4fe] -> dovecot/imap [xx 10.193.97.50 SELECT](command_exec+0x3d) [0x41150d] -> dovec ot/imap [xx 10.193.97.50 SELECT]() [0x41045e] -> dovecot/imap [xx 10.193.97.50 SELECT]() [0x41053d] -> dovecot/imap [xx 10.193.97.50 SELECT] (client_handl Dovecot was upgraded to v2.1.16 this morning. Gilles
Re: [Dovecot] v2.1.16 released
On Friday 05 April 2013 14:08:30 Timo Sirainen wrote: > doveadm batch -A : quota get : mailbox status -t all '*' > > Except looks like subcommand parameters don't work. Fixed: > http://hg.dovecot.org/dovecot-2.2/rev/2918bfacf565 > > Also those commands crash now because it tries to use "table" > formatter.. Giving doveadm -f tab for example fixes the crash. Anyway, > it's not really meant for those types of commands, most importantly the > idea was to run: > > doveadm batch -A : altmove savedbefore 1w : purge > > And maybe some expunge commands to delete old mails from Trash/Spam. Hi Timo, In fact, I need to run those 2 commands from a Python script once a day (to gather data and make some stats about our mailstore usage). We are slowly migrating from an old cyrus server and currently, with only ~7500 users, running these commands takes a while. At the end of the migration we'll have ~34000 users using the mailstore... I fear gathering these infos will take a very long time ;). BTW, thanks for your reply and for the fix ;). Gilles.
Re: [Dovecot] v2.1.16 released
Hi, On Friday 05 April 2013 00:16:17 Timo Sirainen wrote: > + Added "doveadm batch" command to run multiple commands before moving > onto the next user (useful only with -A and -u ) As I understand this new command, it permits to run several doveadm commands in one shot? If this is correct, could you provide an example showing how to execute those two commands at the same time: - doveadm quota get -A - doveadm mailbox status -A -t all '*' Thanks. Regards, Gilles.
Re: [Dovecot] cmd-vacation.c:4:17: fatal error: lib.h: No such file or directory && ./configure: line 11410: -lssl: command not found
On 29/11/2012 16:14, Tobias Hachmer wrote: > On Thursday 29 November 2012 16:09:35 Gilles Chauvin wrote: >> Just have a look at: >> http://www.dovecot.org/list/dovecot/2012-November/069722.html ;). > > Yeah, I saw your post. But Timo asked for another possible errors. I don't > know if this error belongs to your error: > > cmd-vacation.c:4:17: fatal error: lib.h: No such file or directory > > That's why I posted this. > > Greetz, > Tobias Hachmer > Tobias, The post I linked above wasn't my post but I had the exact same problem this morning while trying to compile pigeonhole against dovecot 2.1.11. Manually adding the quotes in the dovecot-config file fixed the issue for me. Regards, Gilles.
Re: [Dovecot] cmd-vacation.c:4:17: fatal error: lib.h: No such file or directory && ./configure: line 11410: -lssl: command not found
On 29/11/2012 15:57, Tobias Hachmer wrote:> Hello Timo, > > Building 2.1.11 was ok, but rebuilding pigeonhole 0.3.3 for dovecot 2.1.11 > show up the following errors: > > [...] > Hi, Just have a look at: http://www.dovecot.org/list/dovecot/2012-November/069722.html ;). Regards, Gilles.-- ========= Gilles CHAUVIN Pôle Système Direction des Systèmes d'information et de l'Informatique Université de Rouen Bâtiment 16 - IRESE-B Place Émile Blondel 76821 MONT-SAINT-AIGNAN CEDEX → http://goo.gl/cYgtX Tél: +33 (0)2 35 14 82 92 Fax: +33 (0)2 35 14 64 64 Mail fonc: syst...@univ-rouen.fr Mail pers: gilles.chau...@univ-rouen.fr =
