Re: v2.2.36 release candidate released

[Gerald Galster]

Hello Timo,

> Am 30.04.2018 um 16:11 schrieb Timo Sirainen :
> 
> https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz
> https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz.sig 
> 
> v2.2.36 is hopefully going to be the last v2.2.x release. Please test this RC 
> well, so we'll have a good final release! v2.3.2 is still going to take a 
> couple of months before it's ready.

I've temporarly upgraded two replicated servers to 2.2.36. The mail duplication 
on pop3 expunge is not fixed yet. Last known good version is 2.2.33.2 (2.2.34 
was not tested).

Summary: mx2a and mx2b are synced via dsync replication (ssh). An email is 
received via mx2a/lmtp and replicated to mx2b. Then it is fetched from mx2b 
with pop3 and expunged. On that expunge it gets duplicated (copied from INBOX) 
so that the mail reapears once again on next pop3 fetch. Duplication does not 
occur when received and deleted on the same host. There are no sieve scripts 
involved.


May 04 18:02:35 mx2b.example.com dovecot[25546]: pop3(popt...@gcore.biz): 
expunge: box=INBOX, uid=29, 
msgid=<228c5d0d-1ad0-44fb-a83f-6a2aa65cb...@example.com>, size=2383, 
subject=test 1802
May 04 18:02:36 mx2b.example.com dovecot[25546]: doveadm: Error: 
dsync-remote(popt...@gcore.biz): Info: copy from INBOX: box=INBOX, uid=30, 
msgid=<228c5d0d-1ad0-44fb-a83f-6a2aa65cb...@example.com>, size=2383, 
subject=test 1802
May 04 18:02:36 mx2b.example.com dovecot[25546]: doveadm: Error: 
dsync-remote(popt...@gcore.biz): Info: expunge: box=INBOX, uid=29, 
msgid=<228c5d0d-1ad0-44fb-a83f-6a2aa65cb...@example.com>, size=2383, 
subject=test 1802

May 04 18:02:42 mx2b.example.com dovecot[25546]: pop3-login: Login: 
user=, method=PLAIN, rip=91.x.x.1, lip=188.x.x.1, 
mpid=25798, TLS, TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
May 04 18:02:42 mx2b.example.com dovecot[25546]: pop3(popt...@gcore.biz): 
expunge: box=INBOX, uid=30, 
msgid=<228c5d0d-1ad0-44fb-a83f-6a2aa65cb...@example.com>, size=2383, 
subject=test 1802
May 04 18:02:42 mx2b.example.com dovecot[25546]: pop3(popt...@gcore.biz): 
Disconnected: Logged out top=0/0, retr=1/2449, del=1/1, size=2432


For more details see my mail: 2018-03-27 16:37 / Duplicate mails on pop3 
expunge with dsync replication on 2.2.35 (2.2.33.2 works)

It would be nice if you could have a look if something changed on dsync 
replication after 2.2.33.2.

Thanks!
Gerald

RE: v2.2.36 release candidate released

[John van der Kamp]

Thanks, that fixes it for me.

 
John

 
From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf Of Timo Sirainen
Sent: Thursday, 3 May, 2018 13:52
To: John van der Kamp 
Cc: Dovecot Mailing List 
Subject: Re: v2.2.36 release candidate released

 
On 2 May 2018, at 12.52, John van der Kamp mailto:jk...@amazon.nl> > wrote:

 
There seems to be a problem with new user directories and the imapc backend. On 
a test setup I get the following error in the log:

May 02 10:09:58 imap(us...@domain.com <mailto:us...@domain.com> ): Error: 
Couldn't create mailbox list lock 
/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com 
<mailto:tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com> /imapc/mailboxes.lock: 
file_create_locked(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com 
<mailto:tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com> /imapc/mailboxes.lock) 
failed: safe_mkstemp(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com 
<mailto:tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com> /imapc/mailboxes.lock) 
failed: No such file or directory

The 'imapc' part in the user directory is not there (yet), and so the 
mailboxes.lock file cannot be created. A [SERVERBUG] Internal error occurred is 
returned to the client.
This may have already been broken in an earlier release.
Older releases don't seem to have the mailbox lock at this point, so they don't 
return an error when the imapc directory is not there yet.

 
Should be fixed by 
https://github.com/dovecot/core/commit/27366c1b4258a8b3becfb1c3e30b0d3aa8798daa

 

Re: v2.2.36 release candidate released

[Timo Sirainen]

On 2 May 2018, at 12.52, John van der Kamp  wrote:
> 
> There seems to be a problem with new user directories and the imapc backend. 
> On a test setup I get the following error in the log:
> 
> May 02 10:09:58 imap(us...@domain.com): Error: Couldn't create mailbox list 
> lock /tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock: 
> file_create_locked(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock)
>  failed: 
> safe_mkstemp(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock)
>  failed: No such file or directory
> 
> The 'imapc' part in the user directory is not there (yet), and so the 
> mailboxes.lock file cannot be created. A [SERVERBUG] Internal error occurred 
> is returned to the client.
> This may have already been broken in an earlier release.
> Older releases don't seem to have the mailbox lock at this point, so they 
> don't return an error when the imapc directory is not there yet.

Should be fixed by 
https://github.com/dovecot/core/commit/27366c1b4258a8b3becfb1c3e30b0d3aa8798daa 

RE: v2.2.36 release candidate released

[John van der Kamp]

There seems to be a problem with new user directories and the imapc backend. On 
a test setup I get the following error in the log:

May 02 10:09:58 imap(us...@domain.com): Error: Couldn't create mailbox list 
lock /tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock: 
file_create_locked(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock)
 failed: 
safe_mkstemp(/tmp/tmp0Y5TMn/dovecot1/users/us...@domain.com/imapc/mailboxes.lock)
 failed: No such file or directory

The 'imapc' part in the user directory is not there (yet), and so the 
mailboxes.lock file cannot be created. A [SERVERBUG] Internal error occurred is 
returned to the client.
This may have already been broken in an earlier release.
Older releases don't seem to have the mailbox lock at this point, so they don't 
return an error when the imapc directory is not there yet.

Client commands to reproduce:

a LOGIN username password
b CREATE INBOX/folder

John

-Original Message-
From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf Of Timo Sirainen
Sent: Monday, 30 April, 2018 16:11
To: dovecot-n...@dovecot.org; Dovecot Mailing List 
Subject: v2.2.36 release candidate released

https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz
https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz.sig 

v2.2.36 is hopefully going to be the last v2.2.x release. Please test this RC 
well, so we'll have a good final release! v2.3.2 is still going to take a 
couple of months before it's ready.

 * login-proxy: If ssl_require_crl=no, allow revoked certificates.
   Also don't do CRL checks for incoming client certificates.
 * stats plugin: Don't temporarily enable PR_SET_DUMPABLE while opening
   /proc/self/io. This may still cause security problems if the process
   is ptrace()d at the same time. Instead, open it while still running
   as root.

 + doveadm: Added mailbox cache decision&remove commands. See
   doveadm-mailbox(1) man page for details.
 + doveadm: Added rebuild attachments command for rebuilding
   $HasAttachment or $HasNoAttachment flags for matching mails. See
   doveadm-rebuild(1) man page for details.
 - dsync: Lock file generation failed if home directory didn't exist
 - Snippet generation for HTML mails didn't ignore &entities inside
   blockquotes, producing strange looking snippets.
 - imapc: Fix assert-crash if getting disconnected and after
   reconnection all mails in the selected mailbox are gone.
 - pop3c: Handle unexpected server disconnections without assert-crash
 - fts: If mails contained NUL characters, the text around it wasn't
   indexed.
 - Obsolete dovecot.index.cache offsets were sometimes used. Trying to
   fetch a field that was just added to cache file may not have always
   found it.

v2.2.36 release candidate released

[Timo Sirainen]

https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz
https://dovecot.org/releases/2.2/rc/dovecot-2.2.36.rc1.tar.gz.sig 

v2.2.36 is hopefully going to be the last v2.2.x release. Please test this RC 
well, so we'll have a good final release! v2.3.2 is still going to take a 
couple of months before it's ready.

 * login-proxy: If ssl_require_crl=no, allow revoked certificates.
   Also don't do CRL checks for incoming client certificates.
 * stats plugin: Don't temporarily enable PR_SET_DUMPABLE while opening
   /proc/self/io. This may still cause security problems if the process
   is ptrace()d at the same time. Instead, open it while still running
   as root.

 + doveadm: Added mailbox cache decision&remove commands. See
   doveadm-mailbox(1) man page for details.
 + doveadm: Added rebuild attachments command for rebuilding
   $HasAttachment or $HasNoAttachment flags for matching mails. See
   doveadm-rebuild(1) man page for details.
 - dsync: Lock file generation failed if home directory didn't exist
 - Snippet generation for HTML mails didn't ignore &entities inside
   blockquotes, producing strange looking snippets.
 - imapc: Fix assert-crash if getting disconnected and after
   reconnection all mails in the selected mailbox are gone.
 - pop3c: Handle unexpected server disconnections without assert-crash
 - fts: If mails contained NUL characters, the text around it wasn't
   indexed.
 - Obsolete dovecot.index.cache offsets were sometimes used. Trying to
   fetch a field that was just added to cache file may not have always
   found it.