Re: Dovecot dsync certificate errors

[Aki Tuomi]

> On 22/02/2021 00:43 Stephan Mending  wrote:
> 
>  
> Hi *, 
> I've setup two dovecot instances. 
> As soon as i send the syncing part of dovecot to work I see the following 
> errors in my maillogs. 
> 
> $ cat /var/log/mail.log 
> doveadm(inbox@sec-level.domain): Error: doveadm server disconnected before 
> handshake: SSL certificate doesn't match expected host name fqdn.of.system: 
> No match to 1 SubjectAltNames
> doveadm(inbox@sec-level.domain): Error: Disconnected from remote: SSL 
> certificate doesn't match expected host name fqdn.of.system: No match to 1 
> SubjectAltNames
> 
> A little context: The certificates on the servers are issued by a private CA. 
> The public CA-Certificate has been added to the keystore, though. Shouldn't 
> be a problem. 
> I can confirm that by connecting using s_client -> 
> 
> $ openssl s_client -connect :12345
> ..
> ...
> 
> 
> Verify return code: 0 (ok)
> 
> So far the certificate seems to be ok. 
> **BUT**: As soon as i start dovecot (on the very same  machine I issued the 
> s_client command  above) ... I am receiving the error messages: 
> 
> doveadm(inbox@sec-level.domain): Error: doveadm server disconnected before 
> handshake: SSL certificate doesn't match expected host name fqdn.of.system: 
> No match to 1 SubjectAltNames
> doveadm(inbox@sec-level.domain): Error: Disconnected from remote: SSL 
> certificate doesn't match expected host name fqdn.of.system: No match to 1 
> SubjectAltNames
> 
> And yes the SubjectAlternativeName in the certificate matches the configured 
> FQDN in 30-dsync.conf.
> 
> My 30-dsync.conf: 
> 
> $ cat /etc/dovecot/conf.d/30-dsync.conf
> 
> service aggregator {
> fifo_listener replication-notify-fifo {
> user = dovecot
> mode = 0666
> }
> unix_listener replication-notify {
> user = dovecot
> mode = 0666
> }
> }
> 
> # Configuring the replicator service
> service replicator {
> process_min_avail = 1
> unix_listener replicator-doveadm {
> user = dovecot
> mode = 0666
> }
> }
> service doveadm {
> user = dovecot
> inet_listener {
> port = 12345
> ssl = yes
> }
> }
> 
> doveadm_port = 12345
> doveadm_password = 
> replication_max_conns = 1
> 
> plugin {
> mail_replica = tcps:fqdn.of.system
> }
> 
> service config {
> unix_listener config {
> user = dovecot
> }
> }
> 
> 
> I'd love to here the answer to this. 
> 
> Thanks alot ! 
> 
> Best regards, 
> Stephan


The certificate provided does not match. You can use 

openssl s_client -connect host:port -verify_hostname fqdn.of.system

to see if it matches, somewhere in the output should be

SSL handshake has read 3086 bytes and written 378 bytes
Verification: OK
Verified peername: fqdn.of.system

If it does match, try 

openssl x509 -text -noout -in /path/to/cert

to see what the name(s) are.

Aki

Dovecot dsync certificate errors

[Stephan Mending]

Hi *, 
I've setup two dovecot instances. 
As soon as i send the syncing part of dovecot to work I see the following 
errors in my maillogs. 

$ cat /var/log/mail.log 
doveadm(inbox@sec-level.domain): Error: doveadm server disconnected before 
handshake: SSL certificate doesn't match expected host name fqdn.of.system: No 
match to 1 SubjectAltNames
doveadm(inbox@sec-level.domain): Error: Disconnected from remote: SSL 
certificate doesn't match expected host name fqdn.of.system: No match to 1 
SubjectAltNames

A little context: The certificates on the servers are issued by a private CA. 
The public CA-Certificate has been added to the keystore, though. Shouldn't be 
a problem. 
I can confirm that by connecting using s_client -> 

$ openssl s_client -connect :12345
..
...


Verify return code: 0 (ok)

So far the certificate seems to be ok. 
**BUT**: As soon as i start dovecot (on the very same  machine I issued the 
s_client command  above) ... I am receiving the error messages: 

doveadm(inbox@sec-level.domain): Error: doveadm server disconnected before 
handshake: SSL certificate doesn't match expected host name fqdn.of.system: No 
match to 1 SubjectAltNames
doveadm(inbox@sec-level.domain): Error: Disconnected from remote: SSL 
certificate doesn't match expected host name fqdn.of.system: No match to 1 
SubjectAltNames

And yes the SubjectAlternativeName in the certificate matches the configured 
FQDN in 30-dsync.conf.

My 30-dsync.conf: 

$ cat /etc/dovecot/conf.d/30-dsync.conf

service aggregator {
fifo_listener replication-notify-fifo {
user = dovecot
mode = 0666
}
unix_listener replication-notify {
user = dovecot
mode = 0666
}
}

# Configuring the replicator service
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
user = dovecot
mode = 0666
}
}
service doveadm {
user = dovecot
inet_listener {
port = 12345
ssl = yes
}
}

doveadm_port = 12345
doveadm_password = 
replication_max_conns = 1

plugin {
mail_replica = tcps:fqdn.of.system
}

service config {
unix_listener config {
user = dovecot
}
}


I'd love to here the answer to this. 

Thanks alot ! 

Best regards, 
Stephan

Re: Dovecot dsync 'ssl_client_ca'

[Thierry]

Bonjour Markus,

Things are working but without SSL.
I will have a look and come back  to you.

Thx

Le mercredi 8 février 2017 à 00:31:08, vous écriviez :

> Dear Thierry,

> (I'm omitting the remainder of your post because the below has a
> separate root cause from what has been assumed.)

>>[...]
>> This morning logs:
>>
>> Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in
> state_dir: ssl-parameters.dat - disabling SSL 360
>> Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters,
> disabling SSL
>>[...]

> Did I miss these lines before or did the messages change?
> In either case, have a look at
> http://wiki.dovecot.org/SSL/DovecotConfiguration#SSL_security_settings
> which explains how to fix this in detail--if you're lucky, your problems
> might be gone afterwards.

> KR, Markus



-- 
Cordialement,
 Thierrye-mail : lenai...@maelenn.org

Re: Dovecot dsync 'ssl_client_ca'

[Markus Ueberall]

Dear Thierry,

(I'm omitting the remainder of your post because the below has a
separate root cause from what has been assumed.)

>[...]
> This morning logs:
>
> Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in
state_dir: ssl-parameters.dat - disabling SSL 360
> Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters,
disabling SSL
>[...]

Did I miss these lines before or did the messages change?
In either case, have a look at
http://wiki.dovecot.org/SSL/DovecotConfiguration#SSL_security_settings
which explains how to fix this in detail--if you're lucky, your problems
might be gone afterwards.

KR, Markus

Re: Dovecot dsync 'ssl_client_ca'

[Thierry]

Bonjour Markus,

> - Have you checked that port 12345 as specified below is open/forwarded
> and actually /used/ by dovecot (e.g., use "netstat -tulpn|grep dovecot")?

Yes of course: 

tcp0  0 0.0.0.0:12345   0.0.0.0:*   LISTEN  
22025/dovecot
tcp6   0  0 :::12345:::*LISTEN  
22025/dovecot


> - Did you retrace your steps and have you verified that synchronisation
> works with ssl disabled?

This  dovecot  is  working  well  with  my email  client and web mail 
interface,  I would prefer not to start playing with this config file 
...

> - Did you verify your certificate files (e.g., "openssl verify -verbose
> -CAfile /etc/ssl/certs/GandiCA2.pem /etc/ssl/certs/key.crt")?

yes:  openssl  verify  -verbose  -CAfile  /etc/ssl/certs/GandiCA2.pem 
/etc/ssl/certs/key.crt
/etc/ssl/certs/key.crt: OK

> Personally, I prefer to use a single, specialised tool to manage
> certificates/encryption (which in my case is stunnel); all other
> programs are set up using (link-)local ip addresses only. If everything
> but encryption works with your setup, this might be a possible
> "workaround". (Apart from that, stunnel debug mode is very detailed and
> can help you to rule out problems with the certificates/connections
> between two nodes.)
> And once the latter works but the dovecot setup below still does not, it
> would also point to a problem with certificate handling by dovecot
> (could be library related).

This morning logs:

Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: 
ssl-parameters.dat - disabling SSL 360
Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling 
SSL
Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: 
ssl-parameters.dat - disabling SSL 360
Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling 
SSL
Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: 
ssl-parameters.dat - disabling SSL 360
Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling 
SSL
Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: 
ssl-parameters.dat - disabling SSL 360
Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling 
SSL
Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: 
ssl-parameters.dat - disabling SSL 360
Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling 
SSL




> KR, Markus

Thx
> Am 06.02.2017 um 07:36 schrieb Thierry:
>> Hi Aki,
>>
>> I do  not have any error message but (on both server):
>>
>> doveadm replicator status '*'
>> doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) 
>> failed: Connection refused
>>
>> Thx
>>
>>
>> Le vendredi 3 février 2017 à 17:09:52, vous écriviez :
>>
>>> Please keep responses in list. rm -f 
>>> /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir.
>>
>>> On 2017-02-03 17:00, Thierry wrote:
 Hi,

 I have removed the '<' :

 ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem

 But now:

 doveadm: Error: Corrupted SSL parameters file in state_dir: 
 ssl-parameters.dat - disabling SSL 360
 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
 doveadm: Error: Corrupted SSL parameters file in state_dir: 
 ssl-parameters.dat - disabling SSL 360
 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL

 Any idea ?

 Thx

> Yes. The ssl_client_ca_file is not actually expecting <, just file name.
> Aki
> On 2017-02-03 15:13, Thierry wrote:
>> Hi,
>>
>> I have made change:
>>
>> ssl_protocols = !SSLv2 !SSLv3
>> ssl = required
>> verbose_ssl = no
>> ssl_key = > ssl_cert = > ssl_client_ca_file = >
>>
>> # Create a listener for doveadm-server
>> service doveadm {
>> user = vmail
>> inet_listener {
>>   port = 12345
>>   ssl= yes
>> }
>> }
>>
>> and  doveadm_port = 12345// mail_replica = tcps:server2.domain.ltd # 
>> use doveadm_port
>>
>> And now:
>>
>> Feb 03 14:11:16 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>> initialize SSL context: Can't load CA certs from directory : 
>> error:02001024:system library:fopen:File name too long
>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in 
>> state_dir: ssl-parameters.dat - disabling SSL 360
>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, 
>> disabling SSL
>>
>> Thx for your support
>>
>>
>>
>>
>> Le vendredi 3 février 2017 à 11:34:43, vous écriviez :
>>
>>> Hello,
>>> On 02/03/2017 08:51 AM, Thierry wrote:
 Hello,

 Still working with my dsync pb.
 I have done a clone (vmware) of my email server.
 Today   

Re: Dovecot dsync 'ssl_client_ca'

[Markus Ueberall]

Dear Thierry,

- Have you checked that port 12345 as specified below is open/forwarded
and actually /used/ by dovecot (e.g., use "netstat -tulpn|grep dovecot")?
- Did you retrace your steps and have you verified that synchronisation
works with ssl disabled?
- Did you verify your certificate files (e.g., "openssl verify -verbose
-CAfile /etc/ssl/certs/GandiCA2.pem /etc/ssl/certs/key.crt")?

Personally, I prefer to use a single, specialised tool to manage
certificates/encryption (which in my case is stunnel); all other
programs are set up using (link-)local ip addresses only. If everything
but encryption works with your setup, this might be a possible
"workaround". (Apart from that, stunnel debug mode is very detailed and
can help you to rule out problems with the certificates/connections
between two nodes.)
And once the latter works but the dovecot setup below still does not, it
would also point to a problem with certificate handling by dovecot
(could be library related).

KR, Markus


Am 06.02.2017 um 07:36 schrieb Thierry:
> Hi Aki,
>
> I do  not have any error message but (on both server):
>
> doveadm replicator status '*'
> doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) 
> failed: Connection refused
>
> Thx
>
>
> Le vendredi 3 février 2017 à 17:09:52, vous écriviez :
>
>> Please keep responses in list. rm -f 
>> /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir.
>
>> On 2017-02-03 17:00, Thierry wrote:
>>> Hi,
>>>
>>> I have removed the '<' :
>>>
>>> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem
>>>
>>> But now:
>>>
>>> doveadm: Error: Corrupted SSL parameters file in state_dir: 
>>> ssl-parameters.dat - disabling SSL 360
>>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
>>> doveadm: Error: Corrupted SSL parameters file in state_dir: 
>>> ssl-parameters.dat - disabling SSL 360
>>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
>>>
>>> Any idea ?
>>>
>>> Thx
>>>
 Yes. The ssl_client_ca_file is not actually expecting <, just file name.
 Aki
 On 2017-02-03 15:13, Thierry wrote:
> Hi,
>
> I have made change:
>
> ssl_protocols = !SSLv2 !SSLv3
> ssl = required
> verbose_ssl = no
> ssl_key =  ssl_cert =  ssl_client_ca_file = 
>
> # Create a listener for doveadm-server
> service doveadm {
> user = vmail
> inet_listener {
>   port = 12345
>   ssl= yes
> }
> }
>
> and  doveadm_port = 12345// mail_replica = tcps:server2.domain.ltd # 
> use doveadm_port
>
> And now:
>
> Feb 03 14:11:16 doveadm(us...@domain.ltd): Error: sync: Couldn't 
> initialize SSL context: Can't load CA certs from directory : 
> error:02001024:system library:fopen:File name too long
> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in 
> state_dir: ssl-parameters.dat - disabling SSL 360
> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, 
> disabling SSL
>
> Thx for your support
>
>
>
>
> Le vendredi 3 février 2017 à 11:34:43, vous écriviez :
>
>> Hello,
>> On 02/03/2017 08:51 AM, Thierry wrote:
>>> Hello,
>>>
>>> Still working with my dsync pb.
>>> I have done a clone (vmware) of my email server.
>>> Today   I   have   two  strictly  identical  emails  servers (server1
>>> (main) and server2 (bck) (except IP, hostname and  mail_replica).
>>>
>>> The ssl config on my both server:
>>>
>>> ssl_protocols = !SSLv2 !SSLv3
>>> ssl = required
>>> verbose_ssl = no
>>> ssl_key = >> ssl_cert = >> ssl_ca = > I think it should be ssl_client_ca_file =
>> >> This  config  is  working   for  my   email  client  and my email web
>>> interface ...
>>>
>>> Are they on the right order ?
>>>
>>> mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd
>>>
>>> There is trafic on my iptables rules on my both  servers:
>>>
>>> 60  3600 ACCEPT tcp  --  *  *   0.0.0.0/0
>>> 0.0.0.0/0tcp dpt:4711
>>>
>>>
>>>
>>> My  error message from server1 (main server):
>>>
>>> Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>>> initialize SSL context: Can't verify remote server certs without 
>>> trusted CAs (ssl_client_ca_* settings)
>>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>>> initialize SSL context: Can't verify remote server certs without 
>>> trusted CAs (ssl_client_ca_* settings)
>>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>>> initialize SSL context: Can't verify remote server certs without 
>>> trusted CAs (ssl_client_ca_* settings)
>>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>>> initialize SSL context: Can't verify remote server certs 

Re: Dovecot dsync 'ssl_client_ca'

[Thierry]

Hi Aki,

I do  not have any error message but (on both server):

doveadm replicator status '*'
doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) 
failed: Connection refused

Thx


Le vendredi 3 février 2017 à 17:09:52, vous écriviez :

> Please keep responses in list. rm -f 
> /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir.


> On 2017-02-03 17:00, Thierry wrote:
>> Hi,
>>
>> I have removed the '<' :
>>
>> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem
>>
>> But now:
>>
>> doveadm: Error: Corrupted SSL parameters file in state_dir: 
>> ssl-parameters.dat - disabling SSL 360
>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
>> doveadm: Error: Corrupted SSL parameters file in state_dir: 
>> ssl-parameters.dat - disabling SSL 360
>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
>>
>> Any idea ?
>>
>> Thx
>>
>>> Yes. The ssl_client_ca_file is not actually expecting <, just file name.
>>> Aki
>>
>>> On 2017-02-03 15:13, Thierry wrote:
 Hi,

 I have made change:

 ssl_protocols = !SSLv2 !SSLv3
 ssl = required
 verbose_ssl = no
 ssl_key = >>> ssl_cert = >>> ssl_client_ca_file = >>>

 # Create a listener for doveadm-server
 service doveadm {
 user = vmail
 inet_listener {
   port = 12345
   ssl= yes
 }
 }

 and  doveadm_port = 12345// mail_replica = tcps:server2.domain.ltd # 
 use doveadm_port

 And now:

 Feb 03 14:11:16 doveadm(us...@domain.ltd): Error: sync: Couldn't 
 initialize SSL context: Can't load CA certs from directory : 
 error:02001024:system library:fopen:File name too long
 Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in 
 state_dir: ssl-parameters.dat - disabling SSL 360
 Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, 
 disabling SSL

 Thx for your support




 Le vendredi 3 février 2017 à 11:34:43, vous écriviez :

> Hello,
> On 02/03/2017 08:51 AM, Thierry wrote:
>> Hello,
>>
>> Still working with my dsync pb.
>> I have done a clone (vmware) of my email server.
>> Today   I   have   two  strictly  identical  emails  servers (server1
>> (main) and server2 (bck) (except IP, hostname and  mail_replica).
>>
>> The ssl config on my both server:
>>
>> ssl_protocols = !SSLv2 !SSLv3
>> ssl = required
>> verbose_ssl = no
>> ssl_key = > ssl_cert = > ssl_ca =  I think it should be ssl_client_ca_file =
> > This  config  is  working   for  my   email  client  and my email web
>> interface ...
>>
>> Are they on the right order ?
>>
>> mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd
>>
>> There is trafic on my iptables rules on my both  servers:
>>
>> 60  3600 ACCEPT tcp  --  *  *   0.0.0.0/0
>> 0.0.0.0/0tcp dpt:4711
>>
>>
>>
>> My  error message from server1 (main server):
>>
>> Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>> initialize SSL context: Can't verify remote server certs without trusted 
>> CAs (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>> initialize SSL context: Can't verify remote server certs without trusted 
>> CAs (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>> initialize SSL context: Can't verify remote server certs without trusted 
>> CAs (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't 
>> initialize SSL context: Can't verify remote server certs without trusted 
>> CAs (ssl_client_ca_* settings)
>>
>> No logs from server2
>>
>> Any ideas ?
>>
>> Thx for your support
>>
>>




-- 
Cordialement,
 Thierrye-mail : lenai...@maelenn.org

Re: Dovecot dsync 'ssl_client_ca'

[Aki Tuomi]

Please keep responses in list. rm -f 
/var/lib/dovecot/ssl-parameters.dat, i think it was in that dir.



On 2017-02-03 17:00, Thierry wrote:

Hi,

I have removed the '<' :

ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem

But now:

doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat 
- disabling SSL 360
doveadm: Error: Couldn't initialize SSL parameters, disabling SSL
doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat 
- disabling SSL 360
doveadm: Error: Couldn't initialize SSL parameters, disabling SSL

Any idea ?

Thx


Yes. The ssl_client_ca_file is not actually expecting <, just file name.
Aki



On 2017-02-03 15:13, Thierry wrote:

Hi,

I have made change:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key = 
Hello,
On 02/03/2017 08:51 AM, Thierry wrote:

Hello,

Still working with my dsync pb.
I have done a clone (vmware) of my email server.
Today   I   have   two  strictly  identical  emails  servers (server1
(main) and server2 (bck) (except IP, hostname and  mail_replica).

The ssl config on my both server:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key = 
I think it should be ssl_client_ca_file =

This  config  is  working   for  my   email  client  and my email web
interface ...

Are they on the right order ?

mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd

There is trafic on my iptables rules on my both  servers:

60  3600 ACCEPT tcp  --  *  *   0.0.0.0/00.0.0.0/0  
  tcp dpt:4711



My  error message from server1 (main server):

Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)

No logs from server2

Any ideas ?

Thx for your support

Re: Dovecot dsync 'ssl_client_ca'

[Aki Tuomi]

Yes. The ssl_client_ca_file is not actually expecting <, just file name.

Aki


On 2017-02-03 15:13, Thierry wrote:

Hi,

I have made change:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key = 
Hello,



On 02/03/2017 08:51 AM, Thierry wrote:

Hello,

Still working with my dsync pb.
I have done a clone (vmware) of my email server.
Today   I   have   two  strictly  identical  emails  servers (server1
(main) and server2 (bck) (except IP, hostname and  mail_replica).

The ssl config on my both server:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key = 
I think it should be ssl_client_ca_file =



This  config  is  working   for  my   email  client  and my email web
interface ...

Are they on the right order ?

mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd

There is trafic on my iptables rules on my both  servers:

60  3600 ACCEPT tcp  --  *  *   0.0.0.0/00.0.0.0/0  
  tcp dpt:4711



My  error message from server1 (main server):

Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)

No logs from server2

Any ideas ?

Thx for your support

Re: Dovecot dsync 'ssl_client_ca'

[Thierry]

Hi,

I have made change:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key =  Hello,


> On 02/03/2017 08:51 AM, Thierry wrote:
>> Hello,
>>
>> Still working with my dsync pb.
>> I have done a clone (vmware) of my email server.
>> Today   I   have   two  strictly  identical  emails  servers (server1
>> (main) and server2 (bck) (except IP, hostname and  mail_replica).
>>
>> The ssl config on my both server:
>>
>> ssl_protocols = !SSLv2 !SSLv3
>> ssl = required
>> verbose_ssl = no
>> ssl_key = > ssl_cert = > ssl_ca =  I think it should be ssl_client_ca_file = 
> >
>> This  config  is  working   for  my   email  client  and my email web
>> interface ...
>>
>> Are they on the right order ?
>>
>> mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd
>>
>> There is trafic on my iptables rules on my both  servers:
>>
>> 60  3600 ACCEPT tcp  --  *  *   0.0.0.0/00.0.0.0/0   
>>  tcp dpt:4711
>>
>>
>>
>> My  error message from server1 (main server):
>>
>> Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>>
>> No logs from server2
>>
>> Any ideas ?
>>
>> Thx for your support
>>
>>



-- 
Cordialement,
 Thierrye-mail : lenai...@maelenn.org

Re: Dovecot dsync 'ssl_client_ca'

[Thierry]

Bonjour Mike,

I  have made the change from 'ssl_ca =' tp 'ssl_client_ca_file =' but 
now I do have:

Error: sync: Couldn't initialize SSL context: Can't load CA certs from 
directory : error:02001024:system library:fopen:File name too long

thx



Le vendredi 3 février 2017 à 11:34:43, vous écriviez :

> Hello,


> On 02/03/2017 08:51 AM, Thierry wrote:
>> Hello,
>>
>> Still working with my dsync pb.
>> I have done a clone (vmware) of my email server.
>> Today   I   have   two  strictly  identical  emails  servers (server1
>> (main) and server2 (bck) (except IP, hostname and  mail_replica).
>>
>> The ssl config on my both server:
>>
>> ssl_protocols = !SSLv2 !SSLv3
>> ssl = required
>> verbose_ssl = no
>> ssl_key = > ssl_cert = > ssl_ca =  I think it should be ssl_client_ca_file = 
> >
>> This  config  is  working   for  my   email  client  and my email web
>> interface ...
>>
>> Are they on the right order ?
>>
>> mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd
>>
>> There is trafic on my iptables rules on my both  servers:
>>
>> 60  3600 ACCEPT tcp  --  *  *   0.0.0.0/00.0.0.0/0   
>>  tcp dpt:4711
>>
>>
>>
>> My  error message from server1 (main server):
>>
>> Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>> Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize 
>> SSL context: Can't verify remote server certs without trusted CAs 
>> (ssl_client_ca_* settings)
>>
>> No logs from server2
>>
>> Any ideas ?
>>
>> Thx for your support
>>
>>



-- 
Cordialement,
 Thierrye-mail : lenai...@maelenn.org

Re: Dovecot dsync 'ssl_client_ca'

[Mike Fröhner]

Hello,


On 02/03/2017 08:51 AM, Thierry wrote:

Hello,

Still working with my dsync pb.
I have done a clone (vmware) of my email server.
Today   I   have   two  strictly  identical  emails  servers (server1
(main) and server2 (bck) (except IP, hostname and  mail_replica).

The ssl config on my both server:

ssl_protocols = !SSLv2 !SSLv3
ssl = required
verbose_ssl = no
ssl_key = 

I think it should be ssl_client_ca_file = 





This  config  is  working   for  my   email  client  and my email web
interface ...

Are they on the right order ?

mail_replica = tcps:serv...@domain.ltd and tcps:serv...@domain.ltd

There is trafic on my iptables rules on my both  servers:

60  3600 ACCEPT tcp  --  *  *   0.0.0.0/00.0.0.0/0  
  tcp dpt:4711



My  error message from server1 (main server):

Feb 03 08:38:08 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)
Feb 03 08:42:35 doveadm(us...@domain.ltd): Error: sync: Couldn't initialize SSL 
context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* 
settings)

No logs from server2

Any ideas ?

Thx for your support

Re: Dovecot dsync tcps sends incomplete certificate chain

[Juri]

In data venerdì 6 gennaio 2017 01:34:48 CET, John Fawcett ha scritto:
> On 01/05/2017 08:55 PM, Juri wrote:
> > 5 Gennaio 2017 01:21, "John Fawcett"  wrote:
> >> On 01/04/2017 08:40 PM, Juri wrote:

> Hi Juri
> 
> if you find validation failing when you have only the root certificate
> in the CA file but a chained server+intermediate in the server
> certificate file, then your analysis makes sense and it seems that the
> intermediate certificate is not being sent by the server. That ties in
> with the different error messages between imap and replication.
> 
> It might be interesting to do a test with -showcerts parameter.
> 
> |openssl s_client -showcerts -connect hostname:|7557
> |
> |openssl s_client -showcerts -connect hostname:993 The bundled version of
> 
> Dovecot on Centos 7 is 2.2.10 but I am not using that version. I am on
> 2.2.26, where I don't have the problem you see and both services send
> the server and intermediate certificate. I was unable to see any
> specific patches to the ssl or doveadm code for this issue, though it
> has undergone a few changes from 2.2.13. John |

I tried what you suggested, and the result is more or less the same as what I 
wrote in the first message (only the last cert sent on port 7557, and both - 
the last and the intermediate one - on port 993).

I tried to recompile the same version (2.2.13) on my Arch Linux home PC, and 
using the same settings and  same certs as on the server, all the certificates 
are correctly being sent on both ports, so I suppose the bug lies in the 
debian patches - I'll try to report to them.

In the meantime, thank you all for the help!

Juri

Re: Dovecot dsync tcps sends incomplete certificate chain

[Aki Tuomi]

> On January 6, 2017 at 2:34 AM John Fawcett  wrote:
> 
> 
> On 01/05/2017 08:55 PM, Juri wrote:
> > 5 Gennaio 2017 01:21, "John Fawcett"  wrote:
> >
> >> On 01/04/2017 08:40 PM, Juri wrote:
> >>
> >>
> > Thank you.
> >
> > In fact I tried both settings, that is
> > |ssl_client_ca_dir = /etc/ssl/certs
> > |ssl_client_ca_file = /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> > but with no luck.
> > Actually, I noticed that with the two settings I get a slightly different 
> > error message (it took me
> > quite a bit to notice it!), that is:
> > |Error: sync: Disconnected from remote: Received invalid SSL certificate: 
> > unable to get issuer
> > certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
> > |Error: sync: Disconnected from remote: Received invalid SSL certificate: 
> > unable to get *local*
> > issuer certificate: /CN=mail.dividebyzero.it
> > (emphasis mine).
> > I suppose that in the first case - as the server is sending only the last 
> > certificate on the chain
> > - the client is unable to find the intermediate, while in the second case 
> > it won't find the root
> > one.
> >
> > I then tried, as you suggested me, to concatenate both the intermediate and 
> > the root certificate in
> > a single file, and it finally worked.
> > In any case the original point still stands: in the sync mode - at least on 
> > my version (2.2.13) -
> > the server sends only the last cert, so the client has to have the rest of 
> > the chain, instead of
> > needing to have only the root certificate.
> >
> > May I ask you which is the version of Dovecot bundled with CentOS, to know 
> > if this may be a bug
> > fixed in a newer version?
> >
> > Juri
> 
> Hi Juri
> 
> if you find validation failing when you have only the root certificate
> in the CA file but a chained server+intermediate in the server
> certificate file, then your analysis makes sense and it seems that the
> intermediate certificate is not being sent by the server. That ties in
> with the different error messages between imap and replication. 
> 
> It might be interesting to do a test with -showcerts parameter.
> 
> |openssl s_client -showcerts -connect hostname:|7557
> 
> |openssl s_client -showcerts -connect hostname:993 The bundled version of
> Dovecot on Centos 7 is 2.2.10 but I am not using that version. I am on
> 2.2.26, where I don't have the problem you see and both services send
> the server and intermediate certificate. I was unable to see any
> specific patches to the ssl or doveadm code for this issue, though it
> has undergone a few changes from 2.2.13. John |

You might want to return from passdb following things, if I understood your 
scenario correctly.

proxy=y host=your-backend-host ssl=any-cert port=993

https://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy

Aki

Re: Dovecot dsync tcps sends incomplete certificate chain

[John Fawcett]

On 01/05/2017 08:55 PM, Juri wrote:
> 5 Gennaio 2017 01:21, "John Fawcett"  wrote:
>
>> On 01/04/2017 08:40 PM, Juri wrote:
>>
>>
> Thank you.
>
> In fact I tried both settings, that is
> |ssl_client_ca_dir = /etc/ssl/certs
> |ssl_client_ca_file = /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> but with no luck.
> Actually, I noticed that with the two settings I get a slightly different 
> error message (it took me
> quite a bit to notice it!), that is:
> |Error: sync: Disconnected from remote: Received invalid SSL certificate: 
> unable to get issuer
> certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
> |Error: sync: Disconnected from remote: Received invalid SSL certificate: 
> unable to get *local*
> issuer certificate: /CN=mail.dividebyzero.it
> (emphasis mine).
> I suppose that in the first case - as the server is sending only the last 
> certificate on the chain
> - the client is unable to find the intermediate, while in the second case it 
> won't find the root
> one.
>
> I then tried, as you suggested me, to concatenate both the intermediate and 
> the root certificate in
> a single file, and it finally worked.
> In any case the original point still stands: in the sync mode - at least on 
> my version (2.2.13) -
> the server sends only the last cert, so the client has to have the rest of 
> the chain, instead of
> needing to have only the root certificate.
>
> May I ask you which is the version of Dovecot bundled with CentOS, to know if 
> this may be a bug
> fixed in a newer version?
>
> Juri

Hi Juri

if you find validation failing when you have only the root certificate
in the CA file but a chained server+intermediate in the server
certificate file, then your analysis makes sense and it seems that the
intermediate certificate is not being sent by the server. That ties in
with the different error messages between imap and replication. 

It might be interesting to do a test with -showcerts parameter.

|openssl s_client -showcerts -connect hostname:|7557

|openssl s_client -showcerts -connect hostname:993 The bundled version of
Dovecot on Centos 7 is 2.2.10 but I am not using that version. I am on
2.2.26, where I don't have the problem you see and both services send
the server and intermediate certificate. I was unable to see any
specific patches to the ssl or doveadm code for this issue, though it
has undergone a few changes from 2.2.13. John |

Re: Dovecot dsync tcps sends incomplete certificate chain

[Juri]

5 Gennaio 2017 01:21, "John Fawcett" <j...@voipsupport.it> wrote:

> On 01/04/2017 08:40 PM, Juri wrote:
> 
>> Hi,
>> I'm trying to configure a Dovecot dsync service between two servers, using a 
>> tcp+ssl connection and
>> a valid Let's Encrypt certificate.
>> I followed the guide on the wiki (http://wiki.dovecot.org/Replication) using 
>> the tcps method, but
>> when I launch the replication it fails writing on the log 
>> (/var/log/mail.err):
>> (Server 1 - sync "client" )| Error: sync: Disconnected from remote: Received 
>> invalid SSL
>> certificate: unable to get local issuer certificate: /CN=mail.dividebyzero.it
>> (Server 2 - sync "server")| Error: doveadm client disconnected before 
>> handshake: 
>> 
>> If I try to connect to the server using openssl s_client, on the port 993 
>> (imaps) the server
>> correctly sends the full chain:
>> $ openssl s_client -connect server1.fqdn:993
>> CONNECTED(0003)
>> depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
>> verify return:1
>> depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
>> verify return:1
>> depth=0 CN = mail.dividebyzero.it
>> verify return:1
>> ---
>> Certificate chain
>> 0 s:/CN=mail.dividebyzero.it
>> i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
>> 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
>> i:/O=Digital Signature Trust Co./CN=DST Root CA X3
>> ...
>> 
>> while on the doveadm port it fails:
>> $ openssl s_client -connect server1.fqdn:7557
>> CONNECTED(0003)
>> depth=0 CN = mail.dividebyzero.it
>> verify error:num=20:unable to get local issuer certificate
>> verify return:1
>> depth=0 CN = mail.dividebyzero.it
>> verify error:num=21:unable to verify the first certificate
>> verify return:1
>> ---
>> Certificate chain
>> 0 s:/CN=mail.dividebyzero.it
>> i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
>> ...
>> 
>> I run Dovecot 2.2.13 on Debian 8.6:
>> $ dovecot -n
>> # 2.2.13: /etc/dovecot/dovecot.conf
>> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6
>> auth_default_realm = dividebyzero.it
>> auth_mechanisms = plain login
>> doveadm_password = (redacted)
>> doveadm_port = 7557
>> mail_location = maildir:~/Maildir
>> mail_plugins = " notify replication"
>> namespace inbox { (removed) }
>> passdb {
>> driver = pam
>> }
>> passdb {
>> args = username_format=%n /etc/vmail/%d/passwd
>> driver = passwd-file
>> }
>> plugin {
>> mail_replica = tcps:otherserver.fqdn
>> }
>> protocols = " imap lmtp"
>> service aggregator {
>> fifo_listener replication-notify-fifo {
>> user = dovecot
>> }
>> unix_listener replication-notify {
>> user = dovecot
>> }
>> }
>> service auth {
>> unix_listener auth-client {
>> group = Debian-exim
>> mode = 0660
>> }
>> unix_listener auth-userdb {
>> user = vmail
>> }
>> }
>> service doveadm {
>> inet_listener {
>> port = 7557
>> ssl = yes
>> }
>> }
>> service imap-login {
>> inet_listener imap {
>> port = 143
>> }
>> inet_listener imaps {
>> port = 993
>> ssl = yes
>> }
>> }
>> service replicator {
>> process_min_avail = 1
>> unix_listener replicator-doveadm {
>> mode = 0666
>> }
>> }
>> ssl = required
>> ssl_cert = > ssl_client_ca_file = /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
>> ssl_key = > userdb {
>> driver = passwd
>> }
>> userdb {
>> args = uid=vmail gid=vmail home=/var/local/vmail/%d/%n
>> driver = static
>> }
>> 
>> Is it a known problem, or has it been resolved in a subsequent version?
>> If it is not, can you suggest me a workaround in the meantime?
>> Thank you.
> 
> I would do those test using the -CAfile parameter to be sure of the
> local certificate file being used:
> 
> openssl s_client -connect server1.fqdn:993 -CAfile
> /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> openssl s_client -connect server1.fqdn:7557 -CAfile
> /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> 
> You should also be able to see the problem using the verify command directly 
> (on the cert copied
> from the remote server)
> openssl verify -CAfile /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> fullchain_copied_from_remote_server.pem
> 
> This error happens when the local CA file or directory that is specified
&

Re: Dovecot dsync tcps sends incomplete certificate chain

[John Fawcett]

On 01/04/2017 08:40 PM, Juri wrote:
> Hi,
> I'm trying to configure a Dovecot dsync service between two servers, using a 
> tcp+ssl connection and
> a valid Let's Encrypt certificate.
> I followed the guide on the wiki (http://wiki.dovecot.org/Replication) using 
> the tcps method, but
> when I launch the replication it fails writing on the log (/var/log/mail.err):
> (Server 1 - sync "client" )| Error: sync: Disconnected from remote: Received 
> invalid SSL
> certificate: unable to get local issuer certificate: /CN=mail.dividebyzero.it
> (Server 2 - sync "server")| Error: doveadm client disconnected before 
> handshake: 
>
> If I try to connect to the server using openssl s_client, on the port 993 
> (imaps) the server
> correctly sends the full chain:
> $ openssl s_client -connect server1.fqdn:993
> CONNECTED(0003)
> depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
> verify return:1
> depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
> verify return:1
> depth=0 CN = mail.dividebyzero.it
> verify return:1
> ---
> Certificate chain
> 0 s:/CN=mail.dividebyzero.it
> i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
> 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
> i:/O=Digital Signature Trust Co./CN=DST Root CA X3
> ...
>
> while on the doveadm port it fails:
> $ openssl s_client -connect server1.fqdn:7557
> CONNECTED(0003)
> depth=0 CN = mail.dividebyzero.it
> verify error:num=20:unable to get local issuer certificate
> verify return:1
> depth=0 CN = mail.dividebyzero.it
> verify error:num=21:unable to verify the first certificate
> verify return:1
> ---
> Certificate chain
> 0 s:/CN=mail.dividebyzero.it
> i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
> ...
>
> I run Dovecot 2.2.13 on Debian 8.6:
> $ dovecot -n
> # 2.2.13: /etc/dovecot/dovecot.conf
> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6
> auth_default_realm = dividebyzero.it
> auth_mechanisms = plain login
> doveadm_password = (redacted)
> doveadm_port = 7557
> mail_location = maildir:~/Maildir
> mail_plugins = " notify replication"
> namespace inbox { (removed) }
> passdb {
> driver = pam
> }
> passdb {
> args = username_format=%n /etc/vmail/%d/passwd
> driver = passwd-file
> }
> plugin {
> mail_replica = tcps:otherserver.fqdn
> }
> protocols = " imap lmtp"
> service aggregator {
> fifo_listener replication-notify-fifo {
> user = dovecot
> }
> unix_listener replication-notify {
> user = dovecot
> }
> }
> service auth {
> unix_listener auth-client {
> group = Debian-exim
> mode = 0660
> }
> unix_listener auth-userdb {
> user = vmail
> }
> }
> service doveadm {
> inet_listener {
> port = 7557
> ssl = yes
> }
> }
> service imap-login {
> inet_listener imap {
> port = 143
> }
> inet_listener imaps {
> port = 993
> ssl = yes
> }
> }
> service replicator {
> process_min_avail = 1
> unix_listener replicator-doveadm {
> mode = 0666
> }
> }
> ssl = required
> ssl_cert =  ssl_client_ca_file = /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
> ssl_key =  userdb {
> driver = passwd
> }
> userdb {
> args = uid=vmail gid=vmail home=/var/local/vmail/%d/%n
> driver = static
> }
>
> Is it a known problem, or has it been resolved in a subsequent version?
> If it is not, can you suggest me a workaround in the meantime?
> Thank you.
I would do those test using the -CAfile parameter to be sure of the
local certificate file being used:

openssl s_client -connect server1.fqdn:993 -CAfile 
/etc/letsencrypt/live/mail.dividebyzero.it/chain.pem
openssl s_client -connect server1.fqdn:7557 -CAfile 
/etc/letsencrypt/live/mail.dividebyzero.it/chain.pem

You should also be able to see the problem using the verify command directly 
(on the cert copied from the remote server) 
openssl verify -CAfile /etc/letsencrypt/live/mail.dividebyzero.it/chain.pem 
fullchain_copied_from_remote_server.pem

This error happens when the local CA file or directory that is specified
does not contain the root certificate or the root certificate and
intermediate ones in the case that the intermediates are not supplied by
the server. My understanding is that Dovecot supplies the intermediate
certificates both for replication and imap services if they are in the
server certificate file. So you should be able to solve this by making
the root certificate available to Dovecot (parameter
ssl_client_ca_file). In the worst case you can concatenate the
intermediate and root certificates.

The certificate you are likely missing is the root certificate:

/O=Digital Signature Trust Co./CN=DST Root CA X3

You can follow the link on this page for it: 
https://le

Dovecot dsync tcps sends incomplete certificate chain

[Juri]

Hi,
I'm trying to configure a Dovecot dsync service between two servers, using a 
tcp+ssl connection and
a valid Let's Encrypt certificate.
I followed the guide on the wiki (http://wiki.dovecot.org/Replication) using 
the tcps method, but
when I launch the replication it fails writing on the log (/var/log/mail.err):
(Server 1 - sync "client" )| Error: sync: Disconnected from remote: Received 
invalid SSL
certificate: unable to get local issuer certificate: /CN=mail.dividebyzero.it
(Server 2 - sync "server")| Error: doveadm client disconnected before 
handshake: 

If I try to connect to the server using openssl s_client, on the port 993 
(imaps) the server
correctly sends the full chain:
$ openssl s_client -connect server1.fqdn:993
CONNECTED(0003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = mail.dividebyzero.it
verify return:1
---
Certificate chain
0 s:/CN=mail.dividebyzero.it
i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
i:/O=Digital Signature Trust Co./CN=DST Root CA X3
...

while on the doveadm port it fails:
$ openssl s_client -connect server1.fqdn:7557
CONNECTED(0003)
depth=0 CN = mail.dividebyzero.it
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = mail.dividebyzero.it
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:/CN=mail.dividebyzero.it
i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
...

I run Dovecot 2.2.13 on Debian 8.6:
$ dovecot -n
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6
auth_default_realm = dividebyzero.it
auth_mechanisms = plain login
doveadm_password = (redacted)
doveadm_port = 7557
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
namespace inbox { (removed) }
passdb {
driver = pam
}
passdb {
args = username_format=%n /etc/vmail/%d/passwd
driver = passwd-file
}
plugin {
mail_replica = tcps:otherserver.fqdn
}
protocols = " imap lmtp"
service aggregator {
fifo_listener replication-notify-fifo {
user = dovecot
}
unix_listener replication-notify {
user = dovecot
}
}
service auth {
unix_listener auth-client {
group = Debian-exim
mode = 0660
}
unix_listener auth-userdb {
user = vmail
}
}
service doveadm {
inet_listener {
port = 7557
ssl = yes
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0666
}
}
ssl = required
ssl_cert = 

Re: [Dovecot] dsync replication errors

[Timo Sirainen]

On 08 Sep 2015, at 11:20, Sergey Schwartz  
wrote:
> 
> I use mdbox and probably have similar issue, but in my case only shared 
> mailboxes were affected.

Yes, shared mailboxes don't work nicely with replication. Replication is 
locking only the original user, so for shared mailboxes multiple dsyncs can be 
running in parallel and messing things up. A bit troublesome to fix this. I've 
had this issue happening for a couple of years now for our mails and I haven't 
bothered fixing it, so it's unlikely I'll do it anytime soon.. Although I 
haven't seen that many duplicates of the mails - just 10 or so.

Re: [Dovecot] dsync replication errors

[Gedalya]

On 02/17/2013 03:21 AM, Timo Sirainen wrote:

Although there's still some mail
duplication problem with maildir that doesn't log any errors about it.
I'm not sure why that happens.


While you're around, Timo :-)

I've had such an issue recently with 2.2.18, using Maildir, where emails 
were being replicated circularly creating more and more duplicate copies.
Replication should have been unidirectional in reality since changes 
were being made on one side only.
Nothing coherent was being logged. Only "Warning: Maildir 
/srv/mail/domains/.../Maildir: Expunged message reappeared, giving a new 
UID .. " appearing on the receiving side.
Is there any intelligence on the matter, or should I isolate this down 
and report it from scratch?

Re: [Dovecot] dsync replication errors

[Timo Sirainen]

On 08 Sep 2015, at 01:16, Gedalya  wrote:
> 
> On 02/17/2013 03:21 AM, Timo Sirainen wrote:
>> Although there's still some mail
>> duplication problem with maildir that doesn't log any errors about it.
>> I'm not sure why that happens.
> 
> While you're around, Timo :-)
> 
> I've had such an issue recently with 2.2.18, using Maildir, where emails were 
> being replicated circularly creating more and more duplicate copies.
> Replication should have been unidirectional in reality since changes were 
> being made on one side only.
> Nothing coherent was being logged. Only "Warning: Maildir 
> /srv/mail/domains/.../Maildir: Expunged message reappeared, giving a new UID 
> .. " appearing on the receiving side.
> Is there any intelligence on the matter, or should I isolate this down and 
> report it from scratch?

dsync bugs usually take a lot of time to debug. Unless there's an easily 
reproducible way to break it, I try to avoid spending time on it. Also in this 
case the bug might be in Maildir code instead of dsync code.

Re: Dovecot dsync not replicating .dovecot.sieve - .sieve/managesieve.sieve / setactive

[Martin Štefany]

Hello Claus,

I've installed dovecot-2.2.15-3.fc20.x86_64.rpm + 
dovecot-pigeonhole-2.2.15-3.fc20.x86_64.rpm from Fedora guys and it 
works like a charm.


Thank you!

Martin


Dňa 12.2.2015 18:20 Claus napísal(a):

Am 12.02.2015 um 15:47 schrieb Martin Štefany:

Hello,

I've ran into problem with Dovecot and dsync replication. Everything 
works perfectly, including replication of sieve scripts, except fact 
that if user activates the 'managesieve' ruleset (I'm using currently 
Roundcubemail) on mail1 host, it wouldn't be activated on mail2 
host, by creating symlink .dovecot.sieve - 
.sieve/managesieve.sieve. I've also tried to use 
'replication_full_sync_interval', but symlink is not created anyway.


I found 2 references already for this problem, but none came to any 
conclusion:


http://dovecot.org/pipermail/dovecot/2014-June/096650.html
http://www.dovecot.org/list/dovecot/2014-September/097857.html


Here is the output from 'doveconf -n' from both hosts for reference ::

mail1 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com
auth_gssapi_hostname = mail.example.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain gssapi
auth_realms = example.com
auth_verbose = yes
doveadm_password = secret
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins =  fts fts_lucene notify quota replication virtual zlib
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave

namespace inbox {
  inbox = yes
  location =
  mailbox All {
auto = create
special_use = \All
  }
  mailbox Archives {
auto = subscribe
special_use = \Archive
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Templates {
auto = subscribe
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-ldap-passdb.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
  mail_replica = tcps:mail2.example.com:10993
  quota = maildir:User quota
  quota_rule = *:storage=4GB
  quota_rule2 = Trash:storage=+50MB
  sieve = ~/.dovecot.sieve
  sieve_after = /srv/sieve/after.d/
  sieve_before = /srv/sieve/before.d/
  sieve_default = /srv/sieve/default.d/dovecot.sieve
  sieve_dir = ~/.sieve
  sieve_global_dir = /srv/sieve/
  zlib_save = gz
  zlib_save_level = 9
}
postmaster_address = postmas...@example.com
protocols = imap lmtp sieve
service aggregator {
  fifo_listener replication-notify-fifo {
group = vmail
mode = 0660
user = vmail
  }
  unix_listener replication-notify {
group = vmail
mode = 0660
user = vmail
  }
}
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
  }
}
service doveadm {
  inet_listener {
port = 10993
ssl = yes
  }
}
service imap-login {
  inet_listener imaps {
port = 0
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 1
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
group = vmail
mode = 0660
user = vmail
  }
}
ssl_ca = /etc/ipa/ca.crt
ssl_cert = /etc/pki/tls/certs/dovecot.pem
ssl_cipher_list = 
EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA

ssl_client_ca_file = /etc/ipa/ca.crt
ssl_client_cert = /etc/pki/tls/certs/dovecot.pem
ssl_client_key = /etc/pki/tls/private/dovecot.key
ssl_key = /etc/pki/tls/private/dovecot.key
ssl_parameters_regenerate = 1 weeks
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = /etc/dovecot/dovecot-ldap-userdb.conf.ext
  driver = ldap
  override_fields = gid=vmail home=/srv/vmail/example.com/%n
}
verbose_ssl = yes
protocol lmtp {
  mail_plugins =  fts fts_lucene notify quota replication virtual 
zlib sieve

}
protocol imap {
  mail_plugins =  fts fts_lucene notify quota replication virtual 
zlib imap_quota imap_zlib

}


mail2 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com

Dovecot dsync not replicating .dovecot.sieve - .sieve/managesieve.sieve / setactive

[Martin Štefany]

Hello,

I've ran into problem with Dovecot and dsync replication. Everything 
works perfectly, including replication of sieve scripts, except fact 
that if user activates the 'managesieve' ruleset (I'm using currently 
Roundcubemail) on mail1 host, it wouldn't be activated on mail2 
host, by creating symlink .dovecot.sieve - .sieve/managesieve.sieve. 
I've also tried to use 'replication_full_sync_interval', but symlink is 
not created anyway.


I found 2 references already for this problem, but none came to any 
conclusion:


http://dovecot.org/pipermail/dovecot/2014-June/096650.html
http://www.dovecot.org/list/dovecot/2014-September/097857.html


Here is the output from 'doveconf -n' from both hosts for reference ::

mail1 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com
auth_gssapi_hostname = mail.example.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain gssapi
auth_realms = example.com
auth_verbose = yes
doveadm_password = secret
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins =  fts fts_lucene notify quota replication virtual zlib
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave

namespace inbox {
  inbox = yes
  location =
  mailbox All {
auto = create
special_use = \All
  }
  mailbox Archives {
auto = subscribe
special_use = \Archive
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Templates {
auto = subscribe
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-ldap-passdb.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
  mail_replica = tcps:mail2.example.com:10993
  quota = maildir:User quota
  quota_rule = *:storage=4GB
  quota_rule2 = Trash:storage=+50MB
  sieve = ~/.dovecot.sieve
  sieve_after = /srv/sieve/after.d/
  sieve_before = /srv/sieve/before.d/
  sieve_default = /srv/sieve/default.d/dovecot.sieve
  sieve_dir = ~/.sieve
  sieve_global_dir = /srv/sieve/
  zlib_save = gz
  zlib_save_level = 9
}
postmaster_address = postmas...@example.com
protocols = imap lmtp sieve
service aggregator {
  fifo_listener replication-notify-fifo {
group = vmail
mode = 0660
user = vmail
  }
  unix_listener replication-notify {
group = vmail
mode = 0660
user = vmail
  }
}
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
  }
}
service doveadm {
  inet_listener {
port = 10993
ssl = yes
  }
}
service imap-login {
  inet_listener imaps {
port = 0
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 1
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
group = vmail
mode = 0660
user = vmail
  }
}
ssl_ca = /etc/ipa/ca.crt
ssl_cert = /etc/pki/tls/certs/dovecot.pem
ssl_cipher_list = 
EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA

ssl_client_ca_file = /etc/ipa/ca.crt
ssl_client_cert = /etc/pki/tls/certs/dovecot.pem
ssl_client_key = /etc/pki/tls/private/dovecot.key
ssl_key = /etc/pki/tls/private/dovecot.key
ssl_parameters_regenerate = 1 weeks
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = /etc/dovecot/dovecot-ldap-userdb.conf.ext
  driver = ldap
  override_fields = gid=vmail home=/srv/vmail/example.com/%n
}
verbose_ssl = yes
protocol lmtp {
  mail_plugins =  fts fts_lucene notify quota replication virtual zlib 
sieve

}
protocol imap {
  mail_plugins =  fts fts_lucene notify quota replication virtual zlib 
imap_quota imap_zlib

}


mail2 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com
auth_gssapi_hostname = mail.example.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain gssapi
auth_realms = example.com
auth_verbose = yes
doveadm_password = secret
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_location = 

Re: Dovecot dsync not replicating .dovecot.sieve - .sieve/managesieve.sieve / setactive

[Claus]

Am 12.02.2015 um 15:47 schrieb Martin Štefany:

Hello,

I've ran into problem with Dovecot and dsync replication. Everything 
works perfectly, including replication of sieve scripts, except fact 
that if user activates the 'managesieve' ruleset (I'm using currently 
Roundcubemail) on mail1 host, it wouldn't be activated on mail2 
host, by creating symlink .dovecot.sieve - 
.sieve/managesieve.sieve. I've also tried to use 
'replication_full_sync_interval', but symlink is not created anyway.


I found 2 references already for this problem, but none came to any 
conclusion:


http://dovecot.org/pipermail/dovecot/2014-June/096650.html
http://www.dovecot.org/list/dovecot/2014-September/097857.html


Here is the output from 'doveconf -n' from both hosts for reference ::

mail1 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com
auth_gssapi_hostname = mail.example.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain gssapi
auth_realms = example.com
auth_verbose = yes
doveadm_password = secret
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins =  fts fts_lucene notify quota replication virtual zlib
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave

namespace inbox {
  inbox = yes
  location =
  mailbox All {
auto = create
special_use = \All
  }
  mailbox Archives {
auto = subscribe
special_use = \Archive
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Templates {
auto = subscribe
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-ldap-passdb.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
  mail_replica = tcps:mail2.example.com:10993
  quota = maildir:User quota
  quota_rule = *:storage=4GB
  quota_rule2 = Trash:storage=+50MB
  sieve = ~/.dovecot.sieve
  sieve_after = /srv/sieve/after.d/
  sieve_before = /srv/sieve/before.d/
  sieve_default = /srv/sieve/default.d/dovecot.sieve
  sieve_dir = ~/.sieve
  sieve_global_dir = /srv/sieve/
  zlib_save = gz
  zlib_save_level = 9
}
postmaster_address = postmas...@example.com
protocols = imap lmtp sieve
service aggregator {
  fifo_listener replication-notify-fifo {
group = vmail
mode = 0660
user = vmail
  }
  unix_listener replication-notify {
group = vmail
mode = 0660
user = vmail
  }
}
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
  }
}
service doveadm {
  inet_listener {
port = 10993
ssl = yes
  }
}
service imap-login {
  inet_listener imaps {
port = 0
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 1
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
group = vmail
mode = 0660
user = vmail
  }
}
ssl_ca = /etc/ipa/ca.crt
ssl_cert = /etc/pki/tls/certs/dovecot.pem
ssl_cipher_list = 
EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA

ssl_client_ca_file = /etc/ipa/ca.crt
ssl_client_cert = /etc/pki/tls/certs/dovecot.pem
ssl_client_key = /etc/pki/tls/private/dovecot.key
ssl_key = /etc/pki/tls/private/dovecot.key
ssl_parameters_regenerate = 1 weeks
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = /etc/dovecot/dovecot-ldap-userdb.conf.ext
  driver = ldap
  override_fields = gid=vmail home=/srv/vmail/example.com/%n
}
verbose_ssl = yes
protocol lmtp {
  mail_plugins =  fts fts_lucene notify quota replication virtual 
zlib sieve

}
protocol imap {
  mail_plugins =  fts fts_lucene notify quota replication virtual 
zlib imap_quota imap_zlib

}


mail2 ::
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-123.20.1.el7.x86_64 x86_64 CentOS Linux release 
7.0.1406 (Core)

auth_cache_size = 5 M
auth_debug = yes
auth_default_realm = example.com
auth_gssapi_hostname = mail.example.com
auth_krb5_keytab = /etc/dovecot/dovecot.keytab
auth_mechanisms = plain gssapi
auth_realms = example.com
auth_verbose = yes
doveadm_password = secret
lmtp_save_to_detail_mailbox = 

[Dovecot] dsync changing source permission to root in backup mode

[Peter Mogensen]

Hi,

We have dsync failing once in a while when running in backup mode.
What's strange is that the result is that the file permissions on the 
*source* machine ends up with the wrong permissions (set to uid 0).


Even though the dsync manual clearly says:
Backup mails from default mail location to location2 (or vice versa, if 
-R parameter is given). No changes are ever done to the source location. 
Any changes done in destination are discarded.


Running: 'dsync -R -o mail_home=/users/maildir backup ssh -c arcfour 
src-host dsync -o mail_home=/users/maildir'


I know it's running as root, but even then ... it shouldn't modify the 
source in backup mode ??


The error message from dsync when failing is:

dsync-remote(root): Error: Cached message size larger than expected 
(5292  5289)
dsync-remote(root): Error: Maildir filename has wrong S value, renamed 
the file from 
/users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S 
to /users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5289:2,S
dsync-remote(root): Error: Corrupted index cache file 
/users/maildir/.Sent/dovecot.index.cache: Broken physical size for mail 
UID 1040
dsync-remote(root): Error: dsync(dst-host): 
read(/users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S) 
failed: Cached message size larger than expected (5292  5289)



/Peter

Re: [Dovecot] dsync changing source permission to root in backup mode

[Peter Mogensen]

Oh ... sorry... I forgot the last log-line. (see below)

btw... tested with versions:
Between 2.2.12 in both ends, and
between dst=2.2.12, src=2.2.13


On 2014-05-27 15:03, Peter Mogensen wrote:

The error message from dsync when failing is:

dsync-remote(root): Error: Cached message size larger than expected
(5292  5289)
dsync-remote(root): Error: Maildir filename has wrong S value, renamed
the file from
/users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S
to /users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5289:2,S
dsync-remote(root): Error: Corrupted index cache file
/users/maildir/.Sent/dovecot.index.cache: Broken physical size for mail
UID 1040
dsync-remote(root): Error: dsync(dst-host):
read(/users/maildir/.Sent/cur/1381224782.M959810P3574.mail,S=5292,W=5411:2,S)
failed: Cached message size larger than expected (5292  5289)


dsync-local(root): Error: dsync(src-host): read() failed: read((fd)) 
failed: dot-input stream ends without '.' line

[Dovecot] dsync incredibly slow

[Alan McGinlay]

Hi!

While performing a dsync from cyrus imap to dovecot 2.2.12, dsync seems 
to stop for perhaps a minute without disk / cpu / memory activity (that 
I can see). This happens several times per sync per user so it takes an 
enormous amount of time to sync just a couple of gigs of mail.


dsync -D -v -o mail_fsync=never mirror -f -R -u u...@example.com imapc:

What could possibly be the cause of this slowness? Is it protocol 
dependent or just dsync it's self?


Thanks!

Re: [Dovecot] dsync incredibly slow

[Marcin Mirosław]

W dniu 23.05.2014 10:48, Alan McGinlay pisze:
 Hi!
 
 While performing a dsync from cyrus imap to dovecot 2.2.12, dsync seems
 to stop for perhaps a minute without disk / cpu / memory activity (that
 I can see). This happens several times per sync per user so it takes an
 enormous amount of time to sync just a couple of gigs of mail.
 
 dsync -D -v -o mail_fsync=never mirror -f -R -u u...@example.com imapc:
 
 What could possibly be the cause of this slowness? Is it protocol
 dependent or just dsync it's self?
Hi!
Use the strace, Luke;)
Use `strace -f -tt -T -s 512 dsync other parameters and look what
dsync does (when does nothing;))

[Dovecot] dsync replication does not replicate new subfolders

[Nikolaos Milas]

Hello,

Although this issue has been raised in another thread:

   https://www.mail-archive.com/dovecot@dovecot.org/msg57281.html

I decided I should create a new one dedicated on it because that thread 
includes other issues as well, and the issue in question still remains 
unresolved although it is an important one (and I am hoping to attract 
Timo's attention on it, too ;-) ).


The problem is that when dsync is configured between two servers 
(master/master), messages do get replicated, but newly created 
subfolders do not get replicated; as a result, when messages are moved 
by users from existing folders to newly created ones, replication 
results in chaos. Manually running a dsync command does not resolve the 
issue. See details of debug log on the above thread.


I remind you that the configs of the two master/master installations 
(2.2.12) are available at:


   Primary Master:
   https://www.mail-archive.com/dovecot@dovecot.org/msg57298.html
   Secondary Master:
   https://www.mail-archive.com/dovecot@dovecot.org/msg57304.html

The questions:

1. Is it an expected dsync behavior that newly created subfolders do
   not get replicated?
2. If yes, how can we replicate those?
3. If not, what is going wrong in the replication process? Does our
   configuration need a fix (which?) or does Dovecot needs one?

Please advise!

Sincerely,
Nick

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Nikolaos Milas]

On 8/4/2014 5:20 μμ, Nikolaos Milas wrote:

Still no subfolder sync (but individual messages on existing 
subfolders get sync'ed): 


Given that I can't make dsync work properly, until this thread provides 
info that might lead to correct operation, which I hope, I am thinking 
of trying syncing using lsyncd  unison (instead of dsync).


Before moving on, I would like to kindly ask list members:

1. Are there any admins around who have setup dsync replication and see 
new subfolder replication work correctly?


2. Is lsyncd  unison expected to work correctly given the particular 
architecture of Dovecot? (I would think it should work.) Anyone has 
tried it?


Thanks,
Nick

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Teemu Huovila]

Hello

Many different dsync issues have come up in this thread. Ill try to answer them 
as best as I can.

1) dsync backup -R
The conclusion reached in the thread was correct. Instead of the backup option, 
doveadm import would be better suited for
merging old mails into an existing mailbox.

2) Maildir + INBOX + backup/sync/replicate
In the test scenarios where the INBOX on one side was to be completely removed, 
e.g. doveadm backup -R the dsync failed and
nothing was synced to the target. This is because before moving the source 
mails to the mailbox, dsync cleans out the old ones (
-R preserves nothing) and in Maildir the INBOX can not be removed. This is a 
feature/not easily solvable, because in Maildir
INBOX is different from other folders.

3) dsync replication / doveadm sync not working as expected.
These came in pretty late in the thread and I did not get a full picture of 
what kind of setups and parameters were used. I
suspect these might be a configuration issue. I think trying with different 
configurations and going through the documentation,
such as it is, once more, is your best bet. Use -D and -v to make dsync more 
verbose, so you do not miss any error messages.

br,
Teemu Huovila

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Nikolaos Milas]

On 8/4/2014 12:38 μμ, Teemu Huovila wrote:


3) dsync replication / doveadm sync not working as expected.
These came in pretty late in the thread and I did not get a full picture of 
what kind of setups and parameters were used. I
suspect these might be a configuration issue. I think trying with different 
configurations and going through the documentation,
such as it is, once more, is your best bet. Use -D and -v to make dsync more 
verbose, so you do not miss any error messages.


Thank you for the reply; I am focusing on the 3rd part, since this is 
the one I can provide feedback about.


My current configuration is exactly as suggested on the wiki2 and I list 
it below for your reference.


Neither using replication nor using dsync from CLI leads to subfolders 
getting replicated, as I have explained. As an example, if a user 
creates subfolder boxtest e.g. under Inbox on either side, it never 
gets created on the other side.


Running dsync with -Dv does not reveal any errors.

For example, here is the output of command:

# dsync -fDv -u imaptester mirror ssh -l root vmail1.example.com 
dsync -u imaptester


Mailbox boxtest under Inbox (on vmail server) -containing one message- 
should get replicated (created) on vmail1 server, but it does not.


If you can spot anything that can help on tracing the problem, please help.

Otherwise, I can't see what is causing the erratic replication.

Thanks,
Nick

*
...
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox changed=0

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=0
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=attributes send_state=changes changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=attributes send_state=changes
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=changes send_state=mail_requests changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=changes send_state=mail_requests
dsync-local(imaptester): Debug: brain M: Import INBOX.boxtest: Import 
change GUID=1396119018.M550517P3113.vmail.example.com,S=1169,W=1194 
UID=1 hdr_hash=
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=changes send_state=mail_requests changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=changes send_state=mail_requests
dsync-local(imaptester): Debug: brain M: Import INBOX.boxtest: Last 
common UID=1
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=mail_requests send_state=mails changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mail_requests send_state=mails
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=mail_requests send_state=mails changed=0

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=0
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mail_requests send_state=mails
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=mails send_state=done changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mails send_state=done

dsync-local(imaptester): Debug: brain M: Import INBOX.boxtest: Saved UIDs:
dsync-local(imaptester): Debug: brain M: Import INBOX.boxtest: Finish 
update: min_next_uid=2 min_first_recent_uid=2 min_highest_modseq=2 
min_highest_pvt_modseq=0
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=recv_last_common send_state=done changed=1

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=recv_last_common send_state=done
dsync-local(imaptester): Debug: brain M: out 

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Teemu Huovila]

On 04/08/2014 03:00 PM, Nikolaos Milas wrote:
 Neither using replication nor using dsync from CLI leads to subfolders 
 getting replicated, as I have explained. As an example,
 if a user creates subfolder boxtest e.g. under Inbox on either side, it 
 never gets created on the other side.
I cant find any errors, but I might be missing something obvious. I only have a 
few suggestions for things to check.

1) You listed the config for one host (vmail i assume). Is the configuration 
similar on the vmail1 side? Especially, can the
command dsync -u user find the correct location for the users mails?

2) For the replicator plugin scenario, does doveadm have access to auth, i.e. 
does doveadm user '*' work on both sides?

3) Are the dovecot instances running on different hosts (dovecot --hostdomain 
is different)?

4) Instead of dsync mirror, try using the v2.2 syntax doveadm sync. Also, i 
_think_ you need to execute dsync-server on the
other side, so your full command becomes:
doveadm sync -u imaptester ssh -l root vmail1.example.com doveadm dsync-server 
-u imaptester
Sadly, there is no man-page for doveadm sync yet.

br,
Teemu Huovila

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Nikolaos Milas]

On 8/4/2014 4:47 μμ, Teemu Huovila wrote:


I cant find any errors, but I might be missing something obvious. I only have a 
few suggestions for things to check.

1) You listed the config for one host (vmail i assume). Is the configuration 
similar on the vmail1 side? Especially, can the
command dsync -u user find the correct location for the users mails?


Yes, mail is replicated in existing folders, but new subfolders don't 
get replicated.


The config I listed was on the vmail (master) side, indeed. Config on 
the vmail1 (replicated) side is identical except the replication parts. 
I list it at the end of this mail for your reference.



2) For the replicator plugin scenario, does doveadm have access to auth, i.e. 
does doveadm user '*' work on both sides?


Yes. Everything is fine: 317 identical accounts on each side (read from 
replicated LDAP).



3) Are the dovecot instances running on different hosts (dovecot --hostdomain 
is different)?


Yes: vmail.example.com vs vmail1.example.com


4) Instead of dsync mirror, try using the v2.2 syntax doveadm sync. Also, 
i_think_  you need to execute dsync-server on the
other side, so your full command becomes:
doveadm sync -u imaptester ssh -l root vmail1.example.com doveadm dsync-server 
-u imaptester
Sadly, there is no man-page for doveadm sync yet.


OK, I ran the command:

doveadm -Dv sync -u imaptester ssh -l root vmail1.example.com doveadm 
dsync-server -u imaptester


The output is similar. Still no subfolder sync (but individual messages 
on existing subfolders get sync'ed):


...
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox
dsync-local(imaptester): Debug: brain M: out box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox changed=0

dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=0
dsync-local(imaptester): Debug: brain M: in state=sync_mails
dsync-local(imaptester): Debug: brain M: in box 'INBOX.boxtest' 
recv_state=mailbox send_state=mailbox
dsync-local(imaptester): Debug: brain M: out box '' recv_state=mailbox 
send_state=mailbox changed=1
dsync-local(imaptester): Debug: brain M: out state=master_send_mailbox 
changed=1

dsync-local(imaptester): Debug: brain M: in state=master_send_mailbox
dsync-local(imaptester): Debug: brain M: out state=sync_mails changed=1
dsync-local(imaptester): Debug: brain M: in state=sync_mails
...

Please suggest any other ideas!

Thanks,
Nick

==   vmail1 Config file 
==


# cat /etc/dovecot/dovecot.conf
# 2.2.12: dovecot.conf

protocols = imap pop3

login_greeting = VMAIL1 POP/IMAP Srv XAPITI XPICTOY

mail_location = maildir:~/Maildir/
mail_gid = 5000
mail_uid = 5000

auth_mechanisms = plain login
auth_username_format = %Lu
auth_verbose = yes
disable_plaintext_auth = no

mail_plugins = quota

protocol imap {
  imap_client_workarounds = delay-newmail  
  mail_plugins = quota imap_quota
}

protocol pop3 {
  mail_max_userip_connections = 3
  mail_plugins = quota
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %08Xu%08Xv
}

protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  info_log_path =
  log_path =
  mail_plugins = quota
  postmaster_address = sysad...@example.com
  sendmail_path = /usr/lib/sendmail
}

userdb {
  args = /etc/dovecot/dovecot-usrdb-ldap.conf
  driver = ldap
}

passdb {
  args = /etc/dovecot/dovecot-passdb-ldap.conf
  driver = ldap
}

plugin {
  quota = maildir:User quota
  quota_rule = *:storage=4G
  quota_rule2 = Trash:storage=+3%%
  quota_warning = storage=75%% quota-warning 75 %u
  quota_warning2 = storage=90%% quota-warning 90 %u
}

service quota-warning {
  executable = script /opt/mail1.sh
  user = vmail
  unix_listener quota-warning {
user = vmail
  }
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
  unix_listener auth-master {
group = vmail
mode = 0660
user = vmail
  }
  user = root
}

service imap-login {
  service_count = 1
  vsz_limit = 64 M
}

service pop3-login {
  service_count = 1
  vsz_limit = 64 M
}

ssl_ca = /etc/pki/tls/certs/chain-2228.pem
ssl_cert = /etc/pki/tls/certs/cert-2228.pem
ssl_key = /etc/pki/tls/private/key-2228.pem

syslog_facility = local1

 

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Benjamin Podszun]

Hey Jiri.

Thanks for getting back.

On Friday, April 4, 2014 4:48:48 PM CEST, Jiri Bourek wrote:

- where did I fail (ignoring the backups, please. That's .. something I
know)


 From the man page: backup - Backup mails from default mail 
location to location2 (or vice versa, if -R parameter is given). 
No changes are ever done to the source location. Any changes 
done in destination are discarded.


Yeah, maybe. That's what I thought _after_ the fact (i.e. that was what I 
hinted at with 'one way sync'). But see below.


The last sentence describes what happened to you: all new mail 
on the new machine is a change and was discarded (by deleting 
new mail.) If I'm not mistaken, this is correct behaviour for 
backup mode - you get exact copy of the source side 
(maildir:/tmp/mail_backup) on destination side 
(d...@darklajid.de)


That would be sort of okay. Except that isn't what happened:

- The target mailbox was killed completely
- Nothing was restored

If what you're suggesting here is true I'd expect a clean copy of my source 
- even if it destroys all other changes. That did NOT happen though. It 
nuked the target and didn't restore a thing.


Plus, dsync mirror did exactly the same: Nuked the (live) mailbox once 
more, same error message, not a single message restored (but also no 
modification to the source).




- Can I use dsync ... for backups? I don't think that this is a good
idea after yesterday night?


AFAIK you can safely use it to make the backup. I'm not sure if 
it can be reliably used to restore data (don't think so but I'm 
not an expert.) I'd use doveadm import for that.


That'd be my experience at this point as well, of course. :-)
The bigger question is if this is well-known / correct and if this should 
be documented in a better fashion. Was I really that naive to expect that 
to work (in that case: ignore the documentation request) or could that 
happen again?


Ben

Re: [Dovecot] dsync replication questions

[Simon Fraser]

On Fri, 2014-04-04 at 15:41 +0300, Teemu Huovila wrote:
 Hi
 
 On 04/04/2014 03:38 PM, Simon Fraser wrote:
  It does help, thank you.  Do you also know what the '-f' option does?
 According to the dsync-man page it:
 
 Makes dsync run in full sync mode rather than fast sync mode. In fast 
 sync mode dsync might skip syncing a mailbox, if both
 locations had modified it equally many times (i.e. highest-modseqs were 
 equal), but with different changes.

Thank you.  Is it still only the changes that are synced each way, or
the entire mailbox? I'm trying to gauge the performance hit for enabling
this on larger mailboxes. (I could, of course, run some tests, but
someone may already have done that)

Simon.




-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Re: [Dovecot] dsync replication questions

[Teemu Huovila]

On 04/07/2014 12:22 PM, Simon Fraser wrote:
 Thank you.  Is it still only the changes that are synced each way, or
 the entire mailbox? I'm trying to gauge the performance hit for enabling
 this on larger mailboxes. (I could, of course, run some tests, but
 someone may already have done that)
Cant say anything certain on this one. I do know that not all the messages are 
sent to the other side. There are optimizations
in place, using the Dovecot transaction logs and some pretty complicated 
application login, but a lot of data still needs to be
processed by the dsync brains.

I think testing with your particular setup and data would give the most 
accurate results.

br,
Teemu Huovila

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Jiri Bourek]

The last sentence describes what happened to you: all new mail on the
new machine is a change and was discarded (by deleting new mail.) If
I'm not mistaken, this is correct behaviour for backup mode - you get
exact copy of the source side (maildir:/tmp/mail_backup) on
destination side (d...@darklajid.de)


That would be sort of okay. Except that isn't what happened:

- The target mailbox was killed completely
- Nothing was restored

If what you're suggesting here is true I'd expect a clean copy of my
source - even if it destroys all other changes. That did NOT happen
though. It nuked the target and didn't restore a thing.


True - if we move from problem is dsync deleted new mail to problem 
is dsync was unable to restore the backup, the described behaviour 
looks like a bug to me too. It may have something to do with the maildir 
format, I recall some discussion regarding folder INBOX, which needs 
special handling (because it's physically stored in maildir root, 
whereas every other folder is stored in folder-named subdirectory)


That said, I tried something along what you did and it failed for me 
too. So I deleted the mailbox completely, recreated it, tried again and 
this time the restore succeeded. It seems the easiest possible way to 
reproduce the faulty behaviour is:


1. create mailbox for testing, here t...@example.com
2. create IMAP folder under INBOX
( namespace inbox { separator = / } )

# doveadm mailbox create -u t...@example.com INBOX/test

3. attempt to restore from backup

# doveadm backup -u t...@example.com -R maildir:/mnt/mail-backups/test/

which yields (on Dovecot 2.2.12)

dsync(t...@example.com): Error: Mailbox INBOX sync: mailbox_delete 
failed: INBOX can't be deleted.


Another try shows that IMAP folder created somewhere else (not under 
INBOX) isn't a problem:


# doveadm mailbox create -u t...@example.com testtest
# doveadm backup -u t...@example.com -R maildir:/mnt/mail-backups/test/

This yields no output, folder testtest is deleted (as expected), INBOX 
is populated from backup.


Another try, this time I used mbox instead of maildir by specifying

-o mail_location=mbox:/path/test/mail

to doveadm. Worked without error even with INBOX/test folder (which got 
deleted during restore)


No idea if this can be considered as a bug, or the test does something 
that is not supposed to be done in the first place (Although different 
results with different storage format suggests a bug to me.)




Plus, dsync mirror did exactly the same: Nuked the (live) mailbox once
more, same error message, not a single message restored (but also no
modification to the source).



I was doing some trial and error testing with doveadm sync (should be 
the same as dsync mirror.) If used on a mailbox which has seen some 
changes, this sync's behaviour is just strange.


Or - to be more precise - it seems strange on first sight, but when you 
think about it, it does what is supposed to do. The sync mode is (AFAIK) 
designed to keep single mailbox synchronized on two hosts. If you 
created new mailbox on the new host, then had some mail delivered to it 
and after some time decided to add mail from old host, then you don't 
have single mailbox - you have two mailboxes with the same name.


In other words this scenario is probably something dsync wasn't designed 
to be used for and there's no surprise mirror mode can't handle it.


And again - I'm no expert, so it's entirely possible everything I wrote 
here is complete and utter nonsense Let's hope someone more 
knowledgeable corrects me if that is the case.

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Nikolaos Milas]

On 7/4/2014 4:01 μμ, Jiri Bourek wrote:

I was doing some trial and error testing with doveadm sync (should be 
the same as dsync mirror.) If used on a mailbox which has seen some 
changes, this sync's behaviour is just strange. 


I can confirm (on 2.2.12) that the behavior is the same using 
replication (mirroring). Creating a folder on either end, e.g. as a 
subfolder of Inbox, does not create an identical new folder on the other 
end. I would expect that the folder is replicated!


Manual dsync from the CLI will not replicate folders as well.

I will agree with Jiri that is a strange (wrong?) behavior in dsync. 
Folders should be replicated, whether new or not. If not, how message 
moves between older and newer folders can be replicated on the fly?


Please advise on how to handle this situation because if folders are 
created/deleted/moved by users, dsync may lead to a chaos!


Nick

[Dovecot] dsync deleted my mailbox - what did I do wrong?

[Benjamin Podszun]

Hi.

Mostly annoying: I migrated from one machine to another, made sure the 
target host worked as expected, updated mx records and - after a couple of 
days - signed it off as good. This is just my private machine, no big deal 
if something goes wrong..


Everything's fine? Good, let's migrate my inbox from the old machine. 
There's no direct connectivity between those servers, so what I did was:


(old server)
sudo -u vmail dsync -u d...@darklajid.de backup maildir:/tmp/mail_backup/

Works fine, got my maildir. Tar'd it up, moved it to the new server. Now 
how do I import those mails? Ah, let's use the same command, with -R?


(new server)
sudo -u vmail dsync -u d...@darklajid.de -R backup maildir:/tmp/mail_backup
Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted.

Wait. What? Sure enough, the last couple of days are gone, the target 
mailbox is completely empty. I read the man page over and over again, but 
failed to see the problem. I even thought for a moment that _maybe_ dsync 
backup is one-way only (even if that failed as well) and tried the same 
command with mirror. Exactly the same output, same result, empty target 
mailbox.


In the end I succeeded to import the mails with doveadm import, completely 
lost a number of days of mails. My fault, sloppy not to back the up again, 
but I still don't think that this should happen. Ever.


My question now is:

- where did I fail (ignoring the backups, please. That's .. something I 
know)


- Can I use dsync ... for backups? I don't think that this is a good idea 
after yesterday night?


- Should dsync EVER try to delete mailboxes? Even 'special' mailboxes? 
Should it warn about that, asking for a --force switch or something?


Any insights would be appreciated. At this point the damage is done, but 
I'd like to learn how to do better.


Ben

Re: [Dovecot] dsync replication questions

[Nikolaos Milas]

On 31/3/2014 12:03 μμ, Simon Fraser wrote:


On Fri, 2014-03-28 at 11:58 +0200, Nikolaos Milas wrote:


Question 2a: What do options -d -N -l 30 -U signify in
replication_dsync_parameters = -d -N -l 30 -U?

I'd also be interested to know the answer to this part. I found mention
of the '-f' option, and adding '-f' to that list appears to have fixed a
replication race condition I was having, but I can't find out a note of
what it does.


There is no one who knows that? Or no documentation whatsoever?

Anyone please?

How can we determine whether we need to configure 
replication_dsync_parameters = -d -N -l 30 -U as indicated on the 
wiki2 (or some variation thereof) or not?


Thanks,
Nick

Re: [Dovecot] dsync replication questions

[Alan McGinlay]

On 2014-04-04 11:42, Nikolaos Milas wrote:

On 31/3/2014 12:03 μμ, Simon Fraser wrote:

On Fri, 2014-03-28 at 11:58 +0200, Nikolaos Milas wrote:

Question 2a: What do options -d -N -l 30 -U signify in
replication_dsync_parameters = -d -N -l 30 -U?
I'd also be interested to know the answer to this part. I found 
mention
of the '-f' option, and adding '-f' to that list appears to have fixed 
a
replication race condition I was having, but I can't find out a note 
of

what it does.

There is no one who knows that? Or no documentation whatsoever?

Anyone please?

How can we determine whether we need to configure
replication_dsync_parameters = -d -N -l 30 -U as indicated on the
wiki2 (or some variation thereof) or not?


It does appear to be completely undocumented, I tried looking in the 
source code but just ended up going in circles (i'm not familiar with it 
at all).


doveadm sync has no documentation that I can find at all. I would 
happily write it if I could find the options in the code lol


/A

Re: [Dovecot] dsync replication questions

[Nikolaos Milas]

On 4/4/2014 1:17 μμ, Alan McGinlay wrote:

It does appear to be completely undocumented, 


We would appreciate if Timo can kindly provide us with -at least- some 
details!


I tried looking in the source code but just ended up going in circles 
(i'm not familiar with it at all). 


I faced the same difficulties... :-(

Nick

Re: [Dovecot] dsync replication questions

[Teemu Huovila]

On 04/04/2014 12:42 PM, Nikolaos Milas wrote:
 Anyone please?
 
 How can we determine whether we need to configure 
 replication_dsync_parameters = -d -N -l 30 -U as indicated on the wiki2 (or
 some variation thereof) or not?

I am definitely not Timo, but I will try a short answer based on my limited 
familiarity with the dsync replication functionality
and code.

-d Use a default location for the replica. As far as I can tell this is 
obtained from userdb variable mail_replica.
-N Sync all (visible?) namespaces (only makes sense when syncing with a remote 
host, with potentially different namespaces)
  (instead of either -n namespace to sync or -x namespaces not to sync)
-l seconds Use a dsync lock file when syncing and the timeout for locking.
-U Update replicator server on the status of the replication

For replicating in a multiple server configuration, they sound like a good idea 
to me.

Hope this helps a little,
Teemu Huovila

Re: [Dovecot] dsync replication questions

[Simon Fraser]

On Fri, 2014-04-04 at 15:34 +0300, Teemu Huovila wrote:

 -d Use a default location for the replica. As far as I can tell this is 
 obtained from userdb variable mail_replica.
 -N Sync all (visible?) namespaces (only makes sense when syncing with a 
 remote host, with potentially different namespaces)
   (instead of either -n namespace to sync or -x namespaces not to sync)
 -l seconds Use a dsync lock file when syncing and the timeout for locking.
 -U Update replicator server on the status of the replication
 
 For replicating in a multiple server configuration, they sound like a good 
 idea to me.
 
 Hope this helps a little,

It does help, thank you.  Do you also know what the '-f' option does?

Simon.




-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Re: [Dovecot] dsync replication questions

[Teemu Huovila]

Hi

On 04/04/2014 03:38 PM, Simon Fraser wrote:
 It does help, thank you.  Do you also know what the '-f' option does?
According to the dsync-man page it:

Makes dsync run in full sync mode rather than fast sync mode. In fast sync 
mode dsync might skip syncing a mailbox, if both
locations had modified it equally many times (i.e. highest-modseqs were equal), 
but with different changes.

br,
Teemu Huovila

Re: [Dovecot] dsync deleted my mailbox - what did I do wrong?

[Jiri Bourek]

Hi



- where did I fail (ignoring the backups, please. That's .. something I
know)


From the man page: backup - Backup mails from default mail location to 
location2 (or vice versa, if -R parameter is given). No changes are ever 
done to the source location. Any changes done in destination are discarded.


The last sentence describes what happened to you: all new mail on the 
new machine is a change and was discarded (by deleting new mail.) If 
I'm not mistaken, this is correct behaviour for backup mode - you get 
exact copy of the source side (maildir:/tmp/mail_backup) on destination 
side (d...@darklajid.de)




- Can I use dsync ... for backups? I don't think that this is a good
idea after yesterday night?


AFAIK you can safely use it to make the backup. I'm not sure if it can 
be reliably used to restore data (don't think so but I'm not an expert.) 
I'd use doveadm import for that.

Re: [Dovecot] dsync replication questions

[Simon Fraser]

On Fri, 2014-03-28 at 11:58 +0200, Nikolaos Milas wrote:

 Question 2a: What do options -d -N -l 30 -U signify in 
 replication_dsync_parameters = -d -N -l 30 -U?

I'd also be interested to know the answer to this part. I found mention
of the '-f' option, and adding '-f' to that list appears to have fixed a
replication race condition I was having, but I can't find out a note of
what it does.

Simon.




-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 

Re: [Dovecot] dsync replication questions

[Nikolaos Milas]

On 28/3/2014 10:40 μμ, Nikolaos Milas wrote:

When I started the server (vmail.example.com), mirroring started and 
completed fine (after a few hours).


However, since then, I am not seeing a continued mirroring between the 
two. I would expect changes to one of the masters to be propagated in 
real time to the other - which does not happen.


Hi,

I found this older thread:

   http://comments.gmane.org/gmane.mail.imap.dovecot/69148

which provided the solution to my issues (I had to enable the 
replication plugin on pop3/imap/lda services).


Things seem to be working fine now. I only wish dsync logging were a bit 
more verbose; I don't know how this can be done. If anyone knows, I 
would appreciate this info.


Case closed for now.

Best regards,
Nick

[Dovecot] dsync replication questions

[Nikolaos Milas]

I am running two servers with Dovecot v2.2.12 on CentOS x86_64 (5.10 and 
6.5 respectively) and users are virtual over ldap.


I have setup our main internal server (vmail.example.com) with dsync 
replication according to the first part of 
http://wiki2.dovecot.org/Replication. The second one 
(vmail1.example.com) will be the failover server which we want to be a 
real-time mirror (but can be manipulated directly and it should 
propagate changes back to vmail.example.com).


When I started the server (vmail.example.com), mirroring started and 
completed fine (after a few hours).


However, since then, I am not seeing a continued mirroring between the 
two. I would expect changes to one of the masters to be propagated in 
real time to the other - which does not happen.


If I manually run (on vmail.example.com):

   dsync -u imaptester mirror ssh -l root vmail1.example.com dsync -u
   imaptester

then these two accounts are synced.

Question 1: Shouldn't mirroring continue automatically in real time 
(having configured it as in the first part of 
http://wiki2.dovecot.org/Replication)? If not, what should I do to 
enable such a real-time (or near-real-time) sync?


Note that I have not configured replication_dsync_parameters as noted 
at the above page.


Question 2: Where can I find documentation about 
replication_dsync_parameters and about the doveadm sync command 
(referred to at the same page)? I have not been able to locate such a 
documentation/wiki page.


Question 2a: What do options -d -N -l 30 -U signify in 
replication_dsync_parameters = -d -N -l 30 -U?


Thanks,
Nick

Re: [Dovecot] dsync replication questions

[Nikolaos Milas]

On 28/3/2014 11:58 πμ, Nikolaos Milas wrote:

When I started the server (vmail.example.com), mirroring started and 
completed fine (after a few hours).


However, since then, I am not seeing a continued mirroring between the 
two. I would expect changes to one of the masters to be propagated in 
real time to the other - which does not happen.


If I manually run (on vmail.example.com):

   dsync -u imaptester mirror ssh -l root vmail1.example.com dsync -u 
imaptester


then these two accounts are synced.


I am trying to figure out whether replication plugin is configured 
correctly (- I guess something is wrong).


I have (in vmail.example.com dovecot.conf):

dsync_remote_cmd = ssh -l root vmail1.example.com doveadm dsync-server -u%u

plugin {
  mail_replica = remote:vm...@vmail1.example.com
}

The question here is, in:

   mail_replica = remote:vm...@vmail1.example.com

the reference: vmail@ refers to the virtual user under whom accounts 
live in the remote system (which I have assumed), or something else?


Also, in the aggregator service, is there something wrong?

service aggregator {
  fifo_listener replication-notify-fifo {
user = vmail
  }
  unix_listener replication-notify {
user = vmail
  }
}

Note that I am not using Director.

Follows the whole configuration on the main master, for your reference 
(I've only changed the true domain name). Dovecot conf on the mirror 
server is exactly the same without the replication bits.


Please advise.

Thanks,
Nick

==
# 2.2.12: dovecot.conf

protocols = imap pop3

login_greeting = Org POP/IMAP Srv XAPITI XPICTOY

mail_location = maildir:~/Maildir/
mail_gid = 500
mail_uid = 500

auth_mechanisms = plain login
auth_username_format = %Lu

auth_verbose = yes
auth_debug = no

disable_plaintext_auth = no

mail_plugins = quota notify replication

protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins = quota imap_quota
}

protocol pop3 {
  mail_max_userip_connections = 3
  mail_plugins = quota
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %08Xu%08Xv
}

protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  info_log_path =
  log_path =
  mail_plugins = quota
  postmaster_address = sysad...@example.com
  sendmail_path = /usr/lib/sendmail
}

userdb {
  args = /etc/dovecot/dovecot-usrdb-ldap.conf
  driver = ldap
}

passdb {
  args = /etc/dovecot/dovecot-passdb-ldap.conf
  driver = ldap
}

dsync_remote_cmd = ssh -l root vmail1.example.com doveadm dsync-server -u%u

plugin {
  mail_replica = remote:vm...@vmail1.example.com
}

plugin {
  quota = maildir:User quota
  quota_rule = *:storage=4G
  quota_rule2 = Trash:storage=+3%%
  quota_warning = storage=75%% quota-warning 75 %u
  quota_warning2 = storage=90%% quota-warning 90 %u
}

service quota-warning {
  executable = script /opt/mail1.sh
  user = vmail
  unix_listener quota-warning {
user = vmail
  }
}

service aggregator {
  fifo_listener replication-notify-fifo {
user = vmail
  }
  unix_listener replication-notify {
user = vmail
  }
}

service replicator {
   unix_listener replicator-doveadm {
 mode = 0600
   }
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
  unix_listener auth-master {
group = vmail
mode = 0660
user = vmail
  }
  user = root
}

service imap-login {
  service_count = 1
  vsz_limit = 64 M
}

service pop3-login {
  service_count = 1
  vsz_limit = 64 M
}

service replicator {
  process_min_avail = 1
}

ssl_ca = /etc/pki/tls/certs/chain-1552.pem
ssl_cert = /etc/pki/tls/certs/cert-1552.pem
ssl_key = /etc/pki/tls/private/key-1552.pem

syslog_facility = local1
==

Re: [Dovecot] dsync in a cronjob to replace offlineimap

[Félix Sipma]

Hi,

I'm in the exact same situation: trying to use a state file in a cron
sync job on my laptop, it quickly becomes corrupted...

Is there a way to use incremental replication in this use case (sync
between server and laptop)?

Thanks,

-- 
Félix


signature.asc
Description: Digital signature

Re: [Dovecot] dsync Error: Mailbox INBOX: Save commit failed: Mailbox was deleted under us

[Alan McGinlay]

On 2014-01-31 10:51, Alan McGinlay - SICS wrote:

Hi,

I am doing a mass migration of users from Cyrus imap on a solaris
server to dovecot 1:2.2.9-1ubuntu1 and am getting this weird issue
with dsync if I try to do a full sync. Debug output below:

dsync(u...@example.com): Error: Mailbox INBOX: Save commit failed:
Mailbox was deleted under us
dsync(u...@example.com): Debug: brain M: out box 'INBOX'
recv_state=recv_last_common send_state=done changed=1
dsync(u...@example.com): Debug: brain M: out state=sync_mails 
changed=1

dsync(u...@example.com): Debug: brain S: in state=sync_mails
dsync(u...@example.com): Debug: brain S: in box 'INBOX'
recv_state=recv_last_common send_state=done
dsync(u...@example.com): Debug: brain S: out box 'INBOX'
recv_state=recv_last_common send_state=done changed=0
dsync(u...@example.com): Debug: brain S: out state=sync_mails 
changed=0

dsync(u...@example.com): Debug: imapc(10.10.10.10:143): Disconnected
[2]+  Killed  /usr/bin/dsync -v -o mail_fsync=never
backup -R -f -u u...@example.com imapc:

(I changed the domain / ip here of course)

I notice that it does the IMAP sync for a while and then shows
connecting info for pop3 (I have configuration for both imap and pop).
The user in question has never connected with POP.

imap migration config:

imapc_host = 10.10.10.10
imapc_user = %u
imapc_master_user = cyrusadmin
imapc_password = blahblahblahpasswordhere
imapc_features = rfc822.size
imapc_features = $imapc_features #fetch-headers
mail_prefetch_count = 20

pop configuration:

pop3c_host = pop.example.com
pop3c_user = %u
pop3c_master_user = cyrusadmin
pop3c_password = blahblahblahpasswordhere

namespace {
  prefix = POP3-MIGRATION-NS/
  location = pop3c:
  list = no
  hidden = yes
}
protocol doveadm {
  mail_plugins = $mail_plugins pop3_migration
}
plugin {
  pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX
}

Thanks in advance for any pointers on this!

/Alan


Still having this issue, if anyone has any ideas I would really 
appreciate it! It's putting a serious crimp in my migration plans :|

[Dovecot] Dsync replication with LDAP and auth_bind=yes

[Michał Franczak]

Hello,

I have working dovecot configuration with LDAP based authentication in 
Active Directory.

I'd like to use dsync replication but dovecot can't do user lookups.
Is it possible to configure replication in such architecture?
doveadm user '*'
Returns only one local dovecot user.

dovecot --version
2.2.10

My dovecot-ldap.conf look as follows:


hosts   = ad.domain.com:389
ldap_version= 3
auth_bind   = yes
dn  = src_mail_ldap
dnpass  = somePass
base= OU=users,DC=domain,DC=com
scope   = subtree
deref   = never
user_filter = 
((userPrincipalName=%u)(objectClass=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))
pass_filter = 
((userPrincipalName=%u)(objectClass=person)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

pass_attrs  = userPassword=password
default_pass_scheme = CRYPT
user_attrs  = 
=home=/var/vmail/vmail1/%Ld/%Ln/Maildir/,=mail=maildir:/var/vmail/vmail1/%Ld/%Ln/Maildir/


Best Regards
--
Michal



smime.p7s
Description: S/MIME Cryptographic Signature

[Dovecot] dsync mirror to mailstores in director setup

[Murray Trainer]

Hi All,

I have dsync mirror command working between a standalone dovecot
server and a backend mailstore in a Dovecot Director setup.   Is it
safe to do the dsync directly to the backend mailstore in this setup?
It doesn't seem to work via the director proxy.

Thanks

Murray

Re: [Dovecot] dsync, a zero-way synchronisation tool?

[Fabiano Sidler]

So schrieb Fabiano Sidler:
 Hi folks!
 
 I have set up dsync replication with SSH according to
 http://wiki2.dovecot.org/Replication with the exception of having system users
 and calling doveadm dsync-server directly from authorized_keys, because the
 wrapper script posted on the above site is needless (at least in 2.2.10).
 
 However, while the two instances connect well to each other, no 
 synchronisation
 is performed at all, the mailboxes happily remain untouched. I've then
 temporarily replaced SSH with socat and captured the traffic.
 
 Any hints what goes wrong? Unfortunately there are no logs to provide...

Hello? Noone with any idea? Timo, as the developer, what do you think about
the captured traffic between the hosts?

Greetings,
Fabiano

Re: [Dovecot] dsync: possible cosmetic bug

[Fabiano Sidler]

Thus wrote Andrei Dobrotsvetov:
 Hello Everyone,
 
 I use dovecot2-2.2.10,
 FreeBSD 9.2-RELEASE.

Same version and OS like me...:)

 Replication was set up according to:
 http://wiki2.dovecot.org/Replication,
 dsync wrapper script is used.
 
 It seems that all is worked as desired,
 but i see the following into log file:
 
 doveadm: Error: dsync-remote(XXX@YYY): Info: save: box=INBOX, ...
 
 There were no such log records
 when i tried replication without wrapper script.

Forget the wrapper script on that site, it's needless or broken. Call doveadm
dsync-server directly from authorized_keys.

But synchronisation works then for you? I'm currently having the problem that
dsync doesn't synch anything at all...

Greetings,
Fabiano

[Dovecot] dsync: possible cosmetic bug

[Andrei Dobrotsvetov]

Hello Everyone,

I use dovecot2-2.2.10,
FreeBSD 9.2-RELEASE.

Replication was set up according to:
http://wiki2.dovecot.org/Replication,
dsync wrapper script is used.

It seems that all is worked as desired,
but i see the following into log file:

doveadm: Error: dsync-remote(XXX@YYY): Info: save: box=INBOX, ...

There were no such log records
when i tried replication without wrapper script.

Best regards,
Andrei 

[Dovecot] dsync, a zero-way synchronisation tool?

[Fabiano Sidler]

Hi folks!

I have set up dsync replication with SSH according to
http://wiki2.dovecot.org/Replication with the exception of having system users
and calling doveadm dsync-server directly from authorized_keys, because the
wrapper script posted on the above site is needless (at least in 2.2.10).

However, while the two instances connect well to each other, no synchronisation
is performed at all, the mailboxes happily remain untouched. I've then
temporarily replaced SSH with socat and captured the traffic.

Any hints what goes wrong? Unfortunately there are no logs to provide...

Greetings,
Fabiano
VERSION dsync   3   2
Hhostname   sync_ns_prefix  sync_boxsync_box_guid   sync_type   
debug   sync_visible_namespaces exclude_mailboxes   send_mail_requests  
backup_send backup_recv lock_timeoutno_mail_sync
no_backup_overwrite purge_remote
Smailbox_guid   last_uidvaliditylast_common_uid last_common_modseq  
last_common_pvt_modseq  changes_during_sync
Nname   existence   mailbox_guiduid_validityuid_next
last_renamed_or_created subscribed  last_subscription_change
Dhierarchy_sep  mailboxes   dirsunsubscribes
Bmailbox_guid   uid_validityuid_nextmessages_count  
first_recent_uidhighest_modseq  highest_pvt_modseq  mailbox_lost
cache_fieldshave_guids  have_save_guids have_only_guid128
Atype   key value   stream  deleted last_change modseq
Ctype   uid guidhdr_hashmodseq  pvt_modseq  save_timestamp  
add_flags   remove_flagsfinal_flags keywords_reset  keyword_changes
Rguid   uid
Mguid   uid pop3_uidl   pop3_order  received_date   stream
cname   decisionlast_used
.
VERSION dsync   3   2
Hhostname   sync_ns_prefix  sync_boxsync_box_guid   sync_type   
debug   sync_visible_namespaces exclude_mailboxes   send_mail_requests  
backup_send backup_recv lock_timeoutno_mail_sync
no_backup_overwrite purge_remote
Smailbox_guid   last_uidvaliditylast_common_uid last_common_modseq  
last_common_pvt_modseq  changes_during_sync
Nname   existence   mailbox_guiduid_validityuid_next
last_renamed_or_created subscribed  last_subscription_change
Dhierarchy_sep  mailboxes   dirsunsubscribes
Bmailbox_guid   uid_validityuid_nextmessages_count  
first_recent_uidhighest_modseq  highest_pvt_modseq  mailbox_lost
cache_fieldshave_guids  have_save_guids have_only_guid128
Atype   key value   stream  deleted last_change modseq
Ctype   uid guidhdr_hashmodseq  pvt_modseq  save_timestamp  
add_flags   remove_flagsfinal_flags keywords_reset  keyword_changes
Rguid   uid
Mguid   uid pop3_uidl   pop3_order  received_date   stream
cname   decisionlast_used
.
Hmail1.example.org           c            
Hmail2.example.org
NINBOX  y   9c788533a3760253b975879d8c251392670371  29
NTrash  y   7888b4327f930253627d879d8c251392670372  1   
   1392675836
.
D.
NINBOX  y   9c788533a3760253b975879d8c251392670371  29
NTrash  y   7888b4327f930253627d879d8c251392670372  1   
   1392677757
.
D.
B9c788533a3760253b975879d8c25   1392670371  29  28  29  
7   0      
flags
ty
t1392926317
ndate.received
tt
t1392927996
ndate.save
tt
t1392927888
nsize.virtual
ty
t1392926317
nsize.physical
ty
t1392926317
nmime.parts
tt
t1392843172
nhdr.BCC
tt
t1392926317
nhdr.CC
tt
t1392926317
nhdr.CONTENT-TYPE
tt
t1392926317
nhdr.DATE
tt
t1392926317
nhdr.FROM
tt
t1392926317
nhdr.IN-REPLY-TO
tt
t1392926317
nhdr.MESSAGE-ID
tt
t1392926317
nhdr.NEWSGROUPS
tt
t1392926317
nhdr.PRIORITY
tt
t1392926317
nhdr.REFERENCES
tt
t1392926317
nhdr.REPLY-TO
tt
t1392926317
nhdr.SUBJECT
tt
t1392926317
nhdr.TO
tt
t1392926317
nhdr.X-PRIORITY
tt
t1392926317
   
B9c788533a3760253b975879d8c25   1392670371  29  28  29  
7   0      
flags
ty
t1392926317
ndate.received
tt
t1392927996
ndate.save
tt
t1392927888
nsize.virtual
ty
t1392926317
nsize.physical
ty
t1392926317
nmime.parts
tt
t1392843172
nhdr.Date
tt
t1392926317
nhdr.BCC
tt
t1392926317
nhdr.CC
tt
t1392926317
nhdr.CONTENT-TYPE
tt
t1392926317
nhdr.FROM
tt
t1392926317
nhdr.IN-REPLY-TO
tt
t1392926317
nhdr.MESSAGE-ID
tt
t1392926317
nhdr.NEWSGROUPS
tt
t1392926317
nhdr.PRIORITY
tt
t1392926317
nhdr.REFERENCES
tt
t1392926317
nhdr.REPLY-TO
tt
t1392926317
nhdr.SUBJECT
tt
t1392926317
nhdr.TO
tt
t1392926317
nhdr.X-PRIORITY
tt
t1392926317
   
B7888b4327f930253627d879d8c25   1392670372  1   0   1   
1   0      flags
tt
B7888b4327f930253627d879d8c25   1392670372  1   0   1   
1   0      flags
tt

Re: [Dovecot] Dsync crash (v2.2.10, sdbox+sis → mbox)

[Gilles Chauvin]

Hi Timo,

I've made some further research on this issue (Dovecot was upgraded to
the latest release in the meantime but, unsurprisingly, to no avail) and
here's what I've found so far.


On 09/02/2014 10:42, Gilles Chauvin wrote:
 dsync(user2): Error:
 read(/zfspool/clone_srv_attachments/ad/0c/ad0cef35cc6f0b2dae2197c4ff2b61a2bd58070d-9e8345192ccbf352c21044c1c7e7-6efa5f2e522db350ed3d94b229f9-15470[base64:18
 b/l]) failed: Stream is larger than expected (194476  194475, eof=1)
 dsync(user2): Error: copy: i_stream_read() failed: Invalid argument
 dsync(user2): Panic: file mail-index-transaction-update.c: line 19
 (mail_index_transaction_lookup): assertion failed: (seq =
 t-first_new_seq  seq = t-last_new_seq)


The original mail got an attachment which is base64 encoded on 72 cols.
The last 3 lines are:

MAAxADMAIAAyADAAOgAwADEAOgA1ADQADQAKAGwAJwB1AHQAaQBsAGkAcwBhAHQAZQB1AHIA
IABkAGUAIABsAG8AZwBpAG4AOgAgAGsAZQBsAGUAbQBhAHIAaQAgAGEAIADpAHQA6QAgAGMA
cgDpAOkAIABsAGUAIAAyADEALwAwADMALwAyADAAMQAzACAAMgAwADoAMAAyADoAMAA0AA0ACgA=


For no good reason, the last line lacks a CR before the final CgA= part.

I guess this is where Dovecot yells about the stream larger than
expected because when it reencodes the attachment, it does it correctly
by adding a proper CR before CgA= hence the one byte difference
(tested using the base64 command line tool).

During my tests, each time dsync failed with this particular error, the
same pattern applied (malformed base64 last line).

Looks like a pretty hard problem to solve but, for now, it prevents us
from restoring a mailbox.


Regards,
Gilles

Re: [Dovecot] Dsync Panic

[Gilles Chauvin]

Hi,

Here is another dsync Panic while using:

$ dsync -Dvf -u user -R backup ssh r...@server.domain.tld dsync -u user


Dovecot 2.2.11 is running on both sides:


dsync-local(user): Debug: brain M: in state=master_recv_handshake
dsync-local(user): Debug: brain M: out state=master_recv_handshake changed=0
dsync-local(user): Debug: brain M: in state=master_recv_handshake
dsync-local(user): Debug: brain M: out state=send_mailbox_tree changed=1
dsync-local(user): Debug: brain M: in state=send_mailbox_tree
dsync-local(user): Debug: brain M: out state=send_mailbox_tree_deletes
changed=1
dsync-local(user): Debug: brain M: in state=send_mailbox_tree_deletes
dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=1
dsync-local(user): Debug: brain M: in state=recv_mailbox_tree
dsync-local(user): Debug: brain M: out state=recv_mailbox_tree changed=0
dsync-local(user): Debug: brain M: in state=recv_mailbox_tree
dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes
changed=1
dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes
dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes
changed=0
dsync-remote(user): Panic: file dsync-mailbox-tree-sync.c: line 401
(sync_rename_node_to_temp): assertion failed: (ctx-sync_type !=
DSYNC_MAILBOX_TREES_SYNC_TYPE_PRESERVE_LOCAL)
dsync-remote(user): Error: Raw backtrace:
/usr/local/lib/dovecot/libdovecot.so.0(+0x68aea) [0x7f616d58aaea] -
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32)
[0x7f616d58abf2] - /usr/local/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f616d54423f] - dsyn() [0x437c06] - dsyn() [0x438122] - dsyn()
[0x438494] - dsyn() [0x43821c] -
dsyn(dsync_mailbox_trees_sync_init+0xe6) [0x439766] -
dsyn(dsync_brain_recv_mailbox_tree_deletes+0x102) [0x42d602] -
dsyn(dsync_brain_run+0x2e6) [0x42afb6] - dsyn() [0x42b910] - dsyn()
[0x43db50] -
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
[0x7f616d59a666] -
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7)
[0x7f616d59b6d7] -
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f616d59a5d8] - dsyn() [0x4282f4] - dsyn() [0x411ca7] -
dsyn(doveadm_mail_try_run+0x238) [0x4120b8] - dsyn(main+0x3d1)
[0x41aaf1] - /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f616d1acd1d]
- dsyn() [0x411429]
dsync-local(user): Debug: brain M: in state=recv_mailbox_tree_deletes
dsync-local(user): Error: read(server.domain.tld) failed: EOF
dsync-local(user): Debug: brain M: out state=recv_mailbox_tree_deletes
changed=0
dsync-local(user): Error: Remote command returned error 25



Regards,
Gilles.

[Dovecot] dsync in a cronjob to replace offlineimap

[Guilhem Moulin]

Hi list,

Replacing offlineimap with dsync for IMAP-to-IMAP synchronization (using
dovecot 2.1.7) is working great, but now that I upgraded to 2.2.9 I'd
like to take advantage of the incremental replication [1].

I don't think having a separate replicator process  notifications suits
my configuration, as I want to sync my server with my laptop which is
not always on or might be behind a firewall.

After reading ‘replication/replicator/dsync-client.c’ I was thinking of
writing a small wrapper around dsync performing a daily full replication 
and updating the state otherwise.


The following works great in a cronjob

  doveadm sync -l30 imap.example.org


but when I try to get the current state

  doveadm sync -l30 -fs '' imap.example.org ~/mail/dsync.state

and later use incremental replication

  state=$( ~/mail/dsync.state) doveadm sync -l30 -s $state imap.example.org 
~/mail/dsync.state

the state seems to somehow quickly become stale and messages stop 
being retrieved from the server.

I understand that dsync's incremental replication is mostly intended to
be used by the replicator process, but I'm wondering whether it's also
possible to use it manually ;-) If so, how to keep a sane state?

Thanks,
cheers,
-- 
Guilhem.

[1] http://wiki2.dovecot.org/Replication


signature.asc
Description: Digital signature

[Dovecot] Dsync crash (v2.2.10, sdbox+sis → mbox)

[Gilles Chauvin]

Hi,

I'm trying to use dsync to convert sdbox + sis mailboxes to mbox (mbox
is chosen here to re-attach the attachments to their original place)


# dsync -Dv -u $LOGIN -o
mail_location=sdbox:/zfspool/clone_srv_mail/$LOGIN -o
mail_attachment_dir=/zfspool/clone_srv_attachments backup
mbox:/zfspool/restore/$LOGIN/mbox:DIRNAME=mBoX-MeSsAgEs:INDEX=/zfspool/restore/$LOGIN/indexes:CONTROL=/zfspool/restore/$LOGIN/control


For 5 users out of a sample of 24, here is what's happening:

dsync(user1): Error:
read(/zfspool/clone_srv_attachments/cb/0a/cb0aad465a4ff95bf6fa6ece0fba94b43e8892cf-19dc51309fc2f3527e3144c1c7e7-b55eb9176ca1b350e56594b229f9-30810[base64:19
b/l]) failed: Stream is larger than expected (163244  163243, eof=1)
dsync(user1): Error: copy: i_stream_read() failed: Invalid argument
dsync(user1): Panic: file mail-index-transaction-update.c: line 19
(mail_index_transaction_lookup): assertion failed: (seq =
t-first_new_seq  seq = t-last_new_seq)
dsync(user1): Error: Raw backtrace:
/usr/local/lib/dovecot/libdovecot.so.0(+0x6889a) [0x7f58a95a189a] -
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32)
[0x7f58a95a19a2] - /usr/local/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f58a955b1cf] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc0287)
[0x7f58a98ca287] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc3145)
[0x7f58a98cd145] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_decision_state_update+0xb6)
[0x7f58a98bcb06] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_lookup_headers+0x91)
[0x7f58a98be5e1] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xa0ac3)
[0x7f58a98aaac3] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0x4a)
[0x7f58a98ab04a] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c021)
[0x7f58a98a6021] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c151)
[0x7f58a98a6151] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_close+0xf5)
[0x7f58a98a6295] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x48)
[0x7f58a98867c8] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x92)
[0x7f58a9880e32] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f)
[0x7f58a9886c2f] - dsync() [0x42f750] -
dsync(dsync_brain_sync_mails+0x459) [0x42e9c9] -
dsync(dsync_brain_run+0x2a1) [0x42ac51] - dsync() [0x42876f] - dsync()
[0x411c97] - dsync(doveadm_mail_try_run+0x238) [0x4120a8] -
dsync(main+0x3d1) [0x41aaa1] - /lib64/libc.so.6(__libc_start_main+0xfd)
[0x7f58a91c3d1d] - dsync() [0x411419]


dsync(user2): Error:
read(/zfspool/clone_srv_attachments/ad/0c/ad0cef35cc6f0b2dae2197c4ff2b61a2bd58070d-9e8345192ccbf352c21044c1c7e7-6efa5f2e522db350ed3d94b229f9-15470[base64:18
b/l]) failed: Stream is larger than expected (194476  194475, eof=1)
dsync(user2): Error: copy: i_stream_read() failed: Invalid argument
dsync(user2): Panic: file mail-index-transaction-update.c: line 19
(mail_index_transaction_lookup): assertion failed: (seq =
t-first_new_seq  seq = t-last_new_seq)
dsync(user2): Error: Raw backtrace:
/usr/local/lib/dovecot/libdovecot.so.0(+0x6889a) [0x7f2e2248d89a] -
/usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32)
[0x7f2e2248d9a2] - /usr/local/lib/dovecot/libdovecot.so.0(i_error+0)
[0x7f2e224471cf] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc0287)
[0x7f2e227b6287] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xc3145)
[0x7f2e227b9145] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_decision_state_update+0xb6)
[0x7f2e227a8b06] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_cache_lookup_headers+0x91)
[0x7f2e227aa5e1] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xa0ac3)
[0x7f2e22796ac3] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0x4a)
[0x7f2e2279704a] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c021)
[0x7f2e22792021] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0x9c151)
[0x7f2e22792151] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_close+0xf5)
[0x7f2e22792295] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x48)
[0x7f2e227727c8] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x92)
[0x7f2e2276ce32] -
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5f)
[0x7f2e22772c2f] - dsync() [0x42f750] -
dsync(dsync_brain_sync_mails+0x459) [0x42e9c9] -
dsync(dsync_brain_run+0x2a1) [0x42ac51] - dsync() [0x42876f] - dsync()
[0x411c97] - dsync(doveadm_mail_try_run+0x238) [0x4120a8] -
dsync(main+0x3d1) [0x41aaa1] - /lib64/libc.so.6(__libc_start_main+0xfd)
[0x7f2e220afd1d] - dsync() [0x411419]


dsync(user3): Error:
read(/zfspool/clone_srv_attachments/23/8a/238a781b53bb4d1b1bee989a5ff38636b616d5c5-41ba47152912f4522c6f44c1c7e7-f3b06c2f5aa1b350d56594b229f9-38650[base64:19
b/l]) failed: Stream is larger than expected (33191  33190, eof=1)
dsync(user3): Error: copy: i_stream_read() failed: Invalid argument

Re: [Dovecot] Dsync Errors

[Timo Sirainen]

On 30.1.2014, at 11.06, Asai a...@globalchangemusic.org wrote:

 Maybe this error sheds some light on it:
 
 Panic: file dsync-brain-mailbox-tree-sync.c: line 111 
 (dsync_brain_mailbox_tree_sync_change): assertion failed: 
 (brain-no_backup_overwrite)

Fixed by http://hg.dovecot.org/dovecot-2.2/rev/fbc8fe46dfce

Re: [Dovecot] Dsync Errors

[Asai]

Timo, you're amazing.  Thank you.

--Asai

On 2/7/14 1:07 PM, Timo Sirainen wrote:

On 30.1.2014, at 11.06, Asai a...@globalchangemusic.org wrote:


Maybe this error sheds some light on it:

Panic: file dsync-brain-mailbox-tree-sync.c: line 111 
(dsync_brain_mailbox_tree_sync_change): assertion failed: 
(brain-no_backup_overwrite)

Fixed by http://hg.dovecot.org/dovecot-2.2/rev/fbc8fe46dfce

Re: [Dovecot] dsync backup; compressed to uncompressed

[Timo Sirainen]

On 29.1.2014, at 15.30, Michael Smith (DF) msm...@datafoundry.com wrote:

 Ok, I think I found an answer.  I don't know if it's the right one...
 
 I duplicated my dovecot.conf to backup.conf.  I then removed this part from 
 plugin {}
 zlib_save = gz
 zlib_save_level = 9
 
 I then add -c backup.conf to my dsync command.  This appears to allow dsync 
 to read the compressed mdbox accounts, but when it writes the backup in 
 maildir format, everything is uncompressed.

Or you could just use doveadm -o plugin/zlib_save= to override the setting.

[Dovecot] dsync Error: Mailbox INBOX: Save commit failed: Mailbox was deleted under us

[Alan McGinlay - SICS]

Hi,

I am doing a mass migration of users from Cyrus imap on a solaris 
server to dovecot 1:2.2.9-1ubuntu1 and am getting this weird issue with 
dsync if I try to do a full sync. Debug output below:


dsync(u...@example.com): Error: Mailbox INBOX: Save commit failed: 
Mailbox was deleted under us
dsync(u...@example.com): Debug: brain M: out box 'INBOX' 
recv_state=recv_last_common send_state=done changed=1

dsync(u...@example.com): Debug: brain M: out state=sync_mails changed=1
dsync(u...@example.com): Debug: brain S: in state=sync_mails
dsync(u...@example.com): Debug: brain S: in box 'INBOX' 
recv_state=recv_last_common send_state=done
dsync(u...@example.com): Debug: brain S: out box 'INBOX' 
recv_state=recv_last_common send_state=done changed=0

dsync(u...@example.com): Debug: brain S: out state=sync_mails changed=0
dsync(u...@example.com): Debug: imapc(10.10.10.10:143): Disconnected
[2]+  Killed  /usr/bin/dsync -v -o mail_fsync=never 
backup -R -f -u u...@example.com imapc:


(I changed the domain / ip here of course)

I notice that it does the IMAP sync for a while and then shows 
connecting info for pop3 (I have configuration for both imap and pop). 
The user in question has never connected with POP.


imap migration config:

imapc_host = 10.10.10.10
imapc_user = %u
imapc_master_user = cyrusadmin
imapc_password = blahblahblahpasswordhere
imapc_features = rfc822.size
imapc_features = $imapc_features #fetch-headers
mail_prefetch_count = 20

pop configuration:

pop3c_host = pop.example.com
pop3c_user = %u
pop3c_master_user = cyrusadmin
pop3c_password = blahblahblahpasswordhere

namespace {
  prefix = POP3-MIGRATION-NS/
  location = pop3c:
  list = no
  hidden = yes
}
protocol doveadm {
  mail_plugins = $mail_plugins pop3_migration
}
plugin {
  pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX
}

Thanks in advance for any pointers on this!

/Alan

Re: [Dovecot] Dsync Errors

[Asai]

Maybe this error sheds some light on it:

Panic: file dsync-brain-mailbox-tree-sync.c: line 111 
(dsync_brain_mailbox_tree_sync_change): assertion failed: 
(brain-no_backup_overwrite)

Re: [Dovecot] Dsync Errors

[Asai]

I see Timo addressed this problem with this reply ( from thread 
[Dovecot] reproducible dsync backup panic (dovecot 2.2.7)/Fri Dec 20 
00:37:21 EET 2013)/, but basically I don't understand how to go about 
doing this:


Can you get it to dump core and use

gdb /usr/bin/doveadm core
bt full
fr 5
p *change


/  The command I'm using to back up individual user's mail is something like:
//  
//  sudo -u vmail dsync -v -uuser at server.com  http://dovecot.org/cgi-bin/mailman/listinfo/dovecot  -R backup ssh -i sshid-filelogin at server.com  http://dovecot.org/cgi-bin/mailman/listinfo/dovecot  sudo -u vmail dsync -v -uuser at server.com  http://dovecot.org/cgi-bin/mailman/listinfo/dovecot

/
Probably an easier way would be if you could copy the local and remote mdboxes 
to e.g. /tmp/mdbox-local and /tmp/mdbox-remote. Then you should be able to do:

gdb --args doveadm -o mail=mdbox:/tmp/mdbox-local backup -R 
mdbox:/tmp/mdbox-remote
run
bt full
fr 5
p *change

Re: [Dovecot] Dsync Errors

[Asai]

The first thing to do when experiencing problems like this is make 
sure you are on the current version of whatever point release you are 
running. In your case that would be 2.2.10. Then if you still 
experience the problem, come back and re-ask...


OK, I upgraded to 2.2.10.  Same thing.  Is there some kind of debug 
routine I can run?

[Dovecot] dsync backup; compressed to uncompressed

[Michael Smith (DF)]

I need to backup some of the mailboxes on our system.  We are currently using 
mdbox with zlib compression plugin.

The backups need to be maildir without compression.  How can I accomplish this 
using dsync?

The following just causes each msg file in maildir to be compressed as well.
dsync -f -u user1 backup maildir:/var/tmp/user1

While this just causes the dsync to fail while spewing a bunch of errors 
because it can no longer read the compressed mdbox files.
dsync -f -u user1 -o mail_plugins= backup maildir:/var/tmp/user1


# doveconf -n
# 2.2.4: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)
auth_debug = yes
auth_debug_passwords = yes
auth_default_realm = domain1.net
auth_master_user_separator = *
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/dovecot-debug.log
disable_plaintext_auth = no
dotlock_use_excl = no
lda_mailbox_autocreate = yes
lock_method = dotlock
mail_access_groups = mail
mail_debug = yes
mail_fsync = always
mail_location = mdbox:~/mail:INDEX=~/index
mail_plugins = quota zlib
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date
mbox_read_locks = dotlock
mbox_write_locks = dotlock
mdbox_rotate_interval = 1 weeks
mdbox_rotate_size = 20 M
mmap_disable = yes
namespace {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql-master.conf.ext
  driver = sql
  master = yes
  pass = yes
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  autosubscribe = Trash
  autosubscribe2 = Junk
  autosubscribe3 = Drafts
  autosubscribe4 = Sent
  autosubscribe5 = INBOX
  mail_max_userip_connections = 10
  managesieve_max_line_length = 65536
  quota = dict:User quota::file:%h/dovecot.quota
  quota2_rule = Trash:storage=+10%%
  quota3_rule = Junk:storage=+20%%
  quota_rule = *:storage=100M:messages=10
  recipient_delimiter = +
  sieve_before = /var/opt/mail/global.sieve/
  zlib_save = gz
  zlib_save_level = 9
}
pop3_reuse_xuidl = yes
protocols = imap pop3 sieve lmtp
service auth {
  inet_listener auth {
port = 113
  }
  unix_listener auth-userdb {
user = nobody
  }
}
service lmtp {
  inet_listener lmtp {
port = 24
  }
  process_min_avail = 10
  service_count = 1
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  process_min_avail = 0
  service_count = 1
  vsz_limit = 64 M
}
ssl = no
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocol lda {
  mail_plugins = quota zlib sieve
}
protocol imap {
  mail_plugins = quota zlib imap_quota
}
protocol sieve {
  mail_max_userip_connections = 10
  mail_plugins = quota zlib
  managesieve_max_line_length = 65536
  managesieve_notify_capability = mailto
  managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date
}
protocol lmtp {
  mail_plugins = quota zlib sieve
}

Re: [Dovecot] dsync backup; compressed to uncompressed

[Michael Smith (DF)]

Ok, I think I found an answer.  I don't know if it's the right one...

I duplicated my dovecot.conf to backup.conf.  I then removed this part from 
plugin {}
  zlib_save = gz
  zlib_save_level = 9

I then add -c backup.conf to my dsync command.  This appears to allow dsync 
to read the compressed mdbox accounts, but when it writes the backup in maildir 
format, everything is uncompressed.


-Original Message-
From: dovecot-boun...@dovecot.org [mailto:dovecot-boun...@dovecot.org] On 
Behalf Of Michael Smith (DF)
Sent: Wednesday, January 29, 2014 12:04 PM
To: 'dovecot@dovecot.org'
Subject: [Dovecot] dsync backup; compressed to uncompressed

I need to backup some of the mailboxes on our system.  We are currently using 
mdbox with zlib compression plugin.

The backups need to be maildir without compression.  How can I accomplish this 
using dsync?

The following just causes each msg file in maildir to be compressed as well.
dsync -f -u user1 backup maildir:/var/tmp/user1

While this just causes the dsync to fail while spewing a bunch of errors 
because it can no longer read the compressed mdbox files.
dsync -f -u user1 -o mail_plugins= backup maildir:/var/tmp/user1


# doveconf -n
# 2.2.4: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)
auth_debug = yes
auth_debug_passwords = yes
auth_default_realm = domain1.net
auth_master_user_separator = *
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/dovecot-debug.log
disable_plaintext_auth = no
dotlock_use_excl = no
lda_mailbox_autocreate = yes
lock_method = dotlock
mail_access_groups = mail
mail_debug = yes
mail_fsync = always
mail_location = mdbox:~/mail:INDEX=~/index
mail_plugins = quota zlib
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date
mbox_read_locks = dotlock
mbox_write_locks = dotlock
mdbox_rotate_interval = 1 weeks
mdbox_rotate_size = 20 M
mmap_disable = yes
namespace {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql-master.conf.ext
  driver = sql
  master = yes
  pass = yes
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  autosubscribe = Trash
  autosubscribe2 = Junk
  autosubscribe3 = Drafts
  autosubscribe4 = Sent
  autosubscribe5 = INBOX
  mail_max_userip_connections = 10
  managesieve_max_line_length = 65536
  quota = dict:User quota::file:%h/dovecot.quota
  quota2_rule = Trash:storage=+10%%
  quota3_rule = Junk:storage=+20%%
  quota_rule = *:storage=100M:messages=10
  recipient_delimiter = +
  sieve_before = /var/opt/mail/global.sieve/
  zlib_save = gz
  zlib_save_level = 9
}
pop3_reuse_xuidl = yes
protocols = imap pop3 sieve lmtp
service auth {
  inet_listener auth {
port = 113
  }
  unix_listener auth-userdb {
user = nobody
  }
}
service lmtp {
  inet_listener lmtp {
port = 24
  }
  process_min_avail = 10
  service_count = 1
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  process_min_avail = 0
  service_count = 1
  vsz_limit = 64 M
}
ssl = no
ssl_cert = /etc/pki/dovecot/certs/dovecot.pem
ssl_key = /etc/pki/dovecot/private/dovecot.pem
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocol lda {
  mail_plugins = quota zlib sieve
}
protocol imap {
  mail_plugins = quota zlib imap_quota
}
protocol sieve {
  mail_max_userip_connections = 10
  mail_plugins = quota zlib
  managesieve_max_line_length = 65536
  managesieve_notify_capability = mailto
  managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date
}
protocol lmtp {
  mail_plugins = quota zlib sieve
}

[Dovecot] dsync failed

[Mario Arnold]

Hello,

Since i updated to 2.2.10 there is a problem with dsync:

dsync -v -u t...@vtlx.de backup $SSH -i $KEY -p $PORT $RHOST dsync -u 
t...@vtlx.de
dsync-remote(t...@vtlx.de): Error: Mailbox INBOX: Failed to set attribute
vendor/vendor.dovecot/pvt/server/sieve/files/t1: Internal attributes cannot be
changed directly

doveconf -n
# 2.2.10 (6018854c8c91): /etc/dovecot/dovecot.conf
# OS: Linux 3.12.6-domU i686 Debian jessie/sid ext3
...
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
...
plugin {
  ...
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
...
protocols =  imap lmtp sieve pop3

Do I need to reconfigure something?

Thank you for any hints

--
中華人民共和國

Re: [Dovecot] Dsync Errors

[Charles Marcus]

On 2014-01-24 10:51 AM, Asai a...@globalchangemusic.org wrote:

We're running Dovecot 2.2.4 and the dsync command is this:
dsync -u username backup 
maildir:/mnt/backups/period/domain/username


When this happens to a particular users account I delete the backup 
and let it rebuild, which works for awhile, but then it happens again, 
and it seems to happen to particular users.


Can you point me in the right direction to start troubleshooting this?


The first thing to do when experiencing problems like this is make sure 
you are on the current version of whatever point release you are 
running. In your case that would be 2.2.10. Then if you still experience 
the problem, come back and re-ask...


If you are unable to update due to some kind of 'LTS' restrictions 
enforced by your chosen OS, then your first line of support should be 
from them - otherwise, what is the point of using that OS?


Also, they would be the ones that would have to back-port any fixes from 
more recent releases to your 'stable' version.


--

Best regards,

Charles

[Dovecot] Dsync Errors

[Asai]

Greetings,

I've seen something like this in another thread, about a month ago, but we're 
running into this error pretty frequently when we run dsync backup.

Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x39ccc5f1c0] - 
/usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x39ccc5f2a5] - 
/usr/lib64/dovecot/libdovecot.so.0 [0x39ccc5ebb3] - dsync(dsync_brain_mailbox_tree_sync_change+0x41c) 
[0x42cc0c] - dsync(dsync_brain_recv_mailbox_tree_deletes+0x10a) [0x42be7a] - 
dsync(dsync_brain_run+0x4fc) [0x429cdc] - dsync [0x42885b] - dsync [0x4119ff] - 
dsync(doveadm_mail_try_run+0x269) [0x411e09] - dsync(main+0x358) [0x41a2c8] - 
/lib64/libc.so.6(__libc_start_main+0xf4) [0x3f4281d994] - dsync [0x4111c9]

We're running Dovecot 2.2.4 and the dsync command is this:
dsync -u username backup maildir:/mnt/backups/period/domain/username

When this happens to a particular users account I delete the backup and let it 
rebuild, which works for awhile, but then it happens again, and it seems to 
happen to particular users.

Can you point me in the right direction to start troubleshooting this?

Thanks.

--
--Asai

Re: [Dovecot] dsync mbox to maildir migration does not delete, deleted e-mails

[A M]

I think I have found solution.

CONVERTING section of dsync man page suggests to use 'mirror'
command.


But I think, right command is 'backup' and not 'mirror'.

With 'backup' as command, I found that it copies only new e-mails and
also deletes DELETED messages (which is exactly what I wanted)

'mirror' is for 2 way syncing but for CONVERTING, in most cases,
one needs 1 way syncing (backup)


Possibly the dsync man page also needs this change.


Hope it helps others.

Regards,

A M




 From: A M ammdispose-dove...@yahoo.com
To: dovecot@dovecot.org dovecot@dovecot.org 
Sent: Wednesday, 15 January 2014 10:38 AM
Subject: [Dovecot] dsync mbox to maildir migration does not delete, deleted 
e-mails
 

Hello,

I am planning to migrate all users from mbox to maildir.

I am trying to do it with minimum downtime. (~100GB data)

All users are currently using POP3.

Dovecot version is 2.2.10 (latest).
Command used is:
dsync -u username mirror maildir:~/Maildir


Process I plan is: (omitting steps related to sendmail / procmail)


1) keep dovecot running (with mail_location as mbox)
2) dsync for all users (this may take 3-4 hours or more)

3) (downtime starts) stop dovecot and sendmail (to stop new e-mails)

4) block pop3, imap ports on firewall (so users can not connect)
5) start dovecot (still with mbox)

6) dsync again to sync e-mails arrived between step 2 and 3

7) dsync again (just to make sure!)
8) (downtime ends) restart dovecot (with mail_location as maildir)


Now, here is my problem.

Lets say there is user joe, who has 50 NEW e-mails in mbox (INBOX).


Step 2 perfectly syncs his 50 e-mails to 'new' folder of maildir.


Now in the mean time, before step 3, he connected via POP3 and
downloaded and deleted 50 e-mails.

Now when we reach step 6 (re-sync), what I expected was dsync will
detect that 50 e-mails are deleted and it will delete 50 e-mails from
'new' directory of maildir.


But that is not happening. 50 e-mails are still there.

I fear that these e-mails will be re-downloaded on his Outlook once
I switch dovecot to maildir.

This will happen for each and every user which will cause huge mess.


So how to tell, dsync to delete non-existent e-mails which are no more
there in mbox (INBOX)?

Thanks in advance,

Regards,

A M

[Dovecot] dsync mbox to maildir migration does not delete, deleted e-mails

[A M]

Hello,

I am planning to migrate all users from mbox to maildir.

I am trying to do it with minimum downtime. (~100GB data)

All users are currently using POP3.

Dovecot version is 2.2.10 (latest).
Command used is:
dsync -u username mirror maildir:~/Maildir


Process I plan is: (omitting steps related to sendmail / procmail)


1) keep dovecot running (with mail_location as mbox)
2) dsync for all users (this may take 3-4 hours or more)

3) (downtime starts) stop dovecot and sendmail (to stop new e-mails)

4) block pop3, imap ports on firewall (so users can not connect)
5) start dovecot (still with mbox)

6) dsync again to sync e-mails arrived between step 2 and 3

7) dsync again (just to make sure!)
8) (downtime ends) restart dovecot (with mail_location as maildir)


Now, here is my problem.

Lets say there is user joe, who has 50 NEW e-mails in mbox (INBOX).


Step 2 perfectly syncs his 50 e-mails to 'new' folder of maildir.


Now in the mean time, before step 3, he connected via POP3 and
downloaded and deleted 50 e-mails.

Now when we reach step 6 (re-sync), what I expected was dsync will
detect that 50 e-mails are deleted and it will delete 50 e-mails from
'new' directory of maildir.


But that is not happening. 50 e-mails are still there.

I fear that these e-mails will be re-downloaded on his Outlook once
I switch dovecot to maildir.

This will happen for each and every user which will cause huge mess.


So how to tell, dsync to delete non-existent e-mails which are no more
there in mbox (INBOX)?

Thanks in advance,

Regards,

A M

[Dovecot] dsync error: gz trailer has wrong CRC value

[Joe Beaubien]

A few days ago by dovecot installation started behaving weirdly. First it
was doveadm fts optimize which would fail (a thread running about that) and
now it's dsync, which means my backups are not working.

This all started a few days ago, which I find a little weird, because I
haven't messed around with the dovecot setup in months. I've been searching
the internet, but didn't find anything useful yet.

Would anyone know how to fix the errors in the log below?

Thanks,

-Joe

dovecot version is 2.1.7

Here is the log:

dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): gz trailer has
wrong CRC value at 16966298
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Error: read(msg input) failed: Invalid argument
dsync(archive): Error:
zlib.read(/mba_data/emails/archive/storage/m.1885): missing gz header
at 16973513
dsync(archive): Panic: file mail-index-transaction-update.c: line 906
(mail_index_update_ext): assertion failed: (seq  0  (seq =
mail_index_view_get_messages_count(t-view) || seq =
t-last_new_seq))
dsync(archive): Error: Raw backtrace:
/opt/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x4203a)
[0x7f3a5150503a] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a)
[0x7f3a515050fa] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f3a514dc3f0] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(mail_index_update_ext+0x1c7)
[0x7f3a51804927] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(+0xa8746)
[0x7f3a517f7746] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(mail_cache_transaction_commit+0x4c)
[0x7f3a517f7a2c] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(+0xa8af3)
[0x7f3a517f7af3] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(+0xa2cea)
[0x7f3a517f1cea] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x84)
[0x7f3a51800754] -
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x8a)
[0x7f3a517f209a] - /opt/dovec
 ot/lib/d
 ovecot/lib20_zlib_plugin.so(+0x3b7a) [0x7f3a4fe8cb7a] -
/opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xb481) [0x7f3a5009b481]
- 
/opt/dovecot-2.1.7/lib/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x3d)
[0x7f3a517c9c4d] - /opt/dovecot/bin/dsync() [0x4297e6] -
/opt/dovecot/bin/dsync() [0x42b222] -
/opt/dovecot/bin/dsync(dsync_worker_select_mailbox+0x2e) [0x4291be] -
/opt/dovecot/bin/dsync() [0x424c75] - /opt/dovecot/bin/dsync()
[0x424db7] - /opt/dovecot/bin/dsync(dsync_brain_msg_sync_more+0x22c)
[0x4245ec] - /opt/dovecot/bin/dsync(dsync_brain_sync+0x459)
[0x422fc9] - /opt/dovecot/bin/dsync() [0x423e61] -
/opt/dovecot/bin/dsync(dsync_brain_sync+0x7f1) [0x423361] -
/opt/dovecot/bin/dsync() [0x423cdc] -
/opt/dovecot/bin/dsync(dsync_brain_sync+0x832) [0x4233a2] -
/opt/dovecot/bin/dsync(dsync_brain_sync_all+0x18) [0x423ee8] -
/opt/dovecot/bin/dsync() [0x420f36] - /opt/dovecot/bin/dsync()
[0x4215b2] - /opt/dovecot/bin/dsync() [0x40e798]

Re: [Dovecot] dsync-local crashes due to assertion failures

[Fabian Groffen]

On 08-12-2013 16:26:15 +0100, Fabian Groffen wrote:
 I have two servers that share nothing, running dovecot version 2.2.9.
 The first server hosts the original mail base, which I want to sync
 using dovecot to the second server.  With some trial and error I managed
 to get synchronisation running using tcp sockets.  Now for some accounts
 I see dsync-local crashing on the server that has the original mail base
 (which is still the only of the two that receives new mail).  The log
 shows:
 
 Dec  8 03:13:14 zeus dovecot: dsync-local(someuser): Panic: file mail-index-tr
 ansaction-export.c: line 203 (log_append_ext_hdr_update): assertion failed: 
 (u32
 .offset + u32.size = ext_hdr_size)

Turns out I didn't remove all index files.  After I did (dovecot.index +
dovecot.list.index) the assertion has gone away.  Interesting side-note:
after removal of the indices, some of my mailboxes showed new emails
from long ago, that I hadn't seen previously.
Now, from the first look and sizes, synchronisation seems to have
succeeded after a night.


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-13 01:10, Joseph Tam wrote:

Alan McGinlay wrote:

Can't believe I'm the only one with this error, googled it and there 
is

nothing. Sorry to whine but I am getting desperate here!

I googled safe_mkstemp permission denied and first hit is this 
thread


dovecot.org/list/dovecot/2010-August/052319.html


Hah! thank you so much, I have been pretty stressed the past week or so 
and of course I googled it, my brain must have been mushed however :)




which mentions this error coming about from chrooting users.  Looking 
back

on your post, I see

http://www.dovecot.org/list/dovecot/2013-December/093900.html

mail_chroot = /var/vmail


It makes immediate sense when you mention the chroot! Annoying that 
dsync and the chroot wont work together however




As long as your desperate, maybe try leaving this setting empty?


Yep, this worked once I changed the maildir location to include 
/var/vmail



(This doesn't explain why it's not deterministic though.)   Try
process traces -- I do when I'm stumped.

Joseph Tam jtam.h...@gmail.com


Thanks! This is a really helpful mailing list!

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-10 14:25, Alan McGinlay - SICS wrote:

On 2013-12-10 14:09, Alan McGinlay - SICS wrote:
On 2013-12-10 11:52, Alan McGinlay - SICS wrote:
On 2013-12-10 11:34, Robert Schetterer wrote:
Am 10.12.2013 11:25, schrieb Alan McGinlay - SICS:
On 2013-12-09 11:21, Alan McGinlay - SICS wrote:
On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail)
failed: No such file or directory (in mail_privileged_group setting)

You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

After much trial and error and following Timos advice, I managed to
get a sync to at least start and it lists folders, then it starts
spamming this:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any
reference to it online or in the docs,

Any ideas?


Still not able to get anywhere with this :( It really feels like a
permissions problem, either with the master user, the unix user i 
start

the dsync with (root) or the user that dsync runs as (vmail). I tried
looking at the code for safe_mkstemp but still couldn't work out the
source of this problem. If anyone has an idea it would be great to 
hear it!


perhaps check
the dsync target directory must be writable by vmail:vmail


Best Regards
MfG Robert Schetterer

Thanks, vmail:vmail owns all mail and parent directories up to
/var/vmail/. I tried changing mail_temp_dir in 10-mail.conf from /tmp/
to /var/vmail/tmp (and i created that directory) but it made no
difference apart from changing the directory in the safe_mkstemp
error message. If I comment out mail_temp_dir then the error changes
to:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied
dsync(alantestu...@whatever.com): Error:
stat(/tmp/dovecot.doveadm.3c303c239d223495) failed: Permission denied

Interestingly, the synchronization actually does seem to work! I
hadn't noticed at first but in spite of the error, mails are synced
across and seemingly are completely intact!

It would be really good to find out the source of this error though!

/A

Another update, only about 1900 of 25000~ mails are actually copied :/


Can't believe I'm the only one with this error, googled it and there is 
nothing. Sorry to whine but I am getting desperate here!


I have upgraded to dovecot 2.1.7 but am still getting this error when 
performing a dsync:


Error: safe_mkstemp(/var/vmail/tmp/dovecot.doveadm.) failed: No such 
file or directory


It seems to fire that error only on some mails being synced but it's 
apparently random. If there are a lot of new mails then it gives that 
error a lot, if only one or two mails have come in since the last sync 
then it might give that error for both, one or none of them.


If i change the dsync command to mirror instead of backup then the 
output becomes:


Error: safe_mkstemp(/var/vmail/tmp/dovecot.doveadm.) failed: No such 
file or directory

Error: Couldn't create temp file
Error: Can't save message to mailbox DNS: Internal error occurred. 
Refer to server log for more information. [2013-12-12 13:16:46]

Error: msg-get failed: box=Junk uid=87595 guid=
Error: msg-get failed: box=Junk uid=87596 guid=
Error: msg-get failed: box=Junk uid=87597 guid=
Error: msg-get failed: box=Junk uid=87598 guid=
Error: msg-get failed: box=Junk uid=87599 guid=
Error: msg-get failed: box=Drafts uid=1339 guid=
Warning: Mailbox changes caused a desync. You may want to run dsync 
again.



syslog, mail.log and mail.err contain nothing except the master user 
logging in / out and no errors or warnings. Debug is enabled in 
10-logging.conf.


Please help!

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Joseph Tam]

Alan McGinlay wrote:


Can't believe I'm the only one with this error, googled it and there is
nothing. Sorry to whine but I am getting desperate here!


I googled safe_mkstemp permission denied and first hit is this thread

dovecot.org/list/dovecot/2010-August/052319.html

which mentions this error coming about from chrooting users.  Looking back
on your post, I see

http://www.dovecot.org/list/dovecot/2013-December/093900.html

mail_chroot = /var/vmail

As long as your desperate, maybe try leaving this setting empty?
(This doesn't explain why it's not deterministic though.)   Try
process traces -- I do when I'm stumped.

Joseph Tam jtam.h...@gmail.com

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-09 11:21, Alan McGinlay - SICS wrote:

On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user 
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail) 
failed: No such file or directory (in mail_privileged_group setting)


You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

After much trial and error and following Timos advice, I managed to
get a sync to at least start and it lists folders, then it starts
spamming this:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any
reference to it online or in the docs,

Any ideas?



Still not able to get anywhere with this :( It really feels like a 
permissions problem, either with the master user, the unix user i start 
the dsync with (root) or the user that dsync runs as (vmail). I tried 
looking at the code for safe_mkstemp but still couldn't work out the 
source of this problem. If anyone has an idea it would be great to hear 
it!

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Robert Schetterer]

Am 10.12.2013 11:25, schrieb Alan McGinlay - SICS:
 On 2013-12-09 11:21, Alan McGinlay - SICS wrote:
 On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
 Actually I do, /var/vmail (contains virtual domain mailboxes) is
 owned by vmail:vmail

 On 2013-12-08 21:49, Timo Sirainen wrote:
 On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

 mail_privileged_group = vmail
 ..
 mail_location = maildir:~/Maildir
 ..
 dsync(alantestu...@whatever.com): Error: user
 alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail)
 failed: No such file or directory (in mail_privileged_group setting)

 You don’t have vmail group in your system? Either create it or remove
 this setting. Most likely you want to remove it, since this setting
 was meant only for mbox format, while you’re using maildir.

 After much trial and error and following Timos advice, I managed to
 get a sync to at least start and it lists folders, then it starts
 spamming this:

 dsync(alantestu...@whatever.com): Error:
 safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
 dsync(alantestu...@whatever.com): Error:
 safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
 dsync(alantestu...@whatever.com): Error:
 safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
 ...

 /tmp/dovecot.doveadm. does indeed not exist but I can't find any
 reference to it online or in the docs,

 Any ideas?

 
 Still not able to get anywhere with this :( It really feels like a
 permissions problem, either with the master user, the unix user i start
 the dsync with (root) or the user that dsync runs as (vmail). I tried
 looking at the code for safe_mkstemp but still couldn't work out the
 source of this problem. If anyone has an idea it would be great to hear it!

perhaps check
the dsync target directory must be writable by vmail:vmail


Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-10 11:34, Robert Schetterer wrote:

Am 10.12.2013 11:25, schrieb Alan McGinlay - SICS:
On 2013-12-09 11:21, Alan McGinlay - SICS wrote:
On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail)
failed: No such file or directory (in mail_privileged_group setting)

You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

After much trial and error and following Timos advice, I managed to
get a sync to at least start and it lists folders, then it starts
spamming this:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any
reference to it online or in the docs,

Any ideas?


Still not able to get anywhere with this :( It really feels like a
permissions problem, either with the master user, the unix user i 
start

the dsync with (root) or the user that dsync runs as (vmail). I tried
looking at the code for safe_mkstemp but still couldn't work out the
source of this problem. If anyone has an idea it would be great to 
hear it!


perhaps check
the dsync target directory must be writable by vmail:vmail


Best Regards
MfG Robert Schetterer


Thanks, vmail:vmail owns all mail and parent directories up to 
/var/vmail/. I tried changing mail_temp_dir in 10-mail.conf from /tmp/ 
to /var/vmail/tmp (and i created that directory) but it made no 
difference apart from changing the directory in the safe_mkstemp error 
message. If I comment out mail_temp_dir then the error changes to:


dsync(alantestu...@whatever.com): Error: 
safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied
dsync(alantestu...@whatever.com): Error: 
stat(/tmp/dovecot.doveadm.3c303c239d223495) failed: Permission denied

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-10 11:52, Alan McGinlay - SICS wrote:

On 2013-12-10 11:34, Robert Schetterer wrote:
Am 10.12.2013 11:25, schrieb Alan McGinlay - SICS:
On 2013-12-09 11:21, Alan McGinlay - SICS wrote:
On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail)
failed: No such file or directory (in mail_privileged_group setting)

You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

After much trial and error and following Timos advice, I managed to
get a sync to at least start and it lists folders, then it starts
spamming this:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any
reference to it online or in the docs,

Any ideas?


Still not able to get anywhere with this :( It really feels like a
permissions problem, either with the master user, the unix user i 
start

the dsync with (root) or the user that dsync runs as (vmail). I tried
looking at the code for safe_mkstemp but still couldn't work out the
source of this problem. If anyone has an idea it would be great to 
hear it!


perhaps check
the dsync target directory must be writable by vmail:vmail


Best Regards
MfG Robert Schetterer

Thanks, vmail:vmail owns all mail and parent directories up to
/var/vmail/. I tried changing mail_temp_dir in 10-mail.conf from /tmp/
to /var/vmail/tmp (and i created that directory) but it made no
difference apart from changing the directory in the safe_mkstemp
error message. If I comment out mail_temp_dir then the error changes
to:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied
dsync(alantestu...@whatever.com): Error:
stat(/tmp/dovecot.doveadm.3c303c239d223495) failed: Permission denied


Interestingly, the synchronization actually does seem to work! I hadn't 
noticed at first but in spite of the error, mails are synced across and 
seemingly are completely intact!


It would be really good to find out the source of this error though!

/A

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-10 14:09, Alan McGinlay - SICS wrote:

On 2013-12-10 11:52, Alan McGinlay - SICS wrote:
On 2013-12-10 11:34, Robert Schetterer wrote:
Am 10.12.2013 11:25, schrieb Alan McGinlay - SICS:
On 2013-12-09 11:21, Alan McGinlay - SICS wrote:
On 2013-12-08 22:08, Alan McGinlay - SICS wrote:
Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail)
failed: No such file or directory (in mail_privileged_group setting)

You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

After much trial and error and following Timos advice, I managed to
get a sync to at least start and it lists folders, then it starts
spamming this:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any
reference to it online or in the docs,

Any ideas?


Still not able to get anywhere with this :( It really feels like a
permissions problem, either with the master user, the unix user i 
start

the dsync with (root) or the user that dsync runs as (vmail). I tried
looking at the code for safe_mkstemp but still couldn't work out the
source of this problem. If anyone has an idea it would be great to 
hear it!


perhaps check
the dsync target directory must be writable by vmail:vmail


Best Regards
MfG Robert Schetterer

Thanks, vmail:vmail owns all mail and parent directories up to
/var/vmail/. I tried changing mail_temp_dir in 10-mail.conf from /tmp/
to /var/vmail/tmp (and i created that directory) but it made no
difference apart from changing the directory in the safe_mkstemp
error message. If I comment out mail_temp_dir then the error changes
to:

dsync(alantestu...@whatever.com): Error:
safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied
dsync(alantestu...@whatever.com): Error:
stat(/tmp/dovecot.doveadm.3c303c239d223495) failed: Permission denied

Interestingly, the synchronization actually does seem to work! I
hadn't noticed at first but in spite of the error, mails are synced
across and seemingly are completely intact!

It would be really good to find out the source of this error though!

/A


Another update, only about 1900 of 25000~ mails are actually copied :/

[Dovecot] dsync verbosity, summary of transfer

[Alan McGinlay - SICS]

Hi!

I am playing with dsync and trying to fix an issue I have mentioned in 
another thread (subject: Re: [Dovecot] Dsync error: Couldn't drop 
privileges: getgrnam) and feel that dsync could use some additional 
informational output. It would be particularly helpful while 
experimenting with a migration if it could output a transfer summary and 
/ or log separately.


A summary could be similar to that which imapsync outputs after 
completion, for example:


 Statistics
Transfer started on   : Thu Dec  5 20:30:05 2013
Transfer ended on : Thu Dec  5 20:32:00 2013
Transfer time : 114.3 sec
Messages transferred  : 7
Messages skipped  : 766
Messages found duplicate on host1 : 0
Messages found duplicate on host2 : 0
Messages void (noheader) on host1 : 0
Messages void (noheader) on host2 : 0
Messages deleted on host1 : 0
Messages deleted on host2 : 9
Total bytes transferred   : 71820 (70.137 KiB)
Total bytes duplicate host1   : 0 (0.000 KiB)
Total bytes duplicate host2   : 0 (0.000 KiB)
Total bytes skipped   : 10070561 (9.604 MiB)
Total bytes error : 0 (0.000 KiB)
Message rate  : 0.1 messages/s
Average bandwidth rate: 0.6 KiB/s
Reconnections to host1: 0
Reconnections to host2: 0
Memory consumption: 90.5 MiB
Biggest message   : 39274 bytes
Detected 0 errors

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

On 2013-12-08 22:08, Alan McGinlay - SICS wrote:

Actually I do, /var/vmail (contains virtual domain mailboxes) is
owned by vmail:vmail

On 2013-12-08 21:49, Timo Sirainen wrote:
On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user 
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail) 
failed: No such file or directory (in mail_privileged_group setting)


You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.


After much trial and error and following Timos advice, I managed to get 
a sync to at least start and it lists folders, then it starts spamming 
this:


dsync(alantestu...@whatever.com): Error: 
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error: 
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory
dsync(alantestu...@whatever.com): Error: 
safe_mkstemp(/tmp/dovecot.doveadm.) failed: No such file or directory

...

/tmp/dovecot.doveadm. does indeed not exist but I can't find any 
reference to it online or in the docs,


Any ideas?

PS: Sorry for previous top-post, I was using webmail on my phone!

[Dovecot] dsync-local crashes due to assertion failures

[Fabian Groffen]

Hi,

I have two servers that share nothing, running dovecot version 2.2.9.
The first server hosts the original mail base, which I want to sync
using dovecot to the second server.  With some trial and error I managed
to get synchronisation running using tcp sockets.  Now for some accounts
I see dsync-local crashing on the server that has the original mail base
(which is still the only of the two that receives new mail).  The log
shows:

Dec  8 03:13:14 zeus dovecot: dsync-local(someuser): Panic: file mail-index-tr
ansaction-export.c: line 203 (log_append_ext_hdr_update): assertion failed: (u32
.offset + u32.size = ext_hdr_size)
Dec  8 03:13:14 zeus dovecot: dsync-local(someuser): Error: Raw backtrace: 
/usr/lib64/dovecot/libdovecot.so.0(+0x70ea0) [0x7f3ee4137ea0] - 
/usr/lib64/dovecot/libdovecot.so.0(+0x70efe) [0x7f3ee4137efe] - 
/usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3ee40ebd2c] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_export+0xbed) 
[0x7f3ee442e89d] - /usr/lib64/dovecot/libdovecot-storage.so.0(+0x89104) 
[0x7f3ee442d104] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0xb2)
 [0x7f3ee442d592] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit+0xc) 
[0x7f3ee442d63c] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0xb6) 
[0x7f3ee44372b6] - /usr/lib64/dovecot/libdovecot-storage.so.0(+0x335d6) 
[0x7f3ee43d75d6] - /usr/lib64/dovecot/libdovecot-storage.so.0(+0x328bc) 
[0x7f3ee43d68bc] - /usr/lib64/dovecot/libdovecot-storage.so.0(+0x32d83) 
[0x7f3ee43d6d83] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xd9) 
[0x7f3ee43d7199] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x2e) 
[0x7f3ee43e7f2e] - 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f3ee43e8047] 
- dovecot/doveadm-server(dsync_brain_master_send_mailbox+0xcf) [0x41f17f] - 
dovecot/doveadm-server(dsync_brain_run+0x2c8) [0x41d738] - 
dovecot/doveadm-server() [0x41dd70] - dovecot/doveadm-server() [0x42eda0] - 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f3ee41495a6] - 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xbd) [0x7f3ee414a62d] 
- /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3ee4149008] - 
dovecot/doveadm-server() [0x41bb6c] - dovecot/doveadm-server() [0x40ee3b] - 
dovecot/doveadm-server() [0x419581] - 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36)
 [0x7f3ee41495a6] - 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xbd) [0x7f3ee414a62d] 
- /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3ee4149008] - 
/usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3ee40f1643]
Dec  8 03:13:14 zeus dovecot: dsync-local(someuser): Fatal: master: 
service(doveadm): child 14116 killed with signal 6 (core dumps disabled)

It seems that despite these crashes, some data is being transferred.
Newer accounts seem not to have problems, this happens with older
accounts only, for as far as I can see.  I tried removing the index
files, but that didn't seem to help.

Any ideas on what I can do here?  Both servers are configured to use
maildir storage.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Timo Sirainen]

On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

 mail_privileged_group = vmail
..
 mail_location = maildir:~/Maildir
..
 dsync(alantestu...@whatever.com): Error: user alantestu...@whatever.com: 
 Couldn't drop privileges: getgrnam(vmail) failed: No such file or directory 
 (in mail_privileged_group setting)

You don’t have vmail group in your system? Either create it or remove this 
setting. Most likely you want to remove it, since this setting was meant only 
for mbox format, while you’re using maildir.

Re: [Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

Actually I do, /var/vmail (contains virtual domain mailboxes) is owned 
by vmail:vmail


On 2013-12-08 21:49, Timo Sirainen wrote:

On 5.12.2013, at 22.18, Alan McGinlay - SICS al...@sics.se wrote:

mail_privileged_group = vmail
..
mail_location = maildir:~/Maildir
..
dsync(alantestu...@whatever.com): Error: user 
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail) 
failed: No such file or directory (in mail_privileged_group setting)


You don’t have vmail group in your system? Either create it or remove
this setting. Most likely you want to remove it, since this setting
was meant only for mbox format, while you’re using maildir.

[Dovecot] Dsync error: Couldn't drop privileges: getgrnam

[Alan McGinlay - SICS]

Hi,

I upgraded (in place upgrade, preserving my dovecot configs) to a newer 
release of ubuntu in order to gain access to slightly newer Dovecot 
release and be able to use Dsync for migration from Cyrus, here is my 
doveconf -n output:


# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.5.0-44-generic x86_64 Ubuntu 12.10
auth_debug = yes
auth_default_realm = whatever.com
auth_master_user_separator = *
auth_socket_path = /var/run/dovecot/auth-master
imapc_features = rfc822.size
imapc_host = oldmail.whatever.com
imapc_master_user = cyradmin
imapc_password = password hidden
mail_chroot = /var/vmail
mail_debug = yes
mail_gid = 5000
mail_location = maildir:~/Maildir
mail_prefetch_count = 20
mail_privileged_group = vmail
mail_uid = 5000
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave

passdb {
  args = /etc/dovecot/master-users
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols =  imap lmtp sieve pop3
service auth-worker {
  user = $default_internal_user
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
mode = 0666
  }
  unix_listener auth-userdb {
group = dovecot
mode = 0600
user = dovecot
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
ssl_cert = cert
ssl_key = cert
userdb {
  args = uid=vmail gid=vmail home=/domain/%1d/%d/%1n/%n 
allow_all_users=yes

  driver = static
}
protocol lmtp {
  postmaster_address = p...@whatever.com
}
protocol imap {
  mail_max_userip_connections = 10
}
protocol pop3 {
  pop3_uidl_format = %v.%u
}

And the command I am using to (attempt) a mailbox sync:

doveadm -o mail_fsync=never backup -R -u alantestu...@whatever.com 
imapc:


Produces this output:

dsync(alantestu...@whatever.com): Error: user 
alantestu...@whatever.com: Couldn't drop privileges: getgrnam(vmail) 
failed: No such file or directory (in mail_privileged_group setting)

dsync(alantestu...@whatever.com): Fatal: User init failed

mail.log shows this:

Dec  5 21:10:54 newmailserver dovecot: auth: Debug: Loading modules 
from directory: /usr/lib/dovecot/modules/auth
Dec  5 21:10:54 newmailserver dovecot: auth: Debug: Loading modules 
from directory: /usr/lib/dovecot/modules/auth
Dec  5 21:10:54 newmailserver dovecot: auth: Debug: Module loaded: 
/usr/lib/dovecot/modules/auth/libauthdb_ldap.so
Dec  5 21:10:54 newmailserver dovecot: auth: Debug: passwd-file 
/etc/dovecot/master-users: Read 1 users in 0 secs
Dec  5 21:10:54 newmailserver dovecot: auth: Debug: master in: 
USER#0111#011alantestu...@whatever.com#011service=doveadm
Dec  5 21:10:54 newmailserver dovecot: auth: Debug: master out: 
USER#0111#011alantestu...@whatever.com#011uid=5000#011gid=5000#011home=/domain/w/whatever.com/a/alantestuser


I followed this doc (which could use a little fleshing out, I will be 
happy to do it once I get a grasp on this!): 
http://wiki2.dovecot.org/Migration/Dsync


Thanks in advance!

/Alan

[Dovecot] Dsync on a medium sized site

[Skeffling]

Hello,

We're looking at changing our current Dovecot setup to use dsync between
2 servers in different data centres to share the load and to provide
resilience. We're quite excited about the possibilities!

We receive about 100,000 emails a day, to about 10,000 mailboxes - a mix
of IMAP and POP3.

We'll use Dovecot Deliver via exim to put mail in to the Maildir.

Has dsync been used on sites of this size? -Any tips?

Do people tend to sync between 2 servers, or do some people use more than 2?

I'm interested in other people's experience before setting up some test
boxes.

Thanks!


Andrew.

Re: [Dovecot] dsync-2.2.7 incorrectly synchronizes subscription status of renamed mailbox

[Karol Jurak]

On Tuesday 19 of November 2013 22:32:15 Timo Sirainen wrote:
 On 19.11.2013, at 16.06, Karol Jurak karol.ju...@gmail.com wrote:
  It seems that dsync-2.2.7 doesn't correctly synchronize subscription
  status of a renamed mailbox.
 
 I don’t think any of the v2.2.x dsyncs have done it correctly the first
 time? Looks like the fix will be a bit complex. I’ll make v2.2.8 release
 first..

You may be right. I remember testing 2.2.2 a few months ago and the behavior 
was the same.

-- 
Karol Jurak

[Dovecot] dsync-2.2.7 incorrectly synchronizes subscription status of renamed mailbox

[Karol Jurak]

Hi,

It seems that dsync-2.2.7 doesn't correctly synchronize subscription status
of a renamed mailbox. The situation is as follows.

There are two servers: A and B, and a test user test_mdbox. Mailboxes 
(mdbox) of this user on both servers are synchronized. Specifically on both 
of them there is a (subscribed) Test1 mailbox. Replication plugin is 
disabled.

I rename Test1 to Test2 on A with Thunderbird. The output of 'doveadm 
mailbox list', contents of subscriptions file and the output of 'doveadm 
dump -t mailboxlog dovecot.mailbox.log' (below) confirm that the rename 
succeeded on A.

#2976: rename 2b84f621c0fd4ba8bd514c5c43ab9a89 (2013-11-19 14:30:35)
#3000: subscribe 2b84f621c0fd4ba8bd514c5c43ab9a89 (2013-11-19 14:30:35)
#3024: unsubscribe 99ea7bf70f6e69ad71659995677b43f8 (2013-11-19 14:30:35)

Subsequently I run 'doveadm sync -u test_mdbox remote:B'. Nothing changes on 
A. On B 'doveadm mailbox list' shows that Test1 was renamed to Test2, but 
subscriptions file still contains Test1 and not Test2. The output of 
'doveadm dump -t mailboxlog dovecot.mailbox.log' confirms that only rename 
was performed:

#2160: rename 2b84f621c0fd4ba8bd514c5c43ab9a89 (2013-11-19 14:57:22)

Only another 'doveadm sync -u test_mdbox remote:B' fixes subscriptions on B. 
The following records are added to dovecot.mailbox.log:

#2184: unsubscribe 99ea7bf70f6e69ad71659995677b43f8 (2013-11-19 15:00:32)
#2208: subscribe 2b84f621c0fd4ba8bd514c5c43ab9a89 (2013-11-19 15:00:32)

-- 
Karol Jurak

Re: [Dovecot] dsync-2.2.7 incorrectly synchronizes subscription status of renamed mailbox

[Timo Sirainen]

On 19.11.2013, at 16.06, Karol Jurak karol.ju...@gmail.com wrote:

 It seems that dsync-2.2.7 doesn't correctly synchronize subscription status
 of a renamed mailbox.

I don’t think any of the v2.2.x dsyncs have done it correctly the first time? 
Looks like the fix will be a bit complex. I’ll make v2.2.8 release first..

[Dovecot] dsync-server panic/fatal errors

[IT geek 31]

Would anyone like to hazard a guess what these errors mean:

Nov 11 18:57:04 server2 dovecot: dsync-server(mark): Panic: file
mbox-lock.c: line 799 (mbox_lock): assertion failed: (lock_type == F_RDLCK
|| mbox-mbox_lock_type != F_RDLCK)
Nov 11 18:57:04 server2 dovecot: dsync-server(mark): Fatal: master:
service(doveadm): child 3119 killed with signal 6 (core not dumped - set
service doveadm { drop_priv_before_exec=yes })

Getting this replication working is driving me nuts! :'-(

Any help would be appreciated.


-Mark

[Dovecot] Dsync: Mailbox changes caused a desync.

[Aleksey Tsvetkov]

Hi!

Here such synchronization error:

dovecot: dsync-local(a...@aaa.com): Warning: Mailbox changes caused a desync. 
You may want to run dsync again.
dovecot: dsync-remote(a...@aaa.com): Warning: 
/var/mail/virtual/aaa.com/alex/.INBOX.System/dovecot-uidlist: Duplicate file 
entry at line 2298: 1380157263.M585262P25253.mail1.aaa.com,S=2476,W=2553 (uid 
3645 - 3662)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3650, 
file=1380157264.M261919P17392.mail2.aaa.com,S=2476,W=2553)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3651, 
file=1380157263.M586977P17315.mail2.aaa.com,S=3119,W=3175)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3652, 
file=1380157264.M261920P17392.mail2.aaa.com,S=2476,W=2553)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3653, 
file=1380157264.M261921P17392.mail2.aaa.com,S=3119,W=3175)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3654, 
file=1380157263.M819006P25260.mail2.aaa.com,S=2476,W=2553:2,)
dovecot: dsync-remote(a...@aaa.com): Warning: Maildir 
/var/mail/virtual/aaa.com/alex/.INBOX.System: Expunged message reappeared, 
giving a new UID (old uid=3655, 
file=1380157264.M261922P17392.mail2.aaa.com,S=3119,W=3175:2,)

As a result, synchronization was, but there were duplicate emails. After this 
synchronization is working fine, no more errors.

dovecot --version
2.2.5

dovecot --build-options
Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192
Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail
SQL drivers: mysql
Passdb: checkpassword pam passwd passwd-file sql
Userdb: checkpassword nss passwd prefetch passwd-file sql

Thanks!

--
Best regards,
Aleksey Tsvetkov
System Administrator
Company Grand Vision
tel. +7(495)933-39-79, ext. 184

Re: [Dovecot] Dsync error: Failed to set attribute vendor/vendor.dovecot/pvt/sieve/default

[Aleksey Tsvetkov]

Thanks!
It works!

On Sun, 22 Sep 2013 11:51:36 +0200
Stephan Bosch step...@rename-it.nl writes:

On 9/22/2013 10:37 AM, Цветков Алексей wrote:
 I installed with a patch. In a log there was my sieve file.

This should fix it:

http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/3163f3696498

Regards,

Stephan.




--
Best regards,
Aleksey Tsvetkov
System Administrator
Company Grand Vision
tel. +7(495)933-39-79, ext. 184