On 4/05/2015 11:06 PM, Teemu Huovila wrote:
On 05/03/2015 01:48 PM, Reuben Farrelly wrote:
Hi all,
I've had an interesting use case come up which - to cut the story short - one
way to solve the problem I am looking at may be to
replicate a small number of mailboxes to a third remote server.
I've currently had replication running between my main dovecot machine and
another remote VM for some time and working well (so
I'm not new to replication and I've got a good working config), but I've a need
to add a third to the mix for a select number of
mailboxes. The arch on both of those is Gentoo x86_64 and with latest 2.1.16
-hg.
I have attempted this so far by rsync'ing the initial Maildirs and then once
the bulk of the data has been transferred rely on
dovecot's replication to keep things in sync. I figure that this should in
theory mean that the subsequent updates in both
directions are incremental and the bulk of the data gets moved while the device
is here on my desk using rsync.
I've attempted to do this using a Raspberry Pi as a remote device, but when I
set it up the dovecot replication process seems to
need to start the replication over from scratch even after the rsync is done.
I know this is happening as the disk utilisation
on the Pi skyrockets once the replication starts and I end up with thousands of
double ups of all the mails ... which defeats
the entire point of the process.
If I do an identical configuration but on a third Gentoo x86_64 VM locally it
all works as expected. No double ups of mails and
the "catchup" between the two devices is practically instant. Same filesystem
even. The only difference appears to be the
system architecture.
So main my question is this. Is there a known architecture/endian limitation
on replication? I guess cross-arch replication
is not something many people try but is it supposed to work anyway?
I think you are bumping against Dovecot index endianess restrictions. I dont
think cross-arch dsync can currently work very
efficiently.
http://wiki2.dovecot.org/Design/Indexes/MainIndex?highlight=%28endian%29
Ok. That explains why the rsync won't work. But if I kick off a
dovecot to dovecot replication (without doing the rsync first) will this
work any better once the system catches up? This assumes (possibly
incorrectly - please correct me if I am wrong) that the index files
themselves aren't dsync'd byte-by-byte but instead the metadata/content
from them is sent instead, and then the indexes are written to remote
disk by the remote dovecot in the right arch and format that the remote
machine can read and understand. Because if that's the case then I can
probably make that work - just taking a hit on the initial sync which
could take longer.
Even if this doesn't end up working I figure I'll get to learn a little
more about the indexes themselves in the process.
Thanks for any advice,
Reuben
