Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-29 Thread ncjeffgus
On Thu, 29 Mar 2012 07:04:26 +0300, Timo Sirainen wrote: On 29.3.2012, at 5.07, Jeff Gustafson wrote: That fixed some errors, but it still is having some sort of trouble with that command: # time doveadm -o mail=maildir:/home/bu/user.mdbox import -u u...@domain.com

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Timo Sirainen
On 22.3.2012, at 23.25, Jeff Gustafson wrote: [root@n24 bu]# time dsync backup -u testu...@domain.com \ mdbox:/home/bu/testuser real1m9.519s user1m7.592s sys 0m1.126s Most of the time is spent on usermode CPU code. I doubt the problem is dsync itself, most likely the problem

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Jeff Gustafson
On Thu, 2012-03-29 at 02:12 +0300, Timo Sirainen wrote: On 22.3.2012, at 23.25, Jeff Gustafson wrote: [root@n24 bu]# time dsync backup -u testu...@domain.com \ mdbox:/home/bu/testuser real1m9.519s user1m7.592s sys 0m1.126s Most of the time is spent on usermode CPU

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Timo Sirainen
On 29.3.2012, at 2.51, Jeff Gustafson wrote: Most of the time is spent on usermode CPU code. I doubt the problem is dsync itself, most likely the problem is mdbox's saving code. Or possibly index/cache code. Try the same dsync backup for: - mbox:/tmp/mbox - mbox:/tmp/mbox:INDEX=MEMORY -

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Jeff Gustafson
On Thu, 2012-03-29 at 03:06 +0300, Timo Sirainen wrote: time doveadm -o mail=mdbox:/tmp/mdbox import mdbox:/path/to/real/mdbox all This tried to write to /root for some reason and failed (dovecot 2.1.3): # time doveadm -o mail=maildir:/home/bu/test.mdbox import

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Timo Sirainen
On 29.3.2012, at 3.48, Jeff Gustafson wrote: On Thu, 2012-03-29 at 03:06 +0300, Timo Sirainen wrote: time doveadm -o mail=mdbox:/tmp/mdbox import mdbox:/path/to/real/mdbox all This tried to write to /root for some reason and failed (dovecot 2.1.3): # time doveadm -o

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Jeff Gustafson
On Thu, 2012-03-29 at 04:07 +0300, Timo Sirainen wrote: On 29.3.2012, at 3.48, Jeff Gustafson wrote: On Thu, 2012-03-29 at 03:06 +0300, Timo Sirainen wrote: time doveadm -o mail=mdbox:/tmp/mdbox import mdbox:/path/to/real/mdbox all This tried to write to /root for some

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-28 Thread Timo Sirainen
On 29.3.2012, at 5.07, Jeff Gustafson wrote: That fixed some errors, but it still is having some sort of trouble with that command: # time doveadm -o mail=maildir:/home/bu/user.mdbox import -u u...@domain.com maildir:/home/users/user%domain.com/Maildir/ all doveadm(u...@domain.com):

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-26 Thread Jeff Gustafson
On Fri, 2012-03-23 at 23:12 -0700, Linda Walsh wrote: Next -- bench cp -ax, against rsync -axHAX when it has to copy 75% of the data (cp ~6-8x speed). But for file speed, 'dd' is king, as it can use large buffers (~16MB gives best results on my local Gbit network), but it misses all

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-26 Thread Jeff Gustafson
On Sat, 2012-03-24 at 14:21 +0100, Maarten Bezemer wrote: On Fri, 23 Mar 2012, Jeff Gustafson wrote: That didn't seem to make much of a difference. On a 3.1GB backup it shaved off 5 seconds. dsync's time was over 6 minutes with or without the mail_fsync=never. rsync copied the same

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-25 Thread Ed W
On 24/03/2012 13:21, Maarten Bezemer wrote: On Fri, 23 Mar 2012, Jeff Gustafson wrote: That didn't seem to make much of a difference. On a 3.1GB backup it shaved off 5 seconds. dsync's time was over 6 minutes with or without the mail_fsync=never. rsync copied the same 3.1GB mailbox in 15

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-24 Thread Linda Walsh
Jeff Gustafson wrote: On Fri, 2012-03-23 at 19:02 +0100, Christoph Bußenius wrote: Hi, maybe try dsync -o mail_fsync=never. That didn't seem to make much of a difference. On a 3.1GB backup it shaved off 5 seconds. dsync's time was over 6 minutes with or without the

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-24 Thread Maarten Bezemer
On Fri, 23 Mar 2012, Jeff Gustafson wrote: That didn't seem to make much of a difference. On a 3.1GB backup it shaved off 5 seconds. dsync's time was over 6 minutes with or without the mail_fsync=never. rsync copied the same 3.1GB mailbox in 15 seconds. It seems to me that

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-23 Thread Christoph Bußenius
Hi, maybe try dsync -o mail_fsync=never. Cheers, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 Raum 00.05.055 Boltzmannstr. 3 Garching

Re: [Dovecot] dsync is SLOW compared to rsync

2012-03-23 Thread Jeff Gustafson
On Fri, 2012-03-23 at 19:02 +0100, Christoph Bußenius wrote: Hi, maybe try dsync -o mail_fsync=never. That didn't seem to make much of a difference. On a 3.1GB backup it shaved off 5 seconds. dsync's time was over 6 minutes with or without the mail_fsync=never. rsync copied the same

[Dovecot] dsync is SLOW compared to rsync

2012-03-22 Thread Jeff Gustafson
Hi all, We are currently using snapshots and rsync to backup a large mail server to a backup mail server. I have been looking into using dsync to replace rsync in hopes that it would make backups more efficient. I decided to test the performance using a single mailbox. Unfortunately dsync