rsync bandwidth usage
Hello all, I'm experiencing about 20MBit/s on a 100MBit/s ethernet connection when rsync'ing a lager number on _new_ files. I don't have much exeprience with rsync. I'dd just like to know if thi is an acceptable bw usage and what could be the limiting factors... in order to try to increase this speed. I'm considering using 1000MBit/s dedicated connections and so I'dd like to know if I will or won't get faster operations, or if I will still have this 20MBit bottleneck. Furthermore... is there a way to ask rsync to be slower ?!? thank you joao -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
At 10:37 22.04.2008 +0100, Joao Ferreira gmail wrote: >Hello all, > >I'm experiencing about 20MBit/s on a 100MBit/s ethernet connection when >rsync'ing a lager number on _new_ files. > >I don't have much exeprience with rsync. I'dd just like to know if thi >is an acceptable bw usage and what could be the limiting factors... in >order to try to increase this speed. > >I'm considering using 1000MBit/s dedicated connections and so I'dd like >to know if I will or won't get faster operations, or if I will still >have this 20MBit bottleneck. > >Furthermore... is there a way to ask rsync to be slower ?!? How do you call rsync? Do you use zipping? What are the used machines (CPU type/freq)? We had the case that rsyncing to a NAS was not that fast because the used CPU couldn't hold up. Another reason might be the used rsync protocol, what rsync version do you use? bye Fabi -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
On Tue, 2008-04-22 at 12:18 +0200, Fabian Cenedese wrote: > At 10:37 22.04.2008 +0100, Joao Ferreira gmail wrote: > >Hello all, > > > >I'm experiencing about 20MBit/s on a 100MBit/s ethernet connection when > >rsync'ing a lager number on _new_ files. > > > >I don't have much exeprience with rsync. I'dd just like to know if thi > >is an acceptable bw usage and what could be the limiting factors... in > >order to try to increase this speed. > > > >I'm considering using 1000MBit/s dedicated connections and so I'dd like > >to know if I will or won't get faster operations, or if I will still > >have this 20MBit bottleneck. > > > >Furthermore... is there a way to ask rsync to be slower ?!? > > How do you call rsync? rsync -av --delete --perms --acls ... > Do you use zipping? I don't ask for it ? Is the default behaviour to use or not ? > What are the used machines > (CPU type/freq)? it's a Via C6 @ 1G Hz > We had the case that rsyncing to a NAS was not that > fast because the used CPU couldn't hold up. Another reason might be > the used rsync protocol, what rsync version do you use? > I'm using 3.0.0 > bye Fabi > > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
For situations where it's all new files, using -W would make it a lot more efficient and less CPU intensive. You can ask rsync to be slower with it's built in bandwidth limiting options. On Tue, Apr 22, 2008 at 12:14:02PM +0100, Joao Ferreira wrote: > > On Tue, 2008-04-22 at 12:18 +0200, Fabian Cenedese wrote: > > At 10:37 22.04.2008 +0100, Joao Ferreira gmail wrote: > > >Hello all, > > > > > >I'm experiencing about 20MBit/s on a 100MBit/s ethernet connection when > > >rsync'ing a lager number on _new_ files. > > > > > >I don't have much exeprience with rsync. I'dd just like to know if thi > > >is an acceptable bw usage and what could be the limiting factors... in > > >order to try to increase this speed. > > > > > >I'm considering using 1000MBit/s dedicated connections and so I'dd like > > >to know if I will or won't get faster operations, or if I will still > > >have this 20MBit bottleneck. > > > > > >Furthermore... is there a way to ask rsync to be slower ?!? > > > > How do you call rsync? > > rsync -av --delete --perms --acls ... > > > Do you use zipping? > > I don't ask for it ? Is the default behaviour to use or not ? > > > What are the used machines > > (CPU type/freq)? > > it's a Via C6 @ 1G Hz > > > We had the case that rsyncing to a NAS was not that > > fast because the used CPU couldn't hold up. Another reason might be > > the used rsync protocol, what rsync version do you use? > > > > I'm using 3.0.0 > > > bye Fabi > > > > > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- /* Jason Philbrook | Midcoast Internet Solutions - Wireless and DSL KB1IOJ| Broadband Internet Access, Dialup, and Hosting http://f64.nu/ | for Midcoast Mainehttp://www.midcoast.com/ */ -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
On Tue, 2008-04-22 at 12:18 +0200, Fabian Cenedese wrote: > At 10:37 22.04.2008 +0100, Joao F > ask rsync to be slower ?!? > > How do you call rsync? rsync -av --delete --perms --acls ... > Do you use zipping? I don't explicitlly ask for it ? What's the default behaviour ? to use zipping or not ? > What are the used machines > (CPU type/freq)? it's a Via C6 @ 1G Hz > We had the case that rsyncing to a NAS was not that > fast because the used CPU couldn't hold up. Another reason might be > the used rsync protocol, what rsync version do you use? we're using 3.0.0 thx joao > > bye Fabi > > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
> > How do you call rsync? > > rsync -av --delete --perms --acls ... You left out the rest. Specifically, what protocol are you using over the ethernet? Are you using an rsync server, ssh, NFS, SMB, or what? A straight rsync server is by far the fastest, I've found. So you'd do something like so: rsync -av ... /local/path/ hostname::modulename/remote/path/ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync bandwidth usage
As an example, I transfered 1930420 KBytes with rsync between two 2.8 GHz Intel P4 machines over gigabit ethernet and got this: transfer method:ssh rshnfs rsync wall time in sec 130.01 79.77 176.03 74.92 MBit/sec116 189 86201 Mind you, this is no benchmark as I only did each on once and wasn't particularly careful that nothing else was running on the two machines or on the network. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
