Hi, Chaitanya,
If the primary goal of this exercize is to get a copy of the same files
onto each machine, then rync (http://rsync.samba.org) may well be the
right tool for the job. I've used rsync for many years now to address a
number of issues from syncing source trees on multiple machines to
deploying static content and even for backups to remote machines. It
supports ssh and compression. I consider it to be fundamental to my
toolkit, right behind ssh and perl.
I recommend that you do some research into how long it takes to rsync on
a few days with varying degrees of change in your code. The first thing
that rsync does it to assess the files on both sides in order to
determine what needs to be copied. This can be an expensive operation
-- in fact, the most expensive part for minor changes. It may
While it can do partial file diffs, it may be a good ide to set it to
transfer entire files ( -W param). Compression is an option to be
explored as well. There are a number of options that might work to
optimize this.
Also, and I've never done this, but it may be helpful to add a level to
the distribution by first distributing to a set of machines that then
push to the others to spread out the load. Consider geography here to
minimize bandwidth.
And one more point that I'll make: I've run into some barriers when
attempting to rsync directory structures with many files ( > 60,000).
Rsync croaked, so I wrote a short script to break it out by doing an
separate rsync for individual top-level directories. This may or may
not be an issue, and it may actually reduce the overall time it takes to
do the rysnc. YMMV.
Helpful rsync development params:
-n - dry run. no files are changed.
-stats - as you would expect....
--progress
Eric
________________________________
From: S Chaitanya [mailto:[EMAIL PROTECTED]
Sent: Friday, October 10, 2008 9:50 AM
To: Berg, Eric
Cc: [EMAIL PROTECTED]; [email protected]
Subject: Re: how can I not have to spend 40 minutes _waiting_ in
themorningfor each of 40 updates?
Hi Eric,
I have the code on 40 different machines from the trunk or t-o-t
or head or whatever you call it. So, no different branches involved
here. I will have a go at understanding rsync and using it for syncing
the changed files from 1 machine to the 39 others.
Btw, did you ever try this or run into similar problems?
Regards,
Chaitanya
On Fri, Oct 10, 2008 at 7:06 PM, <[EMAIL PROTECTED]>
wrote:
Has the suggestion to use rsync been raised here? I'm
not sure if
you're updating to different branches for the various
platforms, but
exporting to one or a few locations and then running
rsync to transfer
just the diffs without having to repeatedly go back to
CVS might be a
workable solution.
As for time servers, you'll want to sync to several
servers. NTP's
algorithm takes travel time into account, though closer
is better.
Eric
_______________________________________________
This e-mail may contain information that is confidential, privileged or
otherwise protected from disclosure. If you are not an intended recipient of
this e-mail, do not duplicate or redistribute it by any means. Please delete it
and any attachments and notify the sender that you have received it in error.
Unless specifically indicated, this e-mail is not an offer to buy or sell or a
solicitation to buy or sell any securities, investment products or other
financial product or service, an official confirmation of any transaction, or
an official statement of Barclays. Any views or opinions presented are solely
those of the author and do not necessarily represent those of Barclays. This
e-mail is subject to terms available at the following link:
www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the
foregoing. Barclays Capital is the investment banking division of Barclays
Bank PLC, a company registered in England (number 1026167) with its registered
office at 1 Churchill Place, London, E14 5HP. This email may relate to or be
sent from other members of the Barclays Group.
_______________________________________________
