If rsync isn't doing the networking you are better off with cp -au instead of rsync. It should be significantly faster and you can do a final pass with rsync to get any files that got truncated by a ^C (cp can only skip files that are newer not files that are not different and a truncated file will be newer since it never got back-dated).
> rsync -av --no-perms --no-owner --no-group /old/storage/dir > /new/storage/directory/dir Note that rsync treats a trailing / on the source parameter differently. If you did rsync ... /old/storage/dir /new/storage/dir then you made /new/storage/dir/dir and duplicated everything into it. The correct syntax is rsync ... /old/storage/dir/ /new/storage/dir OR rsync ... /old/storage/dir /new/storage On 10/18/2017 05:00 PM, Herb Burnswell via rsync wrote: > All, > > I am seeding a new storage environment (Glusterfs on XFS) and would like > to gather advise on best practices. This data is primarily all media > data, so not good with compression. > > I currently have made one pass on at 20TB directory tree into the > environment as: > > - nfs mount from old storage to new storage > - rsync -av /old/storage/* /new/storage/directory > > Once the directories and files were on the new storage, I did: > > - chown -R root:root > - chmod -R 774 > > I'll need to do a couple more sync's prior to full cut over. > > Questions regarding performance: > > - Does anyone have any suggestions on how to achieve the best performace > (speed)? > > - Is a local NFS mount from old storage to new storage the best > option? If so are there specific mount options that should be used? > - Any specific rsync flags (I've tested with and without 'z' flag > and it does not help with this data) or best practices? > > Questions regarding rsync behavior: > > - When I test individual directory resync's within the initial ingest > tree, a command such as: > > rsync -av --no-perms --no-owner --no-group /old/storage/dir > /new/storage/directory/dir > > Lists all of the directories under 'dir' in the shell. But if I rerun > the command immediately thereafter, nothing is listed in the shell. > Where is this 'metadata' of what is 'already on the destination' > stored? Is it only stored while the shell is open? I want to set up a > cron job moving forward and would like to make sure all info is available. > > Any guidance is greatly appreciated. > > Thanks in advance, > > HB > > > > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
signature.asc
Description: OpenPGP digital signature
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html