On Mon, 3 Jan 2005, Wayne Davison wrote: > On Mon, Jan 03, 2005 at 05:39:19PM +0100, Dag Wieers wrote: > > With newer rsyncs, rsync seems to sort the transaction set, so I > > cannot longer use this trick to have the metadata uploaded just after > > the data in the same transaction set. > > Rsync has always sorted the list of files to be sent, so this is not > something that is different between 2.5.x and 2.6.x. I'd be interested > in hearing what you believe to be different in how files are processed.
In the past I could say smt. like: rsync -a dir1/ dir2/ [EMAIL PROTECTED]:/remote-dir/ and it would process first dir1 and then dir2. I'm not sure when this change happened, it may not be when 2.6.x. This way I first added the packages/ dir (which contains hardlinks of only the packages) and then the repository (packages+metadata). Resulting in a much smaller window between the start of mirror the repodata and finishing the hardlinking of all packages. (still 5 to 10 minutes !) If rsync includes the atomic transaction set I explained in a previous post, it can be reduced to an even smaller window (only the renaming of the files strictly). Even the hardlinking can be done in advance. > > I was wondering if it was possible and acceptable to have an rsync > > option to update the whole transaction in a atomic (or near-atomic > > way). > > One way to do this would be to use the --link-dest option to create a > new hierarchy of files (with only the changed files getting sent, and > all unchanged files being hard-linked to the prior files) and then > moving the whole file-set into place all at once. Imagine that there > is a hierarchy you want to update in /dest/cur by running this script: I know this, but I don't control all the mirrors and I can't ask all my mirrors to implement one of these complex alternatives. Besides this requires a per mirror configuration (as it requires a fixed directory per rsync transaction). Something inherent to rsync (the same way current atomic-per-file behaviour works) that kicks in at the end of a transaction set instead of after each transfer (of an individual file) works everywhere with a single option and no fuss. I can ask mirrors to use '--atomic' or '--atomic-ts', bt I can't ask them to re-organise their mirror-scripts just for me. -- dag wieers, [EMAIL PROTECTED], http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power] -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html