What does "module" option do?
Hi, Could anyone please tell me what "module" option do in below rsync call and which case it is mandatory? rsync -av host::module /dest Thanks, Jignesh -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Idle option in rsync
Hi. Is there an option in rsync that would lower loading of the disk, namely loading on a file write/delete? For example, in my case it would be great if after a file writing rsync would sleep for N seconds. My options: rsync -v -a -u -S --delete-during --ignore-errors -b --backup-dir=path --log-file=path --bwlimit=1000 --max-size=size rsync://SRC DST/ -- BRGDS. Alexey Vlasov. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: UDP
Charles Darwin wrote: > My question is does rsync use UDP? If not by defaut, then how do I > enable it? Can I compile rsync with UDP as default protocol? rsync uses a byte stream over TCP, SSH, or any other application implementing a byte stream that you want, using the "-e" option. UDP is not a byte stream protocol and quite inappropriate for rsync. Why do you ask for UDP? -- Jamie -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync 2.6.5
Hello, Currently we are using rsync 2.0 in our system to take backup. But it gives an error when it comes to our one of the folder which has very huge data(50 GB). We got to know that rsync 2.0 is not capable of transfering large amount of data and we were asked to upgrade the version. Now we have got an option to install rsync 2.6.5. Can you please let me know if rsync 2.6.5 can handle the large(50 GB) amount of data. Thanks in advance. Regards, Deepak Sharma Application Consultant IBM Gurgaon Ph.+91 9953140845 -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: What does "module" option do?
On Mon, 2008-12-08 at 17:26 +0530, Jignesh Shah wrote: > Could anyone please tell me what "module" option do in below rsync > call and which case it is mandatory? > > rsync -av host::module /dest A module is used when accessing an rsync daemon. See the "CONNECTING TO AN RSYNC DAEMON" section of the rsync man page. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5954] New: Implement something like --very-fuzzy
https://bugzilla.samba.org/show_bug.cgi?id=5954 Summary: Implement something like --very-fuzzy Product: rsync Version: 3.1.0 Platform: Other OS/Version: Linux Status: NEW Severity: enhancement Priority: P3 Component: core AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] QAContact: [EMAIL PROTECTED] I'd like rsync to be able to compare all files on both sides to each other. All of them. Use case: I have a 70GB music directory. I sync it between home and work. At home I run a retagger which tags and renames pretty much every file and directory. So they've all moved, they're all named differently, AND their contents have changed. This basically would result in 70GB of transfers. If rsync were to calculate block hashes for EVERY FILE on BOTH SIDES, and then use those hashes as the sources of new files, this would not take the 2 weeks it would take otherwise. It might take an hour to calculate hashes. But that'd be fine. So maybe --very-fuzzy, --really-fuzzy, or even --fuzzy --fuzzy --fuzzy. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Feature I'd love to see: --move-dest
Looking over rsync's --compare-dest, --copy-dest, and --link-dest options, there's one thing I really wish I could do, and I've been looking for a few years now for a program that could do it: Instead of copying or hard-linking from the compare directory to the target directory, I'd like to move the files. That is, whenever the file already exists in the compare directory, it is moved forward into the target directory; but if the existing file is different or doesn't exist, then a new file is made in the target directory. The result would be that, after several backups (each to a new directory, perhaps named with the current date) the most recent directory would be a snapshot of the source, and each previous directory would have previous versions of files or copies of deleted files. (This is effectively the opposite result of --compare-dest, which gives you a snapshot of the first version of the files, with each successive directory storing only the changed files.) This makes two very common restore scenarios very simple: restoring the entire directory tree to the most recent version after a drive failure, and restoring a previous version of a single file after it was accidentally over-written, even if the modified version has since been backed up. I looked over the rsync source to see if I could add this functionality myself, but I'm really not a very good programmer and it's definitely beyond me. To someone familiar with the source, however, I have to imagine it would be a fairly straightforward feature to add... just add replace the copy operation in --copy-dest or the link operation in --link-dest with a move operation. -- Josh -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Feature I'd love to see: --move-dest
Hi, I'm curious as to why you need a new option rather than just using --link-dest. Both --link-dest and --copy-dest already allow the "two very common restor scenarios" that you mention and --link-dest uses hardly any extra disk space than your --move-dest suggestion. You also get the added advantage that each previous backup continues to look like a full "snapshot". David 2008/12/9 Josh Hanson <[EMAIL PROTECTED]>: > Looking over rsync's --compare-dest, --copy-dest, and --link-dest > options, there's one thing I really wish I could do, and I've been > looking for a few years now for a program that could do it: Instead of > copying or hard-linking from the compare directory to the target > directory, I'd like to move the files. > > That is, whenever the file already exists in the compare directory, it > is moved forward into the target directory; but if the existing file is > different or doesn't exist, then a new file is made in the target > directory. > > The result would be that, after several backups (each to a new > directory, perhaps named with the current date) the most recent > directory would be a snapshot of the source, and each previous directory > would have previous versions of files or copies of deleted files. (This > is effectively the opposite result of --compare-dest, which gives you a > snapshot of the first version of the files, with each successive > directory storing only the changed files.) This makes two very common > restore scenarios very simple: restoring the entire directory tree to > the most recent version after a drive failure, and restoring a previous > version of a single file after it was accidentally over-written, even if > the modified version has since been backed up. > > I looked over the rsync source to see if I could add this functionality > myself, but I'm really not a very good programmer and it's definitely > beyond me. To someone familiar with the source, however, I have to > imagine it would be a fairly straightforward feature to add... just add > replace the copy operation in --copy-dest or the link operation in > --link-dest with a move operation. > > -- Josh > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5954] Implement something like --very-fuzzy
https://bugzilla.samba.org/show_bug.cgi?id=5954 --- Comment #1 from [EMAIL PROTECTED] 2008-12-08 16:40 CST --- (In reply to comment #0) > If rsync were to calculate block hashes for EVERY FILE on BOTH SIDES, and then > use those hashes as the sources of new files, this would not take the 2 weeks > it would take otherwise. It might take an hour to calculate hashes. But that'd > be fine. --fuzzy chooses one destination file whose name looks the most similar to the source name and uses that as a basis, so it fits nicely into rsync's existing workflow. Your suggestion of using the entire destination as a basis for every transfer is fundamentally different, so I wouldn't call it --very-fuzzy. Implementing that in rsync would be a pain. A technique that accomplishes essentially the same thing is to tar up the source and destination and then delta-transfer the tar file, assuming you have enough disk space. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Feature I'd love to see: --move-dest
David, You make some good points. However, I still think a --move-dest feature would be handy. First of all, --link-dest is nice on platforms and file systems that support hard links, but not all platforms and file systems do, and some that do have tagged them on as an afterthought but don't support them very well, or very efficiently. Second, even on systems that do support it, for frequent backups of large numbers of files which may change only rarely, the size of all the hard links may not be negligible. Finally, it's always nice, given the option, to avoid hard links and keep a one-to-one relationship between the "files" I see (i.e. the links) and the actual allocated regions on the disk. Hard links make it impossible to know just how much space each backup takes up. If my backup disk is filling up, I'd like to be able to see which files are taking the most space; I might decide to manually delete older copies of large files, and I won't want to go through and delete every single link to the file. -- Josh -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Overton Sent: Monday, December 08, 2008 5:35 PM To: Josh Hanson Cc: rsync@lists.samba.org Subject: Re: Feature I'd love to see: --move-dest Hi, I'm curious as to why you need a new option rather than just using --link-dest. Both --link-dest and --copy-dest already allow the "two very common restor scenarios" that you mention and --link-dest uses hardly any extra disk space than your --move-dest suggestion. You also get the added advantage that each previous backup continues to look like a full "snapshot". David 2008/12/9 Josh Hanson <[EMAIL PROTECTED]>: > Looking over rsync's --compare-dest, --copy-dest, and --link-dest > options, there's one thing I really wish I could do, and I've been > looking for a few years now for a program that could do it: Instead of > copying or hard-linking from the compare directory to the target > directory, I'd like to move the files. > > That is, whenever the file already exists in the compare directory, it > is moved forward into the target directory; but if the existing file is > different or doesn't exist, then a new file is made in the target > directory. > > The result would be that, after several backups (each to a new > directory, perhaps named with the current date) the most recent > directory would be a snapshot of the source, and each previous directory > would have previous versions of files or copies of deleted files. (This > is effectively the opposite result of --compare-dest, which gives you a > snapshot of the first version of the files, with each successive > directory storing only the changed files.) This makes two very common > restore scenarios very simple: restoring the entire directory tree to > the most recent version after a drive failure, and restoring a previous > version of a single file after it was accidentally over-written, even if > the modified version has since been backed up. > > I looked over the rsync source to see if I could add this functionality > myself, but I'm really not a very good programmer and it's definitely > beyond me. To someone familiar with the source, however, I have to > imagine it would be a fairly straightforward feature to add... just add > replace the copy operation in --copy-dest or the link operation in > --link-dest with a move operation. > > -- Josh > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Feature I'd love to see: --move-dest
Hi, This seems to be a reappearing request. Interestingly enough, I wanted the exact same thing a while ago and nobody was able to suggest a quick solution without implementing a new feature. I didn't have the time to read into the rsync source code, so I decided to write a little perl script, which works similar to rsnapshot. It - keeps one exact copy of the source which is updated every time - stores the changes between the current and the last run in an extra directory which is labelled interval_date_time - supports different intervals by merging several more frequent ones into one less frequent (7 daily -> 1 weekly) - does not use hard links At the moment, I run it about 5 times a day on a 3TB file system (1.2M files) with about 400MB changes each time and it takes around half an hour. I haven't put an amazing amount of work into it because it was never supposed to be used by anyone else than myself; it's not foolproof but I am willing to share it around and improve it if there is interest. Cheers, Tom Josh Hanson wrote: > David, > > You make some good points. However, I still think a --move-dest > feature would be handy. > > First of all, --link-dest is nice on platforms and file systems that > support hard links, but not all platforms and file systems do, and > some that do have tagged them on as an afterthought but don't support > them very well, or very efficiently. > > Second, even on systems that do support it, for frequent backups of > large numbers of files which may change only rarely, the size of all > the hard links may not be negligible. > > Finally, it's always nice, given the option, to avoid hard links and > keep a one-to-one relationship between the "files" I see (i.e. the > links) and the actual allocated regions on the disk. Hard links make > it impossible to know just how much space each backup takes up. If my > backup disk is filling up, I'd like to be able to see which files are > taking the most space; I might decide to manually delete older copies > of large files, and I won't want to go through and delete every > single link to the file. > > -- Josh > > > > -Original Message- From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of David Overton Sent: > Monday, December 08, 2008 5:35 PM To: Josh Hanson Cc: > rsync@lists.samba.org Subject: Re: Feature I'd love to see: > --move-dest > > Hi, > > I'm curious as to why you need a new option rather than just using > --link-dest. Both --link-dest and --copy-dest already allow the "two > very common restor scenarios" that you mention and --link-dest uses > hardly any extra disk space than your --move-dest suggestion. You > also get the added advantage that each previous backup continues to > look like a full "snapshot". > > David > > 2008/12/9 Josh Hanson <[EMAIL PROTECTED]>: >> Looking over rsync's --compare-dest, --copy-dest, and --link-dest >> options, there's one thing I really wish I could do, and I've been >> looking for a few years now for a program that could do it: Instead >> of copying or hard-linking from the compare directory to the target >> directory, I'd like to move the files. >> >> That is, whenever the file already exists in the compare directory, >> it is moved forward into the target directory; but if the existing >> file is different or doesn't exist, then a new file is made in the >> target directory. >> >> The result would be that, after several backups (each to a new >> directory, perhaps named with the current date) the most recent >> directory would be a snapshot of the source, and each previous >> directory would have previous versions of files or copies of >> deleted files. (This is effectively the opposite result of >> --compare-dest, which gives you a snapshot of the first version of >> the files, with each successive directory storing only the changed >> files.) This makes two very common restore scenarios very simple: >> restoring the entire directory tree to the most recent version >> after a drive failure, and restoring a previous version of a single >> file after it was accidentally over-written, even if the modified >> version has since been backed up. >> >> I looked over the rsync source to see if I could add this >> functionality myself, but I'm really not a very good programmer and >> it's definitely beyond me. To someone familiar with the source, >> however, I have to imagine it would be a fairly straightforward >> feature to add... just add replace the copy operation in >> --copy-dest or the link operation in --link-dest with a move >> operation. >> >> -- Josh -- Please use reply-all for most replies to avoid omitting >> the mailing list. To unsubscribe or change options: >> https://lists.samba.org/mailman/listinfo/rsync Before posting, >> read: http://www.catb.org/~esr/faqs/smart-questions.html >> -- Thomas Gutzler Optical+Biomedical Engineering Laboratory The University of Western Australia http://obel.ee.uwa.edu.au/ phone: +61 8 6488 3916 -
Run rsync through intermediary server with SSH
I'm using rsync, ssh, and cron glued together with Python as a push-based synchronization system. From a single location, I push content out to various offices. I log stdout/stderr on the master server to make sure everything is running smoothly. I would now like for some of our "regional hubs" to take on some of the load (bandwidth-wise), while still retaining my centralized master for command execution and logging. So far, I simply send the desired rsync command as a remote ssh command. It looks like this: ssh [EMAIL PROTECTED] "rsync -ritzOK --delete-after --timeout=600 --progress --stats --verbose --bwlimit='350' --exclude-from='/home/gsync/.gsync/config/office-filters/sr' --filter='. /home/gsync/.gsync/config/general-filters/junk' --filter=':- gsync-filter' '/s/cad/Revit/' [EMAIL PROTECTED]:'\"/s/cad/Revit\"'" The idea is that I can push to our regional offices, and then have them push to their near-by peers using their own bandwidth, yet still collect output at the master. The only problem I'm running into is this: if I kill the ssh session on the master, the remote rsync process continues to run on the intermediary server to the final destination. This can lead to multiple rsync jobs running from the intermediary to the same destination server, flooding it. I'm having difficulty figuring out what the appropriate mojo is to ensure that the rsync jobs die with the ssh connections. Hope this is clear. Any advice? Thanks, Jimmie -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
