Re: Change destination root?
Matt McCutchen-7 wrote: > > On Sun, 2007-12-16 at 07:53 -0800, Max Kipness wrote: >> I've got a source directory of /data with several subdirectories such as >> subdir1/ subdir2/ subdir3/ , etc. I'd like to rsync these to a different >> host under directory /backup, so that I have paths /backup/subdir1, >> /backup/subdir2, etc. I also need to be able to use the option >> --files-from. >> >> So, a source dir of /data on host1, with a --files-from file containing: >> >> subdir1 >> subdir2 >> subdir3 >> >> rsyncing to host2 and ending up with the file structure: >> >> /backup/subdir1 >> /backup/subdir2 >> /backup/subdir3 >> >> Basically, I'd like to substitute /var for /backup. Is this possible? > > This should just be a matter of changing the destination in your rsync > command line from "/backup" to "/var". The original command (assuming a > push) would look like: > > rsync [OPTION...] -r --files-from=THE-FF-LIST /data/ host2:/backup/ > > The new command would look like: > > rsync [OPTION...] -r --files-from=THE-FF-LIST /data/ host2:/var/ > > Am I missing something? > > Matt > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > > I apologize, I explained it wrong. Let me try to explain it correctly with a different example. host1.domain.com Source Directory: /data -files-from file c/contents c/contents I want this to end up on host2.domain.com as /backup/x/contents /backup/x/contents So it's not the root thats the problem but the next level down. I need to be able to list the paths to be backed up via -files-from, but yet need c to end up as x on the destination. Thanks, Max -- View this message in context: http://www.nabble.com/Change-destination-root--tp14362791p14365146.html Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Change destination root?
I'm wondering if it's possible to do the following. I've got a source directory of /data with several subdirectories such as subdir1/ subdir2/ subdir3/ , etc. I'd like to rsync these to a different host under directory /backup, so that I have paths /backup/subdir1, /backup/subdir2, etc. I also need to be able to use the option --files-from. So, a source dir of /data on host1, with a --files-from file containing: subdir1 subdir2 subdir3 rsyncing to host2 and ending up with the file structure: /backup/subdir1 /backup/subdir2 /backup/subdir3 Basically, I'd like to substitute /var for /backup. Is this possible? Thanks, Max -- View this message in context: http://www.nabble.com/Change-destination-root--tp14362791p14362791.html Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
dry-run bytest to be transferred?
Is there anyway on a dry-run to actually see per file how many bytes are to be transferred? On a normal run I use --log-format=/%f/%l/%b/%i which shows the total file size and the actual bytes transferred. Also, during a dry-run, is there anyway to show the Literal Data? Basically, I'm trying to calculate something before the actual run takes place. Dry-run is telling me which files have changed and why, but not telling me how much of the file is different. Thanks, Max -- View this message in context: http://www.nabble.com/dry-run-bytest-to-be-transferred--tf4787455.html#a13695755 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Deleting...log path
> -Original Message- > From: Wayne Davison [mailto:[EMAIL PROTECTED] > Sent: Friday, June 22, 2007 1:39 PM > To: Max Kipness > Cc: rsync@lists.samba.org > Subject: Re: Deleting...log path > > On Fri, Jun 22, 2007 at 07:02:30AM -0500, Max Kipness wrote: > > it's still not showing the 'outside' full path as with files > > that are new or changed. > > There's no such thing as an outside path for a deleted file. That's > happening on the receiving side, and only the sender knows outside > paths. Ok, so is there possibly anyway to log the location of deleted files? Basically I create a report based on the rsync logs. If for example there were two different documents named the same in different directories, as far as the rsync logs are concerned, there is no way of knowing which was deleted. If there is no way, I guess I will be forced NOT to log deleted files. It just seems like this is something you should be able to log. Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Deleting...log path
Max Kipness wrote: > >> On Thu, Jun 21, 2007 at 03:19:23PM -0500, Max Kipness wrote: >> > The %f settings gives you the full path. This is what I want. The %n >> > setting gives you just the file name. >> >> No, %n gives you the path to the file within the transfer. %f > includes >> path info from outside the transfer when output on the sending side or >> by a daemon. > > Thanks for the clarification Wayne. I need the path info from outside > whether the file is new, changed or deleted, so that I can log it > appropriately with my application. > >> > Could this possibly be a bug? >> >> It sounds like you're referring to this bug that was fixed in December >> (after 2.6.9 was released): >> >> - Fixed a problem with the --out-format (aka --log-format) option: > it >>would output superfluous directory information for a non-daemon >> rsync. >> >> If you try the cvs version, you'll probably find that it's fixed. > > > I downloaded and installed the CVS version (Rsync 3.0) and although it > looks better in the fact that it's not showing the path the command was > run from, it's still not showing the 'outside' full path as with files > that are new or changed. Here is a sample: > > First adding the new file test9.txt: > > [EMAIL PROTECTED] www]# rsync -e ssh /test/ xxx.xxx.com://test/ --stats > --compress --recursive --partial --archive --del --times > --log-format=%f:::%l:::%b:::%i > test/.:::4096:::0:::.d..t.. > test/test9.txt:::0:::39::: > Then we delete file test9.txt and run the same command again (note: the > command is run from /var/www): > > [EMAIL PROTECTED] www]# rsync -e ssh /test/ xxx.xxx.com://test/ --stats > --compress --recursive --partial --archive --del --times > --log-format=%f:::%l:::%b:::%i > test/.:::4096:::0:::.d..t.. > test9.txt:::0:::0:::*deleting > > It looks better, but I was really hoping this new version would fix this > bug and print 'test/test9.txt deleting' on the last line. As you can > see it does not. Will this be fixed in the final version, or are there > any other suggestions? Would running a daemon on the other end correct > this? > > Update: I've now tried it with rsync in daemon mode, and get the same results. The full path of the deleted file is still not given. Thanks, Max -- View this message in context: http://www.nabble.com/Deleting...log-path-tf3946219.html#a11256253 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Deleting...log path
> On Thu, Jun 21, 2007 at 03:19:23PM -0500, Max Kipness wrote: > > The %f settings gives you the full path. This is what I want. The %n > > setting gives you just the file name. > > No, %n gives you the path to the file within the transfer. %f includes > path info from outside the transfer when output on the sending side or > by a daemon. Thanks for the clarification Wayne. I need the path info from outside whether the file is new, changed or deleted, so that I can log it appropriately with my application. > > Could this possibly be a bug? > > It sounds like you're referring to this bug that was fixed in December > (after 2.6.9 was released): > > - Fixed a problem with the --out-format (aka --log-format) option: it >would output superfluous directory information for a non-daemon > rsync. > > If you try the cvs version, you'll probably find that it's fixed. I downloaded and installed the CVS version (Rsync 3.0) and although it looks better in the fact that it's not showing the path the command was run from, it's still not showing the 'outside' full path as with files that are new or changed. Here is a sample: First adding the new file test9.txt: [EMAIL PROTECTED] www]# rsync -e ssh /test/ xxx.xxx.com://test/ --stats --compress --recursive --partial --archive --del --times --log-format=%f:::%l:::%b:::%i test/.:::4096:::0:::.d..t.. test/test9.txt:::0:::39:::https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Deleting...log path
> On Tue 19 Jun 2007, Max Kipness wrote: > > > > For some time I've noticed something that seems very strange to me > when > > setting an rsync session for delete. > > > > The destination path of the backup is /backup/Latest/Latest/CORP- > SERVER/ so > > for new/changed files the logs appropriately report for example: > > > > backup/Latest/Latest/CORP-SERVER/ > > > > for each file/directory. > > > > However, when a file does not exist on the source end, and a deletion > occurs > > I get the following bizarre line in the logs for each file: > > > > var/www/CORP-SERVER/C$/Corp/User Files/xxx-company/Admin/Property > > Asstes/Apartments/Sequoia/Photos/ > > Sequoia Bend/DSC03632.JPG:::0:::0:::*deleting > > > > I've added the colons to the logging format to easily parse the data. > > > > But why does it show var/www ? /var/www/html/command is where the > rsync > > %f is basically the full path name, from the module root (/ if you're > %not talking to a daemon, I believe) > > I think you want %n The %f settings gives you the full path. This is what I want. The %n setting gives you just the file name. I did some further testing and it seems that whenever deleting a file, %f gives you the path from where the rsync command was run rather than the path from where the file was deleted. When using %f and the file is new or has changed, it does properly give you the full path of the file. Could this possibly be a bug? > Curious, the manpage doesn't list --log-format as an option :-) I > would > have used --out-format. It seems that the old setting was --log-format and the new is --out-format. Evidently the manpage no longer shows the old option, but it still works. Tried the --out-format setting and it returned the same results. > It helps, BTW, if you paste the exact command used, that makes it clear > whether you're connecting to a daemon, what the module name is, etc. Here is the exact command I using for my tests. I'm running it from the /var/www directory: rsync -e ssh /test/ server1.xxx.com://test/ --stats --compress --recursive --partial --archive --delete --times --out-format=%f:::%l:::%b:::%i When adding a new file (test8.txt), the results are just perfect as it shows the full path of the file: test/.:::4096:::0:::.d..t.. test/test8.txt:::0:::39:::https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Deleting...log path
For some time I've noticed something that seems very strange to me when setting an rsync session for delete. The destination path of the backup is /backup/Latest/Latest/CORP-SERVER/ so for new/changed files the logs appropriately report for example: backup/Latest/Latest/CORP-SERVER/ for each file/directory. However, when a file does not exist on the source end, and a deletion occurs I get the following bizarre line in the logs for each file: var/www/CORP-SERVER/C$/Corp/User Files/xxx-company/Admin/Property Asstes/Apartments/Sequoia/Photos/ Sequoia Bend/DSC03632.JPG:::0:::0:::*deleting I've added the colons to the logging format to easily parse the data. But why does it show var/www ? /var/www/html/command is where the rsync script is running from, but then why didn't it include 'html/command'? This messes up my whole logging process. My parameters are: --stats --compress --recursive --partial --archive --del --times --log-format=%f::: %l:::%b:::%i" Anyone know why it does this or how to correct? This happens on various servers with various versions of rsync so I don't think it's isolated. Thanks, Max -- View this message in context: http://www.nabble.com/Deleting...log-path-tf3946219.html#a11194087 Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync with Plink?
Hello, I've already tried asking at the cwrRsync forums with no luck yet, so I thought I would try my luck here. According to the rsync man pages, with the -e option you can use other remote shells, and I had thought plink was one of them, but I could be wrong. I can't seem to find any information anywhere about the proper syntax. What I've been trying, are all kinds of combination close to the following: rsync.exe -e "plink.exe -l username -pw password" --stats --compress --recursive --archive --delete-after --files-from=backup_selections.txt /cygdrive/c server_address://backup/Current I've tried adding options -batch and -N as suggested by someone else to no avail. I basically want to use rsync without a password or keys and yes, I'm aware of the security implications. Any suggestions would be much appreciated. Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Copy 100gb+ file over LAN?
Hopefully this is not too off topic. I have a server in which I'm creating an Exchange Information Store backup using ntbackup. The .bkf file it creates is 112GB. My goal was first to rsync it over to a local linux server, and then rsync just the changes over the internet to another linux server. I do this with other Exchange .bkf files (in the 4GB to 20GB range) and it seems to work well only transferring the changes over the internet. My usual method has been to map the Windows 2003 drive on the local linux server and then use rsync. When trying this with this system, the transfer speed is super slow from the beginning, at 6 mbit per second on a 10/100 mbit network. If I try the copy using smbclient and get, I actually get 60 mbit p/sec for the first 30gb, then it seems to slow to 6 mbit. A regular cp does basically the same thing as it starts out fast and then slows all of a sudden. This obviously will not be very practical as on the local copy I usually would copy the entire file each night. At that rate it will take days. Any suggestions on how to copy a 100gb file over a local LAN quickly? Or what copy/transfer command might keep the same quick throughput during the entire transfer? Or could there be something wrong on the network? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Busy file and partial transfer??(2nd Post)
Can anyone help on this? Or is this just a bug with rsync on certain files? Hi, I actually wrote about this issue over a year ago and got a response from Wayne which I'm thinking resolved the problem. Now I've got the issue again, even though I removed the option that appeared to be the culprit. Basically I'm using rsync to backup a directory structure on a 2003 Server, and there are a couple of files that are in use and locked. On is a PST file, and a few others are Quickbooks files. The Quickbooks files give the appropriate error as: rsync: send_files failed to open "/cygdrive/E/Users/XGL-Shared/Accounting/ Holdings/ Holdings, Inc.QBW": Device or resource busy (16) rsync: send_files failed to open "/cygdrive/E/Users/XGL-Shared/Accounting/ Holdings/ Holdings, Inc.QBW.TLG": Device or resource Nothing else is logged about these files, and this logging method seems to be appropriate for locked files. However, there is a PST file that reports errors as: rsync: read errors mapping "/cygdrive/E/Users/HomeFolders/mike.x/OutlookFiles/Mike6-24-06backup .pst": Permission denied (13) The error is actually repeated twice via stderr, but not together (the quickbooks errors are in between). Then I get this at the bottom of the stderr stream: ERROR: E/Users/HomeFolders/mike./OutlookFiles/Mike6-24-06backup.pst failed verification -- update discarded. rsync error: some files could not be transferred (code 23) at main.c(791) The strange thing is the via stdout, I get the following which would lead me to believe there was an update and the update was transferred successfully: E/Users/HomeFolders/mike./OutlookFiles/Mike6-24-06backup.pst:::73820 16:::179642:::https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Busy file and partial transfer??
Hi, I actually wrote about this issue over a year ago and got a response from Wayne which I'm thinking resolved the problem. Now I've got the issue again, even though I removed the option that appeared to be the culprit. Basically I'm using rsync to backup a directory structure on a 2003 Server, and there are a couple of files that are in use and locked. On is a PST file, and a few others are Quickbooks files. The Quickbooks files give the appropriate error as: rsync: send_files failed to open "/cygdrive/E/Users/XGL-Shared/Accounting/ Holdings/ Holdings, Inc.QBW": Device or resource busy (16) rsync: send_files failed to open "/cygdrive/E/Users/XGL-Shared/Accounting/ Holdings/ Holdings, Inc.QBW.TLG": Device or resource Nothing else is logged about these files, and this logging method seems to be appropriate for locked files. However, there is a PST file that reports errors as: rsync: read errors mapping "/cygdrive/E/Users/HomeFolders/mike.x/OutlookFiles/Mike6-24-06backup .pst": Permission denied (13) The error is actually repeated twice via stderr, but not together (the quickbooks errors are in between). Then I get this at the bottom of the stderr stream: ERROR: E/Users/HomeFolders/mike./OutlookFiles/Mike6-24-06backup.pst failed verification -- update discarded. rsync error: some files could not be transferred (code 23) at main.c(791) The strange thing is the via stdout, I get the following which would lead me to believe there was an update and the update was transferred successfully: E/Users/HomeFolders/mike./OutlookFiles/Mike6-24-06backup.pst:::73820 16:::179642:::https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Incremental Storage
Hello, I've been trying to figure out how to control the space required for backups using rsync, large files, and an incremental backup scheme. In particular, I've got two customers in which I'm creating Exchange backups using the built-in MS backup, then rsyncing an exchange.bks file nightly. One customer has a 4.3GB exchanges.bks file, and although rsync works wonderfully by only sending 43mb worth of changes to this file across the line, using hard links obviously causes incremental directories of at least 4.3GB daily. With 30 days of incremental, this adds up. I've looked at rdiff-backup, but do not like it as it's so different than rsync. I've got a lot of scripting built around rsync and I like the way it handles file/dir selections, so I will not consider rdiff-backup. I've been experimenting with rdiff itself and trying to figure out a formula for creating delta files of only the changes to the Exchange.bks file and using these delta files in the incremental directories, instead of creating the hard links. I've gotten it to work, but not in a rotational scheme where you always save only the latest copy and the deltas, and can easily restore (patch) using just the latest and the delta. Has anybody found a solution for this? Or is there any other tool out there that automates this? Again, rsync works great for the transfer, but storing data incrementally is becoming the problem. Would be nice if a future version of rsync created rdiff incrmentals as an alternative option to 'cp- al' or 'link-destination'. Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: access of windows shadow copy by rsync?
> Is there any way to access the shadow device by the copy command and to > pipe the files to rsync? > that means to "filter" the files by rsync (to copy only the necessary > files)? > Michael Sorry for the late response. I don't know if you can pipe directly to rsync from the xp copy command, as I don't think rsync can receive its file list from stdin. What I've done is created a script with two file lists, standard rsync list and an open file list. My script runs vshadow.exe creating the volume shadow device, reads the files from 'open file list', then copies them to a temp location. Then rsync runs against its file list which includes the temp location. Then the script deletes everything in the temp directory. I add to the 'open file list', files that I know most likely will be open, like .pst files. I've tested restoring .pst files in which they were open, and only the changes calculated by rsync were transferred over the course of several backups. They restored fine. I had the thought of piping stderr from my rsync script to a separate file, and then copying all files that were 'in use' according to rsync via vshadow and then running rsync for a second time. If I recall though, on windows, rsync doesn't specify which files are 'in use' and rather uses a generic error for any file it cannot copy. I guess it still would not hurt to try getting them through vshadow. Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: access of windows shadow copy by rsync?
> On Tue, May 16, 2006 at 01:00:28AM +0200, [EMAIL PROTECTED] wrote: > > Im trying to access shadow copies under windows XP by rsync. > > Shadow copies may be accessed as a device with the following syntax: > > \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1 > > I think you should map that location to a drive letter, such as X: (do > this in your MS Windows setup), and then try accessing it via the normal > /cygdrive/x/... prefix using rsync. > > ..wayne.. I tried the mapping approach. It did not work as I assumed. The error received was "The network path was not found". Then 'net use' command evidently does not recognize the volume shadow device. So far I've found that only the 'copy' command can recognize it. Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: access of windows shadow copy by rsync?
>Hello >Im trying to access shadow copies under windows XP by rsync. >Shadow copies may be accessed as a device with the following syntax: >\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1 >I try it with rsync but the log always states: >rsync: link_stat >"/cygdrive/\//GLOBALROOT/Device/HarddiskVolumeShadowCopy1/ada" failed: No >such file or directory (2) >rsync error: some files could not be transferred (code 23) at >/home/lapo/packaging/tmp/rsync-2.6.3/main.c(702) >Is rsync not capale to access devices with this syntax? >Or do I make an other error? >Thanks for your help >Michael Apparently not. I've done quite a bit of testing with this. It seems as though the windows xp copy command is the only command that can recognize the temporary volume shadow device. It seems as though xcopy doesn't even recognize the device. What I've done in the past is created a script that creates the volume shadow copy, copies the files to a temp directory, rsyncs them from there, then deletes the files. The rsync algorithm still seems to work fine with only the changes. It would be awesome if rsync could recognize the volume shadow copy device, but I have no idea what that would entail. Also, with Windows 2003 Server, you can create volume shadow copies that are linked to a drive letter that rsync does recognize. Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Issue with hard links, please help!
> Recheck the statistics: > 4GB file something like 4,000,000,000 bytes > Total bytes sent: 87,436,048 -- MUCH LESS than 4 GB > Total bytes received: 539,014 > > > > Total transferred file size: 4077908049 bytes Sorry, got it now. I missed the 'Total bytes sent' stat and was assuming the 'Total transferred file size' meant that is what was transferred. For this test I'm on a 100mb local connection, so it seemed like it was taking long enough for this to be true. Thanks for all the help. Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Issue with hard links, please help!
> > Number of files: 50285 > > Number of files transferred: 38 > > Total file size: 16193254538 bytes > > Total transferred file size: 4077908049 bytes > > Literal data: 86201342 bytes > > Matched data: 3989904700 bytes > > File list size: 945440 > > File list generation time: 6.615 seconds > > File list transfer time: 0.000 seconds > > Total bytes sent: 87436048 > > Total bytes received: 539014 > > > > sent 87436048 bytes received 539014 bytes 97913.26 bytes/sec > > total size is 16193254538 speedup is 184.07 > > > > Well, it ends up that there is a Microsoft backup file (a .bkf file) > > that is around 4GB in size that is being changed daily. > > > > Now my question (I think the final one) is why the entire file seems to > > be transferred even though rsync obviously detects that only a fraction > > of the file has changed. Evidently the Literal Data shows 86201342 of > > changes which appears correct. Also, since I'm using option > > --log-format="%f %l %b", I see on the file in question, the following > > results: > > > > SERVER/E$/exchange.bkf 4076087296 86454659 > > > > Isn't this stating that the file size is 4076087296, and the changes to > > the file are 86454659? > > > > So why is the entire file transferring each day. I'm using the > > --no-whole-files option. Here is the rsync command options I used for > > the latest test: > > Rsync has NO guarantee that the only changes are to the END. > Rsync has to work when the changes are to the beginning or scattered > throughout. > Rsync goes to a lot of trouble to find and transmit only the changes. > This is extremely useful over slow and/or erratic network connections. > This is probably significantly slower over gigabit ethernet. > > Also, be aware that of the times that are representable in Unix, > DOS and derivatives are only capable of represententing half of them. > Depending on whatever, you may have DOS files that are always seen > as being different because the times do not and cannot match. Rsync seems to be detecting what the changes on this large file. Based on what you are saying, rsync in this case knows what the changes are in the file, roughly 86mb, but cannot transmit only the changes and therefore transmits the entire 4Gb file? If there is no way around this, I guess I'll have to live with it. > > rsync /share/ /backup/05-13-2006/ -v --link-dest=/backup/05-12-2006/ > > --stats --recursive --archive --times --modify-window=1 --delete > > --ignore-errors --files-from=/var/www/html/backup/adlist.txt > > --exclude-from=/scripts/file-exclude --no-whole-file --log-format="%f %l > > %b" 2> errors.log 1> stats.log\ > > > > In the previous posts I stated that du showed every incremental > > directory to be around 4-5gb in size. This is because each day the > > exchange.bkf has some change associated with it, so I guess the file > > cannot be linked. So in reality if you have very large files that have > > very small changes applied, hard-links really serve no purpose, correct? > > And I assume there is nothing else that can be done with these large > > files to conserve space? > Hard links are how unix names files (the file itself) > Hard links allow one file to have more than one name. > Any change to the file (by any name) is done to the file and shows up in > all > the other names. > When the last name (actually reference) is deleted, the file is deleted. > There is no "yes, but" associated with hard links. > Hard links will not help save space on similar but not exactly the same > files. That's what I figured, just wanted to clarify. So if you had a directory with 10 1GB files, and each day you made a 10k change to each, all your incremental directories would have 10GB total, nothing saved from hard-linking. Thanks again. Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Issue with hard links, please help!
> You could of course (right after an rsync run) do a > "cd newdir; find . -type f -links 1 -print" and then randomly check a > couple and compare all their attributes such as mtime, permissions to > the previous dir. (I still recommend using the --link-dest thing over > using cp -al first.) Ok, I think I've figured out the problem with this one, although I'm not exactly sure of the reason. I have now started using --link-dest and this works great. Here again is the stat screen: Number of files: 50285 Number of files transferred: 38 Total file size: 16193254538 bytes Total transferred file size: 4077908049 bytes Literal data: 86201342 bytes Matched data: 3989904700 bytes File list size: 945440 File list generation time: 6.615 seconds File list transfer time: 0.000 seconds Total bytes sent: 87436048 Total bytes received: 539014 sent 87436048 bytes received 539014 bytes 97913.26 bytes/sec total size is 16193254538 speedup is 184.07 Well, it ends up that there is a Microsoft backup file (a .bkf file) that is around 4GB in size that is being changed daily. Now my question (I think the final one) is why the entire file seems to be transferred even though rsync obviously detects that only a fraction of the file has changed. Evidently the Literal Data shows 86201342 of changes which appears correct. Also, since I'm using option --log-format="%f %l %b", I see on the file in question, the following results: SERVER/E$/exchange.bkf 4076087296 86454659 Isn't this stating that the file size is 4076087296, and the changes to the file are 86454659? So why is the entire file transferring each day. I'm using the --no-whole-files option. Here is the rsync command options I used for the latest test: rsync /share/ /backup/05-13-2006/ -v --link-dest=/backup/05-12-2006/ --stats --recursive --archive --times --modify-window=1 --delete --ignore-errors --files-from=/var/www/html/backup/adlist.txt --exclude-from=/scripts/file-exclude --no-whole-file --log-format="%f %l %b" 2> errors.log 1> stats.log\ In the previous posts I stated that du showed every incremental directory to be around 4-5gb in size. This is because each day the exchange.bkf has some change associated with it, so I guess the file cannot be linked. So in reality if you have very large files that have very small changes applied, hard-links really serve no purpose, correct? And I assume there is nothing else that can be done with these large files to conserve space? Thanks Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Issue with hard links, please help!
> On Thu 11 May 2006, Max Kipness wrote: > > > [EMAIL PROTECTED] backup]# cp -al Latest/ mtest/ > > [EMAIL PROTECTED] backup]# du --max-depth=1 -h > > 21G ./Latest > > 8.7M./mtest > > 21G . > > [EMAIL PROTECTED] backup]# rm mtest/ -rf > > [EMAIL PROTECTED] backup]# cp -al Latest/ test/ > > [EMAIL PROTECTED] backup]# du --max-depth=1 -h > > 21G ./test > > 8.3M./Latest > > 21G . > > > > The last instance is the problem that happens quite often. Now when I > > No, it's not a problem. It's just that now du encounters the "test" > directory before finding the "Latest" directory. du only counts the > blocks of hardlinked files once, and reports the size under the first > directory such a file is in. > > The 8.3M (or 8.7M) is purely the disk blocks needed for the directories > and any symlinks if applicable, it is *not* related to storage of file > contents. > > If you run "du -s -h test Latest" (or use the --count-links option) you > will see that each directory is handled separately, and both will have > 21G. > > > perform an rsync as such: > > > > rsync /share/ /backup/Latest --stats --recursive --archive --times > > --modify-window=1 --delete --ignore-errors --no-whole-file > > --files-from=/var/www/html/new/var/backup_selections.txt > > --exclude-from=/var/www/html/new/var/file-exclude --progress > > > > I get the following results: > > > > Number of files: 53911 > > Number of files transferred: 52223 > > Total file size: 21654476720 bytes > > Total transferred file size: 21654476720 bytes > > Literal data: 21651840443 bytes > > Matched data: 0 bytes > > File list size: 992872 > > Total bytes sent: 21657710607 > > Total bytes received: 1044480 > > > > And a du gives me: > > > > [EMAIL PROTECTED] backup]# du --max-depth=1 -h > > 21G ./test > > 21G ./Latest > > 41G . > > > > It appears that due to the cp -al command not working right as stated > > above, the literal changes needed was everything minus the 8.3mb, when > > in reality there were very few changes between 'Share' and 'Latest'. > > What's happened is that the files are updated, and the hard link is > lost. Why the files are updated I can't say, it could be due to all > sorts of reasons; perhaps using the --itemize-changes option will help. > > Look into the --link-dest option, you can leave out your cp -al pass in > that case. Thanks for the info. I think I understand better how the hard linking works now. I still can't seem to figure out why the hard links are breaking though. And now I've noticed that I have similar issues on other server with hard links (evidently). In one instance, the server has maybe 600mb or so of changes per day, and a total of about 19GB total files, yet each incremental directory shows 5gb or so when doing a du. So does that mean that there are hard links breaking daily? du --max-depth=1 -h /backup 18G /backup/05-02-2006 5.1G/backup/04-26-2006 5.4G/backup/05-05-2006 5.0G/backup/04-23-2006 5.1G/backup/04-29-2006 5.1G/backup/04-27-2006 5.0G/backup/04-17-2006 5.0G/backup/04-13-2006 5.4G/backup/05-08-2006 3.8G/backup/05-06-2006 5.0G/backup/04-20-2006 3.9G/backup/05-07-2006 5.9G/backup/Current 5.1G/backup/05-03-2006 3.7G/backup/05-01-2006 5.0G/backup/04-21-2006 5.0G/backup/04-19-2006 5.0G/backup/04-25-2006 5.0G/backup/04-14-2006 3.6G/backup/04-15-2006 5.0G/backup/04-24-2006 3.7G/backup/04-28-2006 3.7G/backup/04-30-2006 3.9G/backup/05-09-2006 3.9G/backup/05-11-2006 3.6G/backup/04-16-2006 5.0G/backup/04-18-2006 3.9G/backup/05-10-2006 5.1G/backup/05-04-2006 146G/backup [EMAIL PROTECTED] reports]# du -sh /backup/05-02-2006/ /backup/04-26-2006/ /backup/05-05-2006/ /backup/04-23-2006 /backup/04-29-2006 /backup/04-27-2006 /backup/04-17-2006 /backup/04-13-2006 /backup/05-08-2006 /backup/05-06-2006 /backup/04-20-2006 /backup/05-07-2006 /backup/Current /backup/05-03-2006 /backup/05-01-2006 /backup/04-21-2006 /backup/04-19-2006 /backup/04-25-2006 /backup/04-14-2006 /backup/04-15-2006 /backup/04-24-2006 /backup/04-24-2006 /backup/04-30-2006 /backup/05-09-2006 /backup/05-11-2006 /backup/04-16-2006 /backup/04-18-2006 /backup/05-10-2006 /backup/05-04-2006 18G /backup/05-02-2006/ 18G /backup/04-26-2006/ 19G /backup/05-05-2006/ 18G /backup/04-23-2006 18G /backup/04-29-2006 18G /backup/04-27-2006 18G /backup/04-17-2006 18G /backup/04-13-2006 19G /backup/05-08-2006 19G /backup/05-06-2006 18G /backup/04-20-2006 19G /backup/05-07-2006
Issue with hard links, please help!
Hello, Sometimes when creating hard links to the rsync destination directory, it seems like the new directory (created from the cp -al command) ends up with all the data. This causes a problem in the sense that if the rsync destination directory had 21GB, after the cp -al command, it ends up having only 8mb, then the rsync source directory determines that it now requires 21.98GB to update the destination directory. Here is an example of a test that I was doing. I have no idea why sometimes it works like it should, and sometimes it doesn't. My destination directory is called 'Latest'. [EMAIL PROTECTED] backup]# du --max-depth=1 -h 21G ./Latest 21G . [EMAIL PROTECTED] backup]# cp -al Latest/ ktest/ [EMAIL PROTECTED] backup]# du --max-depth=1 -h 21G ./Latest 8.7M./ktest 21G . [EMAIL PROTECTED] backup]# rm ktest/ -rf [EMAIL PROTECTED] backup]# cp -al Latest/ mtest/ [EMAIL PROTECTED] backup]# du --max-depth=1 -h 21G ./Latest 8.7M./mtest 21G . [EMAIL PROTECTED] backup]# rm mtest/ -rf [EMAIL PROTECTED] backup]# cp -al Latest/ test/ [EMAIL PROTECTED] backup]# du --max-depth=1 -h 21G ./test 8.3M./Latest 21G . The last instance is the problem that happens quite often. Now when I perform an rsync as such: rsync /share/ /backup/Latest --stats --recursive --archive --times --modify-window=1 --delete --ignore-errors --no-whole-file --files-from=/var/www/html/new/var/backup_selections.txt --exclude-from=/var/www/html/new/var/file-exclude --progress I get the following results: Number of files: 53911 Number of files transferred: 52223 Total file size: 21654476720 bytes Total transferred file size: 21654476720 bytes Literal data: 21651840443 bytes Matched data: 0 bytes File list size: 992872 Total bytes sent: 21657710607 Total bytes received: 1044480 And a du gives me: [EMAIL PROTECTED] backup]# du --max-depth=1 -h 21G ./test 21G ./Latest 41G . It appears that due to the cp -al command not working right as stated above, the literal changes needed was everything minus the 8.3mb, when in reality there were very few changes between 'Share' and 'Latest'. Can someone give any guidance on this issue? There are time when this will happen several times throughout the 30 day incremental routine so the disk requirements are very large. How can I keep all the data in 'Latest' consistently after using the cp -al command? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
link-dest or cp -al?
I'm looking at updating some of my rsync scripts. For some time I've used the following inside a perl script to create my 30 day versions of backups: cp -al /backup/Current /backup/$yesterday_date rsync /source/ /Current/ --stats --recursive --archive --times --modify-window=1 --delete --ignore-errors --files-from=/var/www/html/backup/list.txt --exclude-from=/scripts/file-exclude --log-format=%f %l %b I've looked at the rsync man page for a description and can't seem to figure out how the link-dest option might work with the scheme I'm using. Or is there really any reason to use link-dest instead? Any help on this would be much appreciated. I'd like to continue just using rsync, as I'm sure there will be some that will respond to use rsnapshot. But doesn't rsnapshot basically do the same thing anyway? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Stats after network failure?
When rsync fails due to a network connectivity issue to a remote host, is it possible to still receive stats to that point in time? In other words, I'm trying to figure out how I could still receive the amount of sent data at least. As of now, with version 2.6.3 it seems like rsync justs prints the errors and stops. Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Sent larger than literal
Hello - When using the stats option I understand that Literal should be the amount of data that did not match, and that Sent should be the actually amount of bytes that were sent. I always use compress for remote transfers, so I'm used to Sent being smaller (usually around half the size) than Literal. However on one server, the backup from last night had the Sent figure larger than Literal even though the compress option is used as is listed below: Number of files: 191253 Number of files transferred: 42 Total file size: 92217159832 bytes Total transferred file size: 692564897 bytes Literal data: 2813738 bytes Matched data: 689751159 bytes File list size: 4910229 Total bytes sent: 6457999 Total bytes received: 392265 sent 6457999 bytes received 392265 bytes 31208.49 bytes/sec total size is 92217159832 speedup is 13461.84 Anybody have an idea why this would happen? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Question about Domino NSF files
I use Rsync to backup 53Gb worth of nsf files from a domino wintel based server, to a linux server. This works fine, however when trying to use cp al to make incremental copies, each copy always ends up being 53Gb in size. Im not sure if this is because the files are in use by the domino server at the time of the backup or if there is some other reason. Has anybody else come across this or know of a possible solution? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync Windows Corrupt File
I've been using rsync for years on linux based systems but just recently downloaded and installed rsync on an XP computer for a certain application. For the most part it works fine except for one issue pertaining to an open file. When backing up the My Documents directory, there is an Outlook .pst file as part of this directory. This is an in use pst file. If Outlook is closed, the pst file seems to sync fine. I'm able to copy it back from the remote server, and open it without issue. However, when Outlook is open, and therefore the pst is in an open state, rsync throws an error about the pst to stderr about not having permissions, yet in the log, it states that the pst was synced. When copying the file back after this process, the file will no longer open as Outlook states that it's not a valid pst file. So it works fine when the pst is closed, but somehow corrupts the remote file if the local file is open?? With linux I'm used to the file being completely skipped. Any suggestions on how to fix this? Here is the command line options: C:\\Program Files\\cwRsync\\rsync.exe" /cygdrive/c/ [EMAIL PROTECTED]://backup/Current/$hostname/ --stats --compress --recursive --partial --files-from=files.txt --bwlimit=$speed --archive --log-format=%f:::%l:::%b Errors: rsync: read errors mapping "/cygdrive/c/Documents and Settings/xxx/My Documents/Max.pst": Permission denied (13) rsync: read errors mapping "/cygdrive/c/Documents and Settings/xxx/My Documents/Max.pst": Permission denied (13) ERROR: Documents and Settings/xxx/My Documents/Max.pst failed verification -- update retained. rsync error: some files could not be transferred (code 23) at main.c(791) Log: Documents and Settings/xxx/My Documents/Max.pst:::271360:::66 Documents and Settings/xxx/My Documents/Max.pst:::271360:::40 Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Cannot Allocate Memory
Hello - I've just upgraded to 2.6.5. On a new set of servers I'm trying to sync I seem to be running into a log of memory allocation errors. With some of the servers, I start the rsync session, and it counts so many files and then stops. At that point I'm unable to control-c out of the process. From another session I can't issue a kill command either...very strange. On other servers, the first few files/directories issue a 'cannot allocate memory', but strangly enough, it continues on. Does this simply mean that the 256mb memory isn't sufficient? I am trying to sync a very large folder structure. Is there anyway around this using tmp disk space or something until I can get more memory? Thanks, Max -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: DU and Hard Links?
> -Original Message- > From: Wayne Davison [mailto:[EMAIL PROTECTED] > Sent: Saturday, June 19, 2004 3:22 PM > To: Max Kipness > Cc: [EMAIL PROTECTED] > Subject: Re: DU and Hard Links? > > On Sat, Jun 19, 2004 at 01:55:02PM -0500, Max Kipness wrote: > > But what I would like to do is automatically get the size > of the last > > created hard linked directory daily through a script. > > Try the attached perl script. It counts the blocks in the > arguments you specify (like du does), but it leaves out any > files that have more links than the number of instances we > find in our scan (i.e. if we find "foo" > with a link count of 2 and we run across it again in our > scan, we count its blocks; if it had a link count of 3 or > more and we only ran across it twice, we'd skip it). > > ..wayne.. > Wayne, that did the trick. Thanks. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DU and Hard Links?
Hi, I'm doing a 30 day rotational backup using rysnc. If I go to the root of the backup directory and use: du --max-depth=1 -h, it gives me the actual space being taken up by each incremental directory, the space being taken by the current directory, and then the total of all. For example: 44G /Current 1G /06-20-2004 750M/06-19-2004 Etc... Etc.. .. .. 70G Total But what I would like to do is automatically get the size of the last created hard linked directory daily through a script. But in the example above, if I move into one of the sub directories, like /06-20-2004, and run: du -sh, it gives me a total of 44G. This is the size of the actual files including the hard links. I'm looking for the 1G or so figure without having to run du from the root, which understandably takes a long time. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: cp -al issues
On Mon, Apr 12, 2004 at 10:18:53AM -0500, Max Kipness wrote: >> It seems there may be links breaking or something. >As I would expect -- all the files that get updated become uniquely linked. Without that, you wouldn't have any backups in >the older dirs. Thanks for the explanation Wayne. I think I understand now that it doesn't matter which directories are seen as holding more than others. However the total for all the directories should still be accurate, right? Well as of last night the total has now jumped 9GB. Am I to assume that there was 9GB worth of changes in one day? I guess it's possible, even though it seems unusual based on my knowledge of how this data is being used. Is there anything else that could cause this sudden increase? Thanks, Max [EMAIL PROTECTED] backup]# du --max-depth=1 -h 37G ./Current 1.4G./04-08-2004 26M ./04-09-2004 38G . [EMAIL PROTECTED] backup]# du --max-depth=1 -h 29G ./Current 9.1G./04-08-2004 211M./04-09-2004 26M ./04-10-2004 39G . [EMAIL PROTECTED] backup]# du --max-depth=1 -h 37G ./Current 9.2G./04-08-2004 838M./04-09-2004 26M ./04-10-2004 24M ./04-11-2004 47G . -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
cp -al issues
Hi, I'm having some strange issues when using cp in my rsync script. It seems there may be links breaking or something. Before running the actual rsync I run the following command in my script: cp -al /backup/Current /backup/$prevday Then I rsync from the source to /Current So shouldn't all my previous days have hard links back to the original /backup directory? Sometimes the /backup directory will end up holding only around 200mb of data, and one of the date directories will end up holding the bulk of the 37GB, should this happen? Shouldn't /Current always have the actual copy (37Gb)? Here is the results of du from the last two days. The first day it seems normal because there was about 1.4Gb of changes, but on the second day 4-8-2004 now has changed to 9.1Gb and Current is only 29Gb?? Strange. Does this mean that 4-8-2004 is no longer properly hard-linked to /Current? Would there be any difference using --link-destination in rsync and not using cp at all? [EMAIL PROTECTED] backup]# du --max-depth=1 -h 37G ./Current 1.4G./04-08-2004 26M ./04-09-2004 38G . [EMAIL PROTECTED] backup]# du --max-depth=1 -h 29G ./Current 9.1G./04-08-2004 211M./04-09-2004 26M ./04-10-2004 39G . -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: File walking issue?
> I'm having an issue with one particular server and am hoping someone > here has dealt with this. > > > > I'm not sure whether this is a strictly samba issue or relates to the > way rsync walks the file list. > > > > Basically after mounting a Windows 2000 file system using and then > rsyncing the contents of this mount, it seems to take 5 - 8 hours to > complete. I've checked on the log periodically and determined that it's > the 'building of the file list' that is taking 95% of the time. We are > only talking about 140,000 files. I do many samba shares and not of them > have this issue. When doing a manual 'ls' command in various directories > on the mount, I encounter no slowness or anything out of the ordinary. > The samba log doesn't give much of a clue either. > > > > Has anybody come across this? Or does anybody have any ideas of how to > troubleshoot? > > > > Oh, and I'm using Rsync 2.6 Max - what is the OS of the machine you're using to access the Win2K SMB share from? I ask because the reference implementation of Samba - ie, the ftp-alike smbclient - works just dandy under every OS I've looked at, but the kernel implementation of smbfs under FreeBSD (and possibly other BSDs, I'm not sure) is severely broken - it will work fine for small operations from an interactive prompt, but if you ask it to handle several thousand files, it slows to an absolute crawl. This is not an rsync-samba interaction problem, it occurs even with a simple cp /mnt/smbshare/* /home/whoever type command as well, if significant numbers of files are involved. Hope this helps. -- I'm actually using RedHat 9 with Samba 2.2.8a. 3.0 was giving me a slight issue with hanging on the mount command. I've got this same basic setup on several servers but this is the only one giving me this issue. I have one that does 130,000 from a samba mount that takes 8 minutes for both the 'file walk' and the actual syncing of files. I've tried a -vv and it doesn't show much of interest. I may try a triple v. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
File walking issue?
Hello, I'm having an issue with one particular server and am hoping someone here has dealt with this. I'm not sure whether this is a strictly samba issue or relates to the way rsync walks the file list. Basically after mounting a Windows 2000 file system using and then rsyncing the contents of this mount, it seems to take 5 - 8 hours to complete. I've checked on the log periodically and determined that it's the 'building of the file list' that is taking 95% of the time. We are only talking about 140,000 files. I do many samba shares and not of them have this issue. When doing a manual 'ls' command in various directories on the mount, I encounter no slowness or anything out of the ordinary. The samba log doesn't give much of a clue either. Has anybody come across this? Or does anybody have any ideas of how to troubleshoot? Oh, and I'm using Rsync 2.6 Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Building File List takes a long time
I'm using rsync 2.6. Last night I started using the -files-from option. It took 6 hours to complete. Total of 126,000 files, 30gb. This was on a local network. I started another rsync process this morning (same system as above) and it's still building the file list after about 5 hours. This shouldn't be the case with 126,000 files, should it? I hear of delays when the file list is in the millions. Based on what I've read in previous posts, rsync may be having trouble walking the file list? This is on a samba file share, but I've got a few others doing this with samba with around 100,000 files that work fairly quick. The only new thing here is the -files-from option. Is there anything else I could be missing? Here is the command I'm using: system ( "/usr/local/bin/rsync", "/share/", "/backup/Current/", "--stats", "--recursive", "--partial", "--archive", "--files-from=/var/www/html/backup/new-adlist.txt", "--exclude-from=/scripts/include/master-exclude", "-v" ); Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Copying hard-linked tree structure
> > > I have a tree structure on one server similar to the following: > > > > > > /Current > > > /01-04-2003 > > > /01-03-2003 > > > > > > etc... > > > > > > /Current holds the most recent rsynced data, and the date > > > directories are created with cp -al on a daily basis so they are > > > hard-linked. I'm going back 60 days. > > > > > > The question is how can I move this entire structure to a > new server > > > and preserve the links from the date directories to the /Current > > > directory? > > > > Well, I ended up rsyncing the root directory to the new server with > > the -H option and it seemed to work. I have 30 directories > for 30 days > > of rotating backups. > > > > However, I had a dir called /Current that had 12Gbs and > then all the > > /date directories had 120mb, 60mb, etc...the daily changes that > > occurred. Well now the directory called /01-01-2004 has 12Gb and > > /Current has like 100mb. I guess /01-01-2004 went first do > to sorting. > > It has to do with the tool you are using to measure them. > > > Anyway to change /Current back as the real directory? Or > does it even > > matter? > > What do you man "real". With hardlinks all links for an > inode are equal. I'm using du --max-depth=1 -h on the root dir. The actual file(s) has to be stored in some directory, right? And then the hard links point to this directory. Well they are all pointing to /01-01-2004 instead of /Current. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Copying hard-linked tree structure
> I have a tree structure on one server similar to the following: > > /Current > /01-04-2003 > /01-03-2003 > > etc... > > /Current holds the most recent rsynced data, and the date > directories are created with cp -al on a daily basis so they > are hard-linked. I'm going back 60 days. > > The question is how can I move this entire structure to a new > server and preserve the links from the date directories to > the /Current directory? Well, I ended up rsyncing the root directory to the new server with the -H option and it seemed to work. I have 30 directories for 30 days of rotating backups. However, I had a dir called /Current that had 12Gbs and then all the /date directories had 120mb, 60mb, etc...the daily changes that occurred. Well now the directory called /01-01-2004 has 12Gb and /Current has like 100mb. I guess /01-01-2004 went first do to sorting. Anyway to change /Current back as the real directory? Or does it even matter? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Copying hard-linked tree structure
Hope this isn't' too off topic... I have a tree structure on one server similar to the following: /Current /01-04-2003 /01-03-2003 etc... /Current holds the most recent rsynced data, and the date directories are created with cp -al on a daily basis so they are hard-linked. I'm going back 60 days. The question is how can I move this entire structure to a new server and preserve the links from the date directories to the /Current directory? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Copying hard-linked tree structure
Hope this isnt' too off topic... I have a tree structure on one server similar to the following: /Current /01-04-2003 /01-03-2003 etc... /Current holds the most recent rsynced data, and the date directories are created with cp -al on a daily basis so they are hard-linked. I'm going back 60 days. The question is how can I move this entire structure to a new server and preserve the links from the date directories to the /Current directory? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Cons of using --size-only
I'm doing some tests on rsyncing some Sendmail spool files every 15 minutes to another Sendmail server as a backup. It seems to work well and efficiently, but I did notice that when users connect via POP the spool file updates it's time. Using log-format option, I could see that even though the file was 0 bytes, it was considered changed and transferring 40 bytes, which I assume should be normal. By adding the --size-only option, this behavior has changed and only changes in file size are being transferred. Is there any negative effect to using this option? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsyncing Windows directory to a Linux box: problem with spacesin directory names
> > Based on what was just said above, I am using Bash version > 2.05b.0(1), > > and no extra configs for splitting are necessary as it splits > > appropriately at the end of the line. > > No, bash splits on all whitespace by default. Take a file like this: > > one two three > four five six > seven eight nine > > put it in a var: > > foo=`cat file` > > and then expand it: > > echo $foo > > This echo command gets 9 separate args. If you use this: > > echo "$foo" > > This echo command gets just 1 arg (with spaces and newlines included). > > However, if you set IFS to contain just a newline, then the > unquoted $foo reference returns 3 args. So, that's one > possible bash solution. You are right about the splitting above, however I can use spaces in my include file without issue. I don't know whether it is due to Bash or not, but I do this on several Redhat 8/9 servers without modification to the shell. Here is my include file: + BACKOFFICE/ + BACKOFFICE/C$/ + BACKOFFICE/C$/Documents and Settings/ - BACKOFFICE/C$/* - BACKOFFICE/* + RAE-DELL/ + RAE-DELL/C$/ + RAE-DELL/C$/Documents and Settings/ - RAE-DELL/C$/* - RAE-DELL/* + SERVER/ + SERVER/C$/ + SERVER/C$/Documents and Settings/ - SERVER/C$/* + SERVER/E$/ + SERVER/E$/Files/ + SERVER/E$/Files/Office/ + SERVER/E$/Files/Users/ - SERVER/E$/Files/* - SERVER/E$/* - /* And here is a partial listing of the log: building file list ... readlink AMY/C$/hiberfil.sys: Text file busy readlink AMY/C$/pagefile.sys: Text file busy readlink 3RDFLOOR/C$/hiberfil.sys: Text file busy readlink 3RDFLOOR/C$/pagefile.sys: Text file busy readlink RAE-DELL/C$/hiberfil.sys: Text file busy readlink RAE-DELL/C$/pagefile.sys: Text file busy done 3RDFLOOR/C$/Documents and Settings/All Users/ 3RDFLOOR/C$/Documents and Settings/All Users/Application Data/Creative/Media Database/ALBUM. DBF 3RDFLOOR/C$/Documents and Settings/All Users/Application Data/Creative/Media Database/ALBUM0 .CDX 3RDFLOOR/C$/Documents and Settings/All Users/Application Data/Creative/Media Database/ARTIST .DBF Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsyncing Windows directory to a Linux box: problem with spacesin directory names
> On Wed, Dec 17, 2003 at 09:05:50AM +0900, Charles Nadeau wrote:
> > backup=`cat /etc/snapshot/include.text` rsync -va [...] $backup
> > $backuproot/$type.1/
>
> This is entirely a shell issue -- the shell you are using is
> splitting the args at all whitespace, and you need to tell it
> to stop. In fact, in some shells $backup would always refer
> to a single arg unless you run the line through an extra
> "eval ..." sequence (bourne shell and bash does way too much
> word splitting for my tastes, for instance).
>
> One option is to use a more advanced shell expansion of the
> file into args. In zsh (my shell of choice), you can do
> something like this:
>
> rsync -va --OPTIONS "${(f)$( $backuproot/$type.1/
>
> This takes the contents of the include.text file, splits in
> by lines, and quotes each one as a separate arg. If you're
> not using zsh, perhaps someone familar with the shell you are
> using can assist you with something similar.
>
> Another option is to upgrade to the CVS version (available in
> the near future as version 2.6.0) and use the --files-from
> option to specify your files (since it already parses the
> names one per line).
Charles,
Based on what was just said above, I am using Bash version 2.05b.0(1),
and no extra configs for splitting are necessary as it splits
appropriately at the end of the line.
Max
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsyncing Windows directory to a Linux box: problem with spacesin directory names
> > >> > I am trying to rsync folders/files hosted on a Windows > machine to a > >> > Linux machine. Many of the directory names have spaces in them. > >> > How can I list them in a file so that the --include-from > option use > >> > them correctly? > >> > Thanks! > >> > > >> > Charles > >> > >> try\ escaping\ the\ spaces\ like\ this "or perhaps quoting" > > > > I just include them with the spaces and it works fine. This is with > > Redhat 8 or 9. > > Max, > > Here are more details on the problem. > > The complete script I am using can be found there: > http://www.mikerubel.org/computers/rsync_snapshots/contributed > /rob_bos. The command from this script that cause me problem > is: rsync -va --delete > --modify-window=2 --delete-excluded > --exclude-from=$excludefile $backup $backuproot/$type.1/ > > Where: > $backup=`cat /etc/snapshot/include.text` $backuproot=/backup > $excludefile=/etc/snapshot/exclude.text > $type=hourly > > /etc/snapshot/include.text contains: > /diskbox/Mariage/*.tdk > /diskbox/Mariage/*.ppj > /diskbox/Outlook/ > /diskbox/PourCD/ > /diskbox/Toburn/ > /diskbox/Anciens projets/ > > /etc/snapshot/exclude.text contains: > diskbox/Mariage/*.AVI > diskbox/Mariage/*.wmv > diskbox/Mariage/*.rm > diskbox/Mariage/*.mp2 > diskbox/Outlook/Outlook/*.* > > When I execute the script above with a directory name > containing space in include.text, I get the following error message: > > building file list ... link_stat /diskbox/Anciens : No such > file or directory link_stat projets/. : No such file or > directory done IO error encountered - skipping file deletion ./ > > Anything I could do besides trying to escape the space or > putting the whole directory between double quote? Not sure what the problem is. Could it be an OS issue? I noticed a difference in general with the way Redhat 8.0 handled directories/folders with spaces in them vs. Redhat 7.3. I've got many include files with spaces in them, especially with the 'Documents and Settings' directories for all the XP/2000 workstations that I back up. And I'm not using any quotes. The only other thing I can think of is to have your script enclose the entire line in double quotes as in: "/diskbox/Anciens projets/" Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsyncing Windows directory to a Linux box: problem with spaces in directory names
> > I am trying to rsync folders/files hosted on a Windows machine to a > > Linux machine. Many of the directory names have spaces in them. > > How can I list them in a file so that the --include-from option use > > them correctly? > > Thanks! > > > > Charles > > try\ escaping\ the\ spaces\ like\ this > "or perhaps quoting" I just include them with the spaces and it works fine. This is with Redhat 8 or 9. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: [RFC] transfer files listed in a file.
> On Wed, Dec 10, 2003 at 07:49:51PM -0500, James A. Morrison wrote: > > Hi, > > > > We've found a need to list the files to transfer in a > files since our > > command lines got too long. Anyway, the attached patch implements > > reading from a file with two command line options. > > > > The first is from-file, this file contains a list of files to get > > from src. The second is subst-file which contains a list > of files to > > send to dst. > > > > e.g. > > rsync -f rsync.files [EMAIL PROTECTED]::jim/ . > > > > and > > > > rsync -s rsync.files [EMAIL PROTECTED]::jim > > > > The first will grab the files listed in rsync.files from > server::jim > > and put them in '.' . The second will send the files listed in > > rsync.files to server::jim . > > see the --files-from option in CVS What is the proper way to patch rsync? I don't seem to be having any luck with: patch [options] [originalfile [patchfile] I need something like this and would like to play around with it on a dev box. Or is this identical to --files-from option? I'd like to be able to just list the files to be rsynced with the full path in a file. I currently use the --include-from option and understand it pretty well now, but I'm working on an automated process for selecting files and this type of option would make the script logic much easier. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Smoother bandwidth limiting
> I rather like bwlimit... i suffer the same problem as Mikko > in that I have a slow uplink. I haven't experienced his > particular problem, though, and bwlimit seems to do its job well... > > Using some other networking tool or QOS just complicates the > matter, and since rsync excels at doing large transfers over > slow connections, bwlimit seems within the scope of rsync... > > So it's not perfect, nothing is... RSYNC would loose a lot > of appeal if bwlimit got axed, at least for me... I had some of these issues initially when using --bwlimit. I since switched to true traffic shaping with CBQ. It works flawlessly as that is it's job. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: --bwlimit not working right
> I cant get the bwlimit option working right. > If i set this option over 400 kbyte per sec i still only get > 400kbyte per sec, whether wich value i set. I try this option > with a 100MB big file. I use a debian stable System with > rsync version 2.5.6cvs protocol version 26. Can someone tell > me how i can this get working? This really doesn't answer your question, but I wanted to mention that I use CBQ on Redhat for QoS and it does a great job at limiting bandwidth to exactly what it's set to. I believe it's included with Debian as well. I use it on several servers to limit saturation. I think I tried the --bwlimit option a while back and it doesn't seem to work to well if you have many small files. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsync download traffic
> On Thu, Oct 16, 2003 at 09:38:32PM -0500, Max Kipness wrote: > > > > I started monitoring lan traffic with RRDTool on a > linux box the > > > > other day that runs rsync, and I've found what I would > consider a > > > > strange traffic pattern. This linux box rsync > > > about 2Gb of > > > > data to a local samba share that is connected to a Windows 2003 > > > > server. Based on the literal data stat, roughly 20% is > > > changed nightly > > > > and uploaded. However, the strange thing is that the traffic > > > > monitoring shows that there is 3mb/s saturation both ways, > > > upload and > > > > download (somehow the server only had a 10mb ethernet adapter). > > > > > > > > This behavior could not possibly be correct, could it? > Or is this > > > > normal. If needed I can post some stats and command > line options. > > > > > > Why couldn't it? Fast disks and CPU will easily overrun a 10Mb > > > connection for network file service. Don't forget 3Mb/s is only > > > 300KB/s compared to 5-30MB/s for disks. > > > > Sorry, I meant to emphasize the point that saturation occurred both > > ways, inbound and outbound. I can understand the saturation on the > > upload as it is transfering changed files up to the > destination, but > > what is it downloading that is so large? > > I assume by upload you mean write and download, read. > > stat(), readdir, and if --no-whole-file reading so it can do > checksums followed by reading in order to merge matched > blocks with new. > > > > Your description leaves a couple of important things out: > > > version of rsync and how rsync views the transfer with regard > > > to networking. > > > > Rsync version 2.5.6. Both servers are on the same local segment. > > As rsync views the transfer, not the physical layout. > A local smbfs mount does not look to rsync like a network tranfer. > > > > > > Make sure that rsync is doing a --whole-file transfer. > > > Otherwise it will really thrash the destination. > > > > I am not doing --whole-file transfer, but will try that for the > > session tonight. Just to let you know, including the --whole-file made a huge difference. The upload (write) speed was still at the 3mb level, but the read (download) was now at less than 100k, instead of matching the upload (write) speed. Thanks for the help. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsync download traffic
> > I started monitoring lan traffic with RRDTool on a linux > > box the other day that runs rsync, and I've found what I would > > consider a strange traffic pattern. This linux box rsync > about 2Gb of > > data to a local samba share that is connected to a Windows 2003 > > server. Based on the literal data stat, roughly 20% is > changed nightly > > and uploaded. However, the strange thing is that the traffic > > monitoring shows that there is 3mb/s saturation both ways, > upload and > > download (somehow the server only had a 10mb ethernet > > adapter). > > > > This behavior could not possibly be correct, could it? Or > > is this normal. If needed I can post some stats and > > command line options. > > Why couldn't it? Fast disks and CPU will easily overrun a > 10Mb connection for network file service. Don't forget > 3Mb/s is only 300KB/s compared to 5-30MB/s for disks. Sorry, I meant to emphasize the point that saturation occurred both ways, inbound and outbound. I can understand the saturation on the upload as it is transfering changed files up to the destination, but what is it downloading that is so large? > Your description leaves a couple of important things out: > version of rsync and how rsync views the transfer with regard > to networking. Rsync version 2.5.6. Both servers are on the same local segment. > Make sure that rsync is doing a --whole-file transfer. > Otherwise it will really thrash the destination. I am not doing --whole-file transfer, but will try that for the session tonight. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Rsync download traffic
Hello, I started monitoring lan traffic with RRDTool on a linux box the other day that runs rsync, and I've found what I would consider a strange traffic pattern. This linux box rsync about 2Gb of data to a local samba share that is connected to a Windows 2003 server. Based on the literal data stat, roughly 20% is changed nightly and uploaded. However, the strange thing is that the traffic monitoring shows that there is 3mb/s saturation both ways, upload and download (somehow the server only had a 10mb ethernet adapter). This behavior could not possibly be correct, could it? Or is this normal. If needed I can post some stats and command line options. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Never up to date
> >> > The Stats shows a literal data of 0, so it would appear that it > >> > recognizes the data is in sync, but why does it show the > >> filenames? I > >> > thought it should only show the names of the files that > >> have changed? > >> > >> For enlightenment, see the description of the -t (--times) > >> option in the man page. > >> > > Thanks for the info, that seemed to do the trick when > rsyncing from a > > local dir to another local dir. > > > > However, I still have similar, but slightly different results when > > rsyncing from local dir to local samba mount. Some of the file names > > still show up, but not all. When using -vv, of course, the ones that > > showed up with -v do not state 'up to date', while the others do. > > Windows file timestamps have a granularity of 2 seconds instead of 1. > > Use --modify-window=1 whenever rsyncing with samba mount filesystems. Thanks John, --modify-window=1 has fixed the problem. Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Never up to date
> On Wed, Oct 08, 2003 at 09:57:39PM -0500, Max Kipness wrote: > > The Stats shows a literal data of 0, so it would appear that it > > recognizes the data is in sync, but why does it show the > filenames? I > > thought it should only show the names of the files that > have changed? > > For enlightenment, see the description of the -t (--times) > option in the man page. > Thanks for the info, that seemed to do the trick when rsyncing from a local dir to another local dir. However, I still have similar, but slightly different results when rsyncing from local dir to local samba mount. Some of the file names still show up, but not all. When using -vv, of course, the ones that showed up with -v do not state 'up to date', while the others do. Any other thoughts? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Never up to date
I've got a strange issue that I'm hoping someone can help out with. Just installed 2.5.6 on a Redhat 8 system. Was doing a simple test from one local directory to another (in the root) with a couple of text files in the source. Using the following command line: rsync /directory1/ /directory2/ -v --recursive --stats After the first sync all files are copied properly, but then every sync afterwards continues to show the name of the text files because of the -v. The Stats shows a literal data of 0, so it would appear that it recognizes the data is in sync, but why does it show the filenames? I thought it should only show the names of the files that have changed? Also, if I run the same command above but add the -c (checksum) option, then it seems to work correctly. Then it does not show the names of the files with the -v option, unless they have changed. 1. Should I have to use the -c option? 2. And if not, any idea why it doesn't work without it? 3. I'm about to do a very large rsync from one local directory to a local samba mount (Windows 2003). So far my tests have shown the exact results as above with this samba share, unless I use -c. Is it a problem to use -c with a large rsync? I like to only see the changed files so I can parse this data for reporting purposes. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Logging updated files?
> > What I'd like, is to be able to get a listing of all files that > > changed on the local side and that were updated to the > remote side via > > rsync. The literal data info tells me how many bytes were > transferred > > total, but I'd like to get the break down per file, or at the least > > just get a listing of the files rsync saw as different and > therefore > > updated. > > > > I've played with --progress and --stats and not really > found the info. > > The --progress option seems to show every file in the root > regardless > > of whether it's actually different or not. The closest I've > gotten is > > with the --log-format %f%l%b option which tells filename, > file length, > > and bytes transferred. The problem with this is that there > is always > > at least 40 bytes transferred for every file, so I'm not sure how I > > could tell if there is a very slight change in the file. > > -v with or without --progress is only supposed to show the > files being transferred. Unchanged files do not appear in > the list unless you use more --verbose and even then the get > additional labelling. A single -v without --progress should > meet your minimum requirement. If it is listing every file > you may be having a lot of false positives and want to review > the options you are using. > Now that I look at some of the different rsync implementations I'm doing, I notice that linux to linux rsyncs are logging only changed files being transferred as you state above. However, my linux to samba (local mount to W2K), are the ones that seem to be logging everything regardless of change. The only difference in options is that the linux to samba is not using the --archive switch because of change of owner issues. Should the rsync to samba being performing this way? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Logging updated files?
> On Wed, Sep 24, 2003 at 03:38:06PM +1000, Martin Pool wrote: > > Why did you send six copies of this? > > He got NDR delevery delay notices and mistook them for > bounces. > Yes, I continued to get the following error message back: soul_rebel,reflex.at (The name was not found at the remote site. Check that the name has been entered correctly.) However, I later found out that I was having DNS problems on a relay so that is why I did not get any of the rsync list emails. So combine the two, and I had no idea that all six messages were being posted, so I apologize. Does anyone have any suggestions for my original question of getting a log of updated files? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Logging updated files?
Hello - I've been doing some experimenting this morning with logging and can't seem to get exactly what I'm looking for. What I'd like, is to be able to get a listing of all files that changed on the local side and that were updated to the remote side via rsync. The literal data info tells me how many bytes were transferred total, but I'd like to get the break down per file, or at the least just get a listing of the files rsync saw as different and therefore updated. I've played with --progress and --stats and not really found the info. The --progress option seems to show every file in the root regardless of whether it's actually different or not. The closest I've gotten is with the --log-format %f%l%b option which tells filename, file length, and bytes transferred. The problem with this is that there is always at least 40 bytes transferred for every file, so I'm not sure how I could tell if there is a very slight change in the file. Any ideas on how to do this accurately? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Operation not permitted?
Hello, Can someone tell me what the problem is here. I am doing an rsync on a sendmail spool directory to a folder that is a samba mount. Why is rsync trying to change owner? Does it have to? I tried manually changing owner (as root) on a file that is sitting on the samba mount and I got the same operation not permitted error. Does anybody know why this is? Or do I need to ask in a samba mailling list. Here is the error: chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Strange errors
After attempting to rsync the /var/spool/mail folder on a sendmail server, I got the following errors on some of the folders: chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted chown var/spool/mail/[EMAIL PROTECTED] : Operation not permitted Number 1, what is it trying to do? Number 2, is this harmful in any way? Number 3, how do I get this to work? Rsync stopped shortly after this with errors. I'm trying to backup sendmail mail spools. Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Patching?
Hello, Hopefully this isn't too stupid of a question, but how do you patch rsync with the --files-from patch? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Follup Up]: Another Question about include-from
> I know this subject has been discussed time and time again, > but everytime I think I've figured it out, I get stuck again. > > Here is the file structure, root is /: > > /var > /etc > /otherdir1 > /otherdir2 > > I want to rsync the following: > > /var/spool/mail > but nothing else in /var or /var/spool/ > /etc/httpd/ > /etc/mail/ > but nothing else in /etc > > Do I have to do this for example? > + /var/ > +/var/spool/ > + /var/spool/mail/ > - /var/spool/dir1 > - /var/spool/dir2 > - /var/spool/dir3 > - /var/dir1 > - /var/dir2 > -/etc, etc > -/* > > And exclude every other dir at that level? Is there not a > better way? Is this what the --files-from patch is for? Well, I figured out that part of the problem was the beginning slashes that didn't belong. I also figured out that you don't need to exclude every directory, but can just use * for every level. But is rsyncing various combinations at different levels easier to do with the --files-from patch? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Another Question about include-from
I know this subject has been discussed time and time again, but everytime I think I've figured it out, I get stuck again. Here is the file structure, root is /: /var /etc /otherdir1 /otherdir2 I want to rsync the following: /var/spool/mail but nothing else in /var or /var/spool/ /etc/httpd/ /etc/mail/ but nothing else in /etc Do I have to do this for example? + /var/ +/var/spool/ + /var/spool/mail/ - /var/spool/dir1 - /var/spool/dir2 - /var/spool/dir3 - /var/dir1 - /var/dir2 -/etc, etc -/* And exclude every other dir at that level? Is there not a better way? Is this what the --files-from patch is for? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Question on --include-from option.
It's a little early here in Dallas and I'm on my first cup...
I'm not sure if this is mandatory, but I would also add plus/minus
characters as follows:
+ testrsync1/
+ testrsync2/
+ testrsync3/
- /*
That way you are assured to sync what you want, and not what you don't
want.
Max
-Original Message-
From: Loris Serena [mailto:[EMAIL PROTECTED]
Sent: Wednesday, August 13, 2003 7:39 AM
To: Max Kipness
Cc: [EMAIL PROTECTED]
Subject: Re: Question on --include-from option.
Hi Max!
Done that, still no joy... ;-((
ppukweb2% more rsync-include-file
/tmp/loris/testrsync1/
/tmp/loris/testrsync2/
/tmp/loris/testrsync3/
ppukweb2% rsync -azv -e ssh --stats
--include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris
receiving file list ... done
drwxr-xr-x512 2003/08/13 12:15:01 loris
drwxr-xr-x512 2003/08/13 12:15:01 loris/testrsync1
Number of files: 2
Number of files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 79
Total bytes written: 100
Total bytes read: 95
wrote 100 bytes read 95 bytes 130.00 bytes/sec
total size is 0 speedup is 0.00
ppukweb2%
Max Kipness wrote:
Hi Loris,
I believe you will need to add a trailing slash to each
line to get the contents within.
Max
-Original Message-
From: "Loris Serena"<[EMAIL PROTECTED]>
Sent: 8/13/03 7:15:40 AM
To: "[EMAIL PROTECTED]"<[EMAIL PROTECTED]>
Subject: Question on --include-from option.
Hello list,
I am running rsync 2.5.5 on some Solaris 7 and 8 boxes.
I'd like to sync different directories from one box to
another.
This is my include file:
ppukweb2% more rsync-include-file
/tmp/loris/testrsync1
/tmp/loris/testrsync2
/tmp/loris/testrsync3
This is the command I run:
rsync -azv -e ssh --stats
--include-from=/tmp/rsync-include-file
ppukweb8:/tmp/loris
and this is the output I get:
ppukweb2% rsync -azv -e ssh --stats
--include-from=/tmp/rsync-include-file
ppukweb8:/tmp/loris
receiving file list ... done
drwxr-xr-x512 2003/08/13 12:15:01 loris
drwxr-xr-x512 2003/08/13 12:15:01
loris/testrsync1
Number of files: 2
Number of files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 79
Total bytes written: 97
Total bytes read: 95
wrote 97 bytes read 95 bytes 128.00 bytes/sec
total size is 0 speedup is 0.00
ppukweb2%
Basically, nothing gets tranferred from ppukweb2 to
ppukweb8.
Can anybody see what I'n doing wrong?
Thanks in advance
Loris
--
Loris Serena
Unix Systems Administrator
PaddyPower Bookmakers
Phone : +353-1-4049633
http://www.paddypowerplc.com
Mobile: +353-87-9683760
mailto:[EMAIL PROTECTED]
Privileged, confidential and/or copyright information
may be contained in this communication.
This e-mail and any files transmitted with it are
confidential and intended solely for the use
of the individual or entity to whom they are addressed.
If you are not the intended addressee,
you may not copy, forward, disclose or otherwise use
this e-mail or any part of it in any way
whatsoever. To do so is prohibited and may be unlawful.
If you have received this email in error
please notify the sender immediately.
Paddy Power PLC may monitor the content of e-mail sent
and received for the purpose
of ensuring compliance with its policies and procedures.
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read:
http://www.catb.org/~esr/faqs/smart-questions.html
--
To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
RE: Question on --include-from option.
Try: testrsync1/ testrsync2/ testrsync3/ Since /tmp/loris is your root dir. And put a trailing slash after /tmp/loris/ on rsync command line. Max -Original Message- From: Loris Serena [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 13, 2003 7:39 AM To: Max Kipness Cc: [EMAIL PROTECTED] Subject: Re: Question on --include-from option. Hi Max! Done that, still no joy... ;-(( ppukweb2% more rsync-include-file /tmp/loris/testrsync1/ /tmp/loris/testrsync2/ /tmp/loris/testrsync3/ ppukweb2% rsync -azv -e ssh --stats --include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris receiving file list ... done drwxr-xr-x512 2003/08/13 12:15:01 loris drwxr-xr-x512 2003/08/13 12:15:01 loris/testrsync1 Number of files: 2 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 79 Total bytes written: 100 Total bytes read: 95 wrote 100 bytes read 95 bytes 130.00 bytes/sec total size is 0 speedup is 0.00 ppukweb2% Max Kipness wrote: Hi Loris, I believe you will need to add a trailing slash to each line to get the contents within. Max -Original Message- From: "Loris Serena"<[EMAIL PROTECTED]> Sent: 8/13/03 7:15:40 AM To: "[EMAIL PROTECTED]"<[EMAIL PROTECTED]> Subject: Question on --include-from option. Hello list, I am running rsync 2.5.5 on some Solaris 7 and 8 boxes. I'd like to sync different directories from one box to another. This is my include file: ppukweb2% more rsync-include-file /tmp/loris/testrsync1 /tmp/loris/testrsync2 /tmp/loris/testrsync3 This is the command I run: rsync -azv -e ssh --stats --include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris and this is the output I get: ppukweb2% rsync -azv -e ssh --stats --include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris receiving file list ... done drwxr-xr-x512 2003/08/13 12:15:01 loris drwxr-xr-x512 2003/08/13 12:15:01 loris/testrsync1 Number of files: 2 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 79 Total bytes written: 97 Total bytes read: 95 wrote 97 bytes read 95 bytes 128.00 bytes/sec total size is 0 speedup is 0.00 ppukweb2% Basically, nothing gets tranferred from ppukweb2 to ppukweb8. Can anybody see what I'n doing wrong? Thanks in advance Loris -- Loris Serena Unix Systems Administrator PaddyPower Bookmakers Phone : +353-1-4049633 http://www.paddypowerplc.com Mobile: +353-87-9683760 mailto:[EMAIL PROTECTED] Privileged, confidential and/or copyright information may be contained in this communication. This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended addressee, you may not copy, forward, disclose or otherwise use this e-mail or any part of it in any way whatsoever. To do so is prohibited and may be unlawful. If you have received this email in error please notify the sender immediately. Paddy Power PLC may monitor the content of e-mail sent and received for the purpose of ensuring compliance with its policies and procedures. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Loris Serena Unix Systems Ad
RE: Question on --include-from option.
Hi Loris, I believe you will need to add a trailing slash to each line to get the contents within. Max -Original Message- From: "Loris Serena"<[EMAIL PROTECTED]> Sent: 8/13/03 7:15:40 AM To: "[EMAIL PROTECTED]"<[EMAIL PROTECTED]> Subject: Question on --include-from option. Hello list, I am running rsync 2.5.5 on some Solaris 7 and 8 boxes. I'd like to sync different directories from one box to another. This is my include file: ppukweb2% more rsync-include-file /tmp/loris/testrsync1 /tmp/loris/testrsync2 /tmp/loris/testrsync3 This is the command I run: rsync -azv -e ssh --stats --include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris and this is the output I get: ppukweb2% rsync -azv -e ssh --stats --include-from=/tmp/rsync-include-file ppukweb8:/tmp/loris receiving file list ... done drwxr-xr-x512 2003/08/13 12:15:01 loris drwxr-xr-x512 2003/08/13 12:15:01 loris/testrsync1 Number of files: 2 Number of files transferred: 0 Total file size: 0 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 79 Total bytes written: 97 Total bytes read: 95 wrote 97 bytes read 95 bytes 128.00 bytes/sec total size is 0 speedup is 0.00 ppukweb2% Basically, nothing gets tranferred from ppukweb2 to ppukweb8. Can anybody see what I'n doing wrong? Thanks in advance Loris -- Loris Serena Unix Systems Administrator PaddyPower Bookmakers Phone : +353-1-4049633 http://www.paddypowerplc.com Mobile: +353-87-9683760 mailto:[EMAIL PROTECTED] Privileged, confidential and/or copyright information may be contained in this communication. This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended addressee, you may not copy, forward, disclose or otherwise use this e-mail or any part of it in any way whatsoever. To do so is prohibited and may be unlawful. If you have received this email in error please notify the sender immediately. Paddy Power PLC may monitor the content of e-mail sent and received for the purpose of ensuring compliance with its policies and procedures. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Include-from ?
There is more to it than that. I'm working on a perl script that backs up many hosts and I wan't to be able to only modify the include/exclude file for each host. The rsync commands run though a look for each host. Thanks, Max > -Original Message- > From: Johannes Liedtke [mailto:[EMAIL PROTECTED] > Sent: Sunday, August 03, 2003 8:33 AM > To: [EMAIL PROTECTED] > Subject: Re: Include-from ? > > > On Sunday 03 August 2003 15:14, Max Kipness wrote: > > Thanks for the info. I actually did not specify my > configuration right > > so that it is where I was going wrong. However, I'm still > not able to > > only rsync /Test2. > > > > My root directory is: > > > > /Rootdir > > > > And then subdirectories of /Rootdir are: > > > > /Test1 > > /Test2 > > /Test3 > > > > To get only /Test2, I tried the following in the include-only file: > > > > + Rootdir/Test2/ > > - /* > > > > This syncs nothing and gives me: excluding directory > Rootdir because > > of pattern /*. > > Maybe I missed something important here, but what's keeping > you from just > using: > rsync -az /Rootdir/Test2/ /somebackupDir/ > > This would only sync the contents of Test2 to somebackupDir. > For your task (as you described it) there's no need for > --include or --exclude > options... > > Hansen > > -- > Powered by SuSE 8.1pro - KDE 3.0.3 - KMail 1.4.3 > > It never ceases to amaze me how many people can't read instructions. > - Renze de Ruiter on kde-linux > > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, > read: http://www.catb.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Include-from ?
Thanks for the info. I actually did not specify my configuration right so that it is where I was going wrong. However, I'm still not able to only rsync /Test2. My root directory is: /Rootdir And then subdirectories of /Rootdir are: /Test1 /Test2 /Test3 To get only /Test2, I tried the following in the include-only file: + Rootdir/Test2/ - /* This syncs nothing and gives me: excluding directory Rootdir because of pattern /*. Thanks, Max > -Original Message- > From: jw schultz [mailto:[EMAIL PROTECTED] > Sent: Sunday, August 03, 2003 2:05 AM > To: [EMAIL PROTECTED] > Subject: Re: Include-from ? > > > On Sun, Aug 03, 2003 at 01:35:22AM -0500, Max Kipness wrote: > > Hello - > > > > I just finished doing a search of the archives and couldn't seem to > > get this answered. > > > > Example, I have a root directory containing the following > directories: > > > > /Test1 > > /Test2 > > /Test3 > > > > I want to rsync only /Test2 and all subdirs under. So my > include-from > > file looks like: > > > > + Test2/* > > - /* > > > > This doesn't work as nothing is synced, and if I do a -vv > it tells me > > that nothing is being synced because I've excluded all. > > I'm sure it didn't say that. Instead it would have spit out > a bunch of lines including: > excluding directory Test2 because of pattern /* > That is because you never included Test2, only everything > under it. > > | + Test2/ > | - /* > > Would be what you want. > > > > -- > > J.W. SchultzPegasystems Technologies > email address: [EMAIL PROTECTED] > > Remember Cernan and Schmitt > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, > read: http://www.catb.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Include-from ?
Hello - I just finished doing a search of the archives and couldn't seem to get this answered. Example, I have a root directory containing the following directories: /Test1 /Test2 /Test3 I want to rsync only /Test2 and all subdirs under. So my include-from file looks like: + Test2/* - /* This doesn't work as nothing is synced, and if I do a -vv it tells me that nothing is being synced because I've excluded all. What am I doing wrong?? Thanks, Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Questions about exclude-file
Thanks for the response. Concerning the first question of excluding pagefile.sys, yes it would indeed be in use. So you are saying that you cannot exclude a file that is in use? - Max > On Mon, Mar 10, 2003 at 07:28:38AM -0600, Max Kipness wrote: > > Is there anyway to keep rsync from reporting an error when > attempting > > to sync pagefile.sys (through a samba share)? It seems like > any type > > of exclusion of the file gets ignored, why is that? Anyway > around it? > > It could only be because your exlude patterns are wrong. > The error is probably because the file is open and therefor locked. > > > Also, of experimented with the exclude-from syntax quite a bit and > > haven't figure out how you could just backup all .doc files > on a file > > system and exclude everything else. > > I almost told you to check the archives but i couldn't find > it quickly so here: > > rsync -blahblah --include='*/' --include='*.doc' --exclude='*' > > That will create of course every single directory even if it > doesn't have a .doc file at any depth beneath it. > > -- > > J.W. SchultzPegasystems Technologies > email address: [EMAIL PROTECTED] > > Remember Cernan and Schmitt > -- > To unsubscribe or change options: > http://lists.samba.org/mailman/listinfo/rsync > Before posting, > read: http://www.tuxedo.org/~esr/faqs/smart-questions.html > -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Questions about exclude-file
Is there anyway to keep rsync from reporting an error when attempting to sync pagefile.sys (through a samba share)? It seems like any type of exclusion of the file gets ignored, why is that? Anyway around it? Also, of experimented with the exclude-from syntax quite a bit and haven't figure out how you could just backup all .doc files on a file system and exclude everything else. Thanks - Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Questions about exclude-file
Is there anyway to keep rsync from reporting an error when attempting to sync pagefile.sys (through a samba share)? It seems like any type of exclusion of the file gets ignored, why is that? Anyway around it? Also, of experimented with the exclude-from syntax quite a bit and haven't figure out how you could just backup all .doc files on a file system and exclude everything else. Thanks - Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Windows Permissions?
Hello - I do some rsyncing of windows files through a samba share. Is there anyway to preserve windows permissions if needed to be restored? Thanks - Max -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync server with conf file
Title: Message Hello - I have normally been using rsync using only a single : in the command line. What are the benefits of setting up an rsync server on the other side using double colons :: ? Is it more efficient? Here is an example of what I use. The server 'iceland' is on the other side of a WAN link. I'm not using an rsync.conf file. rsync iceland://programs/ /journyx-backup/ --progress --stats --compress --recursive --partial --archive -v Thanks - Max
Rsync Logs?
Title: Message Hello - Is there any way to get an ongoing log of what's happening with rsync (most recent files transferred, etc) while rsync is running in the background? I started a 20GB rsync using cron over a wan last night and had know way of knowing where it was at this morning. Thanks - Max
Stats
Title: Message Hello - Can someone please clarify the meaning of these stats. This is the rsync'ing of one file remotely over a wan. Total file size: 383219712 bytesTotal transferred file size: 383219712 bytesLiteral data: 3143680 bytesMatched data: 380076032 bytes The total file size is definitely correct, but what I don't understand is the transfered size. Is rsync reporting that roughly 380mb matches? It would seem like it to me. But is so, why did it transfer the entire file? Or did it? I actually only took 8 minutes which seemed very short (over a slow WAN) Thanks - Max
Replaced file?
Title: Message Hello - I'm wondering if there is a way around this particular issue? I backup a database into a dump file. I backup this file using rsync. I then make a few minor changes in the database and then create another dump file. I then backup with rsync. Instead of merely backing up the incremental changes, it backs up the entire file. I'm sure this is due to the fact that the creation of the dump file doesn't really modify or update the file, but overwrites with a new file. However, the fact remains that there are only a few changes to the dump file. Is there anyway around this? Is there some way to make rsync think this is the same file? How does rsync know this is indeed a different file? Thanks - Max
Include question
Title: Message Hello All - I've been struggling to figure out whether this is possible, and starting to think it isn't. What I'm trying to do is add all directories to be backed up (source) in the include file and not mention any source directories on the actual command line. For example I've tried something like the following with no luck: rsync 10.0.100.254: /backup/ --include-from=/etc/rsync.include -v --progress --stats --compress --recursive --partial --archive The source directories are on the host 10.0.100.254 and are listed in the rsync.include file. The destination is on the local host in the /backup directory. When this command is run, it syncs the root dir of 10.0.100.254 and ignores the include file. Any suggestions? Thanks - Max Thanks - Max ++---------++ Max Kipness II, MCSE, CCNA + Aztech Solutions, Inc. ++ Senior Network Engineer + 1327 Empire Central, STE 205 ++ [EMAIL PROTECTED] + Dallas, Tx 75204 ++ www.aztechsolutions.com + Phone: 214-630-2223 +++-++ ** Solving Business Problems with Technology Solutions ** ++--+
