Re: With rsync --link-dest, is it possible to determine which files are new?
On 10/21/2011 07:14 PM, Ido Magal wrote: Thanks! Unfortunately, it doesn't look like the perl for DroboFS supports the required libraries for your script, error while loading shared libraries: libperl.so.5.10: cannot open shared object file: No such file or directory but --itemize-changes should be sufficient. On Fri, Oct 21, 2011 at 09:10, Kevin Korbk...@sanitarium.net wrote: --itemize-changes # find /backup/dir -type f -links 1 ? -- -Eric 'shubes' -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
On Fri, 21 Oct 2011 12:10:09 -0400, Kevin Korb wrote: If you want something you can run after the fact here is a tool I wrote a while back that does a sort of diff across 2 --link-dest based backups: http://sanitarium.net/unix_stuff/rspaghetti_backup/diff_backup.pl.txt It will also tell you what files were not included in the newer backup which --itemize-changes will not since it doesn't actually --delete anything. I like how comprehensive this is, but wouldn't it - by definition of what it means to have a new file in the new directory tree - always report a modified file as having a different inode? That takes me a quick and dirty solution for this. Simply do a find looking for files with a -links (link count) of 1 to see what files are newly copied in the newer directory tree. - Andrew -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, if a file is a different inode it will be reported by my script even if that is the only difference. If the inode number is different then the file is in fact different and takes up additional disk space. If that is the only thing that is different you might be using rsync incorrectly. This is also why I use md5sum to check files that have different inode numbers but the same mtime and file size. On 10/22/11 10:12, Andrew Gideon wrote: On Fri, 21 Oct 2011 12:10:09 -0400, Kevin Korb wrote: If you want something you can run after the fact here is a tool I wrote a while back that does a sort of diff across 2 --link-dest based backups: http://sanitarium.net/unix_stuff/rspaghetti_backup/diff_backup.pl.txt It will also tell you what files were not included in the newer backup which --itemize-changes will not since it doesn't actually --delete anything. I like how comprehensive this is, but wouldn't it - by definition of what it means to have a new file in the new directory tree - always report a modified file as having a different inode? That takes me a quick and dirty solution for this. Simply do a find looking for files with a -links (link count) of 1 to see what files are newly copied in the newer directory tree. - Andrew - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6i6kUACgkQVKC1jlbQAQfmXACglTh5+UXDdTSFEDZlmWnauUSh 3NEAni0s5XDg0ntixrdNWy+7nogIW9+r =/7b8 -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Opps, forgot that one. That isn't a perl library either but another shell command. On 10/22/11 11:02, Eric Shubert wrote: On 10/21/2011 07:14 PM, Ido Magal wrote: Thanks! Unfortunately, it doesn't look like the perl for DroboFS supports the required libraries for your script, error while loading shared libraries: libperl.so.5.10: cannot open shared object file: No such file or directory but --itemize-changes should be sufficient. On Fri, Oct 21, 2011 at 09:10, Kevin Korbk...@sanitarium.net wrote: --itemize-changes # find /backup/dir -type f -links 1 ? - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6i6mwACgkQVKC1jlbQAQcRRQCgs4p35+qoNH7OsuzoPMaL3Jee 2AYAoMAF/EeLQ6rJsRZuL5qst0PIYesF =p81Y -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
On Sat, 22 Oct 2011 12:07:33 -0400, Kevin Korb wrote: If that is the only thing that is different you might be using rsync incorrectly. And that's why you left it being reported? Interesting idea. Thanks for explaining. - Andrew -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
On Fri, 21 Oct 2011 19:14:00 -0700, Ido Magal wrote: error while loading shared libraries: libperl.so.5.10: cannot open shared object file: No such file or directory This doesn't appear to be a complaint about something the Perl script is doing, but about Perl itself not working. - Andrew -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I had 2 reasons for writing the script... 1. I wanted to know what was different between 2 backups. Capturing - --itemize-changes can certainly do that (I don't remember if my script pre-dates --itemize-changes or not but it predates my knowledge of it) 2. I wanted to know what files were unique to a backup and therefore consuming disk space and why they were different. If I saw an instance where the only difference was the inode number then I would be concerned and investigate why rsync chose to duplicate that file. I have never actually witnessed this happening but I only checked for it a few times. On 10/22/11 15:17, Andrew Gideon wrote: On Sat, 22 Oct 2011 12:07:33 -0400, Kevin Korb wrote: If that is the only thing that is different you might be using rsync incorrectly. And that's why you left it being reported? Interesting idea. Thanks for explaining. - Andrew - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jGloACgkQVKC1jlbQAQevDQCg5Kh5UyR8ymHgXZmlmcSwmNkK kXIAoJzpMbI8aYZw/OJ9iQEIpee+xpcK =QeCn -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shubert e...@shubes.net wrote: # find /backup/dir -type f -links 1 ? -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
On 10/22/2011 04:04 PM, Ido Magal wrote: 'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shuberte...@shubes.net wrote: # find /backup/dir -type f -links 1 ? Sorry, I'm not familiar with droboFS. So I checked it out a bit. From the looks if things, droboFS may not (from their web site) deliver the best file sharing experience ever. I certainly wouldn't trust my data to BeyondRAID technology alone (give me software Raid-10 any day). You're wise to be doing backups of the thing. Good luck with it. -- -Eric 'shubes' -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
On 10/22/2011 07:12 AM, Andrew Gideon wrote: On Fri, 21 Oct 2011 12:10:09 -0400, Kevin Korb wrote: If you want something you can run after the fact here is a tool I wrote a while back that does a sort of diff across 2 --link-dest based backups: http://sanitarium.net/unix_stuff/rspaghetti_backup/diff_backup.pl.txt It will also tell you what files were not included in the newer backup which --itemize-changes will not since it doesn't actually --delete anything. I like how comprehensive this is, but wouldn't it - by definition of what it means to have a new file in the new directory tree - always report a modified file as having a different inode? That takes me a quick and dirty solution for this. Simply do a find looking for files with a -links (link count) of 1 to see what files are newly copied in the newer directory tree. - Andrew Sorry for re-posting your idea Andrew. Great minds think alike. ;) That sometimes happens, especially in threaded view. -- -Eric 'shubes' -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I would say that not doing backups of RAID at any level is rather unwise. Of course I have seen dual drive failures of a RAID1 array on about 5 occasions. I have also seen a dual drive failure and a controller SNAFU of a RAID5 on 2 occasions. Simply put, RAID is not a backup. RAID is to keep you running when a drive dies. Backups are to get you running again when RAID fails or when something happens that RAID doesn't protect from (file deletion/corruption either accident or on purpose). On 10/22/11 23:03, Eric Shubert wrote: On 10/22/2011 04:04 PM, Ido Magal wrote: 'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shuberte...@shubes.net wrote: # find /backup/dir -type f -links 1 ? Sorry, I'm not familiar with droboFS. So I checked it out a bit. From the looks if things, droboFS may not (from their web site) deliver the best file sharing experience ever. I certainly wouldn't trust my data to BeyondRAID technology alone (give me software Raid-10 any day). You're wise to be doing backups of the thing. Good luck with it. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jhg4ACgkQVKC1jlbQAQdzxgCfdj3zH10QrDXQtEYJ1iT68mnx XkMAoLckQyItx+lWIK0PskJmcNL8bk1q =V9RQ -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, is it possible to determine which files are new?
I agree. Raid is no backup. I expect your experience is extensive to see that many dual drive RAID1 failures. Haven't seen one yet myself (knock wood). Several years ago, I recovered all but a handful of files from a 3-drive raid-5 array where 2 drives had failed. Wasn't pretty. Needless to say, there were no backups (wasn't my doing). I know where you're coming from. ;) -- -Eric 'shubes' On 10/22/2011 08:12 PM, Kevin Korb wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I would say that not doing backups of RAID at any level is rather unwise. Of course I have seen dual drive failures of a RAID1 array on about 5 occasions. I have also seen a dual drive failure and a controller SNAFU of a RAID5 on 2 occasions. Simply put, RAID is not a backup. RAID is to keep you running when a drive dies. Backups are to get you running again when RAID fails or when something happens that RAID doesn't protect from (file deletion/corruption either accident or on purpose). On 10/22/11 23:03, Eric Shubert wrote: On 10/22/2011 04:04 PM, Ido Magal wrote: 'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shuberte...@shubes.net wrote: # find /backup/dir -type f -links 1 ? Sorry, I'm not familiar with droboFS. So I checked it out a bit. From the looks if things, droboFS may not (from their web site) deliver the best file sharing experience ever. I certainly wouldn't trust my data to BeyondRAID technology alone (give me software Raid-10 any day). You're wise to be doing backups of the thing. Good luck with it. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jhg4ACgkQVKC1jlbQAQdzxgCfdj3zH10QrDXQtEYJ1iT68mnx XkMAoLckQyItx+lWIK0PskJmcNL8bk1q =V9RQ -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
On 10/22/2011 12:32 PM, Kevin Korb wrote: 2. I wanted to know what files were unique to a backup and therefore consuming disk space and why they were different. Which begs another question that I've struggled a bit with lately. I've considered making a series of snapshots by using --link-dest, and moving(/copying) the backup directory prior to (in between) each run. Once that's done, and there are several snapshots and thus files with link counts of up to the number of snapshots, how can I tell how much space is being used by all of the snapshots (and current backup) put together, counting each file only once. du doesn't appear to cut it. ;) I think I have a solution, but am curious to know what others may think of the problem. -- -Eric 'shubes' -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 du -shc backup1 backup2 ... du -s will not count the same file twice even if it appears 50 times. On 10/22/11 23:32, Eric Shubert wrote: On 10/22/2011 12:32 PM, Kevin Korb wrote: 2. I wanted to know what files were unique to a backup and therefore consuming disk space and why they were different. Which begs another question that I've struggled a bit with lately. I've considered making a series of snapshots by using --link-dest, and moving(/copying) the backup directory prior to (in between) each run. Once that's done, and there are several snapshots and thus files with link counts of up to the number of snapshots, how can I tell how much space is being used by all of the snapshots (and current backup) put together, counting each file only once. du doesn't appear to cut it. ;) I think I have a solution, but am curious to know what others may think of the problem. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jjU0ACgkQVKC1jlbQAQf23QCcCgPkRirDP+6/9+UEGmDlT/vm F3YAoMiAFM8pyZVtkPO6Zz0WXMODtZ9d =lKhT -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. -- -Eric 'shubes' On 10/22/2011 08:43 PM, Kevin Korb wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 du -shc backup1 backup2 ... du -s will not count the same file twice even if it appears 50 times. On 10/22/11 23:32, Eric Shubert wrote: On 10/22/2011 12:32 PM, Kevin Korb wrote: 2. I wanted to know what files were unique to a backup and therefore consuming disk space and why they were different. Which begs another question that I've struggled a bit with lately. I've considered making a series of snapshots by using --link-dest, and moving(/copying) the backup directory prior to (in between) each run. Once that's done, and there are several snapshots and thus files with link counts of up to the number of snapshots, how can I tell how much space is being used by all of the snapshots (and current backup) put together, counting each file only once. du doesn't appear to cut it. ;) I think I have a solution, but am curious to know what others may think of the problem. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The man page says so. My experience doesn't disagree. I am satisfied. On 10/22/11 23:49, Eric Shubert wrote: How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jj1UACgkQVKC1jlbQAQeEXgCg7444EN2dCXIoXiXeqoZQ7gkS w3sAn2EIO33yFHucDsdut1hcQx4ylbz3 =C7Tp -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
On 10/22/2011 08:49 PM, Eric Shubert wrote: How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. Thanks Kevin. You are the wise one: testdir$ ls -lr * -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile2 -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile1 testdir2: total 1024 -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile21 testdir$ du -shc * 1.1Mtestdir2 1.1Mtotal testdir$ I really need to quit assuming. You'd think at my age I'd've learned better. Long live du. :) -- -Eric 'shubes' -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 lol, I have been doing rsync based backups since sometime in 2000. I have learned a few tricks. On 10/23/11 00:00, Eric Shubert wrote: On 10/22/2011 08:49 PM, Eric Shubert wrote: How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. Thanks Kevin. You are the wise one: testdir$ ls -lr * -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile2 -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile1 testdir2: total 1024 -rw-r--r-- 3 shubes shubes 1048576 2011-10-22 20:52 testfile21 testdir$ du -shc * 1.1Mtestdir2 1.1Mtotal testdir$ I really need to quit assuming. You'd think at my age I'd've learned better. Long live du. :) - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jkh4ACgkQVKC1jlbQAQfU3gCg6h9TCxivRigAFUvT++moWLKP qd8AmwW9HsEf3f8qcQG9HwTZXmw5xov/ =Qppe -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
I've read the man page: -s, --summarize display only a total for each argument Leaves a bit to interpretation, I think. Could go either way, no? After all, the name says: du - estimate file space usage I guess du estimates better than I assumed. Thanks for setting me straight Kevin. :) -- -Eric 'shubes' On 10/22/2011 08:51 PM, Kevin Korb wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The man page says so. My experience doesn't disagree. I am satisfied. On 10/22/11 23:49, Eric Shubert wrote: How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jj1UACgkQVKC1jlbQAQeEXgCg7444EN2dCXIoXiXeqoZQ7gkS w3sAn2EIO33yFHucDsdut1hcQx4ylbz3 =C7Tp -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I just looked at 'man du' and you are right about it. I could swear that it used to say something more useful about the subject but I couldn't quote it. In general the man pages for GNU tools suck. On 10/23/11 00:05, Eric Shubert wrote: I've read the man page: -s, --summarize display only a total for each argument Leaves a bit to interpretation, I think. Could go either way, no? After all, the name says: du - estimate file space usage I guess du estimates better than I assumed. Thanks for setting me straight Kevin. :) - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jlJEACgkQVKC1jlbQAQfltgCgydf1fN0eAG2UJ4PXysJMAitJ lLoAnjqjvx6CxK89QAzvlzA7+jcc1n/V =lGRx -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: handling of final partial block in rsync
On Fri, Oct 21, 2011 at 7:28 PM, Peter A. Friend pafri...@octavian.orgwrote: I've read the thesis and the source again, and it looks like rsync is generating a signature on the final partial block. Would someone be able to confirm this, and is rsync handling this as I describe? Search for remainder in generator.c and sender.c. The checksum summary mentions the number of checksums to expect, and the remainder length if the last sum in the sequence is not a full-size block. The sender makes a note of each sum's length when it is received using a simple heuristic (if this is the last sum, the sum's length is the remainder instead of the block length IIF the remainder is non-zero). When checksums are compared, a sum can only match if the sum's length is the same as the block we are trying to find. ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
On Sun, Oct 23, 2011 at 12:14 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I just looked at 'man du' and you are right about it. I could swear that it used to say something more useful about the subject but I couldn't quote it. info du If two or more hard links point to the same file, only one of the hard links is counted. The FILE argument order affects which links are counted, and changing the argument order may change the numbers that `du' outputs. In general the man pages for GNU tools suck. On 10/23/11 00:05, Eric Shubert wrote: I've read the man page: -s, --summarize display only a total for each argument Leaves a bit to interpretation, I think. Could go either way, no? After all, the name says: du - estimate file space usage I guess du estimates better than I assumed. Thanks for setting me straight Kevin. :) -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: With rsync --link-dest, how can you tell how much space is actually used?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, I am aware of info pages. The problem is that reading them is more inconvenient than reverse engineering the behavior of the program. Luckily we have Google to present actual information in a readable format. On 10/23/11 01:41, Frederick Grose wrote: On Sun, Oct 23, 2011 at 12:14 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I just looked at 'man du' and you are right about it. I could swear that it used to say something more useful about the subject but I couldn't quote it. info du If two or more hard links point to the same file, only one of the hard links is counted. The FILE argument order affects which links are counted, and changing the argument order may change the numbers that `du' outputs. In general the man pages for GNU tools suck. On 10/23/11 00:05, Eric Shubert wrote: I've read the man page: -s, --summarize display only a total for each argument Leaves a bit to interpretation, I think. Could go either way, no? After all, the name says: du - estimate file space usage I guess du estimates better than I assumed. Thanks for setting me straight Kevin. :) - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6jqkAACgkQVKC1jlbQAQd+OgCdG2YzILBh8Fesnco1N7rcTYWU LiMAmwST5U9PYfxlXx09irUkIvZmDtBo =OHcL -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[SCM] The rsync repository. - branch master updated
The branch, master has been updated
via 8dd6ea1 Fix --delete-missing-args when --relative is active.
from 7c8f180 Test asprintf() failure with 0, not = 0.
;a=shortlog;h=master
- Log -
commit 8dd6ea1f1ef743fdc28d00137d7bcb746642ebb6
Author: Wayne Davison way...@samba.org
Date: Sat Oct 22 10:20:08 2011 -0700
Fix --delete-missing-args when --relative is active.
---
Summary of changes:
flist.c |8 ++--
1 files changed, 6 insertions(+), 2 deletions(-)
Changeset truncated at 500 lines:
diff --git a/flist.c b/flist.c
index d3f4f64..6ec3c39 100644
--- a/flist.c
+++ b/flist.c
@@ -102,6 +102,7 @@ int flist_eof = 0; /* all the file-lists are now known */
#define NORMAL_NAME 0
#define SLASH_ENDING_NAME 1
#define DOTDIR_NAME 2
+#define MISSING_NAME 3
/* Starting from protocol version 26, we always use 64-bit ino_t and dev_t
* internally, even if this platform does not allow files to have 64-bit inums.
@@ -1933,7 +1934,9 @@ static void send1extra(int f, struct file_struct *file,
struct file_list *flist)
if (name_type != NORMAL_NAME) {
STRUCT_STAT st;
- if (link_stat(fbuf, st, 1) != 0) {
+ if (name_type == MISSING_NAME)
+ memset(st, 0, sizeof st);
+ else if (link_stat(fbuf, st, 1) != 0) {
interpret_stat_error(fbuf, True);
continue;
}
@@ -2285,7 +2288,8 @@ struct file_list *send_file_list(int f, int argc, char
*argv[])
p = fn;
} else
fn = p;
- send_implied_dirs(f, flist, fbuf, fbuf, p,
flags, name_type);
+ send_implied_dirs(f, flist, fbuf, fbuf, p,
flags,
+ st.st_mode == 0 ?
MISSING_NAME : name_type);
if (fn == p)
continue;
}
--
The rsync repository.
___
rsync-cvs mailing list
rsync-cvs@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync-cvs
