Re: Rsync Failed /proc/kcore 128 TiB NTFS HDD Makes Scary sounds even when not mounted
If your hard disk is making scary noises that could be the drive starting to fail have you tried to determine if the drive is still good or on the way out? On Thu, Apr 30, 2015 at 2:12 AM, Kezhawe kezhaw...@gmail.com wrote: Hi, I tried to backup my gentoobox using rsync into a NTFS External Drive Rsync Failed And now there is no space lesft on my root directory / and kcore file size is 128 TiB and my External NTFS HDD makes scary sounds (Like it is working hardly) even when its not mounted rsync created some files/directories that i cant remove them on my External HDD using rm -rf , Please help my to get my system HDD back to normal life. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: increasing the write block size for high latency
One thing you that im not seeing factored in is rpm speed of the drives. On 26 Jul 2014 15:05, L. A. Walsh rs...@tlinx.org wrote: Adam Edgar wrote: It seems the issue is indeed in the ssh layer. scp has the same issue and some work has been done in “fixing” that: http://www.psc.edu/index.php/hpn-ssh From the papers abstract: Status: O SCP and the underlying SSH2 protocol implementation in OpenSSH is network performance limited by statically defined internal flow control buffers. These buffers often end up acting as a bottleneck for network throughput of SCP, especially on long and high bandwith network links. It is *A* bottle neck over networks. look for extensions to ssh to ship unencrypted data streams. There's a patch for this @ http://www.psc.edu/index.php/hpn-ssh. However, rsync is dog slow locally as well for exactly the reasons you mention. An extract from another note on this topic (came up on suse list this week). Someone suggested compression for a speed up... I responded to that: On a local copy or local network, that usually slows down transfers. [ 1000:1 speed ratio with large vs. small io sizes):] One might ask why rsync is so slow -- copying 800G from 1 partition to another via xfsdump/restore takes a bit under 2 hours, or about 170MB/s, but with rsync, on the same partition with rsync transfering less than 1/1000th as much (700MB), it took ~70-80 minutes... or about 163kB/s. That's on the same system (local drive - another local drive) Transfer speeds depend on many factors. One of the largest is transfer size (how much transfered with 1 write /read. Transfer 1GB, 1-meg at a time, took 2.08s read, and 1.56s to write (using direct io). Transfer it at 4K: 37.28s, to read, and 43.02s to write. So 20-40x can be accounted for just on R/W size (1k buffers were 4x slower). Many desktop apps still think 4k is a good read size Over a network, causes drop from 500MB/s down to less than 200KB/s (as seen in FF and TB) -- 2500X. Optimal i/o size on my sys is between 16M-256M. So -- to answer your question, MANY things can affect speed, but I'd look at the R/W size first. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/ mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [Bug 10557] .ssh/config settings are incompletely applied with -e or --rsh
Just to shed light on the OSX matter, they actually use sudo as well. On Thu, Apr 17, 2014 at 9:37 PM, samba-b...@samba.org wrote: https://bugzilla.samba.org/show_bug.cgi?id=10557 --- Comment #3 from Kevin Korb rs...@sanitarium.net 2014-04-17 19:37:06 UTC --- OK, I was speaking from a Linux perspective. I have no idea what OSX uses as root's home dir. Simply put, under sudo you are running as root and root has a different home dir therefore a different ~/.ssh/config file. Also, note that permissions and usernames matter at both ends. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Re[2]: Rsync performance with large exchange database files
the servers at my day job. This works pretty good overall but on one server it's not working so well :) We have an Exchange 2003 server with 4 separate mail store databases. One of them is roughly 900GB the others are ~200GB, ~160GB, and ~50GB. Rsync seems to spend a lot of time trying to find the differences in the files. On the Windows server where rsync is kicked off there's very little CPU or RAM usage for the rsync client. On the server rsync (rsyncd, no ssh) is using around 70-85% of a cpu (well, half a cpu due to hyper threading). I'm using VSS on the windows server to take a snapshot and expose it then running rsync from that to avoid locking issues. Is there anything I should check to help narrow down problems? or any settings I should try that could help speed things up any? Below is the final output of the last two rsync runs to give you an idea. It's taking 30-40+ hours to finish even though it's only transferring 80-160GB of change. Right now I'm testing this against a local rsync server so it should get pretty fast network performance. Eventually it will be moved to our off-site backup but that connection is still pretty fast (20 MBbit) and the backup is only hitting 800-1000 Kbytes/sec. Number of files: 19 Number of files transferred: 6 Total file size: 1265.74G bytes Total transferred file size: 1057.06G bytes Literal data: 160.67G bytes Matched data: 896.39G bytes File list size: 482 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 160.71G Total bytes received: 73.74M sent 160.71G bytes received 73.74M bytes 991.84K bytes/sec total size is 1265.74G speedup is 7.87 [sender] _exit_cleanup(code=0, file=/home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/main.c, line=1052): about to call exit(0) real 2833m1.324s user 2225m55.906s sys 45m10.015s Number of files: 11 Number of files transferred: 6 Total file size: 1268.78G bytes Total transferred file size: 1251.04G bytes Literal data: 83.43G bytes Matched data: 1167.61G bytes File list size: 216 File list generation time: 1.360 seconds File list transfer time: 0.000 seconds Total bytes sent: 83.48G Total bytes received: 87.25M sent 83.48G bytes received 87.25M bytes 836.85K bytes/sec total size is 1268.78G speedup is 15.18 real 1745m5.647s user 1129m14.000s sys 39m58.875s Thanks (in advance) for the help :) rsync options I'm using on the client are : -rltihv --progress --stats --inplace --modify-window=1 On the windows client I'm using cygwin + rsync 3.0.9 but I'm going to test 3.1.0 there and see if there's a difference. On the server it's rsync 3.1.0 running rsyncd. Perhaps useful bit of information :). -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~' `^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~' `^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlL5TgsACgkQVKC1jlbQAQfNTwCfUV1iYYTujpQLHQxdBD1KFLm8 N9kAoMlhdECrmIT/LwC0tUHgXcYMjDfI =42pY -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/ mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/ mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Cannot build rsync 3.1.0 on MacOSX 10.8.5 with fileflags patch
Not trying to derail the tread here, but if you all like I can help you guys out in someway with testing on mac 10.8.5 if you like? On Sunday 27 October 2013 11:17:19 Wayne Davison wrote: On Thu, Oct 10, 2013 at 3:39 PM, Anthony Morton amor...@fastmail.fm wrote: However, attempting to build the new 3.1.0 release on a brand new Mountain Lion box with my standard 'fileflags' and 'crimes' patches applied, make fails, reporting that 'force_change' is undeclared. Yeah, the variable declaration got stuck in the wrong patch. I'm checking-in a fix for that in git. The OS X patches are not well tested at the moment. I managed to snag an old mac to do some build testing at least, but haven't done any testing other than making sure that it passed the make check tests (which I also fixed). ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
readlink works on aix through the ibm toolkit package of coreutils. I will reevaluate the script tomorrow as its late here at work and im not functioning at 100% capacity. On Fri, Sep 6, 2013 at 12:17 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Either you are completely not getting the concept or your OS is not capable of this functionality. I haven't touched AIX since 1995 so I can't really say which. So, again, I am suggesting you try rsnapshot. Either it will fail and prove that AIX can't do this or it will give you a functioning example of this backup concept to look at and decide if you want to do it differently. On 09/05/13 17:44, Jonathan Aquilina wrote: After sym linking I am still having no luck rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-05--23:40:58 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] I am still getting these errors. On Thu, Sep 5, 2013 at 7:57 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 First you rm the existing current link then you create a new one pointing at the backup you just made. It really is just a reference to use in the script to determine what the most recent backup is. The alternative would be to parse a directory listing to determine what the --link-dest parameter should be. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIpAxMACgkQVKC1jlbQAQcewgCgulcebzhLNM2TYJtszIi/TOBn p7oAn12QS1BmY2VTdl4qlKzttDDtXOgu =sImI -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
After sym linking I am still having no luck rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-05--23:40:58 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] I am still getting these errors. On Thu, Sep 5, 2013 at 7:57 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 First you rm the existing current link then you create a new one pointing at the backup you just made. It really is just a reference to use in the script to determine what the most recent backup is. The alternative would be to parse a directory listing to determine what the --link-dest parameter should be. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
issues syncing between one host and another
I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link-dest=$linkdest --out-format='%i %n%L %l %M' --omit-dir-times $localdir $user@$remotehost:$remotedir ssh $user@$remotehost /bin/rm -f /home/aquj001/recievingDir.current ssh $user@$remotehost ln -sf $remotedir $linkdest now the issue becomes when i run it, I get the following errors. rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-04--15:53:16 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] It still creates the directory on the remote machine but its empty and nothing but sym links. Any ideas as to what th eissue is would be greatly appreciated. This is on an AIX 6.1 system installed from the aix toolbox package. -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) There probably shouldn't be a trailing / on the readlink command. If that doesn't settle it is the path in the error message the target or the link-dest? If it is the target try sending a mkdir first. On 09/04/13 09:56, Jonathan Aquilina wrote: I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link-dest=$linkdest --out-format='%i %n%L %l %M' --omit-dir-times $localdir $user@$remotehost:$remotedir ssh $user@$remotehost /bin/rm -f /home/aquj001/recievingDir.current ssh $user@$remotehost ln -sf $remotedir $linkdest now the issue becomes when i run it, I get the following errors. rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-04--15:53:16 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] It still creates the directory on the remote machine but its empty and nothing but sym links. Any ideas as to what th eissue is would be greatly appreciated. This is on an AIX 6.1 system installed from the aix toolbox package. -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIoD/gACgkQVKC1jlbQAQdpnwCfeh0QQ5pa/ixtETZ+2VGTJycF fF4AoO39qJay1Wp172xOkrGlGPswB9sf =5aV6 -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
How would i need to create the sym link and to what? On Thu, Sep 5, 2013 at 7:03 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 No, it is linking the new target to the previous link-dest On 09/05/13 01:02, Jonathan Aquilina wrote: Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) There probably shouldn't be a trailing / on the readlink command. If that doesn't settle it is the path in the error message the target or the link-dest? If it is the target try sending a mkdir first. On 09/04/13 09:56, Jonathan Aquilina wrote: I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link-dest=$linkdest --out-format='%i %n%L %l %M' --omit-dir-times $localdir $user@$remotehost:$remotedir ssh $user@$remotehost /bin/rm -f /home/aquj001/recievingDir.current ssh $user@$remotehost ln -sf $remotedir $linkdest now the issue becomes when i run it, I get the following errors. rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-04--15:53:16 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] It still creates the directory on the remote machine but its empty and nothing but sym links. Any ideas as to what th eissue is would be greatly appreciated. This is on an AIX 6.1 system installed from the aix toolbox package. -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIoEKQACgkQVKC1jlbQAQdMTwCg8/BmBGHwEM5VzpZhQOaxs33S b+0AoJYaGb5poaDdEx+FSxM2kYGZiYhj =DdM8 -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
So if i basically sym link the empty backup with the sym link loop to the current directory it should fix the issue? what would happen though if i want to remove the oldest directory which was synced after 7 days would i have issues then like i am experiencing now? On Thu, Sep 5, 2013 at 7:06 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The .current symlink is a pointer to the current backup. It should always be from the backup you just made to the fixed location of the path you are using readlink on. On 09/05/13 01:04, Jonathan Aquilina wrote: How would i need to create the sym link and to what? On Thu, Sep 5, 2013 at 7:03 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: No, it is linking the new target to the previous link-dest On 09/05/13 01:02, Jonathan Aquilina wrote: Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) There probably shouldn't be a trailing / on the readlink command. If that doesn't settle it is the path in the error message the target or the link-dest? If it is the target try sending a mkdir first. On 09/04/13 09:56, Jonathan Aquilina wrote: I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link-dest=$linkdest --out-format='%i %n%L %l %M' --omit-dir-times $localdir $user@$remotehost:$remotedir ssh $user@$remotehost /bin/rm -f /home/aquj001/recievingDir.current ssh $user@$remotehost ln -sf $remotedir $linkdest now the issue becomes when i run it, I get the following errors. rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-04--15:53:16 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] It still creates the directory on the remote machine but its empty and nothing but sym links. Any ideas as to what th eissue is would be greatly appreciated. This is on an AIX 6.1 system installed from the aix toolbox package. -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Jonathan Aquilina -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIoEUYACgkQVKC1jlbQAQe7fACfZNVdpmkfBCOEgqTPrpbGF93W tqwAnA8x6Gtvgrcj9nBfVvSgx2gcnGiR =eXwi -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: issues syncing between one host and another
Understood, What do i need to sym link together? On Thu, Sep 5, 2013 at 7:34 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The symlink is only a pointer so that you can readlink it on the next run to determine what to use as your --link-dest parameter. It is simply a pointer to the latest directory. Deleting old stuff is a different concept. You have a list of directories with dates and times in the names. A simple ls will list them in order. Any scripting language will have the ability to sort a list of file names. So, you make a sorted list then delete however many you have that are more than the number of them you want starting with the oldest. The current symlink is only pointing to the newest so it would only matter in the deletions if you are deleting all of the backups. On 09/05/13 01:08, Jonathan Aquilina wrote: So if i basically sym link the empty backup with the sym link loop to the current directory it should fix the issue? what would happen though if i want to remove the oldest directory which was synced after 7 days would i have issues then like i am experiencing now? On Thu, Sep 5, 2013 at 7:06 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: The .current symlink is a pointer to the current backup. It should always be from the backup you just made to the fixed location of the path you are using readlink on. On 09/05/13 01:04, Jonathan Aquilina wrote: How would i need to create the sym link and to what? On Thu, Sep 5, 2013 at 7:03 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: No, it is linking the new target to the previous link-dest On 09/05/13 01:02, Jonathan Aquilina wrote: Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) There probably shouldn't be a trailing / on the readlink command. If that doesn't settle it is the path in the error message the target or the link-dest? If it is the target try sending a mkdir first. On 09/04/13 09:56, Jonathan Aquilina wrote: I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link-dest=$linkdest --out-format='%i %n%L %l %M' --omit-dir-times $localdir $user@$remotehost:$remotedir ssh $user@$remotehost /bin/rm -f /home/aquj001/recievingDir.current ssh $user@$remotehost ln -sf $remotedir $linkdest now the issue becomes when i run it, I get the following errors. rsync: change_dir#4 /home/aquj001/recievingDir.2013-09-04--15:53:16 failed: No such file or directory (2) rsync error: errors selecting input/output files, dirs (code 3) at main.c(877) [receiver=3.0.6] rsync: connection unexpectedly closed (9 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] It still creates the directory on the remote machine but its empty and nothing but sym links. Any ideas as to what th eissue is would be greatly appreciated. This is on an AIX 6.1 system installed from the aix toolbox package. -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http
Re: issues syncing between one host and another
The current pointer being the recieveDir.current correct? On Thu, Sep 5, 2013 at 7:37 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The backup you just made to the current pointer you will readlink on the next run. On 09/05/13 01:36, Jonathan Aquilina wrote: Understood, What do i need to sym link together? On Thu, Sep 5, 2013 at 7:34 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: The symlink is only a pointer so that you can readlink it on the next run to determine what to use as your --link-dest parameter. It is simply a pointer to the latest directory. Deleting old stuff is a different concept. You have a list of directories with dates and times in the names. A simple ls will list them in order. Any scripting language will have the ability to sort a list of file names. So, you make a sorted list then delete however many you have that are more than the number of them you want starting with the oldest. The current symlink is only pointing to the newest so it would only matter in the deletions if you are deleting all of the backups. On 09/05/13 01:08, Jonathan Aquilina wrote: So if i basically sym link the empty backup with the sym link loop to the current directory it should fix the issue? what would happen though if i want to remove the oldest directory which was synced after 7 days would i have issues then like i am experiencing now? On Thu, Sep 5, 2013 at 7:06 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: The .current symlink is a pointer to the current backup. It should always be from the backup you just made to the fixed location of the path you are using readlink on. On 09/05/13 01:04, Jonathan Aquilina wrote: How would i need to create the sym link and to what? On Thu, Sep 5, 2013 at 7:03 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: No, it is linking the new target to the previous link-dest On 09/05/13 01:02, Jonathan Aquilina wrote: Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) There probably shouldn't be a trailing / on the readlink command. If that doesn't settle it is the path in the error message the target or the link-dest? If it is the target try sending a mkdir first. On 09/04/13 09:56, Jonathan Aquilina wrote: I have the following script that I am working on #!/bin/bash date=$(date +%F--%T) #ssh parameters user=aquj001 remotehost=qa4app localdir=/home/aquj001/INFENG/scripts remotedir=/home/aquj001/recievingDir.$date linkdest=$(ssh $user@$remotehost /usr/linux/bin/readlink/ /home/aquj001/recievingDir.current) rsync -a --compress --link
Re: issues syncing between one host and another
What will happen if the first directory which is the oldest gets deleted wouldnt i have to re link the next directory with the .current one? On Thu, Sep 5, 2013 at 7:39 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes. On 09/05/13 01:38, Jonathan Aquilina wrote: The current pointer being the recieveDir.current correct? On Thu, Sep 5, 2013 at 7:37 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: The backup you just made to the current pointer you will readlink on the next run. On 09/05/13 01:36, Jonathan Aquilina wrote: Understood, What do i need to sym link together? On Thu, Sep 5, 2013 at 7:34 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: The symlink is only a pointer so that you can readlink it on the next run to determine what to use as your --link-dest parameter. It is simply a pointer to the latest directory. Deleting old stuff is a different concept. You have a list of directories with dates and times in the names. A simple ls will list them in order. Any scripting language will have the ability to sort a list of file names. So, you make a sorted list then delete however many you have that are more than the number of them you want starting with the oldest. The current symlink is only pointing to the newest so it would only matter in the deletions if you are deleting all of the backups. On 09/05/13 01:08, Jonathan Aquilina wrote: So if i basically sym link the empty backup with the sym link loop to the current directory it should fix the issue? what would happen though if i want to remove the oldest directory which was synced after 7 days would i have issues then like i am experiencing now? On Thu, Sep 5, 2013 at 7:06 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: The .current symlink is a pointer to the current backup. It should always be from the backup you just made to the fixed location of the path you are using readlink on. On 09/05/13 01:04, Jonathan Aquilina wrote: How would i need to create the sym link and to what? On Thu, Sep 5, 2013 at 7:03 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: No, it is linking the new target to the previous link-dest On 09/05/13 01:02, Jonathan Aquilina wrote: Doesnt my script already do that at the very end? On Thu, Sep 5, 2013 at 7:00 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: Ahh, I missed looking at the ln command. You are supposed to be linking the backup you just made to .current. So, ssh $user@$remotehost ln -sf $remotedir /home/aquj001/recievingDir.current On 09/05/13 00:54, Jonathan Aquilina wrote: Hey Kevin The path works as it is after i recreated the .current directory but im still having the issue of a directory on the remote being created with too many symbolic links in it and no files. On Wed, Sep 4, 2013 at 8:27 PM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k
Re: getting a list of conflicting files
At this point im debating if rsync is the right tool. I am trying to determine if lsyncd which is based on rsync would do the trick. On Thu, Aug 1, 2013 at 8:15 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I don't know if conflicting is really the right word but it is a list of differences. I wouldn't bother with the snapshot unless you are going to process the list and then do the real sync. On 08/01/13 02:10, Jonathan Aquilina wrote: I was thinking of using an lvm snapshot. all i really need is just a list of the conflicting files to be emailed in a report. On Thu, Aug 1, 2013 at 8:02 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: It would tell you what rsync would have done without the --dry-run and why it would have done it. But it can't guarantee that nothing will change between the dry run and the real run. Of course if your source is a snapshotted filesystem that won't matter. On 08/01/13 01:59, Jonathan Aquilina wrote: I am in the process of writing a script on some testing machines and I am wondering if having it perform a dry run and itemized the items if it would provide a list of differences such as file size and name etc conflicts -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlH5/OkACgkQVKC1jlbQAQdq7QCeJHU1WGc6qdbBWU/OSTEN5a4U UyMAnRymCx2zBiSI+Gt1bt/EzbkHZt7o =roWO -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: getting a list of conflicting files
so Rsync would be a better solution in this case? On Thu, Aug 1, 2013 at 8:20 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 lsyncd is for real time syncing. Whenever something gets changed it would spawn an rsync to update the other copy. On 08/01/13 02:18, Jonathan Aquilina wrote: At this point im debating if rsync is the right tool. I am trying to determine if lsyncd which is based on rsync would do the trick. On Thu, Aug 1, 2013 at 8:15 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: I don't know if conflicting is really the right word but it is a list of differences. I wouldn't bother with the snapshot unless you are going to process the list and then do the real sync. On 08/01/13 02:10, Jonathan Aquilina wrote: I was thinking of using an lvm snapshot. all i really need is just a list of the conflicting files to be emailed in a report. On Thu, Aug 1, 2013 at 8:02 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: It would tell you what rsync would have done without the --dry-run and why it would have done it. But it can't guarantee that nothing will change between the dry run and the real run. Of course if your source is a snapshotted filesystem that won't matter. On 08/01/13 01:59, Jonathan Aquilina wrote: I am in the process of writing a script on some testing machines and I am wondering if having it perform a dry run and itemized the items if it would provide a list of differences such as file size and name etc conflicts -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlH5/hgACgkQVKC1jlbQAQeSvQCgv1PQQLjSjVfIHKnn9q6DB4tF uh4An2v+lbwNPgM3DeN2QnJaDcBvkWnE =rVu2 -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: getting a list of conflicting files
this is what im getting currently as a test im running cd+ syncDir/ f+ syncDir/rsync-3.0.6-1.aix5.3.ppc.rpm f+ syncDir/vim-common-6.3-1.aix5.1.ppc.rpm f+ syncDir/vim-enhanced-6.3-1.aix5.1.ppc.rpm does this mean that nothing has changed? On Thu, Aug 1, 2013 at 8:22 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you are just looking for a list of what is different then yes. On 08/01/13 02:21, Jonathan Aquilina wrote: so Rsync would be a better solution in this case? On Thu, Aug 1, 2013 at 8:20 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: lsyncd is for real time syncing. Whenever something gets changed it would spawn an rsync to update the other copy. On 08/01/13 02:18, Jonathan Aquilina wrote: At this point im debating if rsync is the right tool. I am trying to determine if lsyncd which is based on rsync would do the trick. On Thu, Aug 1, 2013 at 8:15 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: I don't know if conflicting is really the right word but it is a list of differences. I wouldn't bother with the snapshot unless you are going to process the list and then do the real sync. On 08/01/13 02:10, Jonathan Aquilina wrote: I was thinking of using an lvm snapshot. all i really need is just a list of the conflicting files to be emailed in a report. On Thu, Aug 1, 2013 at 8:02 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: It would tell you what rsync would have done without the --dry-run and why it would have done it. But it can't guarantee that nothing will change between the dry run and the real run. Of course if your source is a snapshotted filesystem that won't matter. On 08/01/13 01:59, Jonathan Aquilina wrote: I am in the process of writing a script on some testing machines and I am wondering if having it perform a dry run and itemized the items if it would provide a list of differences such as file size and name etc conflicts -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Jonathan Aquilina -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlH5/rEACgkQVKC1jlbQAQe1rgCdFc5r2fewzNHiRGMEtkLMvcLR qdkAoPQr0+DuqKFhbMfxYzs7JGC1QWxk =GOsd -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: getting a list of conflicting files
my switches are -ain at this point the problem though is its working but copying them to the same host. I need to test copying to a remote host rsync -ain -e ssh $LOCALDIR $REMOTEDIR On Thu, Aug 1, 2013 at 8:29 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 That is only showing new files in a new directory. The rest depends on the rest of your command line. For instance if you had - --ignore-existing you would never see a changed file. For a difference list I would probably use --archive --itemize-changes - --dry-run and probably not much else. On 08/01/13 02:24, Jonathan Aquilina wrote: this is what im getting currently as a test im running cd+ syncDir/ f+ syncDir/rsync-3.0.6-1.aix5.3.ppc.rpm f+ syncDir/vim-common-6.3-1.aix5.1.ppc.rpm f+ syncDir/vim-enhanced-6.3-1.aix5.1.ppc.rpm does this mean that nothing has changed? On Thu, Aug 1, 2013 at 8:22 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: If you are just looking for a list of what is different then yes. On 08/01/13 02:21, Jonathan Aquilina wrote: so Rsync would be a better solution in this case? On Thu, Aug 1, 2013 at 8:20 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: lsyncd is for real time syncing. Whenever something gets changed it would spawn an rsync to update the other copy. On 08/01/13 02:18, Jonathan Aquilina wrote: At this point im debating if rsync is the right tool. I am trying to determine if lsyncd which is based on rsync would do the trick. On Thu, Aug 1, 2013 at 8:15 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: I don't know if conflicting is really the right word but it is a list of differences. I wouldn't bother with the snapshot unless you are going to process the list and then do the real sync. On 08/01/13 02:10, Jonathan Aquilina wrote: I was thinking of using an lvm snapshot. all i really need is just a list of the conflicting files to be emailed in a report. On Thu, Aug 1, 2013 at 8:02 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: It would tell you what rsync would have done without the --dry-run and why it would have done it. But it can't guarantee that nothing will change between the dry run and the real run. Of course if your source is a snapshotted filesystem that won't matter. On 08/01/13 01:59, Jonathan Aquilina wrote: I am in the process of writing a script on some testing machines and I am wondering if having it perform a dry run and itemized the items if it would provide a list of differences such as file size and name etc conflicts -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Jonathan Aquilina -- Jonathan Aquilina -- Jonathan Aquilina -- Jonathan Aquilina - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlH6AFsACgkQVKC1jlbQAQfztQCglIASOn6SDObLULE6xgnYlCKI zr0AnjfHocidDTXTwcLYbNJqrTMYkP78 =6XtH -END PGP SIGNATURE- -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: getting a list of conflicting files
The problem though comes when i remove the n for the dry run and do an actual run it is copying them to the local host On Thu, Aug 1, 2013 at 8:34 AM, Kevin Korb k...@sanitarium.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - -e ssh is the default so you can drop it if you want. the rest is pretty much what I said I just spelled them out so they would be easier to lookup. You can also add a --delete if you want to see what is missing the dry run prevents it from actually deleting anything. On 08/01/13 02:32, Jonathan Aquilina wrote: my switches are -ain at this point the problem though is its working but copying them to the same host. I need to test copying to a remote host rsync -ain -e ssh $LOCALDIR $REMOTEDIR On Thu, Aug 1, 2013 at 8:29 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net wrote: That is only showing new files in a new directory. The rest depends on the rest of your command line. For instance if you had --ignore-existing you would never see a changed file. For a difference list I would probably use --archive --itemize-changes --dry-run and probably not much else. On 08/01/13 02:24, Jonathan Aquilina wrote: this is what im getting currently as a test im running cd+ syncDir/ f+ syncDir/rsync-3.0.6-1.aix5.3.ppc.rpm f+ syncDir/vim-common-6.3-1.aix5.1.ppc.rpm f+ syncDir/vim-enhanced-6.3-1.aix5.1.ppc.rpm does this mean that nothing has changed? On Thu, Aug 1, 2013 at 8:22 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: If you are just looking for a list of what is different then yes. On 08/01/13 02:21, Jonathan Aquilina wrote: so Rsync would be a better solution in this case? On Thu, Aug 1, 2013 at 8:20 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: lsyncd is for real time syncing. Whenever something gets changed it would spawn an rsync to update the other copy. On 08/01/13 02:18, Jonathan Aquilina wrote: At this point im debating if rsync is the right tool. I am trying to determine if lsyncd which is based on rsync would do the trick. On Thu, Aug 1, 2013 at 8:15 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: I don't know if conflicting is really the right word but it is a list of differences. I wouldn't bother with the snapshot unless you are going to process the list and then do the real sync. On 08/01/13 02:10, Jonathan Aquilina wrote: I was thinking of using an lvm snapshot. all i really need is just a list of the conflicting files to be emailed in a report. On Thu, Aug 1, 2013 at 8:02 AM, Kevin Korb k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net mailto:k...@sanitarium.net wrote: It would tell you what rsync would have done without the --dry-run and why it would have done it. But it can't guarantee that nothing will change between the dry run and the real run. Of course if your source is a snapshotted filesystem that won't matter. On 08/01/13 01:59, Jonathan Aquilina wrote: I am in the process of writing a script on some testing machines and I am wondering if having it perform a dry run and itemized the items if it would provide a list of differences such as file size and name etc conflicts -- Jonathan Aquilina -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting
