Sync files locally?
What's the best way to sync files locally? I'm running rsync over ssh to backup files from a server. Now I'm setting up a new server, so I rsynced to it as well. The files end up in a subdir of my home directory, because my account is being used for the rsync. I've been using cp -R to put them in the right place on the server and the chmod and chown to get the perms right, but I'm thinking there has to be a way to sync the files locally so that only the ones that have changed or are new have to be copied to the right place. I can use find to set the perms, but I'm not sure how to sync the files. This is unix, so there's got to be a built-in utility that does this, but I can't seem to find it. Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/
Re: Sync files locally?
On Sep 27, 2006, at 5:40 PM, Paul Schmehl wrote: What's the best way to sync files locally? I'm running rsync over ssh to backup files from a server. rsync (as root) works just fine to perform local copies preserving permissions and symlinks and so forth. Other people use a tar pipeline, or dump/restore, or cpio... -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Sync files locally?
In the last episode (Sep 27), Paul Schmehl said: > What's the best way to sync files locally? > > I'm running rsync over ssh to backup files from a server. Now I'm > setting up a new server, so I rsynced to it as well. The files end > up in a subdir of my home directory, because my account is being used > for the rsync. I've been using cp -R to put them in the right place > on the server and the chmod and chown to get the perms right, but I'm > thinking there has to be a way to sync the files locally so that only > the ones that have changed or are new have to be copied to the right > place. As long as your rsync path has a leading /, it should start at the root filesystem. e.g. "rsync mydir otherserver:mydir" creates ~/mydir , "rsync mydir otherserver:/mydir" creates /mydir (assuming you have permission). > I can use find to set the perms, but I'm not sure how to sync the > files. This is unix, so there's got to be a built-in utility that > does this, but I can't seem to find it. You could use rsync to do this, but It'd probably be easier to get your remote rsync to drop the files in the right directory in the first place. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Sync files locally?
--On Wednesday, September 27, 2006 18:04:40 -0400 Charles Swiger <[EMAIL PROTECTED]> wrote: On Sep 27, 2006, at 5:40 PM, Paul Schmehl wrote: What's the best way to sync files locally? I'm running rsync over ssh to backup files from a server. rsync (as root) works just fine to perform local copies preserving permissions and symlinks and so forth. Other people use a tar pipeline, or dump/restore, or cpio... I wasn't aware of that. Thanks, Charles. Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/
Re: Sync files locally?
--On Wednesday, September 27, 2006 17:04:39 -0500 Dan Nelson <[EMAIL PROTECTED]> wrote: You could use rsync to do this, but It'd probably be easier to get your remote rsync to drop the files in the right directory in the first place. Hmmm...maybe I don't fully understand rsync. The files are from a webserver and owned by www:www. The rsync runs under my account, so the perms are changed to mine. Maybe there's a switch with rsync that would allow me to sync directly? I'll poke around the man page. Thanks. Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/
Re: Sync files locally?
Paul Schmehl wrote: What's the best way to sync files locally? I'm running rsync over ssh to backup files from a server. Now I'm setting up a new server, so I rsynced to it as well. The files end up in a subdir of my home directory, because my account is being used for the rsync. I've been using cp -R to put them in the right place on the server and the chmod and chown to get the perms right, but I'm thinking there has to be a way to sync the files locally so that only the ones that have changed or are new have to be copied to the right place. I can use find to set the perms, but I'm not sure how to sync the files. This is unix, so there's got to be a built-in utility that does this, but I can't seem to find it. It's really a question of reading far enough in the man-page. I have eventually arrived at the options -Cptuvaz C for CVS omissions (exclude *.bak, *.tmp, *~ etc) p for preserve permisions t for preserving time u for update (using mtime I suppose) a for archive - recursive z for compression and with that, there should be no reason to sync into a separate dir and then copy. Cheers, Erik -- Ph: +34.666334818 web: http://www.locolomo.org X.509 Certificate: http://www.locolomo.org/crt/8D03551FFCE04F0C.crt Key ID: 69:79:B8:2C:E3:8F:E7:BE:5D:C3:C3:B1:74:62:B8:3F:9F:1F:69:B9 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Sync files locally?
Paul Schmehl wrote: What's the best way to sync files locally? I'm running rsync over ssh to backup files from a server. Now I'm setting up a new server, so I rsynced to it as well. The files end up in a subdir of my home directory, because my account is being used for the rsync. I've been using cp -R to put them in the right place on the server and the chmod and chown to get the perms right, but I'm thinking there has to be a way to sync the files locally so that only the ones that have changed or are new have to be copied to the right place. I can use find to set the perms, but I'm not sure how to sync the files. This is unix, so there's got to be a built-in utility that does this, but I can't seem to find it. Well, rsync. When the paths are both local it won't use ssh since it doesn't need it. Not sure why you think the files have to end up under your home dir. Something like rsync -a --delete /path1/ /alt/path2 will make an exact copy under /alt/path2, updating only things that need it. Done as root it would do all the preservation of users, modes etc regardless of ownership. You also have --flags if you need it but it doesn't appear to work with schg flag - maybe nouulnk and variants either. Other than that you can use tar with pipes, which was the traditional method Before Rsync (TM). But obviously not very efficient for keeping a tree up-to-date. These days cp -Rp as root should also do a one-time copy but won't preserve any hard links. --Alex ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Sync files locally?
Paul Schmehl wrote: --On Wednesday, September 27, 2006 17:04:39 -0500 Dan Nelson <[EMAIL PROTECTED]> wrote: You could use rsync to do this, but It'd probably be easier to get your remote rsync to drop the files in the right directory in the first place. Hmmm...maybe I don't fully understand rsync. The files are from a webserver and owned by www:www. The rsync runs under my account, so the perms are changed to mine. Maybe there's a switch with rsync that would allow me to sync directly? I'll poke around the man page. Thanks. 1) rsync changes behaviour depending on whether or not you include a trailing / from the man-page: rsync -av /src/foo /dest rsync -av /src/foo/ /dest/foo works the same way, (and sync locally in this example). Read the man page, there are lots of examples. 2) The options -t and -p preserve time and permissions respectively. Ownership will always change to the user running rsync unless you run as root. This has nothing to do with rsync, you can't run chown as an ordinary user. You can preserve the group if you're in that group on the destination host. 3) The files you are syncing - should they be writeable by www? For security, you may really want something like this: -rw-r- user:wwwfile and have user do the rsync. If you really need to have www write to the file, set group permissions +w. Personally, I have all my web-pages owned by me, and let apache access read-only. Log-files generated are owned by apache, but I don't back these up since they are continuously generated and rotated out. The config file is also only readable by apache, I don't want the risk of some exploit committing changes to the config file. It might be useful to have a separate dir where apache can store files with write permissions and owned by apache. But this depends on the web apps you deploy. IIRC to run rsync over ssh the user doing the syncing must have shell access, running your sync as root is not desirable, it MAY be preferred to have it run as www to preserve owner also, at least you can restrict access for www. Cheers, Erik -- Ph: +34.666334818 web: http://www.locolomo.org X.509 Certificate: http://www.locolomo.org/crt/8D03551FFCE04F0C.crt Key ID: 69:79:B8:2C:E3:8F:E7:BE:5D:C3:C3:B1:74:62:B8:3F:9F:1F:69:B9 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Sync files locally?
--On September 28, 2006 9:33:39 AM +0200 Erik Norgaard <[EMAIL PROTECTED]> wrote: 1) rsync changes behaviour depending on whether or not you include a trailing / from the man-page: rsync -av /src/foo /dest rsync -av /src/foo/ /dest/foo That I know. works the same way, (and sync locally in this example). Read the man page, there are lots of examples. 2) The options -t and -p preserve time and permissions respectively. Ownership will always change to the user running rsync unless you run as root. This has nothing to do with rsync, you can't run chown as an ordinary user. You can preserve the group if you're in that group on the destination host. 3) The files you are syncing - should they be writeable by www? For security, you may really want something like this: -rw-r- user:wwwfile They're user:www, but the user isn't me. :-) I could go into the reasons for that, but it doesn't really matter. and have user do the rsync. If you really need to have www write to the file, set group permissions +w. No, I don't need that and don't want it either. The owners of the site own the files, and I know how to use sudo. :-) IIRC to run rsync over ssh the user doing the syncing must have shell access, running your sync as root is not desirable, it MAY be preferred to have it run as www to preserve owner also, at least you can restrict access for www. I use ssh with keys so I can cron the job, but I'll have to do the extra step of fixing perms and ownership after copying the rsynced files over. No big deal. I'll script that as well. Paul Schmehl ([EMAIL PROTECTED]) Adjunct Information Security Officer The University of Texas at Dallas http://www.utdallas.edu/ir/security/
