Re: untransferred files
On Thu, 2008-03-13 at 18:27 -0700, Wayne Davison wrote: > On Thu, Mar 13, 2008 at 11:01:35AM -0400, Ming Zhang wrote: > > Some time rsync just return error code 23 that some files are not > > transferred. Is there a way to get a list of these files so we can retry > > it later time? > > The list of files was output on stderr during the copy. Rsync will try > them again if you repeat the copy. interesting, i will try. thx. > > ..wayne.. -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: untransferred files
On Thu, Mar 13, 2008 at 11:01:35AM -0400, Ming Zhang wrote: > Some time rsync just return error code 23 that some files are not > transferred. Is there a way to get a list of these files so we can retry > it later time? The list of files was output on stderr during the copy. Rsync will try them again if you repeat the copy. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: error allocating core memory buffers with certain iconv args
On Tue, Mar 11, 2008 at 03:00:25PM +0100, Giuliano Gavazzi wrote: > rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32) This just tells you that the receiver side went away, but we don't know why it went away. If it is crashing, try to get a core dump and report the backtrace (with symbols). ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Possibile rsync --link-dest or --compare-dest bug
On Thu, Mar 13, 2008 at 09:33:42PM +0100, jan wrote: > -bash-3.1$ rsync -av --link-dest=20080313/ > --compare-dest=/data/BOC55/20080313/ 20080313/ /data/BOC55/20080314/ That set of options is impossible since you can't mix --link-dest with --compare-dest. Back in the days of 2.6.3, rsync would silently allow the latter --*-dest arg to replace the former arg, while retaining the idea that you wanted to do a --link-dest (due to the prior arg setting a flag), so that impossible syntax was equivalent to using --link-dest with the second arg. > --compare-dest arg does not exist: 20080313/ That's not a bug, that's rsync being helpful. All modern versions of rsync let you specify more than one --link-dest (or other --*-dest) directory to scan (as long as they are of the same type). Beginning with 3.0.0, if you specify a directory that does not exist, rsync is nice enough to let you know so you can fix it. It looks like you just want a single --link-dest=/data/BOC55/20080313 arg in your above command. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Possibile rsync --link-dest or --compare-dest bug
Hi all, I have just migrated my tooling from version 2.6.9 to 3.0.0. Unfortunately my script generated an error message during the execution. Because this command was working perfectly in the previous version I suspect this could be a bug. This is the command that I am executing: rsync -av --link-dest=20080313/ --compare-dest=/data/20080313/ 20080313/ /data/20080314/ It just creates a copy of directory 20080313 to 20080314, but not with an old fashion copy, but hard linking back to 20080313. This is the error it is producing: === BEGIN SNIP === -bash-3.1$ rsync -av --link-dest=20080313/ --compare-dest=/data/BOC55/20080313/ 20080313/ /data/BOC55/20080314/ sending incremental file list created directory /data/20080314 --compare-dest arg does not exist: 20080313/ <== ERROR HERE sent 128478 bytes received 1512 bytes 3767.83 bytes/sec total size is 659198008 speedup is 5071.14 === END SNIP === The weird thing is (besides the error message) that is works perfectly. Should I create a bug for this ? Or do I need to change the command syntax ? Any help is appreciated. Warm regards, Jan -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[PATCH] More itemize clarifications.
--- Note: From now on, my submitted patches will also be pullable from: http://mattmccutchen.net/rsync/rsync.git/ Matt rsync.yo |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/rsync.yo b/rsync.yo index 82b0fff..9c99645 100644 --- a/rsync.yo +++ b/rsync.yo @@ -1678,8 +1678,8 @@ quote(itemization( (requires bf(--checksum)) or that a symlink, device, or special file has a changed value. Note that if you are sending files to an rsync prior to 3.0.1, this - change flag will be present for checksum-differing regular files. - it() A bf(s) means the size of the file is different and will be updated + change flag will be present only for checksum-differing regular files. + it() A bf(s) means the size of a regular file is different and will be updated by the file transfer. it() A bf(t) means the modification time is different and is being updated to the sender's value (requires bf(--times)). An alternate value of bf(T) -- 1.5.4.3.193.g6dd0e -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Group name vs. GID
On Thu, Mar 13, 2008 at 02:11:28PM +0200, Mark, Oren wrote: > On both A and B I use the rsync daemon version and configuration. See the rsyncd manpage for a discussion of the "use chroot" setting, how it affects user-/group-name mapping, and what you can do about it: http://rsync.samba.org/ftp/rsync/rsyncd.conf.html Note that the "numeric ids" option mentioned therein is new for 3.0.0, so for older daemons, you can consider the "numeric ids" option to always be false. Summary: either turn off "use chroot" or give rsync the mapping files it needs to work (and hide them). Note also that the feature mentioned in the man page that allows you to divide your daemon path into a path internal to the chroot and a path external to the chroot is also new for rsync 3.0.0. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
untransferred files
Hi All Some time rsync just return error code 23 that some files are not transferred. Is there a way to get a list of these files so we can retry it later time? For example, we scan the whole file system and N files are not transferred. Instead of scan the whole file system later again, we can use files-from which is much faster. Thanks. -- Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Suggestions for basic rsync configuration
Whilst it is possible to write a script to mount FTP and copy the files across. This isn't what is usually done with Rsync. Rsync can be a replacement for FTP client. So it collects a list of altered files between two directories and puts these files into a remote directory. No FTP required. All you need to do is open up a port on your server to the Internet, install Rsync as a server(daemon) on one end and then run Rsync on the source whenever you need to do the backup. The cute thing is that Rsync don't care what platform the source or server is or whether or not it's across a LAN or WAN. Here is a batch line I typically use on Windows for backing up a directory across the Internet. rsync -zav --delete --recursive '/cygdrive/c/sourcedirectory/' [EMAIL PROTECTED]::backupdirectory/ Easy. This syncs the source to the backup directory. ie Rsync assumes the source is the 'master'. Once you've got Rsync working, you can then add a SSH layer so the traffic is encrypted. I'd advise getting the basic daemon working first, then on the client PC, issue a Rsync command to list the contents of the remote folder. Once you've got that listing working you know things like firewalls, port forwarding of routers is working. :-) Rsync by default using port 873. But it's easy to change this. Just ensure you do it with the daemon and client. ;-) To list the public remote directories. Rsync domain.dom:: Listing the contents of a directory set up in the daemon confg file is done by: Rsync domain.dom::backup Once you've got the basics, you can then add username and password protection in the confg file. Performing a list of a directory with a username is done like this: Rsync [EMAIL PROTECTED]::backup if a password is required, you're prompted for it. If you want the password auto-inserted, then you set a environmental variable In Windows this is added to the batch file of the client rsync. SET RSYNC_PASSWORD=mypassword A basic daemon conf.cfg file is something like this: log file = rsyncd.log pid file = rsyncd.pid use chroot=false uid=administrator gid=administrators charset = utf-8 strict modes = false secrets file =/cygdrive/c/cwrsync/passwordfile [backup] comment=rsync directory path = /cygdrive/c/mybackup/ read only = false transfer logging = yes list=yes #auth users=username password file is a list of users and passwords "username:password" on a single line. -- Stuart Halliday -Original Message- From: Geofoxer <[EMAIL PROTECTED]> To: rsync@lists.samba.org Date: Thu, 13 Mar 2008 05:39:40 -0700 (PDT) Subject: Suggestions for basic rsync configuration > > Hello, > > I am very new to rsync and am wanting to have it back up more than 4 > remote > linux servers via ftp. I want to write a script that will open up a ftp > connection using "ftp://username:[EMAIL PROTECTED]" with a > server, > then mount the drive containing the backups, copy the data that is new, > unmount the drive, close the connection and move onto the next server > until > every server is successfully backed up to a external drive on my linux > machine which is on my local network. I know that I have to use ssh > with the > connection to the servers for backup, but where to start is where I am > at > right now. Here are a couple of noobish questions: > > 1. Do I have to write my own file from scratch and hope it works? > 2. How should I start to write my file in order to have it correlate > with > backing up multiple servers (syntax, switches, and so on)? > > If anyone has any advise on how I should start writing my config file, > please let me know if you have any suggestions. > > Thanx, > > gxr > > - > Computers are like air conditioners. They both dont work, if you open > windows. > -- > View this message in context: > http://www.nabble.com/Suggestions-for-basic-rsync-configuration-tp16025 > 015p16025015.html > Sent from the Samba - rsync mailing list archive at Nabble.com. > > -- > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: > http://www.catb.org/~esr/faqs/smart-questions.html This email is the property of ECS Technology Ltd. This company is registered in Scotland with company number 212513. VAT registered GB 761 7456 12 http://www.ecs-tech.com/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: (cw)rsync 3.0.0 incompatible with 2.6.9
I have now uploaded a new version of cwrsync including plain rsync with default settings. https://sourceforge.net/project/showfiles.php?group_id=69227&package_id=6808 1&release_id=583975 Tev > -Original Message- > From: Matt McCutchen [mailto:[EMAIL PROTECTED] > Sent: 13. mars 2008 03:39 > To: Mojca Miklavec > Cc: rsync@lists.samba.org; Tevfik Karagülle; Taco Hoekwater > Subject: Re: (cw)rsync 3.0.0 incompatible with 2.6.9 > > On Wed, 2008-03-12 at 20:57 +0100, Mojca Miklavec wrote: > > On Wed, Mar 12, 2008 at 4:46 PM, Matt McCutchen wrote: > > > 3. Have cwRsync 3.0.0 users pass RSYNC_ICONV=- or --no-iconv . > > > > That's what has been suggested by Tevfik Karagülle, but > it's ugly as > > it cannot be written in a portable way. (One would have to ask for > > rsync version first and then fiddle with options.) > > Passing RSYNC_ICONV=- in the environment is portable; it will > suppress cwRsync 3.0.0's default iconv option without > upsetting other versions of rsync. > > > Someone else has suggested me to use > > rsync --protocol=29 > > but I need to test it. > > I tested it and it actually doesn't solve the problem. > > > > 4. Build your own Cygwin rsync (which will have no iconv > setting by > > > default) and use that instead of cwRsync 3.0.0. > > > > I don't have the slightest idea how to do that (I have > tried to build > > other programs on windows, but hacking make files is not my > strength). > > No makefile hacking should be necessary. If you were to > build your own rsync, it should just involve installing > Cygwin (including the necessary compilation tools) using > http://www.cygwin.com/setup.exe , downloading and extracting > the source tarball, and running ./configure && make . > > > 5. Ask the author of cwRsync to do suggestion number 4 (so that > > cwrsync would not fail on most servers). > > > > That would be the ideal scenario from what I have heard or > learnt in > > this thread. If I build cwrsyc for me only, that wil serve noone > > except me. > > Yes, it would be best if Tev changed cwRsync officially, but > I think it's worth knowing how to build your own Cygwin rsync > anyway in case you want to test or use patched versions in the future. > > Matt > -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Can a rsync server use MySQL for authentication
On Thursday 13 March 2008 06:51:57 Daniel Maher wrote: >PAM provides a way to develop programs that are independent of >authentication scheme. Thank you for all that, you make some very good points and I learned more about the capabilities of PAM. I've presented the relevant info to the client and we'll see how he wishes to proceed. -- Cheers, Rob Fargher -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Can a rsync server use MySQL for authentication
On Thu, 13 Mar 2008 04:23:15 -0800 Robert Fargher <[EMAIL PROTECTED]> wrote: > > > >Sounds like a job for PAM. > > > >http://www.google.com/search?q=linux+PAM > > > >:) > > No, specifically not. We are already using MySQL to authenticate > secure FTP, using ProFTPDd. What I'd like to do is to use the *same* > MySQL database/tables to authenticate secure rsync. If rsync can't > do it, that's fine, I just won't offer rsync as a service. > > PAM, as I understand it, requires a user account on the system. > This is specifically what we don't want. All remote ftp/rsync user > info is to be completely contained within the MySQL database Actually, PAM /doesn't/ require user accounts on the system. From the PAM docs : Q1: What exactly is PAM? Basically, it is a flexible mechanism for authenticating users. ... PAM provides a way to develop programs that are independent of authentication scheme. In my environment, for example, when a user logs into a machine via SSH, PAM sends the request off to our Kerberos server (which handles the authentication challenge, of course), which if successful, then queries an LDAP server for the users' profile information. There are absolutely no user entries in /etc/passwd at all. I suppose that you're using a ProFTPd-specific manner of interacting with MySQL. This is, of course, completely valid - but you end up locking yourself into a solution which is clearly not as extensible as you'd like. Instead, you may wish to consider housing your "generic" authentication data in a MySQL backend which PAM challenges at a system level. In this way, any application or service which wants to verify a user can simply query the system and obtain a result - be it ProFTPd, Rsync, or something else entirely. By abstracting the authentication mechanism, you no longer have to worry about whether your particular application "supports" your desired backend. -- Daniel Maher signature.asc Description: PGP signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Suggestions for basic rsync configuration
Hello, I am very new to rsync and am wanting to have it back up more than 4 remote linux servers via ftp. I want to write a script that will open up a ftp connection using "ftp://username:[EMAIL PROTECTED]" with a server, then mount the drive containing the backups, copy the data that is new, unmount the drive, close the connection and move onto the next server until every server is successfully backed up to a external drive on my linux machine which is on my local network. I know that I have to use ssh with the connection to the servers for backup, but where to start is where I am at right now. Here are a couple of noobish questions: 1. Do I have to write my own file from scratch and hope it works? 2. How should I start to write my file in order to have it correlate with backing up multiple servers (syntax, switches, and so on)? If anyone has any advise on how I should start writing my config file, please let me know if you have any suggestions. Thanx, gxr - Computers are like air conditioners. They both dont work, if you open windows. -- View this message in context: http://www.nabble.com/Suggestions-for-basic-rsync-configuration-tp16025015p16025015.html Sent from the Samba - rsync mailing list archive at Nabble.com. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Group name vs. GID
Hi, I use rsync the replicate data between several sites. As far as I know the default rsync operation is to copy group name and not GID. I have two sites, lets say A and B with the same GIDs, and one, C, with different GID. When I replicated data from site A to C it is working fine - it copied group names and the GID is mapped to the one in site C. When I rsync the data from site B to C, it is copying files with the source GID, what causing permissions problem in the destination site, C. On both A and B I use the rsync daemon version and configuration. Any ideas what should I look for? Thanks, Oren Mark Intel - Israel Engineering Computing ISEC Data & Application Hosting [EMAIL PROTECTED] (+) 972-4-865-5987 iNET: 465-5987 - Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Can a rsync server use MySQL for authentication
On Thursday 13 March 2008 02:58:43 Daniel Maher wrote: >On Wed, 12 Mar 2008 17:58:53 -0800 Robert Fargher <[EMAIL PROTECTED]> > >wrote: >> What I'd like to know is if I can do the same thing with rsync? >> Specifically, can I run rsync as a daemon that permits secure >> connections and encrypted transfers from remote users using the same >> MySQL database for authentication that I'm already using for ftp? > >Sounds like a job for PAM. > >http://www.google.com/search?q=linux+PAM > >:) No, specifically not. We are already using MySQL to authenticate secure FTP, using ProFTPDd. What I'd like to do is to use the *same* MySQL database/tables to authenticate secure rsync. If rsync can't do it, that's fine, I just won't offer rsync as a service. PAM, as I understand it, requires a user account on the system. This is specifically what we don't want. All remote ftp/rsync user info is to be completely contained within the MySQL database -- Cheers, Rob -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Can a rsync server use MySQL for authentication
On Wed, 12 Mar 2008 17:58:53 -0800 Robert Fargher <[EMAIL PROTECTED]> wrote: > What I'd like to know is if I can do the same thing with rsync? > Specifically, can I run rsync as a daemon that permits secure > connections and encrypted transfers from remote users using the same > MySQL database for authentication that I'm already using for ftp? Sounds like a job for PAM. http://www.google.com/search?q=linux+PAM :) -- Daniel Maher signature.asc Description: PGP signature -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
