Re: Rsync 3.0.0pre2 released
On Thu 11 Oct 2007, Wayne Davison wrote: I've just released rsync 3.0.0pre2, the second pre-release version for the upcoming 3.0.0 release. Please test this out and email the rsync mailing list with any questions, comments, bug reports, etc. Thanks! One thing that's not 100% clear for me: in the NEWS file you state: - Added the --acls (-A) option to preserve Access Control Lists. This is an improved version of the prior patch that was available, and it even supports OS X ACLs. (If you need to have backward compatibility with old, patched versions of rsync, apply the acls.diff file from the patches dir.) Does applying the acls.diff patch influence the workings of the new standard --acls option, when _not_ talking to an old patched version? If it doesn't harm the standard working, then it's probably a good idea for the Debian packaged version to apply the diff at least until this version of rsync is contained in an official new release of Debian... Paul Slootman -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync 3.0.0pre2 performance and assembler warning message
In testing 3.0.0pre1 and now 3.0.0pre2 I have noticed that the rate of data transfer is ~25% slower than 2.6.9. I am using rsync to mirror 2 filesystems from a single host. On average for large files ( 4gb) under 3.0.0pre2 the transfer of ~75MB/sec. WIth 2.6.9 the average is ~105MB/sec. I am compiling on a ia64 based system running SLES10 SP1 and see this assembler warnings when compiling with gcc 4.1.2. Are these of any concern? ~/rsync/rsync-3.0.0pre2 make gcc -std=gnu99 -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -o mkrounding -I. ./mkrounding.c ./mkrounding rounding.h Rounding file_extras in multiples of 2 (EXTRA_LEN=4, FILE_STRUCT_LEN=24) gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c flist.c -o flist.o /tmp/ccC7MQuD.s: Assembler messages: /tmp/ccC7MQuD.s:2993: Warning: Use of 'mov' may violate WAW dependency 'GR%, % in 1 - 127' (impliedf), specific resource number is 14 /tmp/ccC7MQuD.s:2991: Warning: This is the location of the conflicting usage gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c rsync.c -o rsync.o -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5008] make check fails on Cygwin (default-acls)
https://bugzilla.samba.org/show_bug.cgi?id=5008 --- Comment #2 from [EMAIL PROTECTED] 2007-10-12 10:10 CST --- I don't have access to an MS Win box to test this, but I imagine that that this is just showing that MS ACLs work differently. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
3.0.0pre2 performance and assembler warning
In testing 3.0.0pre1 and now 3.0.0pre2 I have noticed that the rate of data transfer is ~25% slower than 2.6.9. I am using rsync to mirror 2 filesystems from a single host. On average for large files ( 4gb) under 3.0.0pre2 the transfer of ~75MB/sec. WIth 2.6.9 the average is ~105MB/sec. I am compiling on a ia64 based system running SLES10 SP1 and see this assembler warnings when compiling with gcc 4.1.2. Are these of any concern? ~/rsync/rsync-3.0.0pre2 make gcc -std=gnu99 -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -o mkrounding -I. ./mkrounding.c ./mkrounding rounding.h Rounding file_extras in multiples of 2 (EXTRA_LEN=4, FILE_STRUCT_LEN=24) gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c flist.c -o flist.o /tmp/ccC7MQuD.s: Assembler messages: /tmp/ccC7MQuD.s:2993: Warning: Use of 'mov' may violate WAW dependency 'GR%, % in 1 - 127' (impliedf), specific resource number is 14 /tmp/ccC7MQuD.s:2991: Warning: This is the location of the conflicting usage gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt -c rsync.c -o rsync.o -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --detect-renamed question
On 10/11/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
I've started testing the detect-renamed patch with 2.6.9 and soon 3.0.0pre1.
I have an unique situation where I'm rsync'ing to a HSM based filesystem.
I've found that the detect-renamed patch works but it appears to do a copy of
the file to the new destination. This is particular slow since the file in
the HSM based filesystem may only be a stub and all the data is only resident
on tape. The copy waits for the datq to be recalled from tape which
depending on the file size can take a long time. I've looked through the
patch code and am wondering if there is an easy way to have rsync do a move
from the ~.tmp. directory.
This is easy to do, and I have implemented a --trust-detect-renamed
option to do it in the attached patch to the current CVS rsync.
However, it is risky because a false rename detection could cause
rsync to substitute an unrelated but similar-looking destination file
for a new source file. Don't use the option unless you are prepared
for the consequences.
Matt
In combination with detect-renamed.diff, this patch adds an option
--trust-detect-renamed that adopts an apparently unrenamed destination
file without verifying that its data matches that of the source file.
This is risky but it is what Greg [EMAIL PROTECTED] wanted:
http://lists.samba.org/archive/rsync/2007-October/018827.html
This patch is EXPERIMENTAL, though it did work correctly in my single test.
-- Matt McCutchen [EMAIL PROTECTED]
--- old/generator.c
+++ new/generator.c
@@ -80,6 +80,7 @@ extern int compare_dest;
extern int copy_dest;
extern int link_dest;
extern int detect_renamed;
+extern int trust_detect_renamed;
extern int whole_file;
extern int list_only;
extern int read_batch;
@@ -1828,6 +1829,22 @@ static void recv_generator(char *fname,
fnamecmp = partialptr;
fnamecmp_type = FNAMECMP_PARTIAL_DIR;
statret = 0;
+ if (detect_renamed trust_detect_renamed
+ unchanged_file(fnamecmp, file, sx.st)) {
+ /* Adopt the partial file. */
+ finish_transfer(fname, fnamecmp, NULL, NULL, file, 1, 1);
+ handle_partial_dir(partialptr, PDIR_DELETE);
+ if (itemizing)
+itemize(fnamecmp, file, ndx, -1, sx,
+ ITEM_LOCAL_CHANGE, fnamecmp_type, NULL);
+#ifdef SUPPORT_HARD_LINKS
+ if (preserve_hard_links F_IS_HLINKED(file))
+finish_hard_link(file, fname, ndx, sx.st, itemizing, code, -1);
+#endif
+ if (remove_source_files == 1)
+goto return_with_success;
+ goto cleanup;
+ }
}
if (!do_xfers)
--- old/options.c
+++ new/options.c
@@ -81,6 +81,7 @@ int am_starting_up = 1;
int relative_paths = -1;
int implied_dirs = 1;
int detect_renamed = 0;
+int trust_detect_renamed = 0;
int numeric_ids = 0;
int allow_8bit_chars = 0;
int force_delete = 0;
@@ -385,6 +386,7 @@ void usage(enum logcode F)
rprintf(F, -T, --temp-dir=DIR create temporary files in directory DIR\n);
rprintf(F, -y, --fuzzy find similar file for basis if no dest file\n);
rprintf(F, --detect-renamedtry to find renamed files to speed up the transfer\n);
+ rprintf(F, --trust-detect-renamed ... and assume identical to source files (risky!)\n);
rprintf(F, --compare-dest=DIR also compare destination files relative to DIR\n);
rprintf(F, --copy-dest=DIR ... and include copies of unchanged files\n);
rprintf(F, --link-dest=DIR hardlink to files in DIR when unchanged\n);
@@ -564,6 +566,7 @@ static struct poptOption long_options[]
{copy-dest,0, POPT_ARG_STRING, 0, OPT_COPY_DEST, 0, 0 },
{link-dest,0, POPT_ARG_STRING, 0, OPT_LINK_DEST, 0, 0 },
{detect-renamed, 0, POPT_ARG_NONE, detect_renamed, 0, 0, 0 },
+ {trust-detect-renamed,0,POPT_ARG_NONE, trust_detect_renamed, 0, 0, 0 },
{fuzzy, 'y', POPT_ARG_NONE, fuzzy_basis, 0, 0, 0 },
{compress,'z', POPT_ARG_NONE, 0, 'z', 0, 0 },
{no-compress, 0, POPT_ARG_VAL,do_compression, 0, 0, 0 },
@@ -1895,8 +1898,12 @@ void server_options(char **args, int *ar
}
}
/* Both sides need to know in case this disables incremental recursion. */
- if (detect_renamed)
+ if (detect_renamed) {
args[ac++] = --detect-renamed;
+ /* But the addition of --trust-detect-renamed is only the receiver's business. */
+ if (am_sender trust_detect_renamed)
+ args[ac++] = --trust-detect-renamed;
+ }
if (modify_window_set) {
if (asprintf(arg, --modify-window=%d, modify_window) 0)
--- old/rsync.yo
+++ new/rsync.yo
@@ -385,6 +385,7 @@ to the detailed description below for a
-T, --temp-dir=DIR create temporary files in directory DIR
-y, --fuzzy find similar file for basis if no dest file
--detect-renamedtry to find renamed files to speed the xfer
+ --trust-detect-renamed .. assume identical to src files (risky!)
--compare-dest=DIR also compare received files relative to DIR
--copy-dest=DIR ...
detect-renamed.diff fixes and improvements
Wayne,
The detect-renamed.diff in the current CVS rsync appears to be very
badly broken. I have fixed it and made some other improvements:
Crash fixes:
- Move misplaced the_fattr_list initialization hunk back to recv_file_list
- Send --detect-renamed to the sender so it knows to disable incremental
recursion
- With --partial-dir=. , look_for_rename returns rather than crashing
Improvements:
- Make look_for_rename play nicer with --dry-run
- Print found renamed: %s = %s at -vv (rather than -vvv) level
- Improve man page explanation of --detect-renamed
My new detect-renamed.diff is attached for your consideration.
Matt
This patch adds the --detect-renamed option which makes rsync notice files
that either (1) match in size modify-time (plus the basename, if possible)
or (2) match in size checksum (when --checksum was also specified) and use
each match as an alternate basis file to speed up the transfer.
The algorithm attempts to scan the receiving-side's files in an efficient
manner. If --delete[-before] is enabled, we'll take advantage of the
pre-transfer delete pass to prepare any alternate-basis-file matches we
might find. If --delete-before is not enabled, rsync does the rename scan
during the regular file-sending scan (scanning each directory right before
the generator starts updating files from that dir). In this latter mode,
rsync might delay the updating of a file (if no alternate-basis match was
yet found) until the full scan of the receiving side is complete, at which
point any delayed files are processed.
I chose to hard-link the alternate-basis files into a .~tmp~ subdir that
takes advantage of rsync's pre-existing partial-dir logic. This uses less
memory than trying to keep track of the matches internally, and also allows
any deletions or file-updates to occur normally without interfering with
these alternate-basis discoveries.
To use this patch, run these commands for a successful build:
patch -p1 patches/detect-renamed.diff
./configure (optional if already run)
make
TODO:
We need to never return a match from fattr_find() that has a basis
file. This will ensure that we don't try to give a renamed file to
a file that can't use it, while missing out on giving it to a file
that could use it.
--- old/compat.c
+++ new/compat.c
@@ -41,6 +41,7 @@ extern int checksum_seed;
extern int basis_dir_cnt;
extern int prune_empty_dirs;
extern int protocol_version;
+extern int detect_renamed;
extern int protect_args;
extern int preserve_uid;
extern int preserve_gid;
@@ -218,7 +219,7 @@ void setup_protocol(int f_out,int f_in)
} else if (protocol_version = 30) {
if (recurse allow_inc_recurse
!delete_before !delete_after !delay_updates
- !use_qsort !prune_empty_dirs)
+ !use_qsort !prune_empty_dirs !detect_renamed)
inc_recurse = 1;
need_messages_from_generator = 1;
}
--- old/flist.c
+++ new/flist.c
@@ -61,6 +61,7 @@ extern int non_perishable_cnt;
extern int prune_empty_dirs;
extern int copy_links;
extern int copy_unsafe_links;
+extern int detect_renamed;
extern int protocol_version;
extern int sanitize_paths;
extern struct stats stats;
@@ -113,6 +114,8 @@ static int64 tmp_dev, tmp_ino;
#endif
static char tmp_sum[MAX_DIGEST_LEN];
+struct file_list the_fattr_list;
+
static char empty_sum[MAX_DIGEST_LEN];
static int flist_count_offset; /* for --delete --progress */
static int dir_count = 0;
@@ -252,6 +255,45 @@ static int is_excluded(char *fname, int
return 0;
}
+static int fattr_compare(struct file_struct **file1, struct file_struct **file2)
+{
+ struct file_struct *f1 = *file1;
+ struct file_struct *f2 = *file2;
+ int64 len1 = F_LENGTH(f1), len2 = F_LENGTH(f2);
+ int diff;
+
+ if (!f1-basename || !S_ISREG(f1-mode) || !len1) {
+ if (!f2-basename || !S_ISREG(f2-mode) || !len2)
+ return 0;
+ return 1;
+ }
+ if (!f2-basename || !S_ISREG(f2-mode) || !len2)
+ return -1;
+
+ /* Don't use diff for values that are longer than an int. */
+ if (len1 != len2)
+ return len1 len2 ? -1 : 1;
+
+ if (always_checksum) {
+ diff = u_memcmp(F_SUM(f1), F_SUM(f2), checksum_len);
+ if (diff)
+ return diff;
+ } else if (f1-modtime != f2-modtime)
+ return f1-modtime f2-modtime ? -1 : 1;
+
+ diff = u_strcmp(f1-basename, f2-basename);
+ if (diff)
+ return diff;
+
+ if (f1-dirname == f2-dirname)
+ return 0;
+ if (!f1-dirname)
+ return -1;
+ if (!f2-dirname)
+ return 1;
+ return u_strcmp(f1-dirname, f2-dirname);
+}
+
static void send_directory(int f, struct file_list *flist,
char *fbuf, int len, int flags);
@@ -2154,6 +2196,25 @@ struct file_list *recv_file_list(int f)
clean_flist(flist, relative_paths);
+ if (detect_renamed) {
+ int j = flist-used;
+ the_fattr_list.used = j;
+ the_fattr_list.files = new_array(struct file_struct *, j);
+ if (!the_fattr_list.files)
+ out_of_memory(recv_file_list);
+ memcpy(the_fattr_list.files, flist-files,
+ j * sizeof (struct file_struct
Re: Recognizing directory name changes
BTW, the detect-renamed.diff in the current CVS rsync seems to be very badly broken, so don't use it yet. I have been working on a fixed version. Matt On 10/12/07, Frank Thomas [EMAIL PROTECTED] wrote: To be honest, I didn't even think of doing that. I was specifically looking for the patch. I didn't even know that it was available in that tarball. I have really no experience in making or modifying rpm's. New stuff to learn! Frank. From: [EMAIL PROTECTED] on behalf of Matt McCutchen Sent: Thu 10/11/2007 5:58 PM To: Frank Thomas Cc: rsync@lists.samba.org Subject: Re: Recognizing directory name changes On 10/11/07, Frank Thomas [EMAIL PROTECTED] wrote: Thanks Matt again for the info. So now I'm feeling foolish. I tried to just pull the latest version of this from the cvs and I keep getting the following errors when I try to click onto 'download'. Yes, the cvsweb seems to be broken; I have reported the problem to the Samba webmaster. However, I don't understand why you would want to download just detect-renamed.diff from the CVS. For best results, you should use the same version of detect-renamed.diff as the rest of the source files; you can either check out the entire CVS tree with a CVS client or download a recent release or snapshot tarball from http://rsync.samba.org/ftp/rsync/ . And you only need to do any of this if you want to replace the source tarball included in the source RPM with something newer. Matt Warning: This message and any attachments are intended only for the use of the individual or entity to which they are addressed and may contain confidential and personal information which may be subject to the Freedom of Information and Protection of Privacy (FOIPP) Act and other legislations. If you are not the intended recipient, you are hereby notified that any review, retransmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited and may violate privacy and copyright laws. If you are not the intended recipient, please notify the sender immediately by return phone or fax, and delete any electronic or hard copy in your possession. Thank-you. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5008] make check fails on Cygwin (default-acls)
https://bugzilla.samba.org/show_bug.cgi?id=5008 --- Comment #3 from [EMAIL PROTECTED] 2007-10-12 10:56 CST --- Waiting for more information on this, i suggest that you SKIP the test in the Cygwin case. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync 3.0.0pre2 released
On Fri, 2007-10-12 at 10:58 +0200, Paul Slootman wrote: On Thu 11 Oct 2007, Wayne Davison wrote: I've just released rsync 3.0.0pre2, the second pre-release version for the upcoming 3.0.0 release. Please test this out and email the rsync mailing list with any questions, comments, bug reports, etc. Thanks! One thing that's not 100% clear for me: in the NEWS file you state: - Added the --acls (-A) option to preserve Access Control Lists. This is an improved version of the prior patch that was available, and it even supports OS X ACLs. (If you need to have backward compatibility with old, patched versions of rsync, apply the acls.diff file from the patches dir.) Does applying the acls.diff patch influence the workings of the new standard --acls option, when _not_ talking to an old patched version? If it doesn't harm the standard working, then it's probably a good idea for the Debian packaged version to apply the diff at least until this version of rsync is contained in an official new release of Debian... By reading the code, I'd answer: no. It seem they just allow rsync to try exchanging this info with versions of the protocol prior 30. Simo. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync 3.0.0pre2 released
On Thu, 2007-10-11 at 22:10 -0400, Matt McCutchen wrote: On 10/11/07, Wayne Davison [EMAIL PROTECTED] wrote: I've just released rsync 3.0.0pre2, the second pre-release version for the upcoming 3.0.0 release. I have built RPMs (source and i386 binary) for rsync 3.0.0pre2: http://mattmccutchen.net/rsync/#rsync-packages FYI, I am building RPMs for rawhide (will be Fedora 9) as well, enabling the backward compatibility patches for xattrs and acls, as we used these patches in RHEL and Fedora packages previously. Simo. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 5008] make check fails on Cygwin (default-acls)
https://bugzilla.samba.org/show_bug.cgi?id=5008 --- Comment #1 from [EMAIL PROTECTED] 2007-10-12 04:57 CST --- The original report was against 3.0.0pre1. The same behaviour also occurs with 3.0.0pre2. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync on 3 places
Please keep replies on-list for the benefit of others, the archives etc. On 10/12/07, Michael Reynolds [EMAIL PROTECTED] wrote: snip question. I am running on Macintosh OS X operating system . You wrote about installing daemon on machine A and put clients on B and C. I would like to ask you what are benefits (differences) between running rsync as daemon (server) and just by writing script by simple command ? I tried just simple command and looks it works ( I am using switches vctzErR. I am asking about it because if I create just one script on machine A I can simply change it only on one place. Otherwise your solution need the script will be changed on two places (B and C) so I am just asking what is advantage of rsync server in this case. Because of security considerations I tend to avoid excessive daemons. Any extra listening port on a host is an extra security threat. With that in mind, I would go for one daemon on machine A, and the two other machines using simple scripted rsync commands to exchange data with the daemon. Other than that, it's up to you. You're right that if you have two daemons (on hosts B C) and one script (on host A) you have less scripts to maintain. Cheers -A -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync on 3 places
On 10/12/07, Alexandros Papadopoulos [EMAIL PROTECTED] wrote: With that in mind, I would go for one daemon on machine A, and the two other machines using simple scripted rsync commands to exchange data with the daemon. Having the other machines connect to A via ssh may be easier than setting up a daemon. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4977] rsync: failed to set times on
https://bugzilla.samba.org/show_bug.cgi?id=4977 [EMAIL PROTECTED] changed: What|Removed |Added CC||[EMAIL PROTECTED] --- Comment #8 from [EMAIL PROTECTED] 2007-10-12 16:14 CST --- I had the same problem with my backups, and rebuilding it on the same machine didn't help. I suspect it's because support for lutimes is fs-specific, and I have ext3 (which supports lutimes) on /home (where the build was done), and reiserfs (which apparently does not support lutimes) on /volumes/backup (where the backups are stored) A rebuild with HAVE_LUTIMES forcibly undefined does not give the error. I think a real solution would be for rsync to detect lutimes support per-fs at runtime, before attempting to use it. I'd be happy if it never bothered to try, or not unless given something like --have_lutimes. I'm not happy having the default be to throw a bunch of errors about something I don't care about that can't be done on that disk anyway. [EMAIL PROTECTED] ~]$ uname -a Linux seneschal 2.6.21.7-1 #1 SMP Sat Aug 4 18:59:20 UTC 2007 i686 AMD_Athlon(tm)_Processor PLD Linux -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
3.0.0pre2: non-recursive remote listing breakage
3.0.0pre2: non-recursive remote listing breakage Looks like pre1's fix broke something else. I noticed this when listing module/directory contents on servers known to be running versions 2.5.7 2.6.3. Not having (full) access to them, I replicated the errors using 2 machines on my home lan. # the machines lithium.elements.lan: the client. rsync 3.0.0pre2 for all tests. fluorine.elements.lan: the server with slack mirror mockup. All rsync versions from 2.6.9 back to 2.5.5 (to cover all the bases). # the test rsync rsync://fluorine.elements.lan/slackware/ # the results (2.6.9 to 2.6.4) motd module contents displayed as expected. (2.6.3, as seen on lithium) snip motd, if any rsync: on remote machine: -d: unknown option rsync error: requested action not supported (code 4) at clientserver.c(473) rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(755) [receiver=3.0.0pre2] (2.6.2 to 2.5.5, as seen on lithium) snip motd, if any rsync: connection unexpectedly closed (0 bytes received so far) [receiver] rsync error: error in rsync protocol data stream (code 12) at io.c(596) [receiver=3.0.0pre2] (2.6.2 to 2.5.5, as seen in fluorine's logs) Oct 12 18:05:03 fluorine rsyncd[2733]: rsync on slackware/ from lithium.elements.lan (192.168.1.3) Oct 12 18:05:03 fluorine rsyncd[2733]: on remote machine: -d: unknown option Oct 12 18:05:03 fluorine rsyncd[2733]: rsync error: requested action not supported (code 4) at clientserver.c(424) Apologies for not reporting this earlier. Real life got in the way. Erik -- Failure is not an option. (It comes bundled with Windows.) If at first you don't succeed, redefine success. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: 3.0.0pre2: non-recursive remote listing breakage
On Fri, Oct 12, 2007 at 06:28:01PM -0400, Erik Jan Tromp wrote: Looks like pre1's fix broke something else. That is a known limitation of the removal of a really old kluge in the listing code. To quote from the NEWS file: - Requesting a remote file list without specifying -r (--recursive) now sends the -d (--dirs) option to the remote rsync rather than sending -r along with an extra exclude of /*/*. If the remote rsync does not understand the -d option (i.e. it is 2.6.3 or older), you will need to either turn off -d (--no-d), or specify -r --exclude='/*/*' manually. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4977] rsync: failed to set times on
https://bugzilla.samba.org/show_bug.cgi?id=4977 [EMAIL PROTECTED] changed: What|Removed |Added Status|NEW |RESOLVED Resolution||FIXED Version|2.6.9 |3.0.0 --- Comment #9 from [EMAIL PROTECTED] 2007-10-12 20:33 CST --- Why not upgrade to 3.0.0 then? This is one of the things that is improved -- it doesn't bother to output an error about failing to set the time on a symlink. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 4977] rsync: failed to set times on
https://bugzilla.samba.org/show_bug.cgi?id=4977 --- Comment #10 from [EMAIL PROTECTED] 2007-10-12 20:42 CST --- (In reply to comment #8) I suspect it's because support for lutimes is fs-specific, and I have ext3 (which supports lutimes) on /home (where the build was done) Linux seneschal 2.6.21.7-1 #1 SMP Sat Aug 4 18:59:20 UTC 2007 i686 BTW: lutimes isn't going to work on any filesystem with a kernel before 2.6.22. For HAVE_LUTIMES, the configure script tests only whether the lutimes function exists in glibc as a non-stub, not whether it actually works. Thus, if your copy of glibc was built on a machine with kernel = 2.6.22, it will contain a non-stub lutimes and rsync's HAVE_LUTIMES test will pass. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --detect-renamed question
On 10/12/07, Greg Siekas [EMAIL PROTECTED] wrote:
The other option I thought of was to only do the move when the mtime,
size, and filename match. Not really a 'detect-renamed' but a
'detected-moved' type operation.
That's a good idea, and easy to implement too! I have improved the
patch (attached) to provide separate --trust-rename and --trust-move
options. Wayne, please consider adding this to patches/ .
Matt
In combination with detect-renamed.diff, this patch adds an option
--trust-rename that adopts the pre-rename destination file found for a
new source file without verifying that the data is actually the same.
It also adds a variant --trust-move that requires that the basenames
match. These options are somewhat risky but were what Greg Siekas wanted:
http://lists.samba.org/archive/rsync/2007-October/018827.html
This patch is EXPERIMENTAL, though it did work correctly in my light
testing.
FIXME: If a run with --trust-rename stages a different-basename destination
file and then gets interrupted, a subsequent run with --trust-move trusts
the staged file.
-- Matt McCutchen [EMAIL PROTECTED]
--- old/generator.c
+++ new/generator.c
@@ -80,6 +80,7 @@ extern int compare_dest;
extern int copy_dest;
extern int link_dest;
extern int detect_renamed;
+extern int trust_rename;
extern int whole_file;
extern int list_only;
extern int read_batch;
@@ -212,7 +213,9 @@ static int fattr_find(struct file_struct
high = mid - 1;
}
- return good_match = 0 ? good_match : ok_match;
+ return good_match = 0 ? good_match :
+ /* --trust-move doesn't allow non-basename matches */
+ (trust_rename == 1) ? -1 : ok_match;
}
static void look_for_rename(struct file_struct *file, char *fname)
@@ -1826,6 +1829,22 @@ static void recv_generator(char *fname,
fnamecmp = partialptr;
fnamecmp_type = FNAMECMP_PARTIAL_DIR;
statret = 0;
+ if (detect_renamed trust_rename
+ unchanged_file(fnamecmp, file, sx.st)) {
+ /* Adopt the partial file. */
+ finish_transfer(fname, fnamecmp, NULL, NULL, file, 1, 1);
+ handle_partial_dir(partialptr, PDIR_DELETE);
+ if (itemizing)
+itemize(fnamecmp, file, ndx, -1, sx,
+ ITEM_LOCAL_CHANGE, fnamecmp_type, NULL);
+#ifdef SUPPORT_HARD_LINKS
+ if (preserve_hard_links F_IS_HLINKED(file))
+finish_hard_link(file, fname, ndx, sx.st, itemizing, code, -1);
+#endif
+ if (remove_source_files == 1)
+goto return_with_success;
+ goto cleanup;
+ }
}
if (!do_xfers)
--- old/options.c
+++ new/options.c
@@ -81,6 +81,7 @@ int am_starting_up = 1;
int relative_paths = -1;
int implied_dirs = 1;
int detect_renamed = 0;
+int trust_rename = 0;
int numeric_ids = 0;
int allow_8bit_chars = 0;
int force_delete = 0;
@@ -385,6 +386,8 @@ void usage(enum logcode F)
rprintf(F, -T, --temp-dir=DIR create temporary files in directory DIR\n);
rprintf(F, -y, --fuzzy find similar file for basis if no dest file\n);
rprintf(F, --detect-renamedtry to find renamed files to speed up the transfer\n);
+ rprintf(F, --trust-rename ... and assume identical to source files (risky!)\n);
+ rprintf(F, --trust-move... only if basenames match (less risky)\n);
rprintf(F, --compare-dest=DIR also compare destination files relative to DIR\n);
rprintf(F, --copy-dest=DIR ... and include copies of unchanged files\n);
rprintf(F, --link-dest=DIR hardlink to files in DIR when unchanged\n);
@@ -564,6 +567,8 @@ static struct poptOption long_options[]
{copy-dest,0, POPT_ARG_STRING, 0, OPT_COPY_DEST, 0, 0 },
{link-dest,0, POPT_ARG_STRING, 0, OPT_LINK_DEST, 0, 0 },
{detect-renamed, 0, POPT_ARG_NONE, detect_renamed, 0, 0, 0 },
+ {trust-rename, 0, POPT_ARG_VAL,trust_rename, 2, 0, 0 },
+ {trust-move, 0, POPT_ARG_VAL,trust_rename, 1, 0, 0 },
{fuzzy, 'y', POPT_ARG_NONE, fuzzy_basis, 0, 0, 0 },
{compress,'z', POPT_ARG_NONE, 0, 'z', 0, 0 },
{no-compress, 0, POPT_ARG_VAL,do_compression, 0, 0, 0 },
@@ -1895,8 +1900,13 @@ void server_options(char **args, int *ar
}
}
/* Both sides need to know in case this disables incremental recursion. */
- if (detect_renamed)
+ if (detect_renamed) {
args[ac++] = --detect-renamed;
+ /* But the addition of --trust-* is only the receiver's business. */
+ if (am_sender trust_rename)
+ args[ac++] = (trust_rename == 2) ?
+ --trust-rename : --trust-move;
+ }
if (modify_window_set) {
if (asprintf(arg, --modify-window=%d, modify_window) 0)
--- old/rsync.yo
+++ new/rsync.yo
@@ -385,6 +385,8 @@ to the detailed description below for a
-T, --temp-dir=DIR create temporary files in directory DIR
-y, --fuzzy find similar file for basis if no dest file
--detect-renamedtry to find renamed files to speed the xfer
+ --trust-rename ... assume identical to src files (risky!)
+ --trust-move
Re: Recognizing directory name changes
On 10/12/07, Frank Thomas [EMAIL PROTECTED] wrote: Thank you for telling me of it's issues. When the patch approaches testing, please contact me and I can setup a test environment to test transfers in the Gig's. I have fixed the patch and Wayne has committed my fixes to the CVS. You may test it now. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/packaging
Date: Fri Oct 12 14:08:19 2007 Author: wayned Update of /data/cvs/rsync/packaging In directory dp.samba.org:/tmp/cvs-serv20893 Modified Files: release-rsync Log Message: Get the version # right in the changelog. Revisions: release-rsync 1.19 = 1.20 http://www.samba.org/cgi-bin/cvsweb/rsync/packaging/release-rsync?r1=1.19r2=1.20 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/packaging/lsb
Date: Fri Oct 12 14:04:00 2007 Author: wayned Update of /data/cvs/rsync/packaging/lsb In directory dp.samba.org:/tmp/cvs-serv16561/lsb Modified Files: rsync.spec Log Message: Fixed the day of the week. Revisions: rsync.spec 1.42 = 1.43 http://www.samba.org/cgi-bin/cvsweb/rsync/packaging/lsb/rsync.spec?r1=1.42r2=1.43 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/packaging/lsb
Date: Fri Oct 12 13:57:37 2007 Author: wayned Update of /data/cvs/rsync/packaging/lsb In directory dp.samba.org:/tmp/cvs-serv9856/packaging/lsb Modified Files: rsync.spec Log Message: Improved the summary, the description, and the changelog. Revisions: rsync.spec 1.41 = 1.42 http://www.samba.org/cgi-bin/cvsweb/rsync/packaging/lsb/rsync.spec?r1=1.41r2=1.42 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync
Date: Sat Oct 13 05:23:34 2007 Author: wayned Update of /data/cvs/rsync In directory dp.samba.org:/tmp/cvs-serv30472 Modified Files: io.c Log Message: Forward MSG_IO_ERROR to the generator so that it can disable deletions. Revisions: io.c1.247 = 1.248 http://www.samba.org/cgi-bin/cvsweb/rsync/io.c?r1=1.247r2=1.248 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
