[SCM] The rsync repository. - branch master updated
The branch, master has been updated
via ab29ee9c Negotation env lists can specify "client & server"
via d07c2992 A few more simple changes & fixes.
from b1a8b09c Some man page changes.
https://git.samba.org/?p=rsync.git;a=shortlog;h=master
- Log -
commit ab29ee9c44ef8d58bd2a2bf1cf0e911c71a92104
Author: Wayne Davison
Date: Fri Jun 26 16:51:30 2020 -0700
Negotation env lists can specify "client & server"
commit d07c2992d1b672ac8524e462fcb5f4bf223de768
Author: Wayne Davison
Date: Fri Jun 26 13:10:10 2020 -0700
A few more simple changes & fixes.
---
Summary of changes:
NEWS.md| 3 ++-
batch.c| 8 ---
compat.c | 76 ++
ifuncs.h | 2 +-
options.c | 2 +-
rsync.1.md | 21 -
rsync.h| 12 +-
util2.c| 4 ++--
8 files changed, 80 insertions(+), 48 deletions(-)
Changeset truncated at 500 lines:
diff --git a/NEWS.md b/NEWS.md
index 2d81eb66..d579b955 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -17,7 +17,8 @@ Protocol: 31 (unchanged)
### ENHANCEMENTS:
- Allow the server side to restrict checksum & compression choices via
- the same environment variables the client uses.
+ the same environment variables the client uses. Allow the env vars
+ to be divided into "client list & server list" by the "`&`" char.
- Simplify how the negotiation environment variables apply to older rsync
versions.
diff --git a/batch.c b/batch.c
index 805acc6f..582c8c16 100644
--- a/batch.c
+++ b/batch.c
@@ -269,14 +269,6 @@ void write_batch_shell_file(void)
err |= write_opt("--exclude-from", "-");
}
- /* We need to make sure that any protocol-based or negotiated choices
get accurately
-* reflected in the options we save AND that we avoid any need for
--read-batch to
-* do a string-based negotiation (since we don't write them into the
file). */
- if (do_compression)
- err |= write_opt("--compress-choice", compress_choice);
- if (strchr(checksum_choice, ',') || xfersum_type !=
parse_csum_name(NULL, -1))
- err |= write_opt("--checksum-choice", checksum_choice);
-
/* Elide the filename args from the option list, but scan for them in
reverse. */
for (i = raw_argc-1, j = cooked_argc-1; i > 0 && j >= 0; i--) {
if (strcmp(raw_argv[i], cooked_argv[j]) == 0) {
diff --git a/compat.c b/compat.c
index ba14a8c5..bf0b4e6d 100644
--- a/compat.c
+++ b/compat.c
@@ -20,6 +20,7 @@
*/
#include "rsync.h"
+#include "itypes.h"
extern int am_server;
extern int am_sender;
@@ -109,6 +110,9 @@ struct name_num_obj valid_compressions = {
#define CF_INPLACE_PARTIAL_DIR (1<<6)
#define CF_VARINT_FLIST_FLAGS (1<<7)
+#define ENV_CHECKSUM 0
+#define ENV_COMPRESS 1
+
static const char *client_info;
/* The server makes sure that if either side only supports a pre-release
@@ -262,10 +266,14 @@ static void init_nno_saw(struct name_num_obj *nno, int
val)
static int parse_nni_str(struct name_num_obj *nno, const char *from, char
*tobuf, int tobuf_len)
{
char *to = tobuf, *tok = NULL;
- int cnt = 0;
+ int saw_tok = 0, cnt = 0;
while (1) {
- if (*from == ' ' || !*from) {
+ int at_space = isSpace(from);
+ char ch = *from++;
+ if (ch == '&')
+ ch = '\0';
+ if (!ch || at_space) {
if (tok) {
struct name_num_item *nni =
get_nni_by_name(nno, tok, to - tok);
if (nni && !nno->saw[nni->num]) {
@@ -279,9 +287,10 @@ static int parse_nni_str(struct name_num_obj *nno, const
char *from, char *tobuf
}
} else
to = tok - (tok != tobuf);
+ saw_tok = 1;
tok = NULL;
}
- if (!*from++)
+ if (!ch)
break;
continue;
}
@@ -294,10 +303,13 @@ static int parse_nni_str(struct name_num_obj *nno, const
char *from, char *tobuf
to = tok - (tok != tobuf);
break;
}
- *to++ = *from++;
+ *to++ = ch;
}
*to = '\0';
+ if (saw_tok && to == tobuf)
+ return strlcpy(tobuf, "INVALID", MAX_NSTR_STRLEN);
+
return to - tobuf;
}
@@ -349,14 +361,36 @@ static void recv_negotiate_str(int f_in, struct
name_num_obj *nno, char *tmpbuf,
exit_cleanup(RERR_UNSUPPORTED);
}
+static const char *getenv_nstr(int etype)
+{
+
Re: Rsync Daemon Remote Pull
On Thu, Jun 25, 2020 at 3:02 PM Chandrasekar Natarajan wrote: > Unexpected remote arg: user@x.x.x.x:port/module > This means that your non-option args starts with a local arg, contains a remote arg somewhere in the middle, and ends with either a local or remote arg. None of those are valid because all the source args must match in being either local or remote. So, you need to figure out where the local path comes from early in the command-line. For instance, if you copied an "--option" that has some kind of unicode char instead of real dashes, it could look like an option but actually be parsed as a local arg. Or you may have some kind of alias or script that is adding an arg early in your rsync command-line. Be sure to check for shell aliases, popt aliases (the ~/.popt and /etc/popt files), and "rsync" scripts on your path. ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync with mode --delete
On Fri, Jun 26, 2020 at 10:24 AM Budi Janto wrote: > path = /mnt/DATA > Any idea or suggestion to prevent client's data loss? > There are several options: 1. Set the "pre-xfer exec = /script/name" in your daemon module and make the script check if a /mnt/DATA/known-to-exist-subdir is missing and if so, do an "exit 1". 2. If your server rsync is at least 3.2.1, set the "early exec = /script/name" in your daemon module and make the script verify that the drive is mounted & attempt to mount it if it is not (return an error code if unable to mount it). The mount action is not possible in the pre-xfer exec script because rsync has already done a chdir to the empty mount by the time this script runs. 3. Move everything down into a deeper directory and tweak the path in your daemon module to be "path = /mnt/DATA/subdir" so that the transfer will fail when the mount is not there (because the subdir is not there). ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RAM speedup
Hello, As disks are slow and rsync reads and writes so much that for the bus this is the equivalent of context switching galore, would it be possible to use RAM as a buffer? Say, you have 10GB of spare RAM, rsync uses the bus to its peak for reading 10GB, then again for writing it down. This would be more efficient than lot of small read/write operations. Thank you-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[SCM] The rsync repository. - branch master updated
The branch, master has been updated via b1a8b09c Some man page changes. from fe2ef556 A few more tweaks. https://git.samba.org/?p=rsync.git;a=shortlog;h=master - Log - commit b1a8b09c21771902cce094bd061b1cd83887af5f Author: Wayne Davison Date: Fri Jun 26 11:27:27 2020 -0700 Some man page changes. --- Summary of changes: rsyncd.conf.5.md | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) Changeset truncated at 500 lines: diff --git a/rsyncd.conf.5.md b/rsyncd.conf.5.md index 4da26b62..4e9da77c 100644 --- a/rsyncd.conf.5.md +++ b/rsyncd.conf.5.md @@ -905,15 +905,15 @@ the values of parameters. See the GLOBAL PARAMETERS section for more details. - `--server`: Required for rsync to even work. - `--rsh`, `-e`: Required to convey compatibility flags to the server. -- `--log-format`: This is required to convey things like - `--itemize-changes` to a remote receiver. Is an older name for - `--out-format` that is still passed to the server for improved backward - compatibility and should not be confused with `--log-file-format`. +- `--out-format`: This is required to convey output behavior to a remote + receiver. While rsync passes the older alias `--log-format` for + compatibility reasons, this options should not be confused with + `--log-file-format`. - `--sender`: Use "write only" parameter instead of refusing this. - `--dry-run`, `-n`: Who would want to disable this? -- `--protect-args`, `-n`: This actually makes transfers safer. -- `--from0`, `-0`: Make it easier to accept/refuse `--files-from` without - affecting this modifier. +- `--protect-args`, `-s`: This actually makes transfers safer. +- `--from0`, `-0`: Makes it easier to accept/refuse `--files-from` without + affecting this helpful modifier. - `--iconv`: This is auto-disabled based on "charset" parameter. - `--no-iconv`: Most transfers use this option. - `--checksum-seed`: Is a fairly rare, safe option. -- The rsync repository. ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
Re: Rsync Daemon Remote Pull
Hi, Even I tried the same syntax but failed. But push is working fine. rsync: Unexpected remote arg: user@x.x.x.x::module. > On 26-Jun-2020, at 5:22 AM, Kevin Korb wrote: > > user@x.x.x.x:port/ModuleName is not correct syntax. You may have > something in your shell config translating that for you. Correct syntax > is --port and user@host::module > >> On 6/25/20 6:01 PM, Chandrasekar Natarajan via rsync wrote: >> Hi, >> >> I am trying to pull folders from a windows remote machine using daemon >> without SSH. >> Below is the command that used: >> rsync -vrtz --delete user@x.x.x.x:port/ModuleName '/cygdrive/d/backup/' >> >> No issue running the same command in Command Prompt.While executing this >> command via c#.net Process startup, facing some issue. >> >> The exception is ': Unexpected remote arg: user@x.x.x.x:port/module >> rsync error: syntax or usage error (code 1) at main.c(1361) [sender=3.1.2] >> >> >> Kindly help me to resolve this. >> >> -- >> Thanks, >> >> *Chandrasekar N* >> > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >Kevin KorbPhone:(407) 252-6853 >Systems AdministratorInternet: >FutureQuest, Inc.ke...@futurequest.net (work) >Orlando, Floridak...@sanitarium.net (personal) >Web page:https://sanitarium.net/ >PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync with mode --delete
On 6/23/20 12:37 PM, francis.montag...@inria.fr wrote: > If the rsync source path is /mnt/DATA yes, it will remove data on the > client, but if the source path is a sub-directory of /mnt/DATA the > rsync will fail without doing any deletion. > > To verify in dry-run ... of course. > Thanks for first information. This is for more detail, which has already been done. Server side: # zfs list NAME USED AVAIL REFER MOUNTPOINT pool 4.98T 9.07T 4.98T /mnt/DATA # cat /some/path/rsyncd.conf [data] comment = Storage area total approx size: 5.0 TiB (5341197216 KiB). gid = nobody list = yes log file = /var/log/rsyncd.log log format = %o %m %a %f %l (%b) max connections = 20 path = /mnt/DATA read only = true reverse lookup = false timeout = 300 transfer logging = yes uid = nobody use chroot = yes Client side: # crontab -l 0 */3 * * * /path/to/rsync.sh # cat /path/to/rsync.sh rsync -avPH --chmod=D755,F644 --delete rsync://xxx.xxx.xxx.xxx/data/ Any idea or suggestion to prevent client's data loss? Thank you. -- Regards, Budi Janto -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
character devices
Hi, I was running a couple of tests on the latest rsync and noticed a different behavior when it comes to character devices. The test creates a character device on the client side and a simple text file on the server side. When copying from server to the client, previously the character device would get changed to a regular text file after the transaction but with new rsync, the character device is still a character device. Before: # ls -l /var/tmp/client crw-rw-rw-. 1 root root 1, 0 Jun 26 05:41 character.device # ls -l /home/rsynctestuser/rsyncdata -rw-r--r--. 1 root root 12 Jun 26 05:41 character.device # rsync --perms --owner --group --times --update --recursive ::my_data /var/tmp/client # ls -l /var/tmp/client -rw-r--r--. 1 root root 12 Jun 26 05:46 character.device <--- the file has changed to a text file After: # ls -l /var/tmp/client/ crw-rw-rw-. 1 root root 1, 0 Jun 26 05:46 character.device # ls -l /home/rsynctestuser/rsyncdata -rw-r--r--. 1 root root 12 Jun 26 05:46 character.device #rsync --perms --owner --group --times --update --recursive ::my_data /var/tmp/client # ls -l /var/tmp/client crw-rw-rw-. 1 root root 1, 0 Jun 26 05:46 character.device <--- no change in the type of file I tried to run the newer version with --write-devices but manpage says: "This option is refused by an rsync daemon.". I am trying to find out what is the correct behavior in this case. Was the assumption that the character device would change with the older version wrong? Is there a way how to let the server rewrite the file? I tried the 'refuse options' statement in the rsyncd.conf but no luck. I would truly appreciate any ideas about this. Thanks and regards, Michal Ruprich -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error
On 25/06/2020 13:18, Madhurananda Pahar via rsync wrote:
> Hello everybody,
>
> I am having a funny problem while using rsync as a tool to
> back-up my files:
>
> [sender] expand file_list pointer array to 524288 bytes, did move.
>
> I am just wondering if you had this issue before and if you know a way
> to solve this, please share with me.
>
> Many thanks,
> Dr. Madhurananda Pahar.
That's just a debug message, letting the developers know when the file
list had to be expanded, and whether it had to be moved (by realloc(3))
as a result -- because that case introduces the particular hazard that
some other code might have kept a pointer to the old array. So this
message would be useful if one were debugging a random crash, to know
whether it only happened when the file list grew to a certain size, or
when it was relocated during expansion.
> if (DEBUG_GTE(FLIST, 1) && flist->malloced != FLIST_START) {
> rprintf(FCLIENT, "[%s] expand file_list pointer array to %s
> bytes, did%s move\n",
> who_am_i(),
> big_num(sizeof flist->files[0] * flist->malloced),
> (new_ptr == flist->files) ? " not" : "");
> }
>
It's not an error of any sort, and you're only seeing it because you've
enabled DEBUG_GTE. Is there some other problem that you're trying to debug?
.Dave.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12769] error allocating core memory buffers (code 22) depending on source file system
https://bugzilla.samba.org/show_bug.cgi?id=12769 --- Comment #13 from MulticoreNOP --- (In reply to Wayne Davison from comment #12) Hi Wayne, that is great news! Could you shine some light on why there is such a limit in the first place? Personally I think such an arbitrary limit is rather unexpected and many people will have a long running rsync command (for me each try took ~6hrs until I could reproduce the error) fail on them until they could even be aware that there is such a limit. In addition to that, I guess only a small fraction of those people will probably find this parameter as a solution to their problem. To me this is like 'cp -R a b' failing, if a/ contains more than 1337 files. Cheers, Mc NOP -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
