[Bug 15585] New: rsync ends still with error 22 when try to deleting many files

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=15585

Bug ID: 15585
   Summary: rsync ends still with error 22 when try to deleting
many files
   Product: rsync
   Version: 3.2.0
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: wa...@opencoder.net
  Reporter: f...@fkn-systems.de
QA Contact: rsync...@samba.org
  Target Milestone: ---

Problem:

When syncing Souce/Target and must deleting many files, rsync stop still
quiet with Error-Code:

 "22 - Error allocating core memory buffers"

Desc:

When more than ~5-10 million files to delete, rsync ends still quiet, in
case of deleting process. Rsync reads (with --delete-before) the
complete Source and breaks than after deleting many (>1000) files on
Target. Else, (withOUT --delete-before), he does some sync on Target and
some deletes, some sync ... and breaks than after delete some files.

In case of this, Backups are not synced, and may space overflow after
next backup run.

Use of ‐‐max‐delete=100.000.000 are not functional to solve this

Command:

 rsync -vaxHSPAX --delete [--delete-before] $SOURCE $TARGET
 # $SOURCE, $TARGET are real directorys or remote (ramote:dir...)

Versions: It tested fail with:

 * rsync  version 3.2.3  protocol version 31
 * rsync  version 3.2.7  protocol version 31

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Everything working as expected, so shouldn't ERROR be WARNING

[Roger Price via rsync]

On Thu, 18 Jan 2024, Paul Slootman via rsync wrote:

On Thu 18 Jan 2024, Roger Price via rsync wrote:

I get the messages

 sending incremental file list
 ERROR: daemon refused to receive file "rprice/demo.dvi"

I understand that the remote daemon has refused file demo.dvi because I
specifically requested that dvi files not be transferred.  I choose that
myself in a regular configuration file.  So shouldn't it be a WARNING rather
than an ERROR?  I would expect to see


In this case you're in control of both ends of the transfer, so you know
that *.dvi files won't be transferred.

However, it could be that this rsync command is being run by someone who
expects rsync to do what they asked it to do, i.e. transfer the entire
contents of .../rprice to the remote server, and the client rsync can't
fulfil that request; hence the error.

If you don't want *.dvi files to be transferred, then you should add
   --exclude '*.dvi'
to the invocation.


I accept that if the message has to cover all the cases, including hardware 
misconfigurations and errors, then ERROR is necessary.


But it would have been nice to have a specific message for exclude = ... 
effect in remote rsyncd.conf even if the user has no direct access.


Roger

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Everything working as expected, so shouldn't ERROR be WARNING

[Paul Slootman via rsync]

On Thu 18 Jan 2024, Roger Price via rsync wrote:

> I am backing up a user's directories from local machine titan to remote
> machine maria. On the remote machine maria file /etc/rsyncd.conf contains
> 
>  [rprice-home]
>  ...
>  exclude = *.dvi
> 
> I start the backup by using this command on the local machine titan:
> 
>  rprice@titan ~ rsync -av --dry-run /mnt/home/rprice 
> rsync://rprice@maria/rprice-home
> 
> I get the messages
> 
>  sending incremental file list
>  ERROR: daemon refused to receive file "rprice/demo.dvi"
>  ...
> 
> I understand that the remote daemon has refused file demo.dvi because I
> specifically requested that dvi files not be transferred.  I choose that
> myself in a regular configuration file.  So shouldn't it be a WARNING rather
> than an ERROR?  I would expect to see

In this case you're in control of both ends of the transfer, so you know
that *.dvi files won't be transferred.

However, it could be that this rsync command is being run by someone who
expects rsync to do what they asked it to do, i.e. transfer the entire
contents of .../rprice to the remote server, and the client rsync can't
fulfil that request; hence the error.

If you don't want *.dvi files to be transferred, then you should add
--exclude '*.dvi'
to the invocation.


Paul

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Everything working as expected, so shouldn't ERROR be WARNING

[Roger Price via rsync]

I am backing up a user's directories from local machine titan to remote machine 
maria. On the remote machine maria file /etc/rsyncd.conf contains


 [rprice-home]
 ...
 exclude = *.dvi

I start the backup by using this command on the local machine titan:

 rprice@titan ~ rsync -av --dry-run /mnt/home/rprice 
rsync://rprice@maria/rprice-home

I get the messages

 sending incremental file list
 ERROR: daemon refused to receive file "rprice/demo.dvi"
 ...

I understand that the remote daemon has refused file demo.dvi because I 
specifically requested that dvi files not be transferred.  I choose that myself 
in a regular configuration file.  So shouldn't it be a WARNING rather than an 
ERROR?  I would expect to see


 WARNING: daemon refused to receive file "rprice/demo.dvi"

Roger

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rsync error messages always in English (= never translated)?

[rsync--- via rsync]

I want to recognize and handle some rsync error messages
in my log files (containing also the --itemize-changes output)
on different computers with different language/locale settings.

Can I rely on rsync to create only English error messages
to have a stable pattern to recognize?

PS: In the source code at https://github.com/WayneD/rsync I could not see
any indication of message translations...



-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Uncorrectable bit error handling by rsync?

[John Conover via rsync]

Suppose rsync on one system is used to synchronize files to another
system.

Further, suppose rsync encounters an uncorrectable bit error reading a
file that is being transferred.

What's the chances that rsync corrupts the file on the other system?

Thanks,

John

-- 

John Conover, cono...@panix.com, http://www.johncon.com/

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Build error on Solaris 10

[Wayne Davison via rsync]

On Tue, Sep 20, 2022 at 11:21 AM İhsan Doğan via rsync <
rsync@lists.samba.org> wrote:

> ./mkgitver
> ./mkgitver: test: argument expected
>

Looks like your OS's "test" doesn't support "-e".  If you tweak the "if [
-e" to be "if [ -d" you should be fine.

..wayne..
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Build error on Solaris 10

[İhsan Doğan via rsync]

Hi,

While rsync 3.2.5 was building fine on Solaris 10, I’m running into this error 
with 3.2.6:

/opt/SUNWspro/bin/cc -I. -I. -I./zlib -xO3 -m32 -xarch=sparc -DHAVE_CONFIG_H 
-I/opt/csw/include -c uidlist.c -o uidlist.o
/opt/SUNWspro/bin/cc -I. -I. -I./zlib -xO3 -m32 -xarch=sparc -DHAVE_CONFIG_H 
-I/opt/csw/include -c socket.c -o socket.o
/opt/SUNWspro/bin/cc -I. -I. -I./zlib -xO3 -m32 -xarch=sparc -DHAVE_CONFIG_H 
-I/opt/csw/include -c hashtable.c -o hashtable.o
gawk -f ./help-from-md.awk -v hfile=help-rsync.h ./rsync.1.md
gawk -f ./help-from-md.awk -v hfile=help-rsyncd.h ./rsync.1.md
./mkgitver
./mkgitver: test: argument expected
gmake: *** [Makefile:145: git-version.h] Error 1
gmake: Leaving directory 
'/home/ihsan/opencsw/rsync/trunk/work/solaris10-sparc/build-isa-sparcv8plus/rsync-3.2.6‘



Regards

Ihsan

smime.p7s
Description: S/MIME cryptographic signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: 3.2.6: ERROR: rejecting unrequested file-list

[Wayne Davison via rsync]

On Tue, Sep 13, 2022 at 9:06 AM Carlos Carvalho wrote:

> The debug option didn't change the output at all.
>

That seems quite unlikely to me. But ignoring that, the latest git should
hopefully have the issue you're seeing fixed.  Rsync packagers will want this
newest commit

and
this filter-verification fix

.

..wayne..
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: 3.2.6: ERROR: rejecting unrequested file-list

[Carlos Carvalho via rsync]

Wayne Davison (wa...@opencoder.net) wrote on Tue, Sep 13, 2022 at 02:54:37AM 
-03:
> It helps to have some kind of a reproducer.  Also, keep in mind there are 2
> kinds of rejections: unrequested args and unfiltered files.  The latter
> should have a big improvement in the latest git version. If you want some
> debug info on arg rejection, try adding `--debug=filter5` and see what
> auto-created filter rules get added to approve incoming file-list items.
 
The debug option didn't change the output at all.
 
It seems the receiver is rejecting the inclusion of a parent directory in the
update list. Here are 2 samples:
 
ERROR: rejecting unrequested file-list name: 
distrib/cauldron/SRPMS/core/release/media_info
rsync error: protocol incompatibility (code 2) at flist.c(998) [Receiver=3.2.6]
 
ERROR: rejecting unrequested file-list name: distribution/leap
rsync error: protocol incompatibility (code 2) at flist.c(998) [Receiver=3.2.6]
 
The first is from the mageia repository, the second from the opensuse one. The
names above are directories where files inside them appear in the files-from
list. The mentioned directories however are not in the files-from list
because there was no change to the directories themselves.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: 3.2.6: ERROR: rejecting unrequested file-list

[Wayne Davison via rsync]

It helps to have some kind of a reproducer.  Also, keep in mind there are 2
kinds of rejections: unrequested args and unfiltered files.  The latter
should have a big improvement in the latest git version. If you want some
debug info on arg rejection, try adding `--debug=filter5` and see what
auto-created filter rules get added to approve incoming file-list items.
That might show you which file-list item is being mistranslated.

..wayne..
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

3.2.6: ERROR: rejecting unrequested file-list

[Carlos Carvalho via rsync]

I've just upgraded the Debian package to 3.2.6 and now I'm getting many cases
of this error. I see that there have been improvements in the check but it
seems they're either not enough or incompatible with 3.1.[23].

This is always with --files-from, used for mirroring. We're a large free
software mirror and distribute tens of repositories. I had to revert to 3.2.5
for now.

Is there anything I can do? I can try patches.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Patch: more detailed error logging when an ACL unpack error occurs

[Wayne Davison via rsync]

On Sat, Sep 10, 2022 at 1:29 PM Ryan Schmidt wrote:

> Would you be interested in adding this patch to the rsync-patches repo or
> incorporating it into rsync directly?
>

I don't recall seeing that before, so thanks for pointing that out.  I've
committed the change to the rsync git.

..wayne..
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Patch: more detailed error logging when an ACL unpack error occurs

[Ryan Schmidt via rsync]

Hi, I maintain the rsync port in MacPorts.

For ten years, we've been carrying around a one-line patch which was described 
by its contributor this way:

> Occasionally, sync will report an ACL unpack error, but the user will be left 
> in the dark regarding which file(s) was affected. This adds more detailed 
> error logging


https://trac.macports.org/ticket/36169

I don't know if the contributor ever attempted to contribute it to the rsync 
project or only sent it to us at MacPorts.

Would you be interested in adding this patch to the rsync-patches repo or 
incorporating it into rsync directly?

-Ryan


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Invalid dir index: -826 (-101 - -825) [receiver] -- rsync error: protocol incompatibility (code 2) at rsync.c(370) [receiver=3.2.5]

[Budi Janto via rsync]

Hi, folk.

I got some problem when running this command on client side:

# rsync -avPH --chown=root:wheel --chmod=D755,F644 --delete --exclude 
ads --exclude backup rsync://192.168.0.100/data /mnt/DATA

receiving incremental file list
Invalid dir index: -826 (-101 - -825) [receiver]
rsync error: protocol incompatibility (code 2) at rsync.c(370) 
[receiver=3.2.5]

unexpected tag 62 [receiver/inc]

In server/client side, I use similar version OS which's 13.1-RELEASE-p1. 
And here rsync version that I have:


server# pkg info rsync
rsync-3.2.5
Name   : rsync
Version: 3.2.5
Installed on   : Sat Aug 20 13:33:47 2022 WIB
Origin : net/rsync
Architecture   : FreeBSD:13:amd64
Prefix : /usr/local
Categories : net
Licenses   : GPLv3+
Maintainer : rodr...@freebsd.org
WWW: http://rsync.samba.org/
Comment: Network file distribution/synchronization utility
Options:
DOCS   : on
FLAGS  : on
ICONV  : on
POPT_PORT  : off
RENAMED: off
SSH: on
ZLIB_BASE  : on
Shared Libs required:
libzstd.so.1
libxxhash.so.0
liblz4.so.1
libiconv.so.2
Annotations:
FreeBSD_version: 1300139
cpe: cpe:2.3:a:samba:rsync:3.2.5:freebsd13:x64
repo_type  : binary
repository : FreeBSD
Flat size  : 824KiB
Description:
rsync is a replacement for rcp that has many more features.

rsync uses the "rsync algorithm" which provides a very fast method for
bringing remote files into sync. It does this by sending just the
differences in the files across the link, without requiring that both
sets of files are present at one of the ends of the link beforehand.
This makes rsync a good remote file distribution/synchronization utility
in a dialup PPP/SLIP environment.

Note, requires rsync on the destination machine.

There is a Computer Science Technical Report on the rsync algorithm is
included in the distribution, and is available as
ftp://samba.anu.edu.au/pub/rsync/tech_report.ps

WWW: http://rsync.samba.org/

client# pkg info rsync
rsync-3.2.5
Name   : rsync
Version: 3.2.5
Installed on   : Sat Aug 20 13:43:46 2022 WIB
Origin : net/rsync
Architecture   : FreeBSD:13:amd64
Prefix : /usr/local
Categories : net
Licenses   : GPLv3+
Maintainer : rodr...@freebsd.org
WWW: http://rsync.samba.org/
Comment: Network file distribution/synchronization utility
Options:
DOCS   : on
FLAGS  : on
ICONV  : on
POPT_PORT  : off
RENAMED: off
SSH: on
ZLIB_BASE  : on
Shared Libs required:
libzstd.so.1
libxxhash.so.0
liblz4.so.1
libiconv.so.2
Annotations:
FreeBSD_version: 1300139
cpe: cpe:2.3:a:samba:rsync:3.2.5:freebsd13:x64
repo_type  : binary
repository : FreeBSD
Flat size  : 824KiB
Description:
rsync is a replacement for rcp that has many more features.

rsync uses the "rsync algorithm" which provides a very fast method for
bringing remote files into sync. It does this by sending just the
differences in the files across the link, without requiring that both
sets of files are present at one of the ends of the link beforehand.
This makes rsync a good remote file distribution/synchronization utility
in a dialup PPP/SLIP environment.

Note, requires rsync on the destination machine.

There is a Computer Science Technical Report on the rsync algorithm is
included in the distribution, and is available as
ftp://samba.anu.edu.au/pub/rsync/tech_report.ps

WWW: http://rsync.samba.org/

So, this affected to several removing directory on client side. But 
without option --delete is fine. Thanks.


--
Regards,
Budi Janto


OpenPGP_signature
Description: OpenPGP digital signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: deflate on token returned 0 - rsync error: error in protocol data stream

[Heiko Schlittermann via rsync]

Hi,

Heiko Schlittermann via rsync  (Mi 20 Okt 2021 10:56:51 
CEST):
> Hello,
> 
> we're using rsync on SLES 12 SP 5 on both sides (for detailed version
> info see below) and we're experiencing the following issue on the
> sender's side:
…
>  deflate on token returned 0 (22199 bytes left)
>  rsync error: error in rsync protocol data stream (code 12) at token.c(427) 
> [sender=3.1.3]

I manged to create a reproducer:
https://code.schlittermann.de/heiko/rsync-bug/src/commit/d8176333f8ef08b7dbf7957c83055677cddcc080/bug

Several times in a row if successfully reproduced the issue on
SLES12SP5, using Rsync 3.1.3 (connection to localhost, so identical
rsyncs on both sides of the connection)

If you can confirm that this is a bug that is fixed already, then it
would be enough for us for pushing the Distro maintainer to backport the
fix or whatever their policy allows.

==8<--
#!/bin/bash
set -eu

size=1029

cd ${1:?Usage $0 DIR}

echo -
echo rsync: $(rsync --version | grep version)
hostnamectl | grep 'Operating System' | sed 's/^\s*//'
echo --


echo Creating src file ${size}MiB
dd status=progress if=/dev/urandom of=fail-src bs=1M count=$size iflag=fullblock

echo Local copy to avoid initial transfer
cp -a fail-{src,dst}

echo Touch source
touch fail-src

echo Trigger the bug now
rsync -Pv -z fail-src localhost:$PWD/fail-dst
==8<--

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -


signature.asc
Description: PGP signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

deflate on token returned 0 - rsync error: error in protocol data stream

[Heiko Schlittermann via rsync]

Hello,

we're using rsync on SLES 12 SP 5 on both sides (for detailed version
info see below) and we're experiencing the following issue on the
sender's side:

```
 etc/test/windows2019_x86_64_20210929.gz
 deflate on token returned 0 (22199 bytes left)
 rsync error: error in rsync protocol data stream (code 12) at token.c(427) 
[sender=3.1.3]
```

This happens with 1 out of about 4'000'000 files.
Transfer w/o compression works without any issues.

Question: Bug or feature?


The options we're using are:

```
rsync',
--rsh'=> "ssh -q -i $arg{privatekey} -l $arg{user}",
--rsync-path' => "sudo $arg{user}",

# content
'--archive',# implies --links (-l), --recursive (-r)
'--hard-links',
(map { ('--exclude' => $_) } @{ $arg{exclude} }),
$arg{delete} eq 'yes' ? '--delete' : (),

# transfer
'--compress',
'--partial',
'--bwlimit' => $arg{bwlimit},

# logging
'--stats',
'--verbose',
'--log-file'=> $arg{rsynclogfile},
'--log-file-format' => 'Modified %i %n%L',
```


Detailed version information:
```
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc, SLP

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
```

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -


signature.asc
Description: PGP signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #14 from Roland Haberkorn  ---
After the new version made it into my system I can confirm it works like a
charm. Many thanks for the effort.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Skip file transfer on detecton of error code

[hancooper via rsync]

Occasionally rsync issues errors, which result in slowing a system during a 
transfer.
Would it be possible to skip the transfer of the file rather than having rsync 
continue to insist, trying the transfer many times?

rsync: read errors mapping 
"/media/hagbard/hc1/d5-comms/c-periodc/a-exposn/newsci/2011/n2809.pdf": 
Input/output error (5)-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Getting error code 12 when using rsync with ssh in RHEL 8

[Srijan Sivakumar via rsync]

Hi SMEs,

I have been working on a geo replication solution and it uses rsync
internally. Recently when I was running the utility in RHEL 8, I started
facing seeing the following message :


Popen: command returned error cmd=rsync -aR0 --inplace --files-from=-
--super --stats --numeric-ids --no-implied-dirs --existing --xattrs --acls
--ignore-missing-args . -e ssh -oPasswordAuthentication=no
-oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem
-p 22 -oControlMaster=auto -S
/tmp/gsyncd-aux-ssh-i1jbmu2r/89283d9f33e5cfb2abacb855f2e1242e.sock
10.70.41.231:/proc/5886/cwd error=12

On checking the logs, I find that I'm receiving the following error :

2020/10/04 21:28:15 [2861944] rsync: connection unexpectedly closed (0
bytes received so far) [sender]
2020/10/04 21:28:15 [2861944] rsync error: error in rsync protocol data
stream (code 12) at io.c(226) [sender=3.1.3]

Now I have tried all the steps such as
1. Checking if the disk space if full ( it wasn't)
2. Recreate the key value pair and setup the password-less ssh scenario.

One observation I had was when I disabled the selinux enforcing, it started
to work. But even if it had been enforced, I did add an exception for
rsync_client and rsync_full_access which should have taken care of the
rsync operations.

I've tried the remedies mentioned in the manpages and even the internet but
the actual root cause still eludes me as to why it doesn't work in selinux
enforced mode if I had set the access.

Thank you in advance to anyone taking their time to answer this query :)

-- 
Thanks and Regards,

SRIJAN SIVAKUMAR

Associate Software Engineer

Red Hat
<https://www.redhat.com>


<https://www.redhat.com>

T: +91-9727532362 <http://redhatemailsignature-marketing.itos.redhat.com/>

<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Copying TBs -> error -> work around

[Philip Rhoades via rsync]

raf,


On 2020-09-14 10:07, raf via rsync wrote:

On Fri, Sep 11, 2020 at 10:53:14AM +1000, Philip Rhoades via rsync
 wrote:


Roland,


On 2020-09-10 21:27, Roland wrote:
> > with rsync hanging - after breakout on /home for writing I then get:
> > "Read-only file system"
>
> if your filesystem switches to read-only, you have a serious problem
> with your system/storage, not with rsync.
>
> rsync (or the workload) is simply triggering the problem.


Thanks for the response . .

Hmm . . but the drive that goes read-only is being read FROM not TO . 
. it

is hard to see how that should be an issue?
The backstory is that a relatively recent internal 8TB Seagate 
Barracuda had
its 7.2TB sda5 (home) partition corrupted - which itself was 
suspicious but
not impossible of course - so I had to switch temporarily to an 
external USB
4TB drive (which was a backup drive and was already up-to-date) for 
/home.

So now this exercise is rsyncing back to a NEW internal 8TB Seagate
Barracuda (sda5 again) . .

If you are correct about rsync simply triggering an existing problem 
on the
4TB USB drive, would that problem going to be recognised by a fsck 
(ext4)?

I will check this out after I switch over to the new internal sda5 for
/home.

Thanks,
Phil.


file systems can be remounted read only when there are
too many errors. perhaps that applies to read errors as
well, not just write errors. check logs for i/o errors.
if it were i/o errors that caused the kernel to remount
the file system read only, it should have logged those
errors. and you should be able to use fsck with a usb
drive.



Hmm . . well I gave up trying to rsync the nearly whole 4TB at once and 
broke it down into individual dirs like I described in the OP but after 
that I did actually look at the 4TB USB "from" drive and there wasn't 
much wrong with it:


# fsck.ext4 /dev/sdc -y
e2fsck 1.45.3 (14-Jul-2019)
/dev/sdc contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 9860147 extent tree (at level 1) could be shorter.  Optimize? yes

Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/sdc: * FILE SYSTEM WAS MODIFIED *
/dev/sdc: 10583339/244195328 files (0.4% non-contiguous), 
806625360/976754645 blocks


So then, from the corrupted original /dev/sda5 I tried to create an 
image to a second, new 8TB drive with an ext4 partition on the whole 
drive:


# ddrescue /dev/sda5 -d -r3 /mnte/sda5_ddrescue.img 
/root/sda5_ddrescue.mapfile

GNU ddrescue 1.25
Press Ctrl-C to interrupt
 ipos:9253 MB, non-trimmed:0 B,  current rate:   0 
B/s
 opos:9253 MB, non-scraped:0 B,  average rate:  37768 
kB/s
non-tried:7849 GB,  bad-sector:0 B,error rate:   0 
B/s
  rescued:9253 MB,   bad areas:0,run time:  4m  
5s
pct rescued:0.11%, read errors:0,  remaining time: 14h 
47m
  time since last successful read: 
13s

Copying non-tried blocks... Pass 1 (forwards)
ddrescue: Write error: Read-only file system

So I am getting a FS changed to RO in _two different_ situations - I 
think there is some OS (or motherboard?) problem . . after I update from 
Fedora 31 to 32 on one of the new 8TB drives, I might go through the 
exercise again to see if the problem is still there . .


Thanks,

Phil.
--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  p...@pricom.com.au

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Copying TBs -> error -> work around

[raf via rsync]

On Fri, Sep 11, 2020 at 10:53:14AM +1000, Philip Rhoades via rsync 
 wrote:

> Roland,
> 
> 
> On 2020-09-10 21:27, Roland wrote:
> > > with rsync hanging - after breakout on /home for writing I then get:
> > > "Read-only file system"
> > 
> > if your filesystem switches to read-only, you have a serious problem
> > with your system/storage, not with rsync.
> > 
> > rsync (or the workload) is simply triggering the problem.
> 
> 
> Thanks for the response . .
> 
> Hmm . . but the drive that goes read-only is being read FROM not TO . . it
> is hard to see how that should be an issue?
> The backstory is that a relatively recent internal 8TB Seagate Barracuda had
> its 7.2TB sda5 (home) partition corrupted - which itself was suspicious but
> not impossible of course - so I had to switch temporarily to an external USB
> 4TB drive (which was a backup drive and was already up-to-date) for /home.
> So now this exercise is rsyncing back to a NEW internal 8TB Seagate
> Barracuda (sda5 again) . .
> 
> If you are correct about rsync simply triggering an existing problem on the
> 4TB USB drive, would that problem going to be recognised by a fsck (ext4)?
> I will check this out after I switch over to the new internal sda5 for
> /home.
> 
> Thanks,
> Phil.

file systems can be remounted read only when there are
too many errors. perhaps that applies to read errors as
well, not just write errors. check logs for i/o errors.
if it were i/o errors that caused the kernel to remount
the file system read only, it should have logged those
errors. and you should be able to use fsck with a usb
drive.

cheers,
raf

> > regards
> > roland
> > 
> > 
> > Am 10.09.20 um 07:30 schrieb Philip Rhoades via rsync:
> > > People,
> > > 
> > > When I did:
> > > 
> > >   rsync -av /home/ /mntb5/  # about 4TB
> > > 
> > > I got errors like:
> > > 
> > >   'rsync [sender] expand file_list pointer array to xxx bytes, "did
> > > move"'
> > > 
> > > with rsync hanging - after breakout on /home for writing I then get:
> > > 
> > >   "Read-only file system"
> > > 
> > > So after unmounting and remounting /home I did:
> > > 
> > >   cd /home
> > >   find /home/ -type d | sort > ./home_dirs_sorted.txt
> > > 
> > > delete first line "/home/" of ./home_dirs_sorted.txt then:
> > > 
> > >   while read dir ; do echo $dir ; rsync -lptgod "$dir" /mntb5/"$dir" ;
> > > done < ./home_dirs_sorted.txt
> > > 
> > > and:
> > > 
> > >   while read dir ; do echo $dir ; rsync -lptgoD "$dir"/\.[a-zA-Z0-9]*
> > > /mntb5/"$dir"/ ; done < ./home_dirs_sorted.txt
> > > 
> > > and finally with no problems:
> > > 
> > >   rsync -av --exclude-from=/usr/local/bin/nfb_caches.txt /home/
> > > /mntb5/
> > > 
> > > If there was a more sensible / efficient way of getting this done I
> > > would like to know about it!
> > > 
> > > Thanks,
> > > 
> > > Phil.
> > > 
> 
> -- 
> Philip Rhoades
> 
> PO Box 896
> Cowra  NSW  2794
> Australia
> E-mail:  p...@pricom.com.au
> 
> -- 
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: 
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Copying TBs -> error -> work around

[Philip Rhoades via rsync]

Roland,


On 2020-09-10 21:27, Roland wrote:

with rsync hanging - after breakout on /home for writing I then get:
"Read-only file system"


if your filesystem switches to read-only, you have a serious problem
with your system/storage, not with rsync.

rsync (or the workload) is simply triggering the problem.



Thanks for the response . .

Hmm . . but the drive that goes read-only is being read FROM not TO . . 
it is hard to see how that should be an issue?


The backstory is that a relatively recent internal 8TB Seagate Barracuda 
had its 7.2TB sda5 (home) partition corrupted - which itself was 
suspicious but not impossible of course - so I had to switch temporarily 
to an external USB 4TB drive (which was a backup drive and was already 
up-to-date) for /home.  So now this exercise is rsyncing back to a NEW 
internal 8TB Seagate Barracuda (sda5 again) . .


If you are correct about rsync simply triggering an existing problem on 
the 4TB USB drive, would that problem going to be recognised by a fsck 
(ext4)?  I will check this out after I switch over to the new internal 
sda5 for /home.


Thanks,

Phil.



regards
roland


Am 10.09.20 um 07:30 schrieb Philip Rhoades via rsync:

People,

When I did:

  rsync -av /home/ /mntb5/  # about 4TB

I got errors like:

  'rsync [sender] expand file_list pointer array to xxx bytes, "did
move"'

with rsync hanging - after breakout on /home for writing I then get:

  "Read-only file system"

So after unmounting and remounting /home I did:

  cd /home
  find /home/ -type d | sort > ./home_dirs_sorted.txt

delete first line "/home/" of ./home_dirs_sorted.txt then:

  while read dir ; do echo $dir ; rsync -lptgod "$dir" /mntb5/"$dir" ;
done < ./home_dirs_sorted.txt

and:

  while read dir ; do echo $dir ; rsync -lptgoD "$dir"/\.[a-zA-Z0-9]*
/mntb5/"$dir"/ ; done < ./home_dirs_sorted.txt

and finally with no problems:

  rsync -av --exclude-from=/usr/local/bin/nfb_caches.txt /home/ 
/mntb5/


If there was a more sensible / efficient way of getting this done I
would like to know about it!

Thanks,

Phil.



--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  p...@pricom.com.au

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Copying TBs -> error -> work around

[Roland via rsync]

>with rsync hanging - after breakout on /home for writing I then get:
>"Read-only file system"

if your filesystem switches to read-only, you have a serious problem
with your system/storage, not with rsync.

rsync (or the workload) is simply triggering the problem.

regards
roland


Am 10.09.20 um 07:30 schrieb Philip Rhoades via rsync:

People,

When I did:

  rsync -av /home/ /mntb5/  # about 4TB

I got errors like:

  'rsync [sender] expand file_list pointer array to xxx bytes, "did
move"'

with rsync hanging - after breakout on /home for writing I then get:

  "Read-only file system"

So after unmounting and remounting /home I did:

  cd /home
  find /home/ -type d | sort > ./home_dirs_sorted.txt

delete first line "/home/" of ./home_dirs_sorted.txt then:

  while read dir ; do echo $dir ; rsync -lptgod "$dir" /mntb5/"$dir" ;
done < ./home_dirs_sorted.txt

and:

  while read dir ; do echo $dir ; rsync -lptgoD "$dir"/\.[a-zA-Z0-9]*
/mntb5/"$dir"/ ; done < ./home_dirs_sorted.txt

and finally with no problems:

  rsync -av --exclude-from=/usr/local/bin/nfb_caches.txt /home/ /mntb5/

If there was a more sensible / efficient way of getting this done I
would like to know about it!

Thanks,

Phil.



--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Copying TBs -> error -> work around

[Philip Rhoades via rsync]

People,

When I did:

  rsync -av /home/ /mntb5/  # about 4TB

I got errors like:

  'rsync [sender] expand file_list pointer array to xxx bytes, "did 
move"'


with rsync hanging - after breakout on /home for writing I then get:

  "Read-only file system"

So after unmounting and remounting /home I did:

  cd /home
  find /home/ -type d | sort > ./home_dirs_sorted.txt

delete first line "/home/" of ./home_dirs_sorted.txt then:

  while read dir ; do echo $dir ; rsync -lptgod "$dir" /mntb5/"$dir" ; 
done < ./home_dirs_sorted.txt


and:

  while read dir ; do echo $dir ; rsync -lptgoD "$dir"/\.[a-zA-Z0-9]* 
/mntb5/"$dir"/ ; done < ./home_dirs_sorted.txt


and finally with no problems:

  rsync -av --exclude-from=/usr/local/bin/nfb_caches.txt /home/ /mntb5/

If there was a more sensible / efficient way of getting this done I 
would like to know about it!


Thanks,

Phil.

--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  p...@pricom.com.au

--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14463] rsync 3.2.2 server protocol error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14463

--- Comment #4 from Wayne Davison  ---
The pre-release patches aren't guaranteed to be backward compatible, and in
this case the bits that were used in a couple different patches actually
conflicted with each other. So, when --atimes was promoted it became impossible
for anything using the older --fileflags patch to work with a newer rsync (or
the older --crtimes option, which also had to change bits).

Your best bet is to either copy an extra rsync binary onto the non-upgraded
target system (it doesn't have to be installed) and use the --rsync-path option
to tell rsync where to run the newer version OR to copy an older rsync version
onto the upgraded system so that you can use that rsync-old-fileflags binary to
copy to a system with the older patch.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14463] rsync 3.2.2 server protocol error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14463

--- Comment #3 from bumkick...@yahoo.com ---

It's not a trivial exercise to upgrade the rsync version on the target system,
so it would be useful if there was some kind of "back patch" available

| suspect there might be others with similar issues ...

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14463] rsync 3.2.2 server protocol error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14463

--- Comment #2 from bumkick...@yahoo.com ---
(In reply to Wayne Davison from comment #1)

What should we do instead to keep the same functionality?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14463] rsync 3.2.2 server protocol error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14463

Wayne Davison  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |INVALID

--- Comment #1 from Wayne Davison  ---
Yup, the older --file-flags patch isn't compatible with the newer one.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14463] New: rsync 3.2.2 server protocol error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14463

Bug ID: 14463
   Summary: rsync 3.2.2 server protocol error
   Product: rsync
   Version: 3.2.0
  Hardware: All
OS: FreeBSD
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: wa...@opencoder.net
  Reporter: bumkick...@yahoo.com
QA Contact: rsync...@samba.org
  Target Milestone: ---

*** client:freebsd 8.4-STABLE  rsync 3.1.1

root 224 > /usr/local/bin/rsync --address=10.246.110.2 -zz -v --fileflags
--force-change --numeric-ids --force --super -8 toor@droplet02::grizzle/
/bkup/droplet02

rsync error: error in rsync protocol data stream (code 12) at io.c(226}
[Receiver=3.1.1]


*** server:freebsd 11.4-STABLErsync 3.2.2

rsync error: protocol incompatibility (code 2) at compat.c(723) [sender=3.2.2]


This started happening after I upgraded the server from rsync version
3.1.3 to 3.2.2

it works if "--fileflags" is removed

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14076] 3.13 - lib/sysacls.h:118:13: error: array type has incomplete element type 'struct acl'

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14076

Wayne Davison  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--- Comment #2 from Wayne Davison  ---
Rsync 3.2.x builds on Cygwin.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11654] Misleading error message

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=11654

Wayne Davison  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |WORKSFORME

--- Comment #2 from Wayne Davison  ---
I don't see that error in that scenario.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 10372] rsync 3.10 error in protocol data stream while rsync 3.0.9 runs through

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=10372

Wayne Davison  changed:

   What|Removed |Added

 CC||m...@gromco.com

--- Comment #15 from Wayne Davison  ---
*** Bug 10332 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 10579] --xattr internal abbrev error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=10579

Wayne Davison  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #2 from Wayne Davison  ---
There are 2 xattr fixes coming in 3.2.3 that have a good chance of solving
this.  If it's not fixed, feel free to reopen.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: rsync error

[Dave Gordon via rsync]

On 25/06/2020 13:18, Madhurananda Pahar via rsync wrote:
> Hello everybody, 
> 
>          I am having a funny problem while using rsync as a tool to
> back-up my files: 
> 
> [sender] expand file_list pointer array to 524288 bytes, did move.
> 
> I am just wondering if you had this issue before and if you know a way
> to solve this, please share with me. 
> 
> Many thanks,
> Dr. Madhurananda Pahar. 

That's just a debug message, letting the developers know when the file
list had to be expanded, and whether it had to be moved (by realloc(3))
as a result -- because that case introduces the particular hazard that
some other code might have kept a pointer to the old array. So this
message would be useful if one were debugging a random crash, to know
whether it only happened when the file list grew to a certain size, or
when it was relocated during expansion.

>   if (DEBUG_GTE(FLIST, 1) && flist->malloced != FLIST_START) {
>   rprintf(FCLIENT, "[%s] expand file_list pointer array to %s 
> bytes, did%s move\n",
>   who_am_i(),
>   big_num(sizeof flist->files[0] * flist->malloced),
>   (new_ptr == flist->files) ? " not" : "");
>   }
> 

It's not an error of any sort, and you're only seeing it because you've
enabled DEBUG_GTE. Is there some other problem that you're trying to debug?

.Dave.


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #13 from MulticoreNOP  ---
(In reply to Wayne Davison from comment #12)
Hi Wayne,

that is great news!

Could you shine some light on why there is such a limit in the first place?

Personally I think such an arbitrary limit is rather unexpected and many people
will have a long running rsync command (for me each try took ~6hrs until I
could reproduce the error) fail on them until they could even be aware that
there is such a limit.

In addition to that, I guess only a small fraction of those people will
probably find this parameter as a solution to their problem.

To me this is like 'cp -R a b' failing, if a/ contains more than 1337 files.
Cheers,

Mc NOP

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

Wayne Davison  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #12 from Wayne Davison  ---
I fixed the allocation args to be size_t values (and improved a bunch of
allocation error checking while I was at it).

I then added an option that lets you override this allocation sanity-check
value. The default is still 1G per allocation, but you can now specify a much
larger value (up to "--max-alloc=8192P-1").

If you want to make a larger value the default for your copies, export
RSYNC_MAX_ALLOC in the environment with the size value of your choice.

Committed for release in 3.2.2.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rsync error

[Madhurananda Pahar via rsync]

Hello everybody,

 I am having a funny problem while using rsync as a tool to back-up
my files:

[sender] expand file_list pointer array to 524288 bytes, did move.

I am just wondering if you had this issue before and if you know a way to
solve this, please share with me.

Many thanks,
Dr. Madhurananda Pahar.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #11 from MulticoreNOP  ---
I want to add that the original implementation also leads to the following
error:


ERROR: out of memory in flist_expand [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106)
[sender=3.1.2]

For me it hit that message for an
rsync --delete-before -H --moreStuff..
at around the 117 million files mark.

The patch fixed that problem for me.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #10 from MulticoreNOP  ---
(In reply to Simon Matter from comment #7)
#define MALLOC_MAX 0x1

is greater than uint32-MAX, therefore will overflow and result in an
unpredictable and unfriendly manner.

#define MALLOC_MAX 0xD09DC300
(~3,5GiB) leaves some space to detect a "just too big for this implementation"
and will fail gracefully.

Yet, the real culprit here is the use of "unsigned int" as opposed to size_t.

I therefore propose the attached patch that removes MALLOC_MAX in its entirety
and should allow arrays as big as available virtual memory can support.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

MulticoreNOP  changed:

   What|Removed |Added

 CC||multicore...@mailbox.org

--- Comment #9 from MulticoreNOP  ---
Created attachment 16074
  --> https://bugzilla.samba.org/attachment.cgi?id=16074=edit
remove MALLOC_MAX from util2.c

change data types from "unsigned int" to "size_t" as it should be and remove
arbitrary size limit.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 10579] --xattr internal abbrev error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=10579

Haravikk  changed:

   What|Removed |Added

Version|3.1.0   |3.1.3

--- Comment #1 from Haravikk  ---
Just wanted to note that this issue can still occur, and I'm still at a bit of
a loss as to what is causing it; it's being reported as a protocol error, but
it can happen even when performing a transfer locally (no possibility of a
version mismatch).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12838] [PATCH] Log sent/received bytes even in case of error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12838

Ben RUBSON  changed:

   What|Removed |Added

 Resolution|--- |MOVED
 Status|NEW |RESOLVED

--- Comment #2 from Ben RUBSON  ---
Patch moved : https://github.com/WayneD/rsync/pull/5

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12522] [PATCH] Send last error messages to sender

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12522

Ben RUBSON  changed:

   What|Removed |Added

 Resolution|--- |MOVED
 Status|NEW |RESOLVED

--- Comment #4 from Ben RUBSON  ---
Patch moved : https://github.com/WayneD/rsync/pull/2

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #8 from Roland Haberkorn  ---
Is it possible to totally get rid of the restriction? I'd prefer running in out
of memory situations rather than in this restriction. Without I could just
throw some more RAM on the machine, with this restriction I would have to
rebuild rsync whenever there is a newer version.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #7 from Simon Matter  ---
I've patches like this to solve our issues:

--- rsync-3.1.3/util2.c.orig2018-01-15 04:55:07.0 +0100
+++ rsync-3.1.3/util2.c2020-03-11 13:07:07.138141415 +0100
@@ -59,7 +59,7 @@
 return True;
 }

-#define MALLOC_MAX 0x4000
+#define MALLOC_MAX 0x1

 void *_new_array(unsigned long num, unsigned int size, int use_calloc)
 {

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14214] New: --report-special=OPS option to return error code if no data was copied

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14214

Bug ID: 14214
   Summary: --report-special=OPS option to return error code if no
data was copied
   Product: rsync
   Version: 3.1.3
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P5
 Component: core
  Assignee: wa...@opencoder.net
  Reporter: sa...@haravikk.com
QA Contact: rsync...@samba.org

I'm unsure of the best name for the option, but it would take the form:

--report-special=OPS

Where OPS specifies one or more comma-separated special conditions that will
cause rsync to return a reserved non-zero status indicating that one or more of
these occurred. These do not actually indicate errors, and will not interrupt
rsync (indeed, they require it to finish the transfer first), but may be useful
when scripting around rsync to know whether other code should continue or not,
or whether your operation was successful or not.

Options include:

* nodata: Indicates that although rsync ran successfully, no file data was
transferred (this does not mean that no files or directories were created,
metadata modified etc.)
* nonew: Indicates that although rsync ran successfully, no new files were
created/linked (i.e- if any transfer occurred, it was for existing files only)
* nodelete: Indicates that although rsync ran successfully, no files were
removed
* nometa: Indicates that although rsync ran successfully, no metadata changed
(timestamps, permissions, ownership, extended attributes etc.)
* nochange: Shorthand for nodata,nonew,nodelete,nometa effectively indicating
that although rsync ran successfully, no changes occurred at all


There may be others that might be worth adding, but these are the two main ones
IMO. I'm uncertain whether rsync should give each a unique status code, or if
there should just be one status code for any special condition being met.

The idea here is to give us new options when scripting, and avoid the need to
parse rsync's output/log to get this information.

For example, let's say I'm using rsync to copy data onto a file-system that
supports snapshots, I could use the following command:

rsync --report-special=nochange /path/to/source /path/to/target

If I get the reserved status code back it means that no changes were found
between /path/to/source and /path/to/target, in which case there is no need for
me to create a new snapshot, as there would be nothing new to rollback to (or
backup further).


Currently the alternative is to either grab the output of --stats and parse it
for created/deleted files and literal data transferred, or use
--itemize-changes and test the number of lines of output (none means no
change), depending upon what else you may need the output for (logs, printing
to terminal etc.). Neither of these is ideal, and even if they were I think it
would still be nice to have an authoritative response from rsync itself, based
upon what the user/script needs to know.

This information should already be available as it's the same basic data
reported by --stats (though --stats does not appear to report metadata/xattr
only changes, unless they show up as data).

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #6 from Dave Gordon  ---
The hash table doubles each time it reaches 75% full. A hash table for 32m
items @ 16 bytes each (8 byte key, 8 byte void *data) needs 512MB of memory. At
the next doubling (up to 64m items) it hits the array allocator limit in
utils2.c:

#define MALLOC_MAX 0x4000

void *_new_array(unsigned long num, unsigned int size, int use_calloc)
{
if (num >= MALLOC_MAX/size)
return NULL;
return use_calloc ? calloc(num, size) : malloc(num * size);
}
void *_realloc_array(void *ptr, unsigned int size, size_t num)
{
if (num >= MALLOC_MAX/size)
return NULL;
if (!ptr)
return malloc(size * num);
return realloc(ptr, size * num);
}

No single array allocation or reallocation is allowed to exceed MALLOC_MAX
(1GB). Hence rsync can only handle up to 32m items per invocation if a hash
table is required (e.g. for tracking hardlinks when -H is specified).

HTH,
Dave

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[Dave Gordon via rsync]

On 11/10/2019 13:53, just subscribed for rsync-qa from bugzilla via
rsync wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=12769
>
> --- Comment #5 from Simon Matter  ---
> I'm suffering the same problem and was wondering if anyone found a solution or
> work around or other tool to do the job?
>
> First I thought maybe it's a bug which is fixed already and tried with the
> latest release 3.1.3. Unfortunately no joy and I still get the same issue.
>
> Any help would be much appreciated!
The hash table doubles each time it reaches 75% full.
A hash table for 32m items @ 16 bytes each (8 byte key, 8 byte void *
data) needs 512MB of memory.
At the next doubling (up to 64m items) it hits the array allocator limit
in utils2.c:

#define MALLOC_MAX 0x4000

void *_new_array(unsigned long num, unsigned int size, int use_calloc)
{
    if (num >= MALLOC_MAX/size)
    return NULL;
    return use_calloc ? calloc(num, size) : malloc(num * size);
}
void *_realloc_array(void *ptr, unsigned int size, size_t num)
{
    if (num >= MALLOC_MAX/size)
    return NULL;
    if (!ptr)
    return malloc(size * num);
    return realloc(ptr, size * num);
}

No single array allocation or reallocation is allowed to exceed
MALLOC_MAX (1GB).
Hence rsync can only handle up to 32m items per invocation if a hash
table is required (e.g. for tracking hardlinks when -H is specified).

HTH,
Dave



-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #5 from Simon Matter  ---
I'm suffering the same problem and was wondering if anyone found a solution or
work around or other tool to do the job?

First I thought maybe it's a bug which is fixed already and tried with the
latest release 3.1.3. Unfortunately no joy and I still get the same issue.

Any help would be much appreciated!

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #4 from Ovidiu Stanila  ---
We hit the same issue on a CentOS 6 server (kernel 2.6.32-754.18.2.el6.x86_64),
the sync would break with the following error:

# /usr/bin/rsync --debug=HASH --stats --no-inc-recursive -aHn --delete /app/
:/app/
[sender] created hashtable 2013770 (size: 16, keys: 64-bit)
[sender] created hashtable 2015370 (size: 512, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 1024, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 2048, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 4096, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 8192, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 16384, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 32768, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 65536, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 131072, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 262144, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 524288, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 1048576, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 2097152, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 4194304, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 8388608, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 16777216, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 33554432, keys: 64-bit)
ERROR: out of memory in hashtable_node [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106)
[sender=3.1.2]

Both the sender and receiver part have the same rsync and OS versions.

The source we try to transfer is over 3.4Tb (65 million files) with a big
number of hard links between various directories(for avoiding duplicate files).

We initially increased the memory from 16Gb to 32Gb but even with that rsync
would die with the same error. On above example, there were over 22Gb free RAM
at the point rsync stopped working.

We also tried "--inplace" instead of "--no-inc-recursive" with the same result.

Did we hit some kind of limitation inside of rsync ? Is there anything else we
should check ?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14076] 3.13 - lib/sysacls.h:118:13: error: array type has incomplete element type 'struct acl'

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14076

--- Comment #1 from Jari Aalto  ---
Created attachment 15375
  --> https://bugzilla.samba.org/attachment.cgi?id=15375=edit
Cygwin x64 config.status

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 14076] New: 3.13 - lib/sysacls.h:118:13: error: array type has incomplete element type 'struct acl'

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=14076

Bug ID: 14076
   Summary: 3.13 - lib/sysacls.h:118:13: error: array type has
incomplete element type 'struct acl'
   Product: rsync
   Version: 3.1.3
  Hardware: x64
OS: All
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: wa...@opencoder.net
  Reporter: jari.aa...@cante.net
QA Contact: rsync...@samba.org

Created attachment 15374
  --> https://bugzilla.samba.org/attachment.cgi?id=15374=edit
Build Log Cygwin x64

Release 3.13 fails to build on Cygwin. Log attached

gcc -I. -I. -I./zlib -I./popt -static -Wall -Wno-format -Wstrict-prototypes
-Wmissing-prototypes -DGETTEXT_STATIC -DHAV\
E_CONFIG_H -Wall -W  -c acls.c -o acls.o
In file included from acls.c:23:0:
lib/sysacls.h:118:13: error: array type has incomplete element type 'struct
acl'
  struct acl acl[1];
 ^~~
acls.c: In function 'unpack_smb_acl':
lib/sysacls.h:108:26: error: 'USER_OBJ' undeclared (first use in this
function); did you mean 'ACL_USER_OBJ'?
 #define SMB_ACL_USER_OBJ USER_OBJ
  ^
acls.c:286:8: note: in expansion of macro 'SMB_ACL_USER_OBJ'
   case SMB_ACL_USER_OBJ:
^~~~
lib/sysacls.h:108:26: note: each undeclared identifier is reported only once
for each function it appears in
 #define SMB_ACL_USER_OBJ USER_OBJ
  ^
acls.c:286:8: note: in expansion of macro 'SMB_ACL_USER_OBJ'
   case SMB_ACL_USER_OBJ:
^~~~
lib/sysacls.h:110:27: error: 'GROUP_OBJ' undeclared (first use in this
function); did you mean 'ACL_GROUP_OBJ'?
 #define SMB_ACL_GROUP_OBJ GROUP_OBJ
   ^
acls.c:292:8: note: in expansion of macro 'SMB_ACL_GROUP_OBJ'
   case SMB_ACL_GROUP_OBJ:
^
lib/sysacls.h:112:23: error: 'CLASS_OBJ' undeclared (first use in this
function)
 #define SMB_ACL_MASK  CLASS_OBJ
(...)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Multiple errors from rsync - later 'soft' error raised rather than earlier 'hard' error.

[Bell, Robert (IM, Clayton) via rsync]

Peter,

I was about to post the below to the rsync mailing list, but wondered whether 
your code which captures rsync output may have allowed rsync to continue past 
what I think would normally be a fatal error?

Thanks

Rob.

P.S. Should I put the final backups of the old pearcey /home back into their 
correct slots - 5 backups were recycled, but nothing was added to them?  The 
.Snapshots area is pointing to the old locations.
Here are the recycles that need to be reversed:
Recycling /backup/pearcey-user-dumps/home_intel.20190606.seq.1687.recycle to 
/backup/pearcey-user-dumps/home_intel.20190611.seq.1692
Recycling /backup/pearcey-user-dumps/home_intel.20190608.seq.1689.recycle to 
/backup/pearcey-user-dumps/home_intel.20190612.seq.1694
Recycling /backup/pearcey-user-dumps/home_intel.20190605.seq.1686.recycle to 
/backup/pearcey-user-dumps/home_intel.20190612.seq.1693
Recycling /backup/pearcey-user-dumps/home_intel.20190609.seq.1690.recycle to 
/backup/pearcey-user-dumps/home_intel.20190613.seq.1696
Recycling /backup/pearcey-user-dumps/home_intel.20190603.seq.1684.recycle to 
/backup/pearcey-user-dumps/home_intel.20190613.seq.1695


Rsync Folks,

We recently encountered a case with rsync hitting what I regard as a 'hard' 
error (unable to change_dir to the source), but it did not immediately abort, 
but continued and went on to hit other errors (which I regard as soft) and 
returned the error code 23.  Our backup suite continues after error 23, because 
it is common and usually not a problem.
Here is selected output.

  rsync: change_dir "/home_intel" (in backups) failed: No such file or 
directory (2)
  rsync error: some files/attrs were not transferred (see previous 
errors) (code 23) at main.c(1637) [Receiver=3.1.1+csiro-sc.4]
  rsync: read error: Connection reset by peer (104)

Should rsync have aborted after encountering error 2?
 
Regards
 
Rob.
 
Dr Robert C. Bell
Retirement Fellow
CSIRO Information Management and Technology
Scientific Computing
Mobile +61 428 108 333
robert.b...@csiro.au<mailto:robert.b...@csiro.au> | www.csiro.au 
https://confluence.csiro.au/display/SC/Scientific+Computing+Homepage
 
T +61 3 9545 2368 (Clayton) 
Street: CSIRO IMT, Rivett Bldg 001 room RNW 2.12, Clayton Vic 3168, Australia
Office: w3w: http://w3w.co/depend.jumpy.faster
Postal: CSIRO IMT, Private Bag 10, Clayton South Vic 3169, Australia
Parcels: CSIRO IMT, Gate 3 Normanby Road, Clayton Vic 3168, Australia
 
PLEASE NOTE
 
The information contained in this email may be confidential or
privileged.  Any unauthorised use or disclosure is prohibited.  If you
have received this email in error, please delete it immediately and
notify the sender by return email. Thank you.  To the extent permitted
by law, CSIRO does not represent, warrant and/or guarantee that the
integrity of this communication has been maintained or that the
communication is free of errors, virus, interception or interference.  
 
Please consider the environment before printing this email.
 
 

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 13953] New: error message instead of --stats informations (in case of vanished files), using rsync 3.1.1 and rsync 3.0.9

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=13953

Bug ID: 13953
   Summary: error message instead of --stats informations (in case
of vanished files), using rsync 3.1.1 and rsync 3.0.9
   Product: rsync
   Version: 3.1.1
  Hardware: x64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: wa...@opencoder.net
  Reporter: g...@4js.com
QA Contact: rsync...@samba.org

Hello,

The issue I'm reporting is about the stats informations usually shown (by
--stats).

This stats information is missing in case of vanished files. I got an error
about protocol data stream instead.
Probably an issue between the both version/protocol.

It's appened since I have updated the rsync version on FOO (from 3.0.9 to
3.1.1).
Now the rsync version between "client"/"server" are not the same.
In my case, the issue only appears in case of vanished files.

1) before (no issue)

on FOO rsync 3.0.9 (debian 7)
on BAR rsync 3.0.9 (RHEL 7)

2) after (got the issue)

on FOO rsync 3.1.1 (debian 8)
on BAR rsync 3.0.9 (RHEL 7)

I got this error instead of the stats informations :
> rsync: writefd_unbuffered failed to write 162 bytes to socket [sender]: 
> Broken pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(1532) 
> [sender=3.0.9]
> rsync: [generator] write error: Broken pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(820) 
> [generator=3.1.1]


On my backup serveur named FOO trying to get data from a source target named
BAR

The minimal command seems like :
rsync [...] --stats [...] BAR:/path/to/dir/ ./

if you are curious the real used command is :
rsync \
--hard-links --numeric-ids --human-readable --prune-empty-dirs
--archive --one-file-system --force \
--stats \
--compare-dest=/path/to/dir1 --delete
--exclude-from=/path/to/file1.exclude --rsh=ssh \
BAR:/path/to/dir/ ./




Additionnal informations


On 5/9/19 3:10 AM, root wrote:> Thu May  9 01:48:03 CEST 2019
> 
> Number of files: 1565540
> Number of files transferred: 591395
> Total file size: 1181.13G bytes
> Total transferred file size: 136.68G bytes
> Literal data: 129.42G bytes
> Matched data: 7.33G bytes
> File list size: 48.55M
> File list generation time: 9.667 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 37.16M
> Total bytes received: 129.52G
> 
> sent 37.16M bytes  received 129.52G bytes  26.40M bytes/sec
> total size is 1181.13G  speedup is 9.12
> rsync warning: some files vanished before they could be transferred (code 24) 
> at main.c(1536) [generator=3.0.9]
> 
> # there are some file vanished :
> file has vanished: 
> "/path/to/files/xx"
> file has vanished: 
> "/path/to/files/"
> file has vanished: 
> "/path/to/files/"
> file has vanished: 
> "/path/to/files/x"[...]


On 5/15/19 3:20 AM, root wrote:
> Wed May 15 02:31:00 CEST 2019
> 
> rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken 
> pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(1532) 
> [sender=3.0.9]
> rsync: [generator] write error: Broken pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(820) 
> [generator=3.1.1]
> 
> # there are some file vanished :
> file has vanished: 
> "/path/to/files/"
> file has vanished: 
> "/path/to/files/xx"
> file has vanished: 
> "/path/to/files/"
> file has vanished: 
> "/path/to/files/x"
...


On 5/16/19 3:14 AM, root wrote:
> Thu May 16 02:03:05 CEST 2019
>
> rsync: writefd_unbuffered failed to write 162 bytes to socket [sender]: 
> Broken pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(1532) 
> [sender=3.0.9]
> rsync: [generator] write error: Broken pipe (32)
> rsync error: error in rsync protocol data stream (code 12) at io.c(820) 
> [generator=3.1.1]
>
> # there are some file vanished :
> file has vanished: 
> "/path/to/files/"
> file has vanished: 
> "/path/to/files/x

[Bug 13656] New: --link-dest target with symbolic links from different user produces unnecessary error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=13656

Bug ID: 13656
   Summary: --link-dest target with symbolic links from different
user produces unnecessary error
   Product: rsync
   Version: 3.1.3
  Hardware: All
OS: Linux
Status: NEW
  Severity: minor
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: nathan.g...@localsearch.ch
QA Contact: rsync...@samba.org

Created attachment 14531
  --> https://bugzilla.samba.org/attachment.cgi?id=14531=edit
reproducing script

I'm using rsync in a tool which atomically distributes directories by using
symbolic links and atomically overwriting them. to still avoid unnecessary
extra copies while generating a new directory I use --link-dest.

If I have a symbolic link belonging to a different user in the --link-dest
directory, linux does not allow a hardlink on that file. rsync outputs that
error and has a return code 23. The resulting copy though is correct, but with
the symbolic link copied instead of hardlinked. The only difference between the
call with --link-dest and without is the owner of the file, which has to be
from the --link-dest directory if provided but is the current user without
--link-dest. As I did not explicitly ask to copy owners this are both imho
correct results and should therefore not result in a 23 return code.

The attached script reproduces the problem (as written requires root, as we
need a symbolic link from a different user). The output I see on my machine is:

$ sh rsynctest.sh 
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc, SLP

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
Password: 
rsync: failed to hard-link ../src/link with link: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1189) [sender=3.1.3]
23


I would have expected at least a different error code, as the directory got
copied correctly. Alternatively ignoring that error by default or having an
option to ignore it would of course be even more convenient. If the --link-dest
option is removed from the script, the rsync copies the directory without error
and return code is 0.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12522] [PATCH] Send last error messages to sender

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12522

Ben RUBSON <ben.rub...@gmail.com> changed:

   What|Removed |Added

  Attachment #12841|0   |1
is obsolete||

--- Comment #3 from Ben RUBSON <ben.rub...@gmail.com> ---
Created attachment 13959
  --> https://bugzilla.samba.org/attachment.cgi?id=13959=edit
Send last error messages to sender

Modified patch to seamlessly suit rsync 3.1.3.
Thx !

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11215] compression/zlib errors discard the zlib error message

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=11215

--- Comment #2 from George  ---
Anyone experiencing a similar issue may want to have a look at bug 10372 .
( Possibly related: bug 10332. )

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 1442] rsync sender appears to hang when receiver recounters an error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=1442

--- Comment #13 from devuran...@gmx.net ---
There are a lot of bugreports related to rsync hanging mysteriously, some of
which may be duplicates of each other:

https://bugzilla.samba.org/show_bug.cgi?id=1442
https://bugzilla.samba.org/show_bug.cgi?id=2957
https://bugzilla.samba.org/show_bug.cgi?id=9164
https://bugzilla.samba.org/show_bug.cgi?id=10035
https://bugzilla.samba.org/show_bug.cgi?id=10092
https://bugzilla.samba.org/show_bug.cgi?id=10518
https://bugzilla.samba.org/show_bug.cgi?id=10950
https://bugzilla.samba.org/show_bug.cgi?id=11166
https://bugzilla.samba.org/show_bug.cgi?id=12732
https://bugzilla.samba.org/show_bug.cgi?id=13109

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 10719] Error with cached effective process gid

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=10719

Wayne Davison  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #2 from Wayne Davison  ---
I've just committed a fix for this to git.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 10719] Error with cached effective process gid

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=10719

--- Comment #1 from lonerr  ---
Three years later, any updates? :)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

rename error on temporary destination file

[Vangelis Katsikaros via rsync]

Hi

I am seeing from time to time the following error:

rsync: rename "/DEST_DIR/SUB_DIR/.A_FILE.AzmlvG" -> 
"SUB_DIR/.tmp/A_FILE.xml.gz": No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 
23) at main.c(1183) 

I rsync from a few source machines to a single destination machine, with:

/usr/local/bin/rsync-no-vanished \
--bwlimit=512k --recursive --delay-updates --quiet --update \
--include=\"*/\" --include=\"*.gz\" --exclude=\"*\" \
/SOURCE_DIR/ DEST_IP:/DEST_DIR/

The filenames on the 4 source machines are distinct, but they do contain sub 
directories with the same names. 

Could it be that while machine A syncs SUB_DIR/fileA, machine B finishes 
syncing SUB_DIR/fileB and cleans the temporary directory on the destination 
machine? If so, is there a way to avoid it?

Both sources and destination have: rsync  version 3.1.0  protocol version 31

rsync-no-vanished is the one provided by rsync's distribution.

Regards
Vangelis

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: hard-link reference error when trying to delete

[Kevin Korb via rsync]

You should use rrsync for that.

On 08/15/2017 08:58 PM, Jared via rsync wrote:
> Hi, Kevin.  Thank you for the suggestion.  It triggered a memory that I
> had set some restrictions on this rsync copy a while back.  Sure enough,
> in ~/.ssh/authorized_keys:
> 
> command="rsync --server -vulogDtpre.iLsfxC --timeout=600 --bwlimit=5120
> . dest" ssh-rsa
> 
> Tacking on --delete in the appropriate spot fixed the issue.
> 
> That's a big whoops on my part.  :-)  Appreciate the pointer.
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb  Phone:(407) 252-6853
Systems Administrator   Internet:
FutureQuest, Inc.   ke...@futurequest.net  (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page:   http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,



signature.asc
Description: OpenPGP digital signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: hard-link reference error when trying to delete

[Jared via rsync]

Hi, Kevin.  Thank you for the suggestion.  It triggered a memory that I
had set some restrictions on this rsync copy a while back.  Sure enough,
in ~/.ssh/authorized_keys:

command="rsync --server -vulogDtpre.iLsfxC --timeout=600 --bwlimit=5120
. dest" ssh-rsa

Tacking on --delete in the appropriate spot fixed the issue.

That's a big whoops on my part.  :-)  Appreciate the pointer.

-- 
Jared

On 08/15/2017 07:20 PM, Kevin Korb via rsync wrote:
> Have you tried doing the clean shell test?
> 
> ssh -i /path/to/key u...@remote.server.com true > testfile
> if testfile isn't 0 bytes then whatever is producing the data that ended
> up in there is your problem.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: hard-link reference error when trying to delete

[Kevin Korb via rsync]

Have you tried doing the clean shell test?

ssh -i /path/to/key u...@remote.server.com true > testfile
if testfile isn't 0 bytes then whatever is producing the data that ended
up in there is your problem.


On 08/15/2017 04:13 PM, jared via rsync wrote:
> Hello, all.  I've encountered an odd error that I haven't been able to
> identify and resolve myself.  Hoping someone here could provide some
> guidance.
> 
> I'm using the following command to rsync files between two servers, over
> the internet (if that matters), via SSH:
> 
> rsync -ahu --timeout=600 --bwlimit=5120 -v -e "ssh -i /path/to/key"
> --filter=". /path/to/filter" /path/to/source/ u...@remote.server.com:dest
> 
> This works perfectly fine.  However, I recently realized that I forgot
> to the --delete option, so there are a great number of stale files on
> the remote server at this point.
> 
> Adding --delete is when I run into my problem:
> 
> sending incremental file list
> hard-link reference out of range: 105 (10)
> rsync error: protocol incompatibility (code 2) at flist.c(769)
> [Receiver=3.1.2]
> 
> The source and destination servers are running the same versions of
> rsync, ssh, etc., so I think that last line can be ignored.  The
> "hard-link reference out of range" line seems to be the main issue, but
> I cannot figure out what would be causing this.  I enabled various debug
> options in both rsync and ssh, but I can't find anything in the debug
> output that would indicate the problem.  I tried searching, but the only
> result I've found outside of source code
> (https://github.com/xonsh/xonsh/issues/2134) is for something that seems
> unrelated - it's a python project that calls rsync, apparently
> incorrectly, resulting in that error.  In my case I'm calling rsync
> directly from bash, so the given solution in that bug report doesn't
> seem to apply here.
> 
> Any suggestions?  I'm stumped, and I really need to figure out how to
> clean up the stale files on the remote server as it's running low on
> space  I'm happy to provide any additional information regarding debug
> logs or whatever if it would help, just didn't want to spam the list
> with a bunch of potentially irrelevant information.
> 
> Thanks.  Appreciate any and all guidance.
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb  Phone:(407) 252-6853
Systems Administrator   Internet:
FutureQuest, Inc.   ke...@futurequest.net  (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page:   http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,



signature.asc
Description: OpenPGP digital signature
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

hard-link reference error when trying to delete

[jared via rsync]

Hello, all.  I've encountered an odd error that I haven't been able to
identify and resolve myself.  Hoping someone here could provide some
guidance.

I'm using the following command to rsync files between two servers, over
the internet (if that matters), via SSH:

rsync -ahu --timeout=600 --bwlimit=5120 -v -e "ssh -i /path/to/key"
--filter=". /path/to/filter" /path/to/source/ u...@remote.server.com:dest

This works perfectly fine.  However, I recently realized that I forgot
to the --delete option, so there are a great number of stale files on
the remote server at this point.

Adding --delete is when I run into my problem:

sending incremental file list
hard-link reference out of range: 105 (10)
rsync error: protocol incompatibility (code 2) at flist.c(769)
[Receiver=3.1.2]

The source and destination servers are running the same versions of
rsync, ssh, etc., so I think that last line can be ignored.  The
"hard-link reference out of range" line seems to be the main issue, but
I cannot figure out what would be causing this.  I enabled various debug
options in both rsync and ssh, but I can't find anything in the debug
output that would indicate the problem.  I tried searching, but the only
result I've found outside of source code
(https://github.com/xonsh/xonsh/issues/2134) is for something that seems
unrelated - it's a python project that calls rsync, apparently
incorrectly, resulting in that error.  In my case I'm calling rsync
directly from bash, so the given solution in that bug report doesn't
seem to apply here.

Any suggestions?  I'm stumped, and I really need to figure out how to
clean up the stale files on the remote server as it's running low on
space  I'm happy to provide any additional information regarding debug
logs or whatever if it would help, just didn't want to spam the list
with a bunch of potentially irrelevant information.

Thanks.  Appreciate any and all guidance.

-- 
Jared

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #3 from Roland Haberkorn <mss...@fau.de> ---
Ok, I digged somewhat deeper. I've found a second difference between my two
sources. The one is the original data, the other one is a diffential rsync
backup with hard links.
I then built a testcase with about 50 million dummy files with something like
this:

#!/bin/bash
for i in {1..50}
do
mkdir $i
#cd $i
for j in {1..1000}
do
mkdir $i/$j
#cd $j
for k in {1..1000}
do
touch $i/$j/$k
done
done
done

Rsyncing this testfolder work fine from and to any of the tested file systems
(ext4 64Bit, XFS, btrfs). This is true for with and without the Option -H and
as long as in the source file system there is no hard linked copy of the source
folder. 
In the moment when there is at least one hard linked copy the option -H breaks
the run:

roland@msspc25:~$ stat /mnt/rsynctestsource/1/1/1
  File: /mnt/rsynctestsource/1/1/1
  Size: 0   Blocks: 0  IO Block: 4096   regular empty file
Device: 811h/2065d  Inode: 153991349   Links: 2
Access: (0644/-rw-r--r--)  Uid: ( 2001/  roland)   Gid: (  100/   users)
Access: 2017-05-09 10:54:10.341300841 +0200
Modify: 2017-05-08 09:33:51.535967423 +0200
Change: 2017-05-26 16:21:57.610628573 +0200
 Birth: -
roland@msspc25:~$ rsync -rlptgoDxAn --info=name,progress2  --delete
--link-dest=/mnt2/link3/ /mnt/rsynctestsource/ /mnt2/link1/.
  0 100%0.00kB/s0:00:00 (xfr#0, to-chk=0/49049050)   
roland@msspc25:~$ rsync -rlptgoDxHAn --info=name,progress2  --delete
--link-dest=/mnt2/link3/ /mnt/rsynctestsource/ /mnt2/link1/.
  0 100%0.00kB/s0:00:00 (xfr#0, ir-chk=1000/25191050)
ERROR: out of memory in hashtable_node [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106)
[sender=3.1.2]

You can see, the first run without -H works, the last one with -H doesn't. 

So I would have to somewhat rename the bug report into "-H breaks the
incremental recursion on hard linked sources". This is as well true for all the
three file systems tested.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12838] [PATCH] Log sent/received bytes even in case of error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12838

--- Comment #1 from Ben RUBSON  ---
Mmmmh unfortunately numbers are not correct.

# Sender side (which interrupts the transfer receiving SIGUSR2) :
Total file size: 16­.85G bytes 
Total transferred file size: 183­.38M bytes 
Literal data: 182­.32M bytes 

# Receiver side : 
rsyncd: rsync: connection unexpectedly closed (202529 bytes received so far)
[generator]
rsyncd: sent 24749 bytes  received 202529 bytes  total size 16845198594

Perhaps related to bug #10738 where stats are also not correctly calculated.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12838] New: [PATCH] Log sent/received bytes even in case of error

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12838

Bug ID: 12838
   Summary: [PATCH] Log sent/received bytes even in case of error
   Product: rsync
   Version: 3.1.2
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: ben.rub...@gmail.com
QA Contact: rsync...@samba.org

Created attachment 13271
  --> https://bugzilla.samba.org/attachment.cgi?id=13271=edit
rsync_log_daemon

Hello,

This patch allows the daemon to log data sent/received even in case of error,
which can be interesting to have when the transfer has been interrupted by the
client.

Thank you !

Ben

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #2 from Roland Haberkorn  ---
I did some further investigation... 
First thing to add: The ext4 file systems are hard-linked differential rsync
backups of the real data on XFS.
I changed the testcase by deleting the --link-dest option.
When rsyncing from an XFS, the rsync process on the client uses about 3% RAM
(of total 8GB). When rsyncing from an ext4, it uses up to about 50% RAM.
This picture totally changes when I delete the option -H. In this case also
copying from an ext4 uses only less than 2% RAM. 
My guess would be that perhaps -H breaks the incremental recursion when copying
from an ext4.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #1 from Roland Haberkorn  ---
If you want me to run further testings with other file systems I am totally
willing to produce fake data and run tests. I just haven't done yet because of
my lack of knowledge about the underlying mechanisms and because I am not
totally sure whether this is a rsync's problem or a kernel issue.
To add two more things: We saw this issue also when having mounted the data
with NFSv3 or v4. The target file system does not matter, we've had this issue
with btrfs, Ext4 and XFS.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12769] New: error allocating core memory buffers (code 22) depending on source file system

[just subscribed for rsync-qa from bugzilla via rsync]

https://bugzilla.samba.org/show_bug.cgi?id=12769

Bug ID: 12769
   Summary: error allocating core memory buffers (code 22)
depending on source file system
   Product: rsync
   Version: 3.1.0
  Hardware: All
OS: Linux
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: mss...@fau.de
QA Contact: rsync...@samba.org

We run an openSuSE Leap 42.2 and an Ubuntu 14.04.5 on two servers. Copying a
large number of files (in this case about 28 million) leads to different
results depending on the source file system. 
We copy with rsync -rlptgoDAxHnP --info=progress2 --delete
--link-dest=$LINK_DEST root@$SERVER:/$FOLDER /backup/rsynctest/ . Replacing
--delete by --delete-delay doesn't change the behaviour as expected. The error
occurs with and without the option -n, in this case it is just for testing
reasons included.
In case the source is located on an Ext4 file system we run into the following
error message after about 26 million files copied:
ERROR: out of memory in hashtable_node [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(102)
[sender=3.1.0]
In case the source is located on an XFS file system the above command copies
all files without error.
Both of the file systems hold the same data as the one is the backup copy of
the other. The behaviour appears as well when we use rsync via an rsync server
and not via SSH and as well when we copy locally on one of the two machines.
And it appears regardless of the operating system (openSuSE 42.2 or Ubuntu
14.04.5). 
I did not try in the last time but replaying the backup from an Ext4 showed
this error at least one year ago as well. With the change to XFS on the source
file system the error suddenly disappeared.
As the error appears even if just doing a --dry-run it seems to be related to
the way rsync handles metadata. The data size seems to be unimportant.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]

[John Hawkinson via rsync]

> But at first blush, it appeared that adding - made things hang
> forever.

Yes. Confirmed against the git HEAD (9e7b8ab7cf66ecd152002926a7da61d8ad862522).

Running:

rsync -n -iaHJAX $d $b

Does some initial work and then gets to:

bash-3.2# lldb -p 6458
(lldb) process attach --pid 6458
Process 6458 stopped
* thread #1: tid = 0x1d5ff0, 0x7fff8c2d83fa libsystem_kernel.dylib`__select 
+ 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x7fff8c2d83fa libsystem_kernel.dylib`__select + 10
libsystem_kernel.dylib`__select:
->  0x7fff8c2d83fa <+10>: jae0x7fff8c2d8404; <+20>
0x7fff8c2d83fc <+12>: movq   %rax, %rdi
0x7fff8c2d83ff <+15>: jmp0x7fff8c2d3c78; cerror
0x7fff8c2d8404 <+20>: retq   

Executable module set to "/usr/local/bin/rsync".
Architecture set to: x86_64h-apple-macosx.
(lldb) bt
* thread #1: tid = 0x1d5ff0, 0x7fff8c2d83fa libsystem_kernel.dylib`__select 
+ 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x7fff8c2d83fa libsystem_kernel.dylib`__select + 10
frame #1: 0x00010dd6aedc rsync`perform_io(needed=47, flags=4) + 2924 at 
io.c:742
frame #2: 0x00010dd6a0bd rsync`send_msg(code=MSG_INFO, buf="[receiver] 
receiving flist for dir 1319\n", len=40, convert=0) + 349 at io.c:958
frame #3: 0x00010dd5c765 rsync`rwrite(code=FINFO, buf="[receiver] 
receiving flist for dir 1319\n", len=40, is_utf8=0) + 469 at log.c:279
frame #4: 0x00010dd5d3f3 rsync`rprintf(code=FINFO, format="[%s] 
receiving flist for dir %d\n") + 627 at log.c:435
frame #5: 0x00010dd394a6 rsync`read_ndx_and_attrs(f_in=0, f_out=4, 
iflag_ptr=0x7fff51ed0f64, type_ptr="\x80", buf="", 
len_ptr=0x7fff51ed0f60) + 646 at rsync.c:362
frame #6: 0x00010dd4483b rsync`recv_files(f_in=0, f_out=4, 
local_name=0x) + 379 at receiver.c:542
frame #7: 0x00010dd566c3 rsync`do_recv(f_in=0, f_out=4, 
local_name=0x) + 899 at main.c:909
frame #8: 0x00010dd54ac7 rsync`do_server_recv(f_in=0, f_out=1, argc=1, 
argv=0x7fff51ed2328) + 1207 at main.c:1078
frame #9: 0x00010dd5431e rsync`start_server(f_in=0, f_out=1, argc=2, 
argv=0x7fff51ed2320) + 286 at main.c:1112
frame #10: 0x00010dd541ee rsync`child_main(argc=2, 
argv=0x7fff51ed2320) + 46 at main.c:1085
frame #11: 0x00010dd852ad rsync`local_child(argc=2, 
argv=0x7fff51ed2320, f_in=0x7fff51ed4328, f_out=0x7fff51ed4324, 
child_main=(rsync`child_main at main.c:1084)) + 701 at pipe.c:167
frame #12: 0x00010dd58e42 rsync`do_cmd(cmd=0x, 
machine=0x, user=0x, 
remote_argv=0x7fee40c04a50, remote_argc=0, f_in_p=0x7fff51ed4328, 
f_out_p=0x7fff51ed4324) + 2210 at main.c:543
frame #13: 0x00010dd57fd2 rsync`start_client(argc=1, 
argv=0x7fee40c04a40) + 2402 at main.c:1414
frame #14: 0x00010dd5747b rsync`main(argc=2, argv=0x7fee40c04a40) + 
2555 at main.c:1652
frame #15: 0x7fff96cad5c9 libdyld.dylib`start + 1
frame #16: 0x7fff96cad5c9 libdyld.dylib`start + 1
(lldb) up
frame #1: 0x00010dd6aedc rsync`perform_io(needed=47, flags=4) + 2924 at 
io.c:742
   739  tv.tv_sec = select_timeout;
   740  tv.tv_usec = 0;
   741
-> 742 cnt = select(max_fd + 1, _fds, _fds, _fds, );
   743  
   744 if (cnt <= 0) {
   745  if (cnt < 0 && errno == EBADF) {
(lldb) fin

And then never returns fomr that stack frame, seems to hang out in
select() forever (unless interrupted by a breakpoint or whatnot).


Bugzilla account request submitted.

--jh...@mit.edu
  John Hawkinson

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]

[John Hawkinson via rsync]

Sorry to keep replying to myself:

> Because this is a Time Machine backup, and there were 66 snapshots of a
> 1 TB disk consuming about 1.5 TB, there were a *lot* of hard links. Many
> of directories rather than individual files, so it's a little

Err, whoops? No, I was tired and confused. They are not hard links
to directories, that would screw up the universe. Still, lots of hard
linked files.

> Do I need to run this under lldb and set a breakpoint in expand_item_list()?
> Quick inspection suggests running with - might give some useful output:


Err...

the result of this was it processed a few files for a minute or so and
then hung in select() and consumed no cpu and there was no disk
activity. Unfortunately apparently my clang/lldb workflow was broken
and I didn't have functional debugging symbols (...) and I also lost
the stack trace I thought I had (inadequate scrollback), so I'm not
sure what was going on. But at first blush, it appeared that adding -
made things hang forever.

Removing it, and rerunning, it's now happily trucking along and has
been for the past hour actually doing work.

--jh...@mit.edu
  John Hawkinson

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]

[John Hawkinson via rsync]

> I guess I can turn on core dumps and increase (unlimit completely) the 
> stack size...
> 
> Although it doesn't seem to have segfaulted, so I'm not sure having
> core dumps enabled would have helped?

Indeed. I reran it on the 50 remaining individual directories,
it seems to have made it through 10 before failing again overnight:

66596  20:38:03 rsync -vP -iaHJAX  `cat /tmp/unback`  
/Volumes/platinum-barratry/x/Backups.backupdb/pb3/

...
rsync: unpack_smb_acl: sys_acl_get_info(): Undefined error: 0 (0)
rsync: unpack_smb_acl: sys_acl_get_info(): Undefined error: 0 (0)
...
.fog.a.. 
2015-06-19-072520/platinum-bar2/Users/jhawk/.emacs.d/auto-save-list/.saves-1541-platinum-bar2.local~
ERROR: out of memory in expand_item_list [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106) 
[sender=3.1.2]
bash-3.2#  


No segfault, no coredump, no syslogs.

Do I need to run this under lldb and set a breakpoint in expand_item_list()?
Quick inspection suggests running with - might give some useful output:

1680 if (DEBUG_GTE(FLIST, 3)) {
1681 rprintf(FINFO, "[%s] expand %s to %s bytes, did%s move\n",
1682 who_am_i(), desc, big_num(new_size * item_size),
1683 new_ptr == lp->items ? " not" : "");
1684 }
1685 if (!new_ptr)
1686 out_of_memory("expand_item_list");

Although I imagine that output might be voluminous [but maybe not]?

Again, I don't have time to build test cases and reproduce this
carefully, every run is painful and long and slow. But I'd like to do the
responsible thing if someone can tell me what that is.

> p.s.: If I had to start over, I would have spent less time just deleting
> the data and recopying it, rather than trying to fixup the metadata and

Indeed, it's looking like fixing the metadata with rsync is an order
of magnitude slower, even as far as I've gotten. So maybe it's time to find
another method. I don't think fts(3) is optimized any better for large
hardlink farms, so I think maybe I need a homegrown solution? Ug.

--jh...@mit.edu
  John Hawkinson

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

error allocating core memory buffers (code 22) at util2.c(106) [sender=3.1.2]

[John A Hawkinson via rsync]

Hi:

I'm in the middle of recoverying from a tactical error copying
around an Mac OS X 10.10.5 Time Machine backup (turns out Apple's
instructions aren't great...), and I had rsync running for the past 6
hours repairing permissions/acls on 1.5 TB of data (not copying the
data), and then it just died in the middle with:

.Log 2015-03-11-094807/platinum-bar2/usr/local/mysql ->
mysql-5.6.21-osx
10.8-x86_64
ERROR: out of memory in expand_item_list [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106) 
[sender=3.1.2]
rsync: [sender] write error: Broken pipe (32)

It was invoked as 

rsync -iaHJAX platinum-barzoom/Backups.backupdb/pb3/ 
platinum-barratry/x/Backups.backupdb/pb3/

I suspect the situation will be different the next time around, and I'm
also not really inclined to try to wait another 7 hours for it to fail.

Process limits were:

bash-3.2# ulimit -a
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
file size   (blocks, -f) unlimited
max locked memory   (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files  (-n) 256
pipe size(512 bytes, -p) 1
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 709
virtual memory  (kbytes, -v) unlimited


Based on "df -i", there are about 200 million inodes in the scope of the
copy

bash-3.2# df -i 
Filesystem512-blocks   Used  Available Capacity   
iused ifree %iused  Mounted on
/dev/disk2s2  3906353072 3879969656   26383416   100% 
484996205   3297927   99%   /Volumes/platinum-barzoom
/dev/disk1s2  9766869344 3327435312 643943403235% 
207964705 402464627   34%   /Volumes/platinum-barratry

Because this is a Time Machine backup, and there were 66 snapshots of a
1 TB disk consuming about 1.5 TB, there were a *lot* of hard links. Many
of directories rather than individual files, so it's a little
challenging to estimate what the number of files to links is.

Are there any useful tips here?
Is it worth filing a bug report on this thin record?
I guess I can turn on core dumps and increase (unlimit completely) the 
stack size...

Although it doesn't seem to have segfaulted, so I'm not sure having
core dumps enabled would have helped?

This was rsync 3.1.2 as installed via Homebrew which appears to have
applied 3 local patches:
https://github.com/Homebrew/homebrew-core/blob/68fe052398ea0939315ad87b0a08313c9d9a95af/Formula/rsync.rb

apply "patches/fileflags.diff",
  "patches/crtimes.diff",
  "patches/hfs-compression.diff"
  
p.s.: If I had to start over, I would have spent less time just deleting
the data and recopying it, rather than trying to fixup the metadata and
dealing with magic Apple stuff like the inability to modify symlinks
inside a top-level Backups.backupdb directory of a Time Machine hfs
volume (But you can move the top-level directory into another directory
and then modify symlinks inside and then move it back). This has been
an "interesting" experience.

Thanks.

--jh...@mit.edu
  John Hawkinson

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12522] [PATCH] Send last error messages to sender

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=12522

Ben RUBSON <ben.rub...@gmail.com> changed:

   What|Removed |Added

  Attachment #12838|0   |1
is obsolete||

--- Comment #2 from Ben RUBSON <ben.rub...@gmail.com> ---
Created attachment 12841
  --> https://bugzilla.samba.org/attachment.cgi?id=12841=edit
Send last error messages to sender

Error correction on sender side.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12522] New: [PATCH] Send last error messages to sender

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=12522

Bug ID: 12522
   Summary: [PATCH] Send last error messages to sender
   Product: rsync
   Version: 3.1.2
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: ben.rub...@gmail.com
QA Contact: rsync...@samba.org

Created attachment 12832
  --> https://bugzilla.samba.org/attachment.cgi?id=12832=edit
Send last error messages to sender

Hello,

Here is a patch which sends last error messages to sender for a file-transfer
which failed.

It helps especially with protocol < 31.

This can be useful for example to know that a FS is full.

Thank you !

Ben



### Without :

/repository/mybigfile.iso
539.26M   6%   11.21MB/s0:11:01  
rsync: [sender] write error: Broken pipe (32)
rsync error: error in file IO (code 11) at io.c(831) [sender=3.1.2]



### With (of course the interesting message here is the first one) :

/repository/mybigfile.iso
539.26M   6%   11.21MB/s0:11:01  
rsync: write failed on "/repository/mybigfile.iso" (in bkp): Disc quota
exceeded (69)
rsync error: error in file IO (code 11) at receiver.c(400) [receiver=3.1.2]
rsync error: error in file IO (code 11) at io.c(1633) [generator=3.1.2]
rsync: [sender] write error: Broken pipe (32)
rsync error: error in file IO (code 11) at io.c(831) [sender=3.1.2]

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: segfault at 968 Error

[VigneshDhanraj G]

after loading libc symbols, gdb output as follows.

#0  0x7f90001cd790 in __pthread_initialize_minimal_internal () at
nptl-init.c:296
296 nptl-init.c: No such file or directory.
(gdb) bt
#0  0x7f90001cd790 in __pthread_initialize_minimal_internal () at
nptl-init.c:296
#1  0x7f90001cd209 in _init () from
/lib/x86_64-linux-gnu/libpthread.so.0
#2  0x7f8c2000 in ?? ()
#3  0x7f9001289f09 in ?? () from /lib64/ld-linux-x86-64.so.2
#4  0x7f900128a0ce in ?? () from /lib64/ld-linux-x86-64.so.2
#5  0x7f900128e333 in ?? () from /lib64/ld-linux-x86-64.so.2
#6  0x7f9001289ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
#7  0x7f900128db1a in ?? () from /lib64/ld-linux-x86-64.so.2
#8  0x7f9000907760 in do_dlopen (ptr=0x7fffe9600e60) at dl-libc.c:86
#9  0x7f9001289ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
#10 0x7f90009077ff in dlerror_run (operate=0x7f90001cd762
<__pthread_initialize_minimal_internal+34>, args=0x1) at dl-libc.c:47
#11 0x7f90009078f7 in *__GI___libc_dlopen_mode (name=,
mode=) at dl-libc.c:160
#12 0x7f90008e0e94 in *__GI___nss_lookup_function (ni=0x24fce80,
fct_name=0x7f90009437c6 "gethostbyaddr_r") at nsswitch.c:434
#13 0x7f90008e16ab in *__GI___nss_next2 (ni=0x7fffe9600f88,
fct_name=0x7f90009437c6 "gethostbyaddr_r", fct2_name=0x0,
fctp=0x7fffe9600f80,
status=1, all_values=21518080) at nsswitch.c:271
#14 0x7f90008e7666 in __gethostbyaddr_r (addr=0x7fffe9601764, len=4,
type=, resbuf=0x7fffe9601610, buffer=,
buflen=, result=0x7fffe9601640, h_errnop=0x7fffe960164c)
at ../nss/getXXbyYY_r.c:274
#15 0x7f90008ece1d in *__GI_getnameinfo (sa=0x7fffe9601760,
addrlen=, host=0x69ac20 "UNKNOWN", hostlen=100,
serv=0x69aca0 "", servlen=100, flags=) at
getnameinfo.c:223

Is this is known issue??


On Fri, Dec 2, 2016 at 1:02 PM, VigneshDhanraj G <vigneshdhanra...@gmail.com
> wrote:

> Thanks, same file consistently it is crashing.
>
> On Fri, Dec 2, 2016 at 12:44 PM, <devz...@web.de> wrote:
>
>> does it crash reproducable at the same file or is it randomly?
>>
>> Am 2. Dezember 2016 07:50:20 MEZ, schrieb VigneshDhanraj G <
>> vigneshdhanra...@gmail.com>:
>>
>>> Any update on this issue.
>>>
>>> On Wed, Nov 30, 2016 at 6:29 PM, VigneshDhanraj G <
>>> vigneshdhanra...@gmail.com> wrote:
>>>
>>>> Hi Team,
>>>>
>>>> While Running rsync rsync://username@ip:873 , I am getting following
>>>> error.
>>>>
>>>> rsync: safe_read failed to read 1 bytes [Receiver]: Connection reset by
>>>> peer (104)
>>>> rsync error: error in rsync protocol data stream (code 12) at io.c(276)
>>>> [Receiver=3.1.1]
>>>>
>>>> In Remote pc , i can see segmentation fault, dmesg gives following
>>>> error. My remote PC using debian wheezy.
>>>>
>>>> rsync[9022]: segfault at 968 ip 7f90001cd790 sp 7fffe96008a0
>>>> error 4 in libpthread-2.13.so[7f90001c8000+17000]
>>>>
>>>> gdb output: bt
>>>>
>>>> (gdb) bt
>>>> #0  0x7fa75e8ab790 in __pthread_initialize_minimal_internal ()
>>>> from /lib/x86_64-linux-gnu/libpthread.so.0
>>>> #1  0x7fa75e8ab209 in _init () from /lib/x86_64-linux-gnu/libpthre
>>>> ad.so.0
>>>> #2  0x7fa75e6a in ?? ()
>>>> #3  0x7fa75f967f09 in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #4  0x7fa75f9680ce in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #5  0x7fa75f96c333 in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #6  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #7  0x7fa75f96bb1a in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #8  0x7fa75efe5760 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>>>> #9  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>>>> #10 0x7fa75efe57ff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>>>> #11 0x7fa75efe58f7 in __libc_dlopen_mode () from
>>>> /lib/x86_64-linux-gnu/libc.so.6
>>>> #12 0x7fa75efbee94 in __nss_lookup_function () from
>>>> /lib/x86_64-linux-gnu/libc.so.6
>>>> #13 0x7fa75efbf6ab in __nss_next2 () from
>>>> /lib/x86_64-linux-gnu/libc.so.6
>>>> #14 0x7fa75efc5666 in gethostbyaddr_r () from
>>>> /lib/x86_64-linux-gnu/libc.so.6
>>>> #15 0x7fa75efcae1d in getnameinfo () from
>>>> /lib/x86_64-linux-gnu/libc.so.6
>>>> #16 0x00441002 in lookup_name ()
>>>> #17 0x00440e3b in client_name ()
>>>> #18 0x0044d566 in

Re: segfault at 968 Error

[VigneshDhanraj G]

Thanks, same file consistently it is crashing.

On Fri, Dec 2, 2016 at 12:44 PM, <devz...@web.de> wrote:

> does it crash reproducable at the same file or is it randomly?
>
> Am 2. Dezember 2016 07:50:20 MEZ, schrieb VigneshDhanraj G <
> vigneshdhanra...@gmail.com>:
>
>> Any update on this issue.
>>
>> On Wed, Nov 30, 2016 at 6:29 PM, VigneshDhanraj G <
>> vigneshdhanra...@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>> While Running rsync rsync://username@ip:873 , I am getting following
>>> error.
>>>
>>> rsync: safe_read failed to read 1 bytes [Receiver]: Connection reset by
>>> peer (104)
>>> rsync error: error in rsync protocol data stream (code 12) at io.c(276)
>>> [Receiver=3.1.1]
>>>
>>> In Remote pc , i can see segmentation fault, dmesg gives following
>>> error. My remote PC using debian wheezy.
>>>
>>> rsync[9022]: segfault at 968 ip 7f90001cd790 sp 7fffe96008a0
>>> error 4 in libpthread-2.13.so[7f90001c8000+17000]
>>>
>>> gdb output: bt
>>>
>>> (gdb) bt
>>> #0  0x7fa75e8ab790 in __pthread_initialize_minimal_internal () from
>>> /lib/x86_64-linux-gnu/libpthread.so.0
>>> #1  0x7fa75e8ab209 in _init () from /lib/x86_64-linux-gnu/libpthre
>>> ad.so.0
>>> #2  0x7fa75e6a in ?? ()
>>> #3  0x7fa75f967f09 in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #4  0x7fa75f9680ce in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #5  0x7fa75f96c333 in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #6  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #7  0x7fa75f96bb1a in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #8  0x7fa75efe5760 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>>> #9  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>>> #10 0x7fa75efe57ff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>>> #11 0x7fa75efe58f7 in __libc_dlopen_mode () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #12 0x7fa75efbee94 in __nss_lookup_function () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #13 0x7fa75efbf6ab in __nss_next2 () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #14 0x7fa75efc5666 in gethostbyaddr_r () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #15 0x7fa75efcae1d in getnameinfo () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #16 0x00441002 in lookup_name ()
>>> #17 0x00440e3b in client_name ()
>>> #18 0x0044d566 in start_daemon ()
>>> #19 0x0043eb74 in start_accept_loop ()
>>> #20 0x0044da7e in daemon_main ()
>>> #21 0x00425751 in main ()
>>>
>>> strace output which shows segmentation fault.
>>> 7241  read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0
>>> \0\3\0>\0\1\0\0\0\3008\0\0\0\0\0\0"..., 832) = 832
>>> 7241  fstat(5, {st_mode=S_IFREG|0644, st_size=80712, ...}) = 0
>>> 7241  mmap(NULL, 2185864, PROT_READ|PROT_EXEC,
>>> MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fd406a88000
>>> 7241  mprotect(0x7fd406a9b000, 2093056, PROT_NONE) = 0
>>> 7241  mmap(0x7fd406c9a000, 8192, PROT_READ|PROT_WRITE,
>>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x12000) = 0x7fd406c9a000
>>> 7241  mmap(0x7fd406c9c000, 6792, PROT_READ|PROT_WRITE,
>>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd406c9c000
>>> 7241  close(5)  = 0
>>> 7241  mprotect(0x7fd406c9a000, 4096, PROT_READ) = 0
>>> 7241  mprotect(0x7fd406ea2000, 4096, PROT_READ) = 0
>>> 7241  set_tid_address(0x7fd4080809d0)   = 7241
>>> 7241  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
>>> 6080  <... select resumed> )= ? ERESTARTNOHAND (To be
>>> restarted)
>>> 6080  --- SIGCHLD (Child exited) @ 0 (0) ---
>>> 6080  wait4(-1, NULL, WNOHANG, NULL)= 7241
>>> 6080  wait4(-1, NULL, WNOHANG, NULL)= -1 ECHILD (No child processes)
>>> 6080  rt_sigreturn(0x)  = -1 EINTR (Interrupted system
>>> call)
>>> 6080  select(5, [4], NULL, NULL, NULL 
>>>
>>> so, i have upgraded rsync to latest in my remote pc, but problem not get
>>> solved.
>>> Please help me to solve this.
>>>
>>> Regards,
>>> Vigneshdhanraj G
>>>
>>
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: 
>> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>
>>
> --
> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail
> gesendet.
>
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: segfault at 968 Error

[devzero]

does it crash reproducable at the same file or is it randomly?

Am 2. Dezember 2016 07:50:20 MEZ, schrieb VigneshDhanraj G 
<vigneshdhanra...@gmail.com>:
>Any update on this issue.
>
>On Wed, Nov 30, 2016 at 6:29 PM, VigneshDhanraj G <
>vigneshdhanra...@gmail.com> wrote:
>
>> Hi Team,
>>
>> While Running rsync rsync://username@ip:873 , I am getting following
>> error.
>>
>> rsync: safe_read failed to read 1 bytes [Receiver]: Connection reset
>by
>> peer (104)
>> rsync error: error in rsync protocol data stream (code 12) at
>io.c(276)
>> [Receiver=3.1.1]
>>
>> In Remote pc , i can see segmentation fault, dmesg gives following
>error.
>> My remote PC using debian wheezy.
>>
>> rsync[9022]: segfault at 968 ip 7f90001cd790 sp 7fffe96008a0
>error
>> 4 in libpthread-2.13.so[7f90001c8000+17000]
>>
>> gdb output: bt
>>
>> (gdb) bt
>> #0  0x7fa75e8ab790 in __pthread_initialize_minimal_internal ()
>from
>> /lib/x86_64-linux-gnu/libpthread.so.0
>> #1  0x7fa75e8ab209 in _init () from /lib/x86_64-linux-gnu/
>> libpthread.so.0
>> #2  0x7fa75e6a in ?? ()
>> #3  0x7fa75f967f09 in ?? () from /lib64/ld-linux-x86-64.so.2
>> #4  0x7fa75f9680ce in ?? () from /lib64/ld-linux-x86-64.so.2
>> #5  0x7fa75f96c333 in ?? () from /lib64/ld-linux-x86-64.so.2
>> #6  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>> #7  0x7fa75f96bb1a in ?? () from /lib64/ld-linux-x86-64.so.2
>> #8  0x7fa75efe5760 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>> #9  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
>> #10 0x7fa75efe57ff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>> #11 0x7fa75efe58f7 in __libc_dlopen_mode () from
>> /lib/x86_64-linux-gnu/libc.so.6
>> #12 0x7fa75efbee94 in __nss_lookup_function () from
>> /lib/x86_64-linux-gnu/libc.so.6
>> #13 0x7fa75efbf6ab in __nss_next2 () from
>> /lib/x86_64-linux-gnu/libc.so.6
>> #14 0x7fa75efc5666 in gethostbyaddr_r () from
>> /lib/x86_64-linux-gnu/libc.so.6
>> #15 0x7fa75efcae1d in getnameinfo () from
>> /lib/x86_64-linux-gnu/libc.so.6
>> #16 0x00441002 in lookup_name ()
>> #17 0x00440e3b in client_name ()
>> #18 0x0044d566 in start_daemon ()
>> #19 0x0043eb74 in start_accept_loop ()
>> #20 0x0044da7e in daemon_main ()
>> #21 0x00425751 in main ()
>>
>> strace output which shows segmentation fault.
>> 7241  read(5,
>"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3008\0\0\0\0\0\0"...,
>> 832) = 832
>> 7241  fstat(5, {st_mode=S_IFREG|0644, st_size=80712, ...}) = 0
>> 7241  mmap(NULL, 2185864, PROT_READ|PROT_EXEC,
>MAP_PRIVATE|MAP_DENYWRITE,
>> 5, 0) = 0x7fd406a88000
>> 7241  mprotect(0x7fd406a9b000, 2093056, PROT_NONE) = 0
>> 7241  mmap(0x7fd406c9a000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x12000) = 0x7fd406c9a000
>> 7241  mmap(0x7fd406c9c000, 6792, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd406c9c000
>> 7241  close(5)  = 0
>> 7241  mprotect(0x7fd406c9a000, 4096, PROT_READ) = 0
>> 7241  mprotect(0x7fd406ea2000, 4096, PROT_READ) = 0
>> 7241  set_tid_address(0x7fd4080809d0)   = 7241
>> 7241  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
>> 6080  <... select resumed> )= ? ERESTARTNOHAND (To be
>> restarted)
>> 6080  --- SIGCHLD (Child exited) @ 0 (0) ---
>> 6080  wait4(-1, NULL, WNOHANG, NULL)= 7241
>> 6080  wait4(-1, NULL, WNOHANG, NULL)= -1 ECHILD (No child
>processes)
>> 6080  rt_sigreturn(0x)  = -1 EINTR (Interrupted
>system
>> call)
>> 6080  select(5, [4], NULL, NULL, NULL 
>>
>> so, i have upgraded rsync to latest in my remote pc, but problem not
>get
>> solved.
>> Please help me to solve this.
>>
>> Regards,
>> Vigneshdhanraj G
>>
>
>
>
>
>-- 
>Please use reply-all for most replies to avoid omitting the mailing
>list.
>To unsubscribe or change options:
>https://lists.samba.org/mailman/listinfo/rsync
>Before posting, read:
>http://www.catb.org/~esr/faqs/smart-questions.html

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: segfault at 968 Error

[VigneshDhanraj G]

Any update on this issue.

On Wed, Nov 30, 2016 at 6:29 PM, VigneshDhanraj G <
vigneshdhanra...@gmail.com> wrote:

> Hi Team,
>
> While Running rsync rsync://username@ip:873 , I am getting following
> error.
>
> rsync: safe_read failed to read 1 bytes [Receiver]: Connection reset by
> peer (104)
> rsync error: error in rsync protocol data stream (code 12) at io.c(276)
> [Receiver=3.1.1]
>
> In Remote pc , i can see segmentation fault, dmesg gives following error.
> My remote PC using debian wheezy.
>
> rsync[9022]: segfault at 968 ip 7f90001cd790 sp 7fffe96008a0 error
> 4 in libpthread-2.13.so[7f90001c8000+17000]
>
> gdb output: bt
>
> (gdb) bt
> #0  0x7fa75e8ab790 in __pthread_initialize_minimal_internal () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #1  0x7fa75e8ab209 in _init () from /lib/x86_64-linux-gnu/
> libpthread.so.0
> #2  0x7fa75e6a in ?? ()
> #3  0x7fa75f967f09 in ?? () from /lib64/ld-linux-x86-64.so.2
> #4  0x7fa75f9680ce in ?? () from /lib64/ld-linux-x86-64.so.2
> #5  0x7fa75f96c333 in ?? () from /lib64/ld-linux-x86-64.so.2
> #6  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
> #7  0x7fa75f96bb1a in ?? () from /lib64/ld-linux-x86-64.so.2
> #8  0x7fa75efe5760 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #9  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
> #10 0x7fa75efe57ff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #11 0x7fa75efe58f7 in __libc_dlopen_mode () from
> /lib/x86_64-linux-gnu/libc.so.6
> #12 0x7fa75efbee94 in __nss_lookup_function () from
> /lib/x86_64-linux-gnu/libc.so.6
> #13 0x7fa75efbf6ab in __nss_next2 () from
> /lib/x86_64-linux-gnu/libc.so.6
> #14 0x7fa75efc5666 in gethostbyaddr_r () from
> /lib/x86_64-linux-gnu/libc.so.6
> #15 0x7fa75efcae1d in getnameinfo () from
> /lib/x86_64-linux-gnu/libc.so.6
> #16 0x00441002 in lookup_name ()
> #17 0x00440e3b in client_name ()
> #18 0x0044d566 in start_daemon ()
> #19 0x0043eb74 in start_accept_loop ()
> #20 0x0044da7e in daemon_main ()
> #21 0x00425751 in main ()
>
> strace output which shows segmentation fault.
> 7241  read(5, 
> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3008\0\0\0\0\0\0"...,
> 832) = 832
> 7241  fstat(5, {st_mode=S_IFREG|0644, st_size=80712, ...}) = 0
> 7241  mmap(NULL, 2185864, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
> 5, 0) = 0x7fd406a88000
> 7241  mprotect(0x7fd406a9b000, 2093056, PROT_NONE) = 0
> 7241  mmap(0x7fd406c9a000, 8192, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x12000) = 0x7fd406c9a000
> 7241  mmap(0x7fd406c9c000, 6792, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd406c9c000
> 7241  close(5)  = 0
> 7241  mprotect(0x7fd406c9a000, 4096, PROT_READ) = 0
> 7241  mprotect(0x7fd406ea2000, 4096, PROT_READ) = 0
> 7241  set_tid_address(0x7fd4080809d0)   = 7241
> 7241  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> 6080  <... select resumed> )= ? ERESTARTNOHAND (To be
> restarted)
> 6080  --- SIGCHLD (Child exited) @ 0 (0) ---
> 6080  wait4(-1, NULL, WNOHANG, NULL)= 7241
> 6080  wait4(-1, NULL, WNOHANG, NULL)= -1 ECHILD (No child processes)
> 6080  rt_sigreturn(0x)  = -1 EINTR (Interrupted system
> call)
> 6080  select(5, [4], NULL, NULL, NULL 
>
> so, i have upgraded rsync to latest in my remote pc, but problem not get
> solved.
> Please help me to solve this.
>
> Regards,
> Vigneshdhanraj G
>
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

segfault at 968 Error

[VigneshDhanraj G]

Hi Team,

While Running rsync rsync://username@ip:873 , I am getting following error.

rsync: safe_read failed to read 1 bytes [Receiver]: Connection reset by
peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(276)
[Receiver=3.1.1]

In Remote pc , i can see segmentation fault, dmesg gives following error.
My remote PC using debian wheezy.

rsync[9022]: segfault at 968 ip 7f90001cd790 sp 7fffe96008a0 error
4 in libpthread-2.13.so[7f90001c8000+17000]

gdb output: bt

(gdb) bt
#0  0x7fa75e8ab790 in __pthread_initialize_minimal_internal () from
/lib/x86_64-linux-gnu/libpthread.so.0
#1  0x7fa75e8ab209 in _init () from
/lib/x86_64-linux-gnu/libpthread.so.0
#2  0x7fa75e6a in ?? ()
#3  0x7fa75f967f09 in ?? () from /lib64/ld-linux-x86-64.so.2
#4  0x7fa75f9680ce in ?? () from /lib64/ld-linux-x86-64.so.2
#5  0x7fa75f96c333 in ?? () from /lib64/ld-linux-x86-64.so.2
#6  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
#7  0x7fa75f96bb1a in ?? () from /lib64/ld-linux-x86-64.so.2
#8  0x7fa75efe5760 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#9  0x7fa75f967ba6 in ?? () from /lib64/ld-linux-x86-64.so.2
#10 0x7fa75efe57ff in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#11 0x7fa75efe58f7 in __libc_dlopen_mode () from
/lib/x86_64-linux-gnu/libc.so.6
#12 0x7fa75efbee94 in __nss_lookup_function () from
/lib/x86_64-linux-gnu/libc.so.6
#13 0x7fa75efbf6ab in __nss_next2 () from
/lib/x86_64-linux-gnu/libc.so.6
#14 0x7fa75efc5666 in gethostbyaddr_r () from
/lib/x86_64-linux-gnu/libc.so.6
#15 0x7fa75efcae1d in getnameinfo () from
/lib/x86_64-linux-gnu/libc.so.6
#16 0x00441002 in lookup_name ()
#17 0x00440e3b in client_name ()
#18 0x0044d566 in start_daemon ()
#19 0x0043eb74 in start_accept_loop ()
#20 0x0044da7e in daemon_main ()
#21 0x00425751 in main ()

strace output which shows segmentation fault.
7241  read(5,
"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\3008\0\0\0\0\0\0"..., 832)
= 832
7241  fstat(5, {st_mode=S_IFREG|0644, st_size=80712, ...}) = 0
7241  mmap(NULL, 2185864, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
5, 0) = 0x7fd406a88000
7241  mprotect(0x7fd406a9b000, 2093056, PROT_NONE) = 0
7241  mmap(0x7fd406c9a000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x12000) = 0x7fd406c9a000
7241  mmap(0x7fd406c9c000, 6792, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd406c9c000
7241  close(5)  = 0
7241  mprotect(0x7fd406c9a000, 4096, PROT_READ) = 0
7241  mprotect(0x7fd406ea2000, 4096, PROT_READ) = 0
7241  set_tid_address(0x7fd4080809d0)   = 7241
7241  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
6080  <... select resumed> )= ? ERESTARTNOHAND (To be restarted)
6080  --- SIGCHLD (Child exited) @ 0 (0) ---
6080  wait4(-1, NULL, WNOHANG, NULL)= 7241
6080  wait4(-1, NULL, WNOHANG, NULL)= -1 ECHILD (No child processes)
6080  rt_sigreturn(0x)  = -1 EINTR (Interrupted system call)
6080  select(5, [4], NULL, NULL, NULL 

so, i have upgraded rsync to latest in my remote pc, but problem not get
solved.
Please help me to solve this.

Regards,
Vigneshdhanraj G
-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: error code 255 - not mentioned in manpage?

[Paul Slootman]

On Tue 25 Oct 2016, devz...@web.de wrote:
> 
> is there a reason why error code 255 is not mentioned in the manpage 
> and wouldn`t it make sense to add "255     Unexplained Error" there 
> for completeness ?

It wouldn't be unexplained then anymore, would it? :-)


Paul

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

error code 255 - not mentioned in manpage?

[devzero]

hi, 

is there a reason why error code 255 is not mentioned in the manpage 
and wouldn`t it make sense to add "255 Unexplained Error" there 
for completeness ?

I`m writing a script which checks exit values and while testing it, i 
got value 255, looked into the manpage and scratched my head what`s the 
meaning of it. 

ok - it`s obvious if you take a look at rsync error output, but i think 
all exit values should be documented.

regards
roland


# cat rsync.err
ssh: connect to host .. port 22: Connection refused
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 12132] New: Error on attributes when directory read-only

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=12132

Bug ID: 12132
   Summary: Error on attributes when directory read-only
   Product: rsync
   Version: 3.1.2
  Hardware: All
OS: Mac OS X
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: dan+mozi...@pasteur.fr
QA Contact: rsync...@samba.org

When making an rsync of a directory with mode 0555 and attributes,
there is an error on any included file creation: 

failed: Permission denied (13)

Smallest test directory:

#   coriolis:/tmp/test  % ls -@alO
total 0
drwxr-xr-x   4 dan   wheel  - 136 Aug  8 19:16 .
drwxrwxrwt  15 root  wheel  - 544 Aug  8 19:01 ..
dr-xr-xr-x@  2 dan   wheel  - 102 Aug  8 17:54 dir+attr
com.apple.FinderInfo 32 
dr-xr-xr-x   2 dan   wheel  - 102 Aug  8 19:13 dir+noattr



0.  Versions in use

Note: this is note the installed version with MacOS X (2.6.9) but
the latest one built from MacPorts (3.1.2).

The OS is OS X 10.10.5 (on source and destination)

rsync --version
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, no prealloc, file-flags,
HFS-compression

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

ssh doppler /opt/local/bin/rsync --version
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, no prealloc, file-flags,
HFS-compression

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.



1.  Case with the directory without attributes

#   coriolis:/tmp/test  % rsync -aArvX --rsync-path=/opt/local/bin/rsync
dir+noattr doppler:/tmp
sending incremental file list
dir+noattr/
dir+noattr/index.xml.gz

sent 102,608 bytes  received 39 bytes  41,058.80 bytes/sec
total size is 102,449  speedup is 1.00



2.  Case with the directory with attributes

#   coriolis:/tmp/test  % rsync -aArvX --rsync-path=/opt/local/bin/rsync
dir+attr doppler:/tmp 
sending incremental file list
dir+attr/
dir+attr/index.xml.gz
rsync: mkstemp "/tmp/dir+attr/.index.xml.gz.E5wKN9" failed: Permission denied
(13)

sent 102,671 bytes  received 127 bytes  68,532.00 bytes/sec
total size is 102,449  speedup is 1.00
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1249) [sender=3.1.2]

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

Armin Mohring  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|REOPENED|RESOLVED

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11654] New: Misleading error message

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11654

Bug ID: 11654
   Summary: Misleading error message
   Product: rsync
   Version: 3.1.1
  Hardware: x64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: taso.ha...@gmail.com
QA Contact: rsync...@samba.org

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11654] Misleading error message

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11654

--- Comment #1 from Taso Hatzi <taso.ha...@gmail.com> ---
rsync bombs with "rsync error: syntax or usage error (code 1) at main.c(873)
[Receiver=3.1.1]" if it doesn't have write permissions on the destination.
This is somewhat misleading as everything works OK with the --dry-run switch.

Platform is: Linux phenom 4.2.0-22-generic #27-Ubuntu SMP Thu Dec 17 22:57:08
UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

rsync version is: 3.1.1-3

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Command Failed -1 error

[Kacperski, Emil]

Hello,

In the past and present we have received an error of "Command failed with -1:" 
when running rsync.  -1 is not a valid response that I can find.  Is there a 
way to debug this or has anyone come across a -1 error?

We receive messages with a 12, 23, 24 which makes sense but this -1 is a 
mystery?

Thanks!


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

Armin Mohring <armin_mohr...@web.de> changed:

   What|Removed |Added

   Severity|normal  |minor
 Resolution|INVALID |---
   Priority|P3  |P2
 Status|RESOLVED|REOPENED

--- Comment #6 from Armin Mohring <armin_mohr...@web.de> ---
Rsync does not detect, if a drive is not mounted.

In my case: I have an external USB3.0 drive with 2TB.
If the drive is not mounted, the following error occurs:

rsync error: error in file io (code 11) at main.c (656) [Receiver=3.1.1]

Improvemnt suggestion:

Rsync should detect, if the drive is mounted or not.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #7 from Kevin Korb  ---
Rsync has no idea (or way) to know that you are trying to rsync to a separate
disk.  You give it a source and a target and it uses what is there.  If you
need to check for a mounted filesystem then you want a wrapper script.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #9 from Kevin Korb  ---
You have no idea how many people would complain if rsync didn't create a
non-existing target directory.  It is already a very common complaint that
rsync won't create more than one deep.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #8 from Armin Mohring <armin_mohr...@web.de> ---
Why should rsync that know.

There is an error, if the specified folder does not exist. 
Could be an external disk or partition or something else.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

Armin Mohring  changed:

   What|Removed |Added

 Resolution|--- |INVALID
 Status|NEW |RESOLVED

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] New: Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

Bug ID: 11578
   Summary: Rsync does start with an error directly connecting an
USB drive to a port
   Product: rsync
   Version: 3.1.1
  Hardware: x64
OS: Linux
Status: NEW
  Severity: normal
  Priority: P5
 Component: core
  Assignee: way...@samba.org
  Reporter: armin_mohr...@web.de
QA Contact: rsync...@samba.org

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

Armin Mohring <armin_mohr...@web.de> changed:

   What|Removed |Added

   Priority|P5  |P3

--- Comment #1 from Armin Mohring <armin_mohr...@web.de> ---
error output:
rsync error: error in file io (code 11) at main.c (656) [Receiver=3.1.1]

This error occurs, when I connect an USB drive [USB HDD, 2TB) to an USB 3.0
port and try to start an incremental file transfer.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #2 from Kevin Korb  ---
There should be additional errors.  If not from rsync then from dmesg.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #3 from Armin Mohring  ---
This is the relevant part from dmesg after connecting the drive to an USB 3.0
port:

 5773.601144] Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
[ 5777.829686] usb 4-3: new SuperSpeed USB device number 6 using xhci_hcd
[ 5777.841134] usb 4-3: New USB device found, idVendor=0bc2, idProduct=3320
[ 5777.841139] usb 4-3: New USB device strings: Mfr=2, Product=3,
SerialNumber=1
[ 5777.841142] usb 4-3: Product: Expansion Desk
[ 5777.841144] usb 4-3: Manufacturer: Seagate
[ 5777.841146] usb 4-3: SerialNumber: NA4MVF15
[ 5777.843264] scsi8 : uas
[ 5777.843690] scsi 8:0:0:0: Direct-Access Seagate  Expansion Desk   0712
PQ: 0 ANSI: 6
[ 5777.844184] sd 8:0:0:0: Attached scsi generic sg8 type 0
[ 5777.844251] sd 8:0:0:0: [sdh] Spinning up disk...
[ 5778.843059] ...ready
[ 5784.839976] sd 8:0:0:0: [sdh] 488378645 4096-byte logical blocks: (2.00
TB/1.81 TiB)
[ 5784.857762] sd 8:0:0:0: [sdh] Write Protect is off
[ 5784.857767] sd 8:0:0:0: [sdh] Mode Sense: 4f 00 00 00
[ 5784.857935] sd 8:0:0:0: [sdh] Write cache: enabled, read cache: enabled,
doesn't support DPO or FUA
[ 5784.858486] sd 8:0:0:0: [sdh] 488378645 4096-byte logical blocks: (2.00
TB/1.81 TiB)
[ 5784.881361]  sdh: sdh1
[ 5784.881816] sd 8:0:0:0: [sdh] 488378645 4096-byte logical blocks: (2.00
TB/1.81 TiB)
[ 5784.882346] sd 8:0:0:0: [sdh] Attached SCSI disk

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 11578] Rsync does start with an error directly connecting an USB drive to a port

[samba-bugs]

https://bugzilla.samba.org/show_bug.cgi?id=11578

--- Comment #4 from Kevin Korb  ---
There are no device, block, or filesystem errors after that?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html