rsync performance weirdness

2019-07-12 Thread Roland --- via rsync
hi, i observe some weirdness in rsync file transfer i cannot explain. i'm transferring data from 2 freenas storages to a linux vm with zfsonlinux. "fnask" is older freenas with rsync 3.1.1 running in daemon mode, freenas-bnkw is more recent with rsync 3.1.2, also daemon-mode. on the linux vm

rsync --inplace dead slow

2019-07-15 Thread Roland --- via rsync
hello, i observe a weird performance problem while trying to rsync .vhd image files from xenserver to linux backupserver. i have broken the problem down to a single 500gb file which gets pathological slow on subsequent runs. rsync begins with decent speed (250MB/s), i can see that with

Re: Do grsync/rsync work well on Mac?

2019-04-21 Thread Roland --- via rsync
have a look at this one: http://osxdaily.com/2016/04/17/speed-up-time-machine-by-removing-low-process-priority-throttling/ i also recommend taking a look at borgbackup and https://github.com/borgbase/vorta roland Am 21.04.19 um 05:32 schrieb Peng Yu via rsync: Hi, I experience a slow backup

osx permission issue

2020-03-28 Thread Roland via rsync
hello, does somebody know how to circumvent that "extra file access restriction feature" introduced in osx some time ago ? i already tried adding rsync binary to programms with "full disk access" privilege ( system-prefs -> security & privacy -> privacy -> full disk access)  , since running as

Re: osx permission issue

2020-03-28 Thread Roland via rsync
that or something different ? roland Am 28.03.20 um 16:20 schrieb Ben Bass: Hi Roland. You might have to give ssh full disk access on the remote Mac, as that is what is accessing the files according to the Mac. On Sat, Mar 28, 2020 at 9:12 AM Roland via rsync mailto:rsync@lists.samba.org>>

Re: Copying TBs -> error -> work around

2020-09-10 Thread Roland via rsync
>with rsync hanging - after breakout on /home for writing I then get: >"Read-only file system" if your filesystem switches to read-only, you have a serious problem with your system/storage, not with rsync. rsync (or the workload) is simply triggering the problem. regards roland Am 10.09.20

Re: [PATCHv2] SSE2/SSSE3 optimized version of get_checksum1() for x86-64

2020-05-20 Thread Roland via rsync
would it perhaps make sense to have a "--disable-sse2/3" commandline switch in rsync, too - at least for some timeframe until this is considered "rock solid" ? i dislike having automatic cpu feature switching code in a tool which needs to be reliable for me, this new optimization may have issues

--exclude vs --filter

2020-07-11 Thread Roland via rsync
Hello, i have a backup script where i first used "--exclude-from" only. users of my script can set exclude statements in backup client specific property files. later on, i needed to extend the script because of special requirement , so i needed to introduce usage of "--filter='merge

Re: [PATCH] Reduce memory usage

2021-12-23 Thread Roland via rsync
hello, it's fantastic to see that such optimizations still being found. out of curiosity - what is the status of this?  will it get merged ? roland Am 28.09.21 um 12:05 schrieb Jindřich Makovička via rsync: In 2004, an allocation optimization has been added to the file list handling code,

bug#282 : timeout option does not work as expected when remote system is hung

2022-02-01 Thread Roland via rsync
hello, I have opened https://github.com/WayneD/rsync/issues/282 looks similar to https://github.com/WayneD/rsync/issues/164 and https://github.com/WayneD/rsync/issues/8 roland -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options:

rsync timeout non-effective

2022-09-01 Thread Roland via rsync
hello, i do some backup via rsync/ssh and pull data from remote machine to local machine. whenever on remote machine there is some network attached subdir getting "stuck" (i.e. read/write is blocking, eg. by suspended zfs pool because of error), rsync backup also hangs forever. i wonder what's

Re: rsync timeout non-effective

2022-09-01 Thread Roland via rsync
t for its connection to an rsync daemon to succeed.  If the timeout is reached, rsync exits with an error. Am 01.09.22 um 17:32 schrieb Kevin Korb via rsync: You are using rsync over ssh.  The connection timeout (and port) options don't matter if rsync isn't doing the networking. On 9/1/22 08:

BUG?: rsync timeout non-effective

2022-09-04 Thread Roland via rsync
Am 01.09.22 um 17:32 schrieb Kevin Korb via rsync: You are using rsync over ssh.  The connection timeout (and port) options don't matter if rsync isn't doing the networking. On 9/1/22 08:49, Roland via rsync wrote: hello, i do some backup via rsync/ssh and pull data from remote machine to loca

Re: Fwd: [3.1.3] --remove-source-files $SRC/ $SRC/ - erases all files

2022-10-17 Thread Roland via rsync
why not avoid using "--remove-source-files" and delete files manually/via extra step afterwards ? Am 17.10.22 um 08:59 schrieb Sridhar Sarnobat via rsync: 90% of my data losses are caused by rsync'ing from dir A to dir A (accidental incorrect copy and paste, or where dir B is a symlink to dir

Re: syncing huge files/devices: bmapfs

2023-02-11 Thread Roland via rsync
nice, do you have performance comparison vs. plain fail on the same hardware setup ? thank you roland Am 10.02.23 um 07:56 schrieb Heiko Schlittermann via rsync: fuse driver that maps the huge file (or device) as a directory of files representing blocks of the backing file (or device).

Re: BUG? rsync ends without message by delete files

2024-02-06 Thread Roland via rsync
and then, it stops totally quiet. you mean it simply exits without any message? what's the return code ( echo $? ) roland Am 06.02.24 um 22:18 schrieb Franke via rsync: Hi Kevin, Am 06.02.24 um 20:55 schrieb Kevin Korb: The other likely cause is your $SOURCE being something that contains

Re: BUG? rsync ends without message by delete files

2024-02-06 Thread Roland via rsync
here is another report of this behaviour. https://unix.stackexchange.com/questions/754923/rsync-just-stops nothing appropriate in bugzilla, besides https://bugzilla.samba.org/show_bug.cgi?id=13317 do you use zfs or is there full-space/quota condition while running? if you can't resolve, please

process --files-from filelist as given

2023-12-31 Thread Roland via rsync
hello, apparently,  rsync sorts the list of files  provided to "--files-from". how can i avoid sorting of that list ? I want to copy a list of files in specific order regards Roland -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change

Re: process --files-from filelist as given

2023-12-31 Thread Roland via rsync
to bugzilla thanks roland Am 31.12.23 um 20:56 schrieb francis.montag...@inria.fr: Hi. On Sun, 31 Dec 2023 20:28:21 +0100 Roland via rsync wrote: apparently, rsync sorts the list of files  provided to "--files-from". how can i avoid sorting of that list ? According

Re: Archiving to vfat

2024-01-21 Thread Roland via rsync
it's most likely because of vfat timestamp limitation try --modify-window   When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than  the modify-window   value.   This  is  normally  0 (for an exact match), but you may find