Re: rsync: "-c" option clarification

2017-03-23 Thread Kevin Korb via rsync
a case of a file with matching mtime+size but not matching checksum due to gzip's metadata even though the uncompressed result is identical. I would not consider this to be a case worth updating the remote copy but I am sure someone will disagree. On 03/23/2017 03:49 PM, Kevin Korb via rsync

Re: rsync: "-c" option clarification

2017-03-23 Thread Kevin Korb via rsync
The -c option causes rsync to checksum EVERY file on both ends BEFORE rsync does anything else. It checksums files that are on only 1 end. It checksums files that are different sizes. It will not catch a hardware problem preventing rsync from writing a file correctly. On 03/23/2017 03:12 PM,

Re: rsync 3.1.1: --ignore-missing-args / --delete-missing args problem

2017-04-07 Thread Kevin Korb via rsync
Those options are for handling files/dirs that are specified on the actual command line they have nothing to do with files vanishing while rsync is running. The correct solution is to simply ignore exit code 24. BTW, this is what filesystem snapshots are for. On 04/07/2017 08:58 AM, Georgy

Re: Rsync slow with hard links

2017-04-12 Thread Kevin Korb via rsync
In Linux I would handle this with a pvmove. I don't know how to make it happen in Windows other than using a disk image. BTW, you will probably find -i to be much more useful than a second (or even a first) -v On 04/12/2017 07:31 PM, Rene J Suarez-Soto via rsync wrote: > I am running rsync

Re: [Bug 12732] hard links can cause rsync to block or to silently skip files

2017-04-05 Thread Kevin Korb via rsync
Note that in all --*-dest options if the path is relative it is relative to the target dir not the "$PWD". I like to always use absolute paths because of this. But essentially, the command with the instances of $PWD vs without them the paths aren't the same. If you want all relative to . paths

Re: modification times questions

2017-04-07 Thread Kevin Korb via rsync
If you are sure the content is correct you can run rsync with both --times and --size-only. This will cause rsync to "fix" the timestamps on files that are the same size on both ends. On 04/07/2017 02:53 PM, McDowell, Blake via rsync wrote: > How do I transfer just the modification times with

Re: modification times questions

2017-04-07 Thread Kevin Korb via rsync
I guess I should also mention that if both trees are local you can use: find . -print -exec touch "/path/to/wrong/times/{}" --reference "{}" \; On 04/07/2017 02:58 PM, Kevin Korb via rsync wrote: > If you are sure the content is correct you can run rsync with both &g

Re: modification times questions

2017-04-07 Thread Kevin Korb via rsync
gt; to transfer the files because of times. When I run rsync a second time with > your suggestion the times do transfer over. I don't know why... > > B > > From: rsync [rsync-boun...@lists.samba.org] on behalf of Kevin Korb via rsync &g

Re: rsync 3.1.1: --ignore-missing-args / --delete-missing args problem

2017-04-07 Thread Kevin Korb via rsync
Exit code 2 is "Protocol incompatibility". Also, sounds like what you really want is --files-from On 04/07/2017 10:01 AM, Axel Kittenberger via rsync wrote: > With this two options on a very live system you may need to take into > account this bug as well I reported a while ago: > >

Re: hard-link reference error when trying to delete

2017-08-15 Thread Kevin Korb via rsync
You should use rrsync for that. On 08/15/2017 08:58 PM, Jared via rsync wrote: > Hi, Kevin. Thank you for the suggestion. It triggered a memory that I > had set some restrictions on this rsync copy a while back. Sure enough, > in ~/.ssh/authorized_keys: > > command="rsync --server

Re: How to set absolute path for rsync?

2017-08-15 Thread Kevin Korb via rsync
With rsyncd you can't. You would have to switch to rsync over ssh... rsync -a --delete --delay-updates --partial-dir=/tmp -P /home/joe/rsync/ 10.148.34.28:/home/joe/rsync/ Also, when you use --partial-dir you probably want to also use --temp-dir. On 08/15/2017 07:19 PM, Joe Qiao via rsync

Re: hard-link reference error when trying to delete

2017-08-15 Thread Kevin Korb via rsync
Have you tried doing the clean shell test? ssh -i /path/to/key u...@remote.server.com true > testfile if testfile isn't 0 bytes then whatever is producing the data that ended up in there is your problem. On 08/15/2017 04:13 PM, jared via rsync wrote: > Hello, all. I've encountered an odd error

Re: Can rsync server mark deleted source files as extraneous?

2017-07-25 Thread Kevin Korb via rsync
There are two different ways to handle this... #1 rsync --link-dest. Each rsync is to a new empty timestamped dir. Files that have not changed are hard linked from the previous run. The result is many timestamps complete trees that can be restored as was at the time of the run. #2 rsync

Re: A small addition to the manpage and "clone mode" suggestion

2017-07-02 Thread Kevin Korb via rsync
Note that --sparse does not guarantee that the files will be the same size. Just that they will be as sparse as rsync can make them regardless of how/if the source file is sparse. OTOH, I would suggest lengthening that line even more by showing the long options. Most of those options have very

Re: rsync -aAXv to a FAT formatted directory?

2017-05-16 Thread Kevin Korb via rsync
A FAT filesystem can't handle -X or -goplD from -a. When you are stuck on FAT you are pretty much limited to -rt instead of -a. Also, it doesn't handle timestamps very well. FAT has a 2 second time resolution and it has no concept of a locale so the timestamps all change whenever your TZ

Re: Bug: rsync erroneously changes modification time

2017-06-11 Thread Kevin Korb via rsync
Whenever you use --times (included in --archive) rsync will fix incorrect time stamps. The only thing --size-only is doing is keeping the incorrect data instead of replacing it. The purpose of these options is to "fix" a copy done in a way that did not preserve timestamps but the data is known

Re: rsync with delete option creates and deletes dot files

2017-05-01 Thread Kevin Korb via rsync
--delete means delete from the target what isn't on the source (except what is excluded). Those look like temp files. Possibly from another rsync that is running at the same time. On 05/01/2017 12:36 AM, Chandana De Silva via rsync wrote: > All, > I have an rsync based backup process which uses

Re: rsync: "-c" option clarification

2017-05-19 Thread Kevin Korb via rsync
original message. > > ____ > From: rsync [rsync-boun...@lists.samba.org] on behalf of Kevin Korb via rsync > [rsync@lists.samba.org] > Sent: Thursday, March 23, 2017 1:10 PM > To: rsync@lists.samba.org > Subject: Re: rsync: "-c" opt

Re: Is it possible to transfer a large, dynamic file in a piecemeal fashion? Yes it is.

2017-09-04 Thread Kevin Korb via rsync
You probably want --inplace for this rather than partial. The upside of --partial is that the existing file isn't replaced until there is a complete file to replace it with. The downside is building up that entire new file. On 09/05/2017 12:01 AM, Don Kuenz via rsync wrote: > Subject: Re: Is it

Re: Should I worry about the "vanished files" warning?

2017-09-07 Thread Kevin Korb via rsync
All it means is that rsync saw a file that needed transferring but the file was gone when rsync actually tried to open it. So look at the filenames and decide if it is a problem or not. On 09/07/2017 10:27 AM, Vangelis Katsikaros via rsync wrote: > Hi > > I would like to ask, when the "vanished

Re: read dir list with write only option

2017-09-06 Thread Kevin Korb via rsync
--list-only (or no target specified) are the only listings without transferring that rsync supports. The restriction would not apply to the list of modules (IIRC there is another option for that). On 09/06/2017 04:00 AM, Pavel Kasparek via rsync wrote: > Hello all, > > when the "write only =

Re: rsync ingest to new storage environment

2017-10-18 Thread Kevin Korb via rsync
If rsync isn't doing the networking you are better off with cp -au instead of rsync. It should be significantly faster and you can do a final pass with rsync to get any files that got truncated by a ^C (cp can only skip files that are newer not files that are not different and a truncated file

Re: Solution for rsync overall progress data display

2017-11-26 Thread Kevin Korb via rsync
--info=progress2 On 11/26/2017 09:27 AM, netbsd--- via rsync wrote: > Hello, > > I looking for a solution to display overall rsync progress on an LCD > display as a bargraph. > I have found 2 parameters: > >   --progress >   This  option  tells  rsync  to  print  information  >

Re: Rsync to just upload new/updated files without comparing source/destination

2017-11-20 Thread Kevin Korb via rsync
The "ssh" in that command is a second source parameter to be copied (and then deleted). using ssh as the transport has been the default for at least 15 years and the option to tell it to use ssh was either -e ssh or --rsh=ssh BTW, -r is included in -a. Also, -v isn't of much use without -i. I

Re: Rsync to just upload new/updated files without comparing source/destination

2017-11-19 Thread Kevin Korb via rsync
If you remove the source files as rsync copies them (--remove-source-files) then rsync will have nothing to compare. It will just go through the tree looking for files that aren't on the target which would probably be any file it finds. Just don't ever use --delete in that scenario. Adding

Re: Rsync to just upload new/updated files without comparing source/destination

2017-11-19 Thread Kevin Korb via rsync
If the source only has 50 files then rsync only needs to check for those 50 files on the target. If one exists rsync will compare the timestamp to see if it needs copying. Yes, rsync verifies a hash of the file after transfer. On 11/19/2017 02:41 PM, Amit via rsync wrote: > Thanks for info. I

Re: some files vanished before... but which?

2017-11-15 Thread Kevin Korb via rsync
That is what rsync says at the end of the run in case you missed the file names fly by in the other output. The names should be in the rest of the output from when rsync found the problem. On 11/15/2017 04:53 AM, devzero--- via rsync wrote: > Hi ! > > I`m getting "rsync warning: some files

Re: [Bug 11414] rsync: chgrp "/.filename.5afK5X" (in dirdir) failed: Operation not permitted (1)

2017-11-06 Thread Kevin Korb via rsync
It isn't running as root (it probably never should). It is launched as root. It is running as tester:tester just like you set it to. On 11/06/2017 10:31 AM, just subscribed for rsync-qa from bugzilla via rsync wrote: > https://bugzilla.samba.org/show_bug.cgi?id=11414 > > --- Comment #2 from

Re: Fwd: rsync ingest to new storage environment

2017-10-20 Thread Kevin Korb via rsync
Simply put, when you network mount rsync operates as a local copy with extra features. --whole-file is forced (and forcing it off makes things worse) and rsync performs worse than cp. If you are only transferring new files then NFS+cp is probably about the same as rsync>rsyncd and probably

Re: is "unintended library loading" actually possible?

2018-06-09 Thread Kevin Korb via rsync
Chroot is not a secure function. If not implemented very carefully it can seriously undermine security rather than enhancing it. If your concern is security maybe you shouldn't be using rsyncd at all. Rsync over ssh works very well and has far superior authentication. It can also be restricted

Re: link-dest and batch-file

2018-06-26 Thread Kevin Korb via rsync
I don't believe there is anything you can do with the batch options for this. If you added a --write-batch to each of those you would get 3 batch files that wouldn't be read without a --read-batch. If you also did a --read-batch that would contain differences between a backup and the backup

Re: link-dest and batch-file

2018-06-26 Thread Kevin Korb via rsync
I don't know how the OP manages their backups. I write out a backupname.current symlink pointing to the new backup once it is completed. That is what I use as the --link-dest parameter and what I would restore from. If a backup is aborted in the middle, doesn't happen at all, or fails the

Re: Getting --delete to work, simply

2018-05-02 Thread Kevin Korb via rsync
--delete only works when you sync a dir not individual files. So... rsync -vai --delete --include='g*' --include='G*' --exclude='*' /share/Public/videos/ /share/Backup-HDD-03 On 05/03/2018 12:08 AM, Mike Brown via rsync wrote: > rsync version 3.1.2 protocol version 31 > > I've run into an

Re: Getting --delete to work, simply

2018-05-03 Thread Kevin Korb via rsync
of all dirs for matching files. On 05/03/2018 02:50 PM, Mike Brown via rsync wrote: > On Thu, May 03, 2018 at 01:09:30AM -0400, Kevin Korb via rsync wrote: >> --delete only works when you sync a dir not individual files. So... >> rsync -vai --delete --include='g*' --include

Re: Yet another filter question

2017-10-26 Thread Kevin Korb via rsync
If a directory is empty (after filtering) then --prune-empty-dirs prunes it. That is the point. The option exists to keep rsync from copying directories when all of the contents of the directory have been filtered out. On 10/25/2017 10:09 PM, dave_g via rsync wrote: > Given the structure… >

Re: downloading only specific directories from directory tree

2018-01-10 Thread Kevin Korb via rsync
1. You don't appear to be including anything that is within the colon dirs so they are being left out by --prune-empty-dirs. 2. Includes only override excludes that follow them. Since your debug excludes are exceptions to the openSUSE_Leap_42.2/*** include they must be before it not after it.

Re: How should I be using rsync with FAT32/VFAT

2018-01-16 Thread Kevin Korb via rsync
#1 yes, you should use -rt instead of -a as everything else in -a is incompatible with vfat. #2 timestamps will always be a problem on vfat. It has a 1 or 2 second resolution so --modify-window=2 is a common solution. However, if you live somewhere that has yearly clock changes (we call it

Re: glob exclude vs include behaviour

2018-01-24 Thread Kevin Korb via rsync
* isn't a shell glob in this case it is a simple pattern wildcard meaning "anything". On 01/24/2018 02:23 PM, Ken Chase via rsync wrote: > not a bug, buy colour me confused: > > /tmp/foo$ mkdir a > /tmp/foo$ touch a/foo > /tmp/foo$ touch a/.baz > /tmp/foo$ cd .. > /tmp$ rsync -avP

Re: downloading only specific directories from directory tree

2018-01-11 Thread Kevin Korb via rsync
o make it clear, for example, > I want to mirror games, games:, graphics, graphics:, hamradio, hamradio:, > hardware, hardware:, only the openSUSE_Leap_42.2 directory containing > paths. > What would be the rules for this? > > Thanks again, > > Istvan > > > &

Re: combining two back ups...

2018-02-22 Thread Kevin Korb via rsync
When in doubt, --dry-run --itemize-changes. In fact, don't even bother with -v and not --itemize-changes. Also, see --update. On 02/22/2018 06:42 AM, Morgan Read via rsync wrote: > Hello list, > > See verbose background below[1] > > I've just backed up my oldmachine that I have been using

Re: What is exit code 5888?

2018-02-22 Thread Kevin Korb via rsync
It is an exit 0 (success) that is being multiplied by 23 for some reason and whatever (shell?) is running rsync doesn't recognize that. On 02/22/2018 02:09 PM, James Moe via rsync wrote: > rsync v3.1.0 > linux v4.4.104-39-default x86_64 > > Found in the system log: > >

Re: What is exit code 5888?

2018-02-22 Thread Kevin Korb via rsync
, if something is running rsync directly and not via a shell wrapping it into a shell would probably solve the problem. On 02/22/2018 03:21 PM, Kevin Korb via rsync wrote: > It is an exit 0 (success) that is being multiplied by 23 for some reason > and whatever (shell?) is running rsync doesn't rec

Re: Troubleshooting RSYNC

2018-08-04 Thread Kevin Korb via rsync
More verbose is really only helpful when debugging include/exclude rules otherwise verbose itself is pretty useless. Use --itemize-changes instead or with just 1 -v. On 08/04/2018 05:28 PM, Leon via rsync wrote: > Greetings, > >   > > I am having a problem with rsync deleting entire folders

Re: Sync multiple devices through a server running rsync in daemon mode

2018-08-14 Thread Kevin Korb via rsync
It sounds like you are assuming that rsync can do a 2-way sync which it can't and you are asking about collision detection which it doesn't have. Maybe you want something like unison or owncloud. Also, if you are doing anything that requires write access or authentication you probably don't want

Re: Aw: Re: link-dest and batch-file

2018-07-20 Thread Kevin Korb via rsync
Correct. Unless you have the resources to handle ZFS's dedup feature (needs tons of RAM) On 07/20/2018 02:50 PM, devz...@web.de wrote: > But don‘t forget —inplace, otherwise snapshots would not be efficient > > >> Gesendet: Mittwoch, 18. Juli 2018 um 21:53 Uhr >> Von:

Re: link-dest and batch-file

2018-07-18 Thread Kevin Korb via rsync
If you are using ZFS then forget --link-dest. Just rsync to the same zfs mount every time and do a zfs snapshot after the rsync finishes. Then delete old backups with a zfs destroy. On 07/18/2018 03:42 PM, Дугин Сергей via rsync wrote: > Hello. > > I need that during today's backup, the

Re: How should I be using rsync with FAT32/VFAT

2018-01-16 Thread Kevin Korb via rsync
> I'll use -rt and maybe ignore timestamps entirely rather than the > window. If I find something wrong on the USB stick I can always copy it > again. I love the -c option, but it takes forever on slow media. > > John > > > On Tue, 2018-01-16 at 13:27 -0500, Kevin Korb via

Re: What does this mean? select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)

2018-01-22 Thread Kevin Korb via rsync
From man 2 select: int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout); So, it is waiting for file descriptor #1 to become available with a 60 second timeout which it is hitting. Use lsof to find out what file descriptor #1 is. On 01/22/2018 10:07

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Kevin Korb via rsync
--no-whole-file would only make it even worse. It would have to read the remote file over the network in order to do the diff then it would write the whole file over the network anyway (--inplace would help a little). Local copies force --whole-file for a good reason. On 03/14/2018 10:05 AM,

Re: Comparison of huge files ineffective?

2018-03-14 Thread Kevin Korb via rsync
is the only part of rsync stupid enough to leave one end idle potentially for hours. On 03/14/2018 10:14 AM, Heiko Schlittermann via rsync wrote: > Kevin Korb via rsync <rsync@lists.samba.org> (Mi 14 Mär 2018 14:52:55 CET): >> Your observation would be right if you are using --che

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Kevin Korb via rsync
. --inplace helps with that somewhat but you will probably still write all of the file after the first difference. On 03/14/2018 10:12 AM, Remi Gauvin via rsync wrote: > On 2018-03-14 10:07 AM, Kevin Korb via rsync wrote: >> --no-whole-file would only make it even worse. It would hav

Re: Comparison of huge files ineffective?

2018-03-14 Thread Kevin Korb via rsync
Your observation would be right if you are using --checksum which you shouldn't be. Otherwise, unless you are using --whole-file rsync will use its differential algorithm to compare the files. If you are using --progress you will see it step through the file at a faster speed than the network is

Re: rsync -- new feature option... (request)

2018-04-01 Thread Kevin Korb via rsync
You can kind of do that with only-write-batch. Then replay the batch file. On April 1, 2018 16:02:14 Andrew McGlashan via rsync wrote: Hi, NB: I am only subscribed because the list doesn't allow non-subscribers to post by default. I find that a bit strange,

Re: Rsync between 2 datacenters not working

2018-03-25 Thread Kevin Korb via rsync
Since your problem is really that your ssh connection is dropping maybe you need more verbosity/debug on the ssh part (possibly even on both ends). Also, if you are using --checksum don't. On 03/25/2018 09:43 AM, Marc Roos via rsync wrote: > > I still stuck with these errors > >

Re: Rsync between 2 datacenters not working

2018-03-25 Thread Kevin Korb via rsync
Note that if you do this you are stuck with --whole-file On 03/25/2018 04:36 PM, Dan Stromberg via rsync wrote: > You could try using an automounter, like autofs, in combination with > sshfs. It'll be slower, possibly a lot slower, but it should be more > reliable over an unreliable connection.

Re: rsync time machine backup permissions

2018-03-19 Thread Kevin Korb via rsync
Is this disk formatted FAT or similar that doesn't support such features? On 03/19/2018 11:16 AM, Andre Althoff via rsync wrote: > sh-3.2# /Users/andre/Desktop/Skripte/rsync_tm > > Kopiere Time Machine Backup ... > > rsync: chgrp "/Volumes/G-DRIVE Thunderbolt >

Re: Very slow to start sync with millions of directories and files

2018-03-19 Thread Kevin Korb via rsync
The performance of rsync with a huge number of files is greatly determined by every option you are using. So, what is your whole command line? On 03/19/2018 09:05 AM, Bráulio Bhavamitra via rsync wrote: > Hi all, >   > I'm using rsync 3 to copy all files from one disk to another. The files >

Re: rsync time machine backup permissions

2018-03-19 Thread Kevin Korb via rsync
What is the error then? On 03/19/2018 10:59 AM, Andre Althoff via rsync wrote: > It doesn‘t work without „—fake-user“. Time Machine is off. The copy process > doesn‘t work. > -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb

Re: rsync time machine backup permissions

2018-03-19 Thread Kevin Korb via rsync
--fake-user means store that metadata in file xattrs rather than on the files themselves. I douby Time Machine understands that. On 03/19/2018 06:22 AM, Andre Althoff via rsync wrote: > Dear rsync users, > > I'm trying to copy my Time Machine Backup from my old USB hard drive to > a new one. I

Re: Very slow to start sync with millions of directories and files

2018-03-20 Thread Kevin Korb via rsync
Nothing there should be preventing incremental indexing. That means it should start copying as soon as it finds a file that needs to be copied. On 03/20/2018 02:33 PM, Bráulio Bhavamitra wrote: > > > Em seg, 19 de mar de 2018 11:34, Kevin Korb via rsync > <rsync@lists.samba.or

Re: Very slow to start sync with millions of directories and files

2018-03-20 Thread Kevin Korb via rsync
s first?  >   > > On 03/20/2018 02:33 PM, Bráulio Bhavamitra wrote: > > > > > > Em seg, 19 de mar de 2018 11:34, Kevin Korb via rsync > > <rsync@lists.samba.org <mailto:rsync@lists.samba.org> > <mailto:rsync@lists.samba.org

Re: Aw: Re: rsync very very slow with multiple instances at the same time.

2018-03-23 Thread Kevin Korb via rsync
storage is'n ssd, is it? there are even more, iscsi is introducing > additional latencies... >   > regards > roland >   >   > *Gesendet:* Freitag, 23. März 2018 um 17:52 Uhr > *Von:* "Jayce Piel via rsync" <rsync@lists.samba.org> > *An:* "Kevin Korb via rsync

Re: rsync very very slow with multiple instances at the same time.

2018-03-21 Thread Kevin Korb via rsync
When rsync has a lot of files to look through but not many to actually transfer most of the work will be gathering information from the stat() function call. You can simulate just the stat call with: find /path -type f -ls > /dev/null You can run one then a few of those to see if your storage has

Re: Security issues when rsyncing directories as root

2018-10-18 Thread Kevin Korb via rsync
Use rrsync. It comes with rsync (some silly Linux distros install it as documentation instead of a helper script so you have to decompress it and chmod +x it). It is a perl script with all the documentation in the comments. Yes, it can be done with rsyncd as you described. The rsyncd.conf file

Re:

2018-10-19 Thread Kevin Korb via rsync
I would highly recommend that you use rsync over ssh for anything that isn't read-only much less anything with root access. Anyway, is there anything in the main section of the config that would interfere? Are there any ACLs or selinux on /data or / ? On 10/19/2018 08:45 AM, Bruce Adams via

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-21 Thread Kevin Korb via rsync
Your rsync must be a bit old. The current --stats output is about twice as long as that. Anyway, there isn't that much change in the file. If you were using rsync over ssh it would probably take around 30 minutes. I would guess cp would take longer but hard to say how much longer but it would

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Kevin Korb via rsync
First, add --stats to find out how much is being transferred. Second, if these are fixed size or sparse image files then --sparse will be a big help. Finally, if they are not fixed size or sparse then --inplace is probably making things worse. --no-whole-file is the default btw unless you are

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Kevin Korb via rsync
I don't see any --stats output. It will tell you exactly how much data was involved. A local copy means that rsync isn't networking. Rsync speeds things up by running on 2 separate computers and communicating using the rsync protocol. If you are running rsync on a single computer with the

Re: Is there a way to get an rsync client to communicate over stdin/stdout?

2018-10-10 Thread Kevin Korb via rsync
Take a look at the lsh scripts that come with rsync. On 10/10/2018 02:53 PM, Parke via rsync wrote: > Hello, > > Is there a way to get an rsync client to communicate over stdin/stdout? > > I found these two emails from 2004. The second email mentions that > there might be a patch. > >

Re: --partial does not "unhide" the files

2018-12-31 Thread Kevin Korb via rsync
I can't say I have any idea why rsync would just skip that step and I can't duplicate it myself. Your only recourse might be to use --inplace on that system. On 12/31/18 12:33 PM, Heiko Schlittermann via rsync wrote: > Kevin Korb via rsync (So 30 Dez 2018 23:56:44 CET): >> I think

Re: --dry-run won't say where the files are going to go

2018-12-30 Thread Kevin Korb via rsync
/tmp/Maildir/new/1546157908.1392_1.jidanni7 Also, don't use -z on a local copy. Rsync is dumb enough to do what you tell it to. On 12/30/18 3:40 AM, 積丹尼 Dan Jacobson via rsync wrote: > Try as we may (--dry-run), it still won't tell us where it intends to put the > files, > > $ rsync --dry-run

Re: Exclude and delete or not delete

2018-12-04 Thread Kevin Korb via rsync
You need to switch to filters where you have the additional options hide, show, risk, and protect. See the filter rules section of man rsync. On 12/4/18 8:36 AM, Fabian Cenedese via rsync wrote: > Hello > > I'm looking for a way to exclude and delete some items but not delete > other items. Use

Re: --link-dest. Time to 'building file list' incrementing

2019-01-03 Thread Kevin Korb via rsync
It does normally take some time to analyze large trees of files. It has to call stat() on each file to get the size and timestamp. However, 15 hours seems a bit excessive even though I have never tried to do this on Windows or a NAS system. Just to be clear, is your --link-dest parameter a

Re: rsync remote raw block device with --inplace

2018-12-30 Thread Kevin Korb via rsync
There have been addons to rsync in the past to do that but rsync really isn't the correct tool for the job. Neither is dd. The right tool is something that understands the filesystem within the block device such as ntfsclone (what I use) or partimage (if you have ever used Clonezilla this is

Re: Aw: Re: rsync remote raw block device with --inplace

2018-12-30 Thread Kevin Korb via rsync
es in sync ? > > maybe these links are interesting in that context: > > https://lists.samba.org/archive/rsync/2010-June/025164.html > > https://github.com/dop251/diskrsync > > roland > >> Gesendet: Sonntag, 30. Dezember 2018 um 19:53 Uhr >> Von: "Kevin Korb

Re: Output of --list-only not as I expected for symlinks

2018-09-12 Thread Kevin Korb via rsync
--list-only is simply a way to list remote files when you only have access via rsync. It ignores the rest of the command line. You want --dry-run (--itemize-changes would be a good idea too). On 09/12/2018 11:06 AM, Michael Hipp via rsync wrote: > The output of --list-only isn't as I expected

Re: Output of --list-only not as I expected for symlinks

2018-09-12 Thread Kevin Korb via rsync
equently. But I was trying to > find something that would give me a correct list of the candidate source > files but without having to make an expensive connect to the remote > server. Any suggestions? > > Thanks, > Michael Hipp > > > > On 2018-09-12 11:17, Kev

Re: Output of --list-only not as I expected for symlinks

2018-09-12 Thread Kevin Korb via rsync
sync's filters/exclusions. > > Thanks, > Michael > > On 2018-09-12 20:54, Kevin Korb via rsync wrote: >> How would it know what needs to be transferred without looking at both >> source and target?  You are correct that it doesn't pay any attention to >> the target b

Re: Ensuring that rsync doesn't try to write to an unmounted drive

2018-09-11 Thread Kevin Korb via rsync
--timeout is about network connection timeouts. You aren't using the network so it doesn't apply at all. Even if you were networking an unmounted filesystem is an empty directory as far as rsync is concerned and rsync would treat it that way with no idea that you intended to have something

Re: Ensuring that rsync doesn't try to write to an unmounted drive

2018-09-11 Thread Kevin Korb via rsync
would suggest adding --itemize-changes as -v isn't very useful without it. I would also suggest looking into rsync's --link-dest as that is the real power of rsync based backups. On 09/11/2018 02:06 PM, Chris via rsync wrote: > On Tue, 2018-09-11 at 12:56 -0400, Kevin Korb via rsync wrote: >>

Re: Possible not expected behaviour with "delete" option

2019-04-04 Thread Kevin Korb via rsync
It doesn't seem like the trailing / on the target should matter. It only means something special on the source. However, if there is any error then rsync will abort the deletions for safety. On 4/3/19 2:10 PM, lugarci1 via rsync wrote: > Hi, I was trying to make a mirror backup of a folder in

Re: --server options

2019-02-07 Thread Kevin Korb via rsync
I don't remember your original question but if I didn't suggest looking at rrsync I should have. It comes with rsync in the contrib dir and it knows which options to allow/deny and it can restrict the transfer to a specific dir and read-only or write-only. Since it comes with rsync it is kept

Re: rsync rewrites all blocks of large files although it uses delta transfer

2019-02-13 Thread Kevin Korb via rsync
With --backup in order to end up with 2 files it has to write out a whole new file. Sure, it only sent the differences (normally that means over the network but there is no network here) but the writing end was told to duplicate the file being updated before updating it. On 2/13/19 10:47 AM,

Re: rsync rewrites all blocks of large files although it uses delta transfer

2019-02-13 Thread Kevin Korb via rsync
It can't do what you want. The closest thing would be --compare-dest. On 2/13/19 5:26 PM, Delian Krustev wrote: > On Wednesday, February 13, 2019 11:29:44 AM EET Kevin Korb via rsync > wrote: >> With --backup in order to end up with 2 files it has to write out a >> whole

Re: Improved output (dry run)

2019-04-14 Thread Kevin Korb via rsync
Having dry-run not act like the same command without --dry-run would make less sense than people occasionally running it and discovering that they need more options. -v doesn't show things it didn't touch. So, it touched them for some reason. Without --itemize-changes you are left in the dark

Re: "WARNING: failed verification -- update retained (will try again)" message

2019-06-03 Thread Kevin Korb via rsync
Usually it means that the file changed while rsync was running. But it can also mean that a hardware problem (usually RAM) is causing the hashing to return bogus results. On 6/3/19 6:09 AM, sambalist--- via rsync wrote: > Hi there! > > On a daily backup script, I am getting a lot of these

Re: rsync many processes and slow backup

2019-07-12 Thread Kevin Korb via rsync
In addition to the cron problems that others have pointed out there are rsync problems... #1 rsync's -c is almost always a bad idea and is extremely slow #2 if rsync isn't networking then -z is a waste of time On 7/12/19 4:56 AM, Hannes Hutmacher via rsync wrote: > Hi all! :-) >   > I have a

Re: --delete-missing-args doesn't delete

2019-04-23 Thread Kevin Korb via rsync
--files-from will delete files from the destination that are listed but not existing in the source. It isn't delete what isn't listed. On 4/23/19 5:29 AM, MI via rsync wrote: > I'm generating a list of files to sync, and would like all the files not > in my list to be deleted from the

Re: Advanced rsync includes and excludes

2019-08-07 Thread Kevin Korb via rsync
I believe you can shorten that to: + /some/very/ + /some/very/deep/ + /some/very/deep/path/ + /some/very/deep/path/to/ + /some/very/deep/path/to/save/*** - /some/* You could also exclude /some and then use /some/very/deep/path/to/save as an additional source. I don't know if rsnapshot can

Re: difference in behaviour of --filter --delete: cmdline vs. file

2019-08-07 Thread Kevin Korb via rsync
It is a little confusing but in order to have an exclude dir merge filter do both things (prevent transfer and prevent deletion) then it needs to be on both ends of the transfer. The file on each end only has partial effect on what rsync does. Think of it like a file you (as a regular user) can

Re: How Can I save the original permissions while setting g+rX for all files

2019-12-08 Thread Kevin Korb via rsync
When I try it the chmod works on both the real permissions and the permissions in the xatttr. Maybe the behavior has changed since whatever version you have (3.1.3 here) but this probably wouldn't help you either. On 12/8/19 7:56 PM, Dimitrios Apostolou via rsync wrote: > Hello list! > >

Re: Seemingly impossible bug: -v not always listing every copied file

2019-10-29 Thread Kevin Korb via rsync
It does seem impossible. I would suggest adding --itemize-changes (-v isn't really all that useful without it anyway). If entries are still missing then I would suspect that either log files are missing (maybe duplicate file names replacing the occasional log file?) or something other than rsync

Re: Skip creating files in --backup-dir if content has not changed

2019-12-18 Thread Kevin Korb via rsync
The reason for that is pretty simple... Rsync isn't reading the existing file to find out that it is the same. Doing so would normally be a waste of rsync's time. A better question is why are your files changing timestamps when the data is the same. On 12/18/19 6:47 PM, Wayne Piekarski via

Re: Exclude-from file format?

2020-01-03 Thread Kevin Korb via rsync
Either format is correct. However, there is no quoting or escaping in the file. So: - filename1 - file name 1 or without the - On 1/3/20 1:45 PM, @lbutlr via rsync wrote: > I have seen two main styles in examples for using rsync with exclude-from. > > The first is simply a list of filename,

Re: Excluding a . directory in a subdirectory

2019-12-28 Thread Kevin Korb via rsync
Order matters. --include overrides --exclude listed later. So, your exclude is nullified by both of your includes. So move the exclude before the includes. Of course then your includes will be irrelevant so just remove them unless you are leaving out details. On 12/28/19 9:15 PM, H via rsync

Re: Would you expect --perms -M--fake-super to set the file mode to the original one?

2020-03-12 Thread Kevin Korb via rsync
t; > On Thursday, March 12, 2020 6:06:34 PM CET, Kevin Korb via rsync wrote: >> Permissions don't require super.  Any place where permissions can't be >> stored certainly can't handle xattrs either.  So, I wouldn't expect >> --fake-super to affect --perms at all. >> >&

Re: Would you expect --perms -M--fake-super to set the file mode to the original one?

2020-03-12 Thread Kevin Korb via rsync
Permissions don't require super. Any place where permissions can't be stored certainly can't handle xattrs either. So, I wouldn't expect --fake-super to affect --perms at all. On 3/12/20 12:46 PM, Dimitrios Apostolou via rsync wrote: > rsync --perms -M--fake-super src dst > > For me, this

Re: How to diff two directories?

2020-05-05 Thread Kevin Korb via rsync
Rsync != diff. However, if that command lists a file something is different about it. add --itemize-changes to find out what. On 5/5/20 8:47 PM, Peng Yu via rsync wrote: > Hi, > > I use this command. But it also shows the files that are the same. Is > there an option that can be specified so

Re: Question/comment about -n (dry run) flag of rsync

2020-03-10 Thread Kevin Korb via rsync
If you used -v then the very last line rsync outputs is: total size is ### speedup is ### (DRY RUN) -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet:

Re: Would you expect --perms -M--fake-super to set the file mode to the original one?

2020-03-16 Thread Kevin Korb via rsync
file mode? > > On Thursday, March 12, 2020 6:26:18 PM CET, Kevin Korb via rsync wrote: >> I would expect that the sending rsync would only send the perms provided >> modified by the --chmod.  I wouldn't expect the receiver to even know >> the other permissions. >> >> On 3

  1   2   >