Re: prevent filter excluded files from being deleted
Robin Lee Powell via rsync (Di 07 Mär 2023 16:20:39 CET): > First of all, I disagree that rsync handles very large files badly, > but that's not super relevant. :) Consdier --partial for large > files. I'm using --inplace (because the receiving side does snapshots of the underlying file system), which implies --partial. But that's another place I need to work on, as written in one of my previous posts. There I need to investigate a bit more. > (1) my .rsync-filter file has "+ .rsync-filter" in it > (2) --delete-after means that the .rsync-filter is present on *both > sides* when the delete runs Boah… got it and read again "PER-DIRECTORY RULES and DELETE". Now it is clear and works as expected. Thanks a lot for clarification and pointing. -- Heiko signature.asc Description: PGP signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: prevent filter excluded files from being deleted
On Tue, Mar 07, 2023 at 01:16:07PM +0100, Heiko Schlittermann via rsync wrote: > Robin Lee Powell via rsync (Di 07 Mär 2023 07:07:01 > CET): > > Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And > > don't feel bad, it took me a while to figure it out myself). > > I did, but left with some uncertainty. > > "H" hides the files from the transfer? What does it mean? > "P" protects files from being deleted? > > I even tried '+r *', no luck. > > > Short version: per-directory rules only apply on the side they're > > *specified on*, but you need the exclusion to apply to *both* sides. > > Hm. Maybe more reasoning behind. > > I'm syncing a quite large tree, but some directories contain huuge VM images, > which I need to transfer using another mechanism. Because rsync doesn't > seem to handle filesizes of about 100…500GiB well. > > These directories should contain a `.rsync-filter` to prevent them from > the sync, but rsync should not remove these same-named files on the > remote side. First of all, I disagree that rsync handles very large files badly, but that's not super relevant. :) Consdier --partial for large files. Anyway. Note that: (1) my .rsync-filter file has "+ .rsync-filter" in it and (2) --delete-after means that the .rsync-filter is present on *both sides* when the delete runs Exclusion on the *sender* side means "don't send this"; exclusion on the *receiver* side means "don't delete this". *Normally*, they're the same, like if you do --exclude=foo in the command, that affects *both* the sender and receeiver side, but with -F, the sender side uses the .rsync-filter in its directory, and the receiver side uses the .rsync-filter in *its* directory. So if you want the files to be excluded from sending *and* deleting, the .rsync-filter that excludes the files has to be present on *both* sides. You can do that by just manually making them in both places, or by copying what I had in my email. > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: prevent filter excluded files from being deleted
Hello Kevin, Kevin Korb via rsync (Di 07 Mär 2023 00:01:27 CET): > I am not 100% sure I am interpreting this correctly but I think you are > complaining that the file was being deleted in the first command? If so, > instead of -F try --include='*/' --exclude='*'. Otherwise, maybe you want a > second -F? For the command line it works, but I'd like to have in a .rsync-filter on a per directory basis. > > a > > ├── a-file > > └── .rsync-filter > > > > b > > └── a-file Following your suggestion, I tried: # .rsync-filter + */ - * combined with this command rsync --del -F -inav a/ b/ sending incremental file list *deleting a-file .d..t.. ./ Your suggestion (include/exclude on the command line) would work, but as mentioned, it is not on a per directory basis rsync --del --include='*/' --exclude='*' -inav a/ b/ sending incremental file list .d..t.. ./ Even rsync --del --filter='- *' -inav a/ b/ sending incremental file list .d..t.. ./ does not remove the excluded files So, why does rsync remove the files excluded via the .rsync-filter file, but doesn't touch the files excluded with command line filters? -- Heiko signature.asc Description: PGP signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: prevent filter excluded files from being deleted
Robin Lee Powell via rsync (Di 07 Mär 2023 07:07:01 CET): > Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And > don't feel bad, it took me a while to figure it out myself). I did, but left with some uncertainty. "H" hides the files from the transfer? What does it mean? "P" protects files from being deleted? I even tried '+r *', no luck. > Short version: per-directory rules only apply on the side they're > *specified on*, but you need the exclusion to apply to *both* sides. Hm. Maybe more reasoning behind. I'm syncing a quite large tree, but some directories contain huuge VM images, which I need to transfer using another mechanism. Because rsync doesn't seem to handle filesizes of about 100…500GiB well. These directories should contain a `.rsync-filter` to prevent them from the sync, but rsync should not remove these same-named files on the remote side. -- Heiko signature.asc Description: PGP signature -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: prevent filter excluded files from being deleted
Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And
don't feel bad, it took me a while to figure it out myself).
Short version: per-directory rules only apply on the side they're
*specified on*, but you need the exclusion to apply to *both* sides.
The following works, for the reasons specified there:
rlpowell@stodi> find a b -ls
703572129 0 drwxr-xr-x 2 rlpowell rlpowell 39 Mar 6 22:06 a
704372871 0 -rw-r--r-- 1 rlpowell rlpowell0 Mar 6 21:56 a/file
704372873 4 -rw-r--r-- 1 rlpowell rlpowell 20 Mar 6 22:06
a/.rsync-filter
703572158 0 drwxr-xr-x 2 rlpowell rlpowell 39 Mar 6 22:06 b
704372870 0 -rw-r--r-- 1 rlpowell rlpowell0 Mar 6 22:06 b/file
704372881 4 -rw-r--r-- 1 rlpowell rlpowell 20 Mar 6 22:06
b/.rsync-filter
rlpowell@stodi> cat a/.rsync-filter
+ .rsync-filter
- *
rlpowell@stodi> rsync -F --delete-after -iav a/ b/
building file list ... done
sent 102 bytes received 11 bytes 226.00 bytes/sec
total size is 20 speedup is 0.18
rlpowell@stodi>
On Mon, Mar 06, 2023 at 10:04:52PM +0100, Heiko Schlittermann via rsync wrote:
> Hello,
>
> given are 2 directories:
>
> a
> ├── a-file
> └── .rsync-filter
>
> b
> └── a-file
>
> I'd like to sync a/ -> b/, but I'd like to *exclude* all files. But I do
> not want to delete the excluded files. (The real scenario is a way more
> complex, the above is my reproducer.)
>
> and the following rsync command: `rsync -F --del a/ b/`
> Note, there is *no* `--delete-excluded`:
>
> $ rsync -F --del -inav a/ b/
> sending incremental file list
> *deleting a-file
> .d..t.. ./
>
> sent 62 bytes received 29 bytes 182.00 bytes/sec
> total size is 0 speedup is 0.00 (DRY RUN)
>
> Why? Because it is excluded? Using the `--exclude` option does what I
> expect:
>
> $ rsync --exclude 'a-file' --del -inav a/ b/
> sending incremental file list
> .d..t.. ./
> >f+ .rsync-filter
>
> sent 92 bytes received 22 bytes 228.00 bytes/sec
> total size is 4 speedup is 0.04 (DRY RUN)
>
> even does what I expect:
>
> $ rsync --filter '- *' --del -inav a/ b/
> sending incremental file list
> .d..t.. ./
>
> sent 58 bytes received 19 bytes 154.00 bytes/sec
> total size is 0 speedup is 0.00 (DRY RUN)
>
> Version:
> rsync version 3.2.3 protocol version 31
>
> Best regards from Dresden/Germany
> Viele Grüße aus Dresden
> Heiko Schlittermann
> --
> SCHLITTERMANN.de internet & unix support -
> Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
> gnupg encrypted messages are welcome --- key ID: F69376CE -
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: prevent filter excluded files from being deleted
I am not 100% sure I am interpreting this correctly but I think you are
complaining that the file was being deleted in the first command? If
so, instead of -F try --include='*/' --exclude='*'. Otherwise, maybe
you want a second -F?
On 3/6/23 16:04, Heiko Schlittermann via rsync wrote:
Hello,
given are 2 directories:
a
├── a-file
└── .rsync-filter
b
└── a-file
I'd like to sync a/ -> b/, but I'd like to *exclude* all files. But I do
not want to delete the excluded files. (The real scenario is a way more
complex, the above is my reproducer.)
and the following rsync command: `rsync -F --del a/ b/`
Note, there is *no* `--delete-excluded`:
$ rsync -F --del -inav a/ b/
sending incremental file list
*deleting a-file
.d..t.. ./
sent 62 bytes received 29 bytes 182.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)
Why? Because it is excluded? Using the `--exclude` option does what I
expect:
$ rsync --exclude 'a-file' --del -inav a/ b/
sending incremental file list
.d..t.. ./
>f+ .rsync-filter
sent 92 bytes received 22 bytes 228.00 bytes/sec
total size is 4 speedup is 0.04 (DRY RUN)
even does what I expect:
$ rsync --filter '- *' --del -inav a/ b/
sending incremental file list
.d..t.. ./
sent 58 bytes received 19 bytes 154.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)
Version:
rsync version 3.2.3 protocol version 31
Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --- key ID: F69376CE -
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone:(407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. ke...@futurequest.net (work)
Orlando, Floridak...@sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
prevent filter excluded files from being deleted
Hello,
given are 2 directories:
a
├── a-file
└── .rsync-filter
b
└── a-file
I'd like to sync a/ -> b/, but I'd like to *exclude* all files. But I do
not want to delete the excluded files. (The real scenario is a way more
complex, the above is my reproducer.)
and the following rsync command: `rsync -F --del a/ b/`
Note, there is *no* `--delete-excluded`:
$ rsync -F --del -inav a/ b/
sending incremental file list
*deleting a-file
.d..t.. ./
sent 62 bytes received 29 bytes 182.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)
Why? Because it is excluded? Using the `--exclude` option does what I
expect:
$ rsync --exclude 'a-file' --del -inav a/ b/
sending incremental file list
.d..t.. ./
>f+ .rsync-filter
sent 92 bytes received 22 bytes 228.00 bytes/sec
total size is 4 speedup is 0.04 (DRY RUN)
even does what I expect:
$ rsync --filter '- *' --del -inav a/ b/
sending incremental file list
.d..t.. ./
sent 58 bytes received 19 bytes 154.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)
Version:
rsync version 3.2.3 protocol version 31
Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --- key ID: F69376CE -
signature.asc
Description: PGP signature
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
