Re: Delete remote folder

[Fabian Cenedese]

At 10:27 02.06.2016, Fabian Cenedese wrote:
>Content-Transfer-Encoding: 7bit
>
>Hi
>
>I'm trying to build a backup system based on rsync. It should
>work with different destinations (local, share, remote etc) that's
>why I'm looking for ways how every operation can be controlled
>from the client side.
>
>I want to create full backups and incremental backups. For the
>incremental backups I use --compare-dest lastfullbackup. This
>works nicely except that the whole directory tree is created
>even if no files are copied. This is mentioned several times
>on the Internet and the usual answer is that there's no way
>around it because of the way that rsync works. I don't mind
>that they are created if there's a (rsync-)way to delete them
>again afterwards, something like
>
>rsync -a --delete --prune-empty-dirs dest dest
>
>Unfortunately at least one place has to be local, doing a
>remote-remote is not possible.

So far I haven't found a solution to this problem. I've now seen
this release note for the last version 3.1.2:

- Don't create an empty backup dir for a transferred file that doesn't
  exist yet.

I'm wondering now if that would help in my case with --compare-dest.
But I don't have it compiled for Windows. Can anybody confirm if the
newest rsync does not create empty folders anymore?

Thanks

bye  Fabi


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Steven Levine]

In <20160602082722.8F11829233B7@macserver.private>, on 06/02/16
   at 10:27 AM, Fabian Cenedese  said:

Hi Fabian,

>rsync -a --delete empty_local_folder remote_folder_to_delete

>to get rid of the contents. However the folder itself still remains. So
>after a while I'd have a lot of empty folders.

This comes up on the list every now and then.

Here's a snip from an archive message:

  Here is how you would write an "rsyncrm" script that can be invoked like
  "rsyncrm -ni remote::volume/directory".  Note that this reveals the
  attributes of the current directory to the server.

  #!/bin/bash
  victim="${@: -1:1}"
  rsync -d --del --filter="R /$(basename -- "$victim")" --filter='- /*' \
  "${@:1:$#-1}" . "$(dirname -- "$victim")"

This works because it deletes the unwanted directory from the parent
directory.

Steven

-- 
--
"Steven Levine"   Warp/DIY/BlueLion etc.
www.scoug.com www.arcanoae.com www.warpcave.com
--


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Fabian Cenedese]

At 13:22 02.06.2016, you wrote:
>Content-Transfer-Encoding: 7bit
>
>Fabian Cenedese  wrote:
>
>> This script is bash and also uses the "remote shell hacks" using SSH.
>> As I want to run it also from Windows I'm looking for a rsync solution.
>
>Assuming you have control of the server, can you do a bit of semaphore ?
>
>Eg, do your backup with rsync, then when it's complete, touch a file (or put 
>something in it) and rsync that to the server. The server then runs a process 
>that does some action.
>
>Eg, you touch a file "/etc/ivedoneabackup" and sync it, when the server sees 
>that, it runs a process which links/moves/whatever stuff around to suit.
>The client side can be scripted in whatever language is available on that 
>platform.

There are several ways to solve it on the server, from simple cron jobs
to do the cleanup to more sophisticated solutions. But what if I don't
have a server but e.g. just a USB disk? Then it has to be done from
the client. Of course now you have full access to the backups and can
do whatever you want. However that depends again on the operating
system and maybe even file system or so. I don't want to (but maybe
have to) implement various solutions depending on local/remote,
source Windows/Linux, destination Windows/Linux etc.

But thanks for your inputs, I may have to do some more thinking.

bye  Fabi


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Simon Hobson]

Fabian Cenedese  wrote:

> This script is bash and also uses the "remote shell hacks" using SSH.
> As I want to run it also from Windows I'm looking for a rsync solution.

Assuming you have control of the server, can you do a bit of semaphore ?

Eg, do your backup with rsync, then when it's complete, touch a file (or put 
something in it) and rsync that to the server. The server then runs a process 
that does some action.

Eg, you touch a file "/etc/ivedoneabackup" and sync it, when the server sees 
that, it runs a process which links/moves/whatever stuff around to suit.
The client side can be scripted in whatever language is available on that 
platform.


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Simon Hobson]

Fabian Cenedese  wrote:

> Are there pure rsync ways to solve these two problems?

Short answer - no I don't think there is.
My feeling is that rsync (at least, rsync on it's own) isn't the right tool for 
the job.

One think I would comment on though is that, IMO, making backup policies under 
the control of the client is "asking for trouble" for the general case. Your 
use case may well be different, but IME you can't trust clients to do the right 
thing - unless they are completely under your control, in which case there's 
little difference between client or server controlled policies.

Personally I use a combination of rsync to maintain a clone of each client onto 
one server, and then StoreBackup (on that one server) to manage multiple 
generations of backups with file-level dedupe etc.


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Fabian Cenedese]

At 10:52 02.06.2016, Hans-Christian Jehg wrote:
>Content-Transfer-Encoding: 7bit
>
>Try having a look at this, for inspiration.
>
>It does require the capability of hard-links on the remote end though... and 
>it does not resolve your "delete target directory" either, but it might save 
>you some fiddeling around :-)
>
>https://blog.interlinked.org/tutorials/rsync_time_machine.html

Thanks, I already saw that before. I looked for existing backup programs
but none really did what I wanted that's why I'm trying to do it myself.

This script is bash and also uses the "remote shell hacks" using SSH.
As I want to run it also from Windows I'm looking for a rsync solution.

bye  Fabi


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: Delete remote folder

[Hans-Christian Jehg]

Try having a look at this, for inspiration.

It does require the capability of hard-links on the remote end though... 
and it does not resolve your "delete target directory" either, but it 
might save you some fiddeling around :-)


https://blog.interlinked.org/tutorials/rsync_time_machine.html

HC

On 02-06-2016 10:27, Fabian Cenedese wrote:

Hi

I'm trying to build a backup system based on rsync. It should
work with different destinations (local, share, remote etc) that's
why I'm looking for ways how every operation can be controlled
from the client side.

I want to create full backups and incremental backups. For the
incremental backups I use --compare-dest lastfullbackup. This
works nicely except that the whole directory tree is created
even if no files are copied. This is mentioned several times
on the Internet and the usual answer is that there's no way
around it because of the way that rsync works. I don't mind
that they are created if there's a (rsync-)way to delete them
again afterwards, something like

rsync -a --delete --prune-empty-dirs dest dest

Unfortunately at least one place has to be local, doing a
remote-remote is not possible.

The second problem is that I want to delete older backups.
I can use

rsync -a --delete empty_local_folder remote_folder_to_delete

to get rid of the contents. However the folder itself still remains.
So after a while I'd have a lot of empty folders.

As I want this to work on various systems I can't (and don't
want to) use remote shell hacks like "do_something && rsync ".
Also the policy of what and when to delete should be configurable
from the client.

Are there pure rsync ways to solve these two problems?

Thanks

bye  Fabi





--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Delete remote folder

[Fabian Cenedese]

Hi

I'm trying to build a backup system based on rsync. It should
work with different destinations (local, share, remote etc) that's
why I'm looking for ways how every operation can be controlled
from the client side.

I want to create full backups and incremental backups. For the
incremental backups I use --compare-dest lastfullbackup. This
works nicely except that the whole directory tree is created
even if no files are copied. This is mentioned several times
on the Internet and the usual answer is that there's no way
around it because of the way that rsync works. I don't mind
that they are created if there's a (rsync-)way to delete them
again afterwards, something like

rsync -a --delete --prune-empty-dirs dest dest

Unfortunately at least one place has to be local, doing a
remote-remote is not possible.

The second problem is that I want to delete older backups.
I can use

rsync -a --delete empty_local_folder remote_folder_to_delete

to get rid of the contents. However the folder itself still remains.
So after a while I'd have a lot of empty folders.

As I want this to work on various systems I can't (and don't
want to) use remote shell hacks like "do_something && rsync ".
Also the policy of what and when to delete should be configurable
from the client.

Are there pure rsync ways to solve these two problems?

Thanks

bye  Fabi


-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html