On Wed, 2024-01-17 at 14:52 -0500, Default User wrote:
> On Wed, 2024-01-17 at 10:29 -0800, Kushal Kumaran wrote:
> > On Wed, Jan 17 2024 at 11:19:39 AM, Default User
> > <hunguponcont...@gmail.com> wrote:
> > > Hello!
> > >
> > > Opinions, please.
> > >
> > > I use rsync to copy my primary backup drive to a secondary backup
> > > drive
> > > , so that the secondary backup drive is theoretically always an
> > > exact
> > > copy of the primary backup drive.
> > >
> > > Here is the rsync command I use:
> > >
> > > time sudo rsync -aAXHxvv --delete-after --numeric-ids --
> > > info=progress2,stats2,name2 --
> > > exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/m
> > > edia
> > > /*","/lost+found"} /media/default/MSD0001/ /media/default/MSD0002/
> > >
> > > Question:
> > > I use rsync --delete-after because it might seem to be "safer", so
> > > in
> > > case of a "glitch" of any kind, no file ever disappears from both
> > > the
> > > source drive and the destination drive.
> > >
> >
> > What do you mean by "glitch"? Irrespective of whether you use --
> > delete
> > or --delete-after, deleted files on the source are deleted on the
> > destination once your rsync is complete (which is what I'd assume you
> > want when you want an exact copy). I'd presume if you're ok with
> > that,
> > you are also fine with the deletion happening earlier in the rsync
> > process?
> >
> > If you're concerned about accidental deletions, you should just not
> > use
> > any of the `--delete*` options (and give up on the exact copy
> > requirement). You can look at alternatives to bare rsync that keep
> > track of multiple backed-up images (rsnapshot is a very simple
> > wrapper
> > over rsync that can do this, for example).
> >
> > > However, I have read that using rsync --delete instead of rsync --
> > > delete-after is faster and uses less memory, and so is more
> > > efficient.
> > >
> > > Note: The current copy process time varies, but takes a long time -
> > > last night 131 minutes.
> > > :(
> >
> > You can try using --delete for a couple of runs and see if it
> > actually
> > affects performance in your situation.
> >
> > >
> > > Disk space used is not currently an issue.
> > >
> > > But, is rsync --delete AS SAFE as rsync --delete-after?
> >
> > You'll need to define what safety means for you.
> >
>
>
>
> Hi, Kushal!
> Thanks for replying.
>
> By "glitch", I mean anything that could interfere with the rsync copy
> process. Possible causes:
> - electrical outages, voltage spikes, voltage drops, "brownouts"
> - mechanical failure
> - earthquake
> - lightning
> - cat walking on keyboard
> - out of memory errors
> - out of disk space errors
> - PEBKAC errors
> - etc.
>
> By safe, may I try to explain using a story?
>
> I once read that centuries ago, a king wanted his crown to be safe. So
> four guards watched his crown at all times. The guards were not
> allowed to take their eves off of the crown, even for a second, until
> the the their replacements, the next group of guards, all said "I see
> the crown".
>
> I am sorry if I can not come up with a better, technical explanation.
> I use this story because it has always been meaningful to me, and seems
> to point to the essence of what I am getting at.
>
> I am writing as someone who has lost data more than once over time, for
> various reasons. The loss has ranged from slightly annoying, to soul-
> rending catastrophe. It is NEVER appreciated.
>
> I do intend to try doing rsync --delete instead of rsync --delete
> after, to see if it "seems to work".
>
> But I just wanted to ask first ask, as it would seem better to hear
> someone say "How stupid are you? I can't believe you were going to do
> that!", than to have them say "How stupid are you? I can't believe you
> did that!"
>
>
On Wed, 2024-01-17 at 14:52 -0500, Default User wrote:
> [...]
> By "glitch", I mean anything that could interfere with the rsync copy
> process. Possible causes:
> - electrical outages, voltage spikes, voltage drops, "brownouts"
> - mechanical failure
> - earthquake
> - lightning
> - cat walking on keyboard
> - out of memory errors
> - out of disk space errors
> - PEBKAC errors
> - etc.
>
> By safe, may I try to explain using a story?
>
> I once read that centuries ago, a king wanted his crown to be safe. So
> four guards watched his crown at all times. The guards were not
> allowed to take their eves off of the crown, even for a second, until
> the the their replacements, the next group of guards, all said "I see
> the crown".
>
> I am sorry if I can not come up with a better, technical explanation.
> I use this story because it has always been meaningful to me, and seems
> to point to the essence of what I am getting at.
>
> I am writing as someone who has lost data more than once over time, for
> various reasons. The loss has ranged from slightly annoying, to soul-
> rending catastrophe. It is NEVER appreciated.
>
> I do intend to try doing rsync --delete instead of rsync --delete
> after, to see if it "seems to work".
>
> But I just wanted to ask first ask, as it would seem better to hear
> someone say "How stupid are you? I can't believe you were going to do
> that!", than to have them say "How stupid are you? I can't believe you
> did that!"
Always use an UPS.
Always use ECC RAM, especially when having lots of data.
Always use redundancy to store data for a running system, like some
form of RAID. It won't hurt to use RAID for backups as well, though I
don't think that's required when you use it for the data you're
backing up.
Make sure you're notified when a disk in a RAID fails so you can
replace it before more disks fail. Otherwise you won't notice before
it's too late.
Use multiple generations of backups. Store some of them offsite.
I use --delete-before when backup space is tight so it's less likely
to run out of space[1]. If you have plenty of space, you can use
--delete-after. I don't use --delete for backups.
Keep in mind that the data of the running system tends to be more
relevant than the data in backups because the data of the running
system is likely more up to date than the data in backups. Also,
RAID, while _not_ being a substitute for backups, doesn't only protect
your data (which may be replacable) against disk failures, it also
saves you from the hassle, time and effort involved with downtimes and
having to re-aquire the data.
[1]:
I'm assuming that --delete may first copy and then delete, and that it
probably will effectively delete after because it won't delete files
it encounters only later in the process before encountering them,
while it will copy files being encountered when they are found and not
some time later after it first deleted other files to make room for
the encountered ones, which is yet another way in which you can run
out of space.
If you either use --delete-before or --delete-after, at least you know
what you can expect.
And don't ignore extend attributes etc. in the backups.
