Re: [BackupPC-users] V4 Infinite Incrementals

[G.W. Haywood via BackupPC-users]

Hi there,

On Sat, 25 Sep 2021, Kenneth Porter wrote:


On 9/24/2021 8:25 AM, Stan Larson wrote:



BTW, I'm also running weekly air-gap backups to external media that
is stored offsite.? Having critical data backed up both locally and
offsite is reassuring, but no guarantee against an organized
ransom-ware attack.


The incremental nature of BackupPC should address the ransomware
issue, provided your backups (and your air gap swap) go back before
the incident.


Also provided that you don't do insane things like permitting shell
access to the backup server from client boxes.

--

73,
Ged.


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] V4 Infinite Incrementals

[Kenneth Porter]

On 9/24/2021 8:25 AM, Stan Larson wrote:

BTW, I'm also running weekly air-gap backups to external media that is
stored offsite.  Having critical data backed up both locally and offsite
is reassuring, but no guarantee against an organized ransom-ware attack.


The incremental nature of BackupPC should address the ransomware issue, 
provided your backups (and your air gap swap) go back before the 
incident. So the hard part is having enough air gap backup media to go 
back to that point in time.





___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] V4 Infinite Incrementals

[Stan Larson]

Dave,

Thanks for responding.

I am using rsync for all backups.

My experience has matched yours in regards to the first full backup of a
large server.  The initial backup can sometimes take several days.  In
my case, subsequent full backups were/are also taking multiple hours and
sometimes over a day to complete.

After reviewing your comments and re-reading the documentation, I now
understand the difference between the FullPeriod and FillCycle
settings.  I've pushed my FullPeriod out to 100 days, and set my
FillCycle to 7 days.

For simplicity, I didn't mention in my OP that I have two BackupPC
servers running at two different physical locations.  For critical
client systems, I'm backing up locally and offsite. Since the BPC
servers are independent of each other, occasionally both BPC servers
would concurrently run full backups up the same client system.  This was
where I was seeing the most significant impact to the performance of the
client systems.  By changing the full backup schedule from 1 week to 100
days, I hope to reduce this situation from happening.

BTW, I'm also running weekly air-gap backups to external media that is
stored offsite.  Having critical data backed up both locally and offsite
is reassuring, but no guarantee against an organized ransom-ware attack.


On 9/24/21 3:07 AM, Dave Sherohman wrote:

On 9/23/21 7:09 PM, Stan Larson wrote:


A few of the servers that are being backed up take many hours to run
a full backup, which can modestly impact the end users of those
servers.  Currently my FullPeriod value is set at the default 6.97
days.  Since V4 uses reverse-delta, I should be able to stretch out
the FullPeriod to a much longer length of time. What is a reasonable
FullPeriod value?


You didn't state what your XferMethod is, but, based on what I've seen
with BPC4, this kind of optimization is not necessary when using
ssh+rsync backups.  I have hosts which took 5-6 days to complete their
initial full backups, but subsequent full backups finish in a matter
of minutes.

My assumption is that this is because, even though the fulls examine
and compare checksums for every file on the host, rsync still only
transfers changed data, unlike other transport methods which need to
send the entire content over the network regardless of whether it is
changed or not.


BTW, FillPeriod is set to 0.  My understanding is that the only
difference between using FullPeriod and FillPeriod is
cosmetic/verbage.  Is that correct?


Not really.  A full backup and a filled backup aren't the same thing.
Every full backup is filled, but the reverse is not the case.

A full backup makes a complete copy of the target host to the backup
server (modulo rsync optimizations for unchanged data).

A filled backup has entries in the backup server's index for every
file on the target host.  This naturally happens with a full, because
it gets every file as part of the backup, but an incremental can also
be filled by adding index entries for any files which aren't included
in the incremental, to produce an index that "looks like" a full
backup, even though it isn't.  This is basically a way to address the
tradeoff you mentioned between backup times and restore times, since
restoring from an incremental only needs to look back to the most
recent filled backup instead of the most recent full.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

All commodities purchased from Freedom Sales are to be handled in accordance 
with US law including but not limited to the Export Administration Regulations, 
International Traffic in Arms Regulations, US Department of State, US 
Department of Homeland Security, US Department of Commerce, and US Office of 
Foreign Assets Control. Diversion Contrary to US law is prohibited.


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] V4 Infinite Incrementals

[Dave Sherohman]

On 9/23/21 7:09 PM, Stan Larson wrote:

A few of the servers that are being backed up take many hours to run a 
full backup, which can modestly impact the end users of those 
servers.  Currently my FullPeriod value is set at the default 6.97 
days.  Since V4 uses reverse-delta, I should be able to stretch out 
the FullPeriod to a much longer length of time. What is a reasonable 
FullPeriod value?


You didn't state what your XferMethod is, but, based on what I've seen 
with BPC4, this kind of optimization is not necessary when using 
ssh+rsync backups.  I have hosts which took 5-6 days to complete their 
initial full backups, but subsequent full backups finish in a matter of 
minutes.


My assumption is that this is because, even though the fulls examine and 
compare checksums for every file on the host, rsync still only transfers 
changed data, unlike other transport methods which need to send the 
entire content over the network regardless of whether it is changed or not.


BTW, FillPeriod is set to 0.  My understanding is that the only 
difference between using FullPeriod and FillPeriod is 
cosmetic/verbage.  Is that correct?


Not really.  A full backup and a filled backup aren't the same thing.  
Every full backup is filled, but the reverse is not the case.


A full backup makes a complete copy of the target host to the backup 
server (modulo rsync optimizations for unchanged data).


A filled backup has entries in the backup server's index for every file 
on the target host.  This naturally happens with a full, because it gets 
every file as part of the backup, but an incremental can also be filled 
by adding index entries for any files which aren't included in the 
incremental, to produce an index that "looks like" a full backup, even 
though it isn't.  This is basically a way to address the tradeoff you 
mentioned between backup times and restore times, since restoring from 
an incremental only needs to look back to the most recent filled backup 
instead of the most recent full.




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] V4 Infinite Incrementals

[Guillermo Rozas]

>
> Lost data is obviously a no-go, but if the reverse-delta process is
> reliable, I would prefer to minimize the full backups to a few times a year.
>

One point you should consider in planning the schedule is that full backups
thoroughly check the source for changes (using checksum for rsync). So,
even if the reverse-delta for incrementals is reliable, if the change was
not picked up because it did not change metadata (size and time for rsync)
it will be lost.

The checksum has also the added benefit of checking for corruption for
"unchanged" files.

Regards,
Guillermo
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

[BackupPC-users] V4 Infinite Incrementals

[Stan Larson]

Freedom  
Stan Larson | IT Manager
Freedom | www.freedomsales.com 
11225 Challenger Avenue Odessa, FL 33556
PH: 813-855-2671 x206 |
Direct Line: 1-727-835-1157



Hi,

I've been using BackupPC for many years.  I'm currently using V4.

A few of the servers that are being backed up take many hours to run a 
full backup, which can modestly impact the end users of those servers.  
Currently my FullPeriod value is set at the default 6.97 days.  Since V4 
uses reverse-delta, I should be able to stretch out the FullPeriod to a 
much longer length of time. What is a reasonable FullPeriod value?


The documentation mentions infinite incrementals.  Is this realistic?  
If I go with a very large FullPeriod, let's say 1000, do I need to 
change any of my incremental values such as IncrKeepCnt or IncrAgeMax?


I realize that there are trade-offs between backup times and restore 
times.  I've never had to do a full restore, and rarely have to do large 
restores, so I'm OK with a longer restore time if it can keep my backup 
times faster.  Lost data is obviously a no-go, but if the reverse-delta 
process is reliable, I would prefer to minimize the full backups to a 
few times a year.


BTW, FillPeriod is set to 0.  My understanding is that the only 
difference between using FullPeriod and FillPeriod is cosmetic/verbage.  
Is that correct?


Thanks for all the great work that has gone into creating, maintaining, 
and supporting BackupPC.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/