Re: [BackupPC-users] BlackoutPeriods seemingly ignored

2011-06-02 Thread martin f krafft
also sprach Holger Parplies  [2011.06.01.2010 +0200]:
> > To get my desired behaviour, I set BlackoutGoodCnt to 0 globally and
> > deleted all global BlackoutPeriods.
[…]
> As I understand the concept of blackouts in BackupPC, I think it
> is meant to automagically adapt backup strategy to avoid the
> (global) blackout window ("working hours") for all hosts that can
> be backed up outside this window. That would be the "global
> blackout" case above.
> 
> Actually, I would modify your suggestion in that I'd set
> BlackoutGoodCnt to 0 *locally* in the pc config file, together
> with a local BlackoutPeriods setting. In the global config,
> BlackoutGoodCnt should be != 0. That way, you would get "expected"
> behaviour for both cases.

Yes, I understand the concept too, and it makes sense for an office.
However, I am using BackupPC chiefly to back up globally spread
machines. There are no office hours really. The only "office hours"
that exist are for the machines behind my DSL link, because I cannot
seem to get bandwidth shaping to work properly. Hence, the backups
should not start during the day but run during the night so that
they do not affect our normal usage.

PS: It's still suboptimal, because now at 23:00 hours, BackupPC
tries to back up all hosts at home, which congests the link
— I think it would be faster and more effective in this case to
back each of them up in series, rather than in parallel.
However, I do not want to set the global limit of concurrent
backups to 1. This is at the heart of the feature suggestion
I made here: http://bugs.debian.org/607761

> > It wouldn't hurt if BackupPC had some sort of MaxAgeBeforeIgnoringBlackout
> > setting to force-ensure some backups.
> 
> If you don't have a backup yet, are you supposed to ignore the blackout window
> or not? You can't really say, "respect it for three days, then ignore it if
> there is still no backup", at least not without defining a concept of when a
> host was added to the BackupPC configuration (timestamp of the host.pl file?).
> In any case, you have the email reminders that are supposed to alert you to
> the fact that there is no recent backup (or no backup at all). If such a case
> happens, you will need to consider whether your blackout window needs
> adjustment anyway. Automatically degrading from one backup per IncrPeriod to
> one backup per MaxAgeBefore... doesn't seem like a sufficient resolution.

Sure, you just forgot half a dozen parameters to allow tweaking of
all aspects. ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"good advice is something a man gives
 when he is too old to set a bad example.
  -- la rouchefoucauld
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see	http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BlackoutPeriods seemingly ignored

2011-06-02 Thread Christian Völker
Hi Martin,

I had a similar issue here with a remote site where I have a couple of
hosts to backup. Only one host at a time should be backuped as otherwise
the slow remote connection will be congested.

What I did was to reduce (or raise?) the ping timeout time. As BackupPC
tries to ping the destination hosts before attempting to back up I know
the ping times to the remote office go up to (let's say) 2,000ms instead
of the default ~100ms.
So BackupPC now pings one remote host, notice a good ping and starts to
backup this guy. Then it attempts to backup a second host on the remote
office, but as the ping time is above 1,500ms this guy is not backed up
and I have only a single host backed up from the remote office.

In parallel it's backing up the local hosts without any issues.

Greetings

Christian




--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


[BackupPC-users] backuppc keeping too many fulls

2011-06-02 Thread Matthew Ayres
I current have backuppc set up as this
 
$Conf{FullPeriod} = '6.97'
$Conf{FullKeepCnt} = ‘1’
$Conf{FullKeepCntMin}=’1’
$Conf{FullAgeMax}=’90’
 
But my system is saving 3 full backups how can I make only keep 1 full 
 
Thanks 
 
 
Matthew Ayres
ProObject Systems Administrator
PH 410-993-1699 Fax 410-993-1691
mgay...@proobject.com--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BlackoutPeriods seemingly ignored

2011-06-02 Thread Tod Detre
On Thu, Jun 2, 2011 at 4:28 AM, martin f krafft  wrote:
> Yes, I understand the concept too, and it makes sense for an office.
> However, I am using BackupPC chiefly to back up globally spread
> machines. There are no office hours really. The only "office hours"
> that exist are for the machines behind my DSL link, because I cannot
> seem to get bandwidth shaping to work properly. Hence, the backups
> should not start during the day but run during the night so that
> they do not affect our normal usage.

Don't forget about rsync's internal '--bwlimit' option. It works great
to keep a single rsync from flooding a connection. The only problem
I've run into with it is the filelist stage is not limited, but the
transfer is.

-- 
Tod

--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backuppc keeping too many fulls

2011-06-02 Thread Bowie Bailey
On 6/2/2011 9:58 AM, Matthew Ayres wrote:
>
> I current have backuppc set up as this
>
> $Conf{FullPeriod} = '6.97'
>
> $Conf{FullKeepCnt} = ‘1’
>
> $Conf{FullKeepCntMin}=’1’
>
> $Conf{FullAgeMax}=’90’
>
> But my system is saving 3 full backups how can I make only keep 1 full
>

BackupPC will always keep the full backups that are needed by the
incrementals. What are your settings for incremental backups?

-- 
Bowie

--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BlackoutPeriods seemingly ignored

2011-06-02 Thread John Rouillard
On Thu, Jun 02, 2011 at 10:31:57AM -0400, Tod Detre wrote:
> On Thu, Jun 2, 2011 at 4:28 AM, martin f krafft  wrote:
> > Yes, I understand the concept too, and it makes sense for an office.
> > However, I am using BackupPC chiefly to back up globally spread
> > machines. There are no office hours really. The only "office hours"
> > that exist are for the machines behind my DSL link, because I cannot
> > seem to get bandwidth shaping to work properly. Hence, the backups
> > should not start during the day but run during the night so that
> > they do not affect our normal usage.
> 
> Don't forget about rsync's internal '--bwlimit' option. It works great
> to keep a single rsync from flooding a connection. The only problem
> I've run into with it is the filelist stage is not limited, but the
> transfer is.

Also you can implement a semaphore/queuing system similar to:

http://www.mail-archive.com/backuppc-users@lists.sourceforge.net/msg13698.html

So only 1 backup runs at a time for a site.

-- 
-- rouilj

John Rouillard   System Administrator
Renesys Corporation  603-244-9084 (cell)  603-643-9300 x 111

--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backuppc keeping too many fulls

2011-06-02 Thread Matthew Ayres
Thanks for the response 

My settings for incremental backups 

$Conf{IncrPeriod} = '2.97'
$Conf{IncrKeepCnt} = 3;
Conf{IncrKeepCntMin} = 1;
$Conf{IncrAgeMax} = 30;


Matthew Ayres
ProObject Systems Administrator
PH 410-993-1699 Fax 410-993-1691
mgay...@proobject.com
-Original Message-
From: Bowie Bailey [mailto:bowie_bai...@buc.com] 
Sent: Thursday, June 02, 2011 10:47 AM
To: backuppc-users@lists.sourceforge.net
Subject: Re: [BackupPC-users] backuppc keeping too many fulls

On 6/2/2011 9:58 AM, Matthew Ayres wrote:
>
> I current have backuppc set up as this
>
> $Conf{FullPeriod} = '6.97'
>
> $Conf{FullKeepCnt} = ‘1’
>
> $Conf{FullKeepCntMin}=’1’
>
> $Conf{FullAgeMax}=’90’
>
> But my system is saving 3 full backups how can I make only keep 1 full
>

BackupPC will always keep the full backups that are needed by the
incrementals. What are your settings for incremental backups?

-- 
Bowie

--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


--
Simplify data backup and recovery for your virtual environment with vRanger. 
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today. 
http://p.sf.net/sfu/quest-sfdev2dev
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backuppc keeping too many fulls

2011-06-02 Thread Bowie Bailey
On 6/2/2011 9:58 AM, Matthew Ayres wrote:

> >
> > I current have backuppc set up as this
> >
> > $Conf{FullPeriod} = '6.97'
> >
> > $Conf{FullKeepCnt} = ‘1’
> >
> > $Conf{FullKeepCntMin}=’1’
> >
> > $Conf{FullAgeMax}=’90’
> >
> > But my system is saving 3 full backups how can I make only keep 1 full
> >
>

On 6/2/2011 10:56 AM, Matthew Ayres wrote:
> Thanks for the response 
>
> My settings for incremental backups 
>
> $Conf{IncrPeriod} = '2.97'
> $Conf{IncrKeepCnt} = 3;
> Conf{IncrKeepCntMin} = 1;
> $Conf{IncrAgeMax} = 30;

So you are doing a full backup every 7 days and an incremental every 3
days. You keep 3 incrementals, so you will have 9 days worth of
incrementals. Your summary screen probably looks something like this:

day 1 - full
day 7 - incr
day 8 - full
day 11 - incr
day 14 - incr
day 15 - full

Since you are keeping 3 incrementals, the first full cannot be deleted
since the "day 7" incremental depends on it. On the next incremental
run, they "day 7" incremental will be removed and the "day 1" full will
be also be removed since it is no longer needed. Your screen will then
look like this:

day 8 - full
day 11 - incr
day 14 - incr
day 15 - full
day 18 - incr

Each time you run an incremental backup, the oldest incremental will be
removed. The full backups can only be removed when there are no more
incrementals left that depend on them. Because of this, you will never
be able to maintain only one full. If you do incrementals more often or
keep fewer of them, you might be able to limit it to no more than 2 fulls.

On the other hand, does it make that much difference? Due to the
pooling, the fulls should not take up any more disk space than the
incrementals, they just take longer to run.

-- 
Bowie

--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today. 
http://p.sf.net/sfu/quest-dev2dev2 
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backuppc keeping too many fulls

2011-06-02 Thread Matthew Ayres
Thanks I will try that out you were a big help


Matthew Ayres
ProObject Systems Administrator
PH 410-993-1699 Fax 410-993-1691
mgay...@proobject.com

-Original Message-
From: Bowie Bailey [mailto:bowie_bai...@buc.com] 
Sent: Thursday, June 02, 2011 11:31 AM
To: backuppc-users@lists.sourceforge.net
Subject: Re: [BackupPC-users] backuppc keeping too many fulls

On 6/2/2011 9:58 AM, Matthew Ayres wrote:

> >
> > I current have backuppc set up as this
> >
> > $Conf{FullPeriod} = '6.97'
> >
> > $Conf{FullKeepCnt} = ‘1’
> >
> > $Conf{FullKeepCntMin}=’1’
> >
> > $Conf{FullAgeMax}=’90’
> >
> > But my system is saving 3 full backups how can I make only keep 1 full
> >
>

On 6/2/2011 10:56 AM, Matthew Ayres wrote:
> Thanks for the response 
>
> My settings for incremental backups 
>
> $Conf{IncrPeriod} = '2.97'
> $Conf{IncrKeepCnt} = 3;
> Conf{IncrKeepCntMin} = 1;
> $Conf{IncrAgeMax} = 30;

So you are doing a full backup every 7 days and an incremental every 3
days. You keep 3 incrementals, so you will have 9 days worth of
incrementals. Your summary screen probably looks something like this:

day 1 - full
day 7 - incr
day 8 - full
day 11 - incr
day 14 - incr
day 15 - full

Since you are keeping 3 incrementals, the first full cannot be deleted
since the "day 7" incremental depends on it. On the next incremental
run, they "day 7" incremental will be removed and the "day 1" full will
be also be removed since it is no longer needed. Your screen will then
look like this:

day 8 - full
day 11 - incr
day 14 - incr
day 15 - full
day 18 - incr

Each time you run an incremental backup, the oldest incremental will be
removed. The full backups can only be removed when there are no more
incrementals left that depend on them. Because of this, you will never
be able to maintain only one full. If you do incrementals more often or
keep fewer of them, you might be able to limit it to no more than 2 fulls.

On the other hand, does it make that much difference? Due to the
pooling, the fulls should not take up any more disk space than the
incrementals, they just take longer to run.

-- 
Bowie

--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today. 
http://p.sf.net/sfu/quest-dev2dev2 
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today. 
http://p.sf.net/sfu/quest-dev2dev2 
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/