[BackupPC-users] Queue Order / Queue Pools
Hello, we backup several Servers with BackupPC - on every server we got several Directories - everyone as an own Host ... (dir1, dir2, dir3, ... and mapped those names to the IP in /etc/hosts) Is it possible that backuppc allows you to put your hosts into groups (aka queues) so that you could limit the number of backup jobs running from any specific queue/host group? Thank you Andreas Schnederle-Wagner Futureweb OG Innsbrucker Str. 4 6380 St. Johann schnede...@futureweb.atmailto:schnede...@futureweb.at www.futureweb.athttp://www.futureweb.at www.ortsinfo.athttp://www.ortsinfo.at Fon: +43 (0) 5352 65335-0 Fax: +43 (0) 5352 65335-777 Gratis über Skype anrufen | Skype-ID: futureweb -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
On Thu, Aug 28, 2014 at 7:01 AM, Andreas Schnederle-Wagner - Futureweb.at schnede...@futureweb.at wrote: Is it possible that backuppc allows you to put your hosts into groups (aka queues) so that you could limit the number of backup jobs running from any specific queue/host group? No you can't do that by groups, but unless you have a mix of fast and slow networks/hosts you will probably get the best performance by setting the overall concurrency (MaxBackups) to 2 or so since you have the same problem server-side. You can have some control over timing by setting the blackout periods differently and you can initially skew them by doing a 'stop' (even if a backup isn't running) and setting a number of hours to defer the next run.The approximately 24-hour scheduling will maintain the skew afterward. -- Les Mikesell lesmikes...@gmail.com -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
This is a feature I've wanted as well. I have virtual machines running on multiple hypervisors, and I'd prefer to only run one backup per hypervisor at a time. If I could do that, I could get away with backups during the day... but if I back up the main file server and the groupware server at the same time, tech support starts getting calls about things being slow. On 08/28/2014 08:03 AM, Les Mikesell wrote: On Thu, Aug 28, 2014 at 7:01 AM, Andreas Schnederle-Wagner - Futureweb.at schnede...@futureweb.at wrote: Is it possible that backuppc allows you to put your hosts into groups (aka queues) so that you could limit the number of backup jobs running from any specific queue/host group? No you can't do that by groups, but unless you have a mix of fast and slow networks/hosts you will probably get the best performance by setting the overall concurrency (MaxBackups) to 2 or so since you have the same problem server-side. You can have some control over timing by setting the blackout periods differently and you can initially skew them by doing a 'stop' (even if a backup isn't running) and setting a number of hours to defer the next run.The approximately 24-hour scheduling will maintain the skew afterward. -- Carl D Cravens (ccrav...@excelii.com), Ext 228 (620.327.1228) Lead System Architect -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
On Thu, Aug 28, 2014 at 8:11 AM, Carl Cravens ccrav...@excelii.com wrote: This is a feature I've wanted as well. I have virtual machines running on multiple hypervisors, and I'd prefer to only run one backup per hypervisor at a time. If I could do that, I could get away with backups during the day... but if I back up the main file server and the groupware server at the same time, tech support starts getting calls about things being slow. Have you tried taking MaxBackups down to 1 to see if backups will still complete in the available time? Or skewing the blackout periods for the host and guest systems? -- Les Mikesell lesmikes...@gmail.com -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
I can't complete all the backups in a day with a maxbackups of 1. It's unreasonable to micromanage the blackout periods for 70+ hosts to keep certain ones from overlapping. I've coped with the problem, but it's something that an only-one-host-at-a-time-from-this-group feature would make a lot easier to deal with. On 08/28/2014 09:45 AM, Les Mikesell wrote: On Thu, Aug 28, 2014 at 8:11 AM, Carl Cravens ccrav...@excelii.com wrote: This is a feature I've wanted as well. I have virtual machines running on multiple hypervisors, and I'd prefer to only run one backup per hypervisor at a time. If I could do that, I could get away with backups during the day... but if I back up the main file server and the groupware server at the same time, tech support starts getting calls about things being slow. Have you tried taking MaxBackups down to 1 to see if backups will still complete in the available time? Or skewing the blackout periods for the host and guest systems? -- Carl D Cravens (ccrav...@excelii.com), Ext 228 (620.327.1228) Lead System Architect -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
On Thu, Aug 28, 2014 at 10:36 AM, Carl Cravens ccrav...@excelii.com wrote: I can't complete all the backups in a day with a maxbackups of 1. It's unreasonable to micromanage the blackout periods for 70+ hosts to keep certain ones from overlapping. I've coped with the problem, but it's something that an only-one-host-at-a-time-from-this-group feature would make a lot easier to deal with. Oh, I agree that it would be a very useful feature. I've wanted it myself to accommodate a mix of remote and local targets where I'd like to group by the network paths. -- Les Mikesell lesmikes...@gmail.com -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
On Thu, Aug 28, 2014 at 10:36:50AM -0500, Carl Cravens wrote: I can't complete all the backups in a day with a maxbackups of 1. It's unreasonable to micromanage the blackout periods for 70+ hosts to keep certain ones from overlapping. I've coped with the problem, but it's something that an only-one-host-at-a-time-from-this-group feature would make a lot easier to deal with. You may also want to see my post for how I dealt with it using an external queing/semaphore mechanism. It allows one or multiple hosts from a group to run in parallel. http://sourceforge.net/p/backuppc/mailman/backuppc-users/thread/20090319212300.gc20...@renesys.com/ -- -- rouilj John Rouillard System Administrator Dyn Corporation 603-244-9084 (cell) 603-643-9300 x 111 -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Queue Order / Queue Pools
On 29/08/14 01:36, Carl Cravens wrote:
I can't complete all the backups in a day with a maxbackups of 1. It's
unreasonable to micromanage the blackout periods for 70+ hosts to keep
certain ones from overlapping.
I've coped with the problem, but it's something that an
only-one-host-at-a-time-from-this-group feature would make a lot easier to
deal with.
This has come up a few times, and I've just had a thought (a dangerous
prospect) but in combination with the other thread about variables in
config files, it might work.
Can the perl code embedded in the host config files create (read and
modify) a global variable which will be available to another hosts
config file.
In other words, something like this in both of host1 and host2 to make
sure that these two don't run at the same time:
if(++$myGroup 1)
{
$myGroup--;
exit 1;
}
Failing that, I suppose you would need to resort to reading/writing the
value into a file, which could be as simple as (in sh script because I
can't be bothered to get correct syntax for perl):
if [ -f /somedir/myGroup ]
then
exit 1
else
echo $pid /somedir/myGroup
# Could ensure that the entire content of myGroup is the pid (ie,
another host didn't add it's pid in a race condition)
fi
Also need to ensure that exit 1 is even a valid method to say don't
backup now, another option might be to set the pingcmd to /bin/false or
similar.
Regards,
Adam
On 08/28/2014 09:45 AM, Les Mikesell wrote:
On Thu, Aug 28, 2014 at 8:11 AM, Carl Cravens ccrav...@excelii.com wrote:
This is a feature I've wanted as well. I have virtual machines running on
multiple hypervisors, and I'd prefer to only run one backup per hypervisor
at a time. If I could do that, I could get away with backups during the
day... but if I back up the main file server and the groupware server at
the same time, tech support starts getting calls about things being slow.
Have you tried taking MaxBackups down to 1 to see if backups will
still complete in the available time? Or skewing the blackout
periods for the host and guest systems?
--
Adam Goryachev Website Managers www.websitemanagers.com.au
--
Slashdot TV.
Video for Nerds. Stuff that matters.
http://tv.slashdot.org/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
