Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-18 Thread Kern Sibbald 
On 11/03/2015 07:24 AM, Wesley Render wrote:
> I have recently started using Bacula and have a couple of questions  
> regarding volume pools.  I am using 7.2 version, and we have 4 Linux  
> servers with a total of about 100GB of data to backup.
>
> 1.  We have two different storage devices that are in different  
> locations (because of bandwidth limitations).  Should we be creating  
> different Volume Pools for each storage location?

This is not required but would be easier to manage.  What is *required*
is to have different MediaType for each
Storage Device.

Best regards,
Kern

>
> I've tried testing using one pool, and two pools.  When I use one pool  
> and a backup job is run it displays an error "Marking Volume  
> "Vol-0001" in Error in Catalog." and then it continues to run.   When  
> I setup two different volume pools it doesn't display this error.
>
> 2.   I've noticed that some people recommend setting up different  
> volume pools for Full, Differential and Incremental jobs. Is this  
> still a recommended strategy for Bacula with backing up to disk, and  
> if so when would someone use this strategy?
>
> The documentation here  
> http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf doesn't  
> mention this.  I don't want to set things up, and then our volumes  
> grow too large and have to re-do everything.
>
>
> Thank you,
>


--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-05 Thread Wesley Render 

In case this helps anyone. I ended up having problems with setting  
volume limits. For example I suddenly decided to adjust our  
incremental backups to run every 4 hours, and started reaching volume  
limits with errors.

 From what I have read in the documentation setting "Volume Use  
Duration" will effectively cause new volumes to be created, and old  
ones to be recycled based on the Volume Retention.  Since I monitor  
servers disk space and this is just disk based backup, if I get an  
alert that the storage space on the backup space is filling up I will  
then look at reducing the retention periods on the volume sets.

Here is what I have so far:

Pool {
   Name = office-p-monthly
   Pool Type = Backup
   Volume Retention = 6 months
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = office-p-monthly-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}
Pool {
   Name = office-p-weekly
   Pool Type = Backup
   Volume Retention = 1 months
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = office-p-weekly-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}
Pool {
   Name = office-p-daily
   Pool Type = Backup
   Volume Retention = 14 days
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = office-p-daily-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}
Pool {
   Name = datacenter-p-monthly
   Pool Type = Backup
   Volume Retention = 6 months
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = datacenter-p-monthly-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}
Pool {
   Name = datacenter-p-weekly
   Pool Type = Backup
   Volume Retention = 1 months
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = datacenter-p-weekly-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}
Pool {
   Name = datacenter-p-daily
   Pool Type = Backup
   Volume Retention = 14 days
   Recycle = yes
   AutoPrune = yes
   Action On Purge = Truncate
   LabelFormat = datacenter-p-daily-
   Volume Use Duration = 23h
   Maximum Volume Bytes = 100G
}

Here are samples of the jobs:

Job {
   Name = web221-domainname
   Type = Backup
   Level = Incremental
   Client = web221.domainname.com-fd
   FileSet = OurFileSet
   Schedule = WeeklyCycle
   Storage = horde-sd
   Pool = Default
   Full Backup Pool = datacenter-p-monthly
   Incremental Backup Pool = datacenter-p-daily
   Differential Backup Pool = datacenter-p-weekly
   Accurate = Yes
   Messages = Standard
}

Job {
   Name = web220-domainname
   Type = Backup
   Level = Incremental
   Client = web220.domainname.com-fd
   FileSet = OurFileSet
   Schedule = WeeklyCycle
   Storage = office-sd
   Pool = Default
   Full Backup Pool = office-p-monthly
   Incremental Backup Pool = office-p-daily
   Differential Backup Pool = office-p-weekly
   Accurate = Yes
   Messages = Standard
}



-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-05 Thread Dimitri Maziuk 
On 11/05/2015 11:02 AM, Wesley Render wrote:
> 
> In case this helps anyone. I ended up having problems with setting  
> volume limits. For example I suddenly decided to adjust our  
> incremental backups to run every 4 hours, and started reaching volume  
> limits with errors.
> 
>  From what I have read in the documentation setting "Volume Use  
> Duration" will effectively cause new volumes to be created, and old  
> ones to be recycled based on the Volume Retention.  Since I monitor  
> servers disk space and this is just disk based backup, if I get an  
> alert that the storage space on the backup space is filling up I will  
> then look at reducing the retention periods on the volume sets.

FWIW I've been using

Pool {
...
Maximum Volumes = X
Maximum Volume Bytes = Y
Recycle = yes
AutoPrune = yes
Purge Oldest Volume = yes
Recycle Oldest Volume = yes
}

(where X*Y = 90-ish% of storage disk size) for years without problems.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-04 Thread Josh Fisher 


On 11/3/2015 6:38 PM, Wesley Render wrote:
> Should each storage daemon/geographic storage location have it's own
> set of Volume Pools?  Or can I share one set of Volume Pools between
> all of the storage daemons/storage locations?

You may also use Media Type to tie volumes to particular devices. For 
example, if disk storage is being used at both locations, then setting 
the Media Type for the Device at location 1 to "File-1" and the Media 
Type for the Device at location 2 to "File-2", then volumes written at 
both sites can belong to the same pool. When Bacula selects a volume 
from a pool, it will not select a volume that does not have the same 
Media Type as the Storage resource being used by the job.

> I am  using auto labelling as well and it works great.
>
>
>


--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-04 Thread Wesley Render 
It seems to be working a lot better with different volume pools at  
each storage location. I have one server backing up to the server at  
our office, and 2 servers backing up to the datacenter. Here is what I  
have so far:

Pool {
   Name = office-p-monthly
   Pool Type = Backup
   Volume Retention = 6 months
   Recycle = yes
   AutoPrune = yes
   LabelFormat = office-p-monthly-
   Maximum Volume Jobs = 1
   Maximum Volumes = 9
}

Pool {
   Name = office-p-weekly
   Pool Type = Backup
   Maximum Volume Jobs = 1
   Volume Retention = 1 months
   Recycle = yes
   AutoPrune = yes
   LabelFormat = office-p-weekly-
   Maximum Volumes = 7
}
Pool {
   Name = office-p-daily
   Pool Type = Backup
   Maximum Volume Jobs = 6
   Volume Retention = 14 days
   Recycle = yes
   AutoPrune = yes
   LabelFormat = office-p-daily-
   Maximum Volumes = 6
}
Pool {
   Name = datacenter-p-monthly
   Pool Type = Backup
   Volume Retention = 6 months
   Recycle = yes
   AutoPrune = yes
   LabelFormat = datacenter-p-monthly-
   Maximum Volume Jobs = 2
   Maximum Volumes = 9
}
Pool {
   Name = datacenter-p-weekly
   Pool Type = Backup
   Maximum Volume Jobs = 2
   Volume Retention = 1 months
   Recycle = yes
   AutoPrune = yes
   LabelFormat = datacenter-p-weekly-
   Maximum Volumes = 7
}
Pool {
   Name = datacenter-p-daily
   Pool Type = Backup
   Maximum Volume Jobs = 2
   Volume Retention = 14 days
   Recycle = yes
   AutoPrune = yes
   LabelFormat = datacenter-p-daily-
   Maximum Volumes = 6
}


Quoting Carlo Filippetto :

> I think you have to use one set of pool for every storage.
> I think that you can write volumes of the same pool into different storage,
> the problem may arrive when you have to restore...
>
> Try to a restore job...
>
>
> This is my Schedule:
> Schedule {
>Name = "Custom"
>Run = Level=Full Storage=ST-data Pool=P-Monthly 1st sat at 21:30
>Run = Level=Differential Storage=ST-data Pool=P-Weekly 2nd-5th sat
> at 21:30
>Run = Level=Incremental Storage=ST-data Pool=P-Daily sun-fri at
> 22:00
> }
>
> As you can see I set the Storage and Pool into every line, you can change
> it.
>
> Why do you like to use a single pool?
> If you have 2 storage may be more clear and easy to find every single
> volumes if you have different pools..
>
> Bye
>
>
>
>
>
>
> 2015-11-04 0:38 GMT+01:00 Wesley Render :
>
>> Should each storage daemon/geographic storage location have it's own
>> set of Volume Pools?  Or can I share one set of Volume Pools between
>> all of the storage daemons/storage locations?
>>
>> I am  using auto labelling as well and it works great.
>>
>>
>>
>> --
>> Wesley Render, Consultant
>> OtherData
>>
>>
>> --
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>



-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-04 Thread Wesley Render 
Ok. Thanks Josh.  I've already created Pools for each storage location  
and done the initial full backups so I will most likely stick with  
this method.

So far the backups appear to run a lot better using different pools  
for each storage location.

Thanks,


-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-04 Thread Carlo Filippetto 
I think you have to use one set of pool for every storage.
I think that you can write volumes of the same pool into different storage,
the problem may arrive when you have to restore...

Try to a restore job...


This is my Schedule:
Schedule {
   Name = "Custom"
   Run = Level=Full Storage=ST-data Pool=P-Monthly 1st sat at 21:30
   Run = Level=Differential Storage=ST-data Pool=P-Weekly 2nd-5th sat
at 21:30
   Run = Level=Incremental Storage=ST-data Pool=P-Daily sun-fri at
22:00
}

As you can see I set the Storage and Pool into every line, you can change
it.

Why do you like to use a single pool?
If you have 2 storage may be more clear and easy to find every single
volumes if you have different pools..

Bye






2015-11-04 0:38 GMT+01:00 Wesley Render :

> Should each storage daemon/geographic storage location have it's own
> set of Volume Pools?  Or can I share one set of Volume Pools between
> all of the storage daemons/storage locations?
>
> I am  using auto labelling as well and it works great.
>
>
>
> --
> Wesley Render, Consultant
> OtherData
>
>
> --
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Wesley Render 

I have recently started using Bacula and have a couple of questions  
regarding volume pools.  I am using 7.2 version, and we have 4 Linux  
servers with a total of about 100GB of data to backup.

1.  We have two different storage devices that are in different  
locations (because of bandwidth limitations).  Should we be creating  
different Volume Pools for each storage location?

I've tried testing using one pool, and two pools.  When I use one pool  
and a backup job is run it displays an error "Marking Volume  
"Vol-0001" in Error in Catalog." and then it continues to run.   When  
I setup two different volume pools it doesn't display this error.

2.   I've noticed that some people recommend setting up different  
volume pools for Full, Differential and Incremental jobs. Is this  
still a recommended strategy for Bacula with backing up to disk, and  
if so when would someone use this strategy?

The documentation here  
http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf doesn't  
mention this.  I don't want to set things up, and then our volumes  
grow too large and have to re-do everything.


Thank you,

-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Wesley Render 
Should each storage daemon/geographic storage location have it's own  
set of Volume Pools?  Or can I share one set of Volume Pools between  
all of the storage daemons/storage locations?

I am  using auto labelling as well and it works great.



-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Carlo Filippetto 
Hi,
for question #2:

Yes you have to create different pool, the firts reason is that usually
incremental bck lives less than a Full backup, the duration is into the
Pool information

So in my case I have
- p-daily  INCREMENTAL with 6 volumes (duration 7 days)
- p-weekly DIFFERENTIAL with 4 volumes (duration 1 month)
- p-monthly FULL with 6 volumes (duration 6 month)

In this way I'm able to cover 6 month


Hope to be usefull




2015-11-03 16:24 GMT+01:00 Wesley Render :

>
> I have recently started using Bacula and have a couple of questions
> regarding volume pools.  I am using 7.2 version, and we have 4 Linux
> servers with a total of about 100GB of data to backup.
>
> 1.  We have two different storage devices that are in different
> locations (because of bandwidth limitations).  Should we be creating
> different Volume Pools for each storage location?
>
> I've tried testing using one pool, and two pools.  When I use one pool
> and a backup job is run it displays an error "Marking Volume
> "Vol-0001" in Error in Catalog." and then it continues to run.   When
> I setup two different volume pools it doesn't display this error.
>
> 2.   I've noticed that some people recommend setting up different
> volume pools for Full, Differential and Incremental jobs. Is this
> still a recommended strategy for Bacula with backing up to disk, and
> if so when would someone use this strategy?
>
> The documentation here
> http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf doesn't
> mention this.  I don't want to set things up, and then our volumes
> grow too large and have to re-do everything.
>
>
> Thank you,
>
> --
> Wesley Render, Consultant
> OtherData
>
>
> --
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Wesley Render 

Thanks Carlo. This is very helpful. I also found this here which I  
missed before:

http://www.bacula.org/7.0.x-manuals/en/main/Automated_Disk_Backup.html

-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Wesley Render 
Is anyone able to clarify question number 1?

I should be all set after that.

Thanks!

-- 
Wesley Render, Consultant
OtherData

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Carlo Filippetto 
I don't like to use "label-format" I prefer to set the label by myself.
In my installation all the jobs that runs during the same night gone to the
same volumes.

bye

2015-11-03 17:14 GMT+01:00 Wesley Render :

>
> Thanks Carlo. This is very helpful. I also found this here which I missed
> before:
>
> http://www.bacula.org/7.0.x-manuals/en/main/Automated_Disk_Backup.html
>
>
> --
> Wesley Render, Consultant
> OtherData
>
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about Volume Pools and Strategy

2015-11-03 Thread Phil Stracchino 
On 11/03/15 11:23, Carlo Filippetto wrote:
> I don't like to use "label-format" I prefer to set the label by myself.
> In my installation all the jobs that runs during the same night gone to
> the same volumes.

I use auto-labeling for my disk volumes, to achieve exactly that end.
Each volume created has a 24-hour use window, and then it's done, with
14 days expiration for daily incrementals and 60 days for weekly
differentials.  My full backups go to a fixed set of 24 statically
labelled LTO4 tapes which are used in rotation.


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: 603.293.8485

--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users