[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Managed to finally create a decent scheme for what i needed, using 2 pools for my job. Thank you everyone for your help!! Regards, Larry +-- |This was sent by larryb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hello Larryboy 2016-05-27 6:40 GMT-03:00 Larrybwoy : > Hello again, > > A new question : Is there a way I can set up Bacula so that it recycles > only the differential jobs from a volume? For example, I have one volume > that has the initial full backup in it and 1 other differential job. Can I > set it up so that only the differential job in there gets recycled every > hour? > This case is not possible, because all backup (Full and Differential) are in the same volume, if you recycle volume, bacula will recycle entire volume. > > OR, say i have the flag "User Volume Once = yes", and I have 1 volume > which is the first full backup, then 1 other volume which represents the > differential backup, can I set up recycling only on the 2nd volume? I don't > want to be doing a full backup every hour because of retention times, and I > only want to do a full backup ONCE, then only differential. > Yes. You need to create a Pool for the Full backups with a long retention and another Pool for the Differential backups with a shorter retention. > I have tried this (without the Use Volume Once flag) with a 3 hour > retention period (since it takes about 2 and a half hours to do a full > backup for my filesystem, and if I set the retention period lower than that > then bacula runs the next differential job(which is hourly) as a full > job...) and the problem is that after those 3 hours are up bacula creates > a new full job in the first volume, since it sees it as being expired. > > I can't seem to figure out a decent plan using Bacula for my specific > needs, it's driving me crazy. > > Best Regards *Wanderlei Hüttel* http://www.huttel.com.br -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hello again, A new question : Is there a way I can set up Bacula so that it recycles only the differential jobs from a volume? For example, I have one volume that has the initial full backup in it and 1 other differential job. Can I set it up so that only the differential job in there gets recycled every hour? OR, say i have the flag "User Volume Once = yes", and I have 1 volume which is the first full backup, then 1 other volume which represents the differential backup, can I set up recycling only on the 2nd volume? I don't want to be doing a full backup every hour because of retention times, and I only want to do a full backup ONCE, then only differential. I have tried this (without the Use Volume Once flag) with a 3 hour retention period (since it takes about 2 and a half hours to do a full backup for my filesystem, and if I set the retention period lower than that then bacula runs the next differential job(which is hourly) as a full job...) and the problem is that after those 3 hours are up bacula creates a new full job in the first volume, since it sees it as being expired. I can't seem to figure out a decent plan using Bacula for my specific needs, it's driving me crazy. +-- |This was sent by larryb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On 05/23/16 11:09, Larrybwoy wrote:
> I see your point. Interesting. So I can make it so that each new differential
> backup that gets created writes itself in a totally new volume for that
> particular job ? Is there an option for this I need to add to the pool or ?
>
Here's how I do it. This example has a 23-hour use duration:
Pool {
Name = Diff-Disk
Storage = babylon4-file
Pool Type = Backup
Recycle = no
Recycle Oldest Volume = no
Recycle Current Volume = no
AutoPrune = yes
Volume Retention = 2 months
File Retention = 2 months
Maximum Volume Jobs = 0
Volume Use Duration = 23h
Label Format =
"DIFF-$Year${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}:${Minute:p/2/0/r}"
RecyclePool = Scratch
}
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: 603.293.8485
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On 05/23/2016 11:15 AM, shouldbe q931 wrote: > Several years ago I did something similar where there was 24 hours of > hourly rsync "backups" and daily runs of Bacula to tape Thankfully you don't have to deal with rsync anymore: there's now zfs with COW snapshots and the ability to mirror snapshot deltas to other hosts. All you need is a storage server with enough disk bays somewhere offsite and a small shell script to do snapshot-transfer-collect garbage in an hourly cron job. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital signature -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On Mon, May 23, 2016 at 9:20 AM, Larrybwoy wrote: I do not need to have backups that are old; in case of a disaster, I need to be able to bring back the data that was lost during the past hour at most, so that the people working with the applications only lose 1 hour of work in the worst case scenario. Being very much aware that this is a Bacula mailing list If you need an recovery point of no longer than one hour, then I might consider just using Bacula for "historic backups", and rsync to keep a current copy for DR purposes, if you wanted multiple DR "copies", something like http://www.mikerubel.org/computers/rsync_snapshots/ or http://www.dirvish.org/ could be a good starting point. Several years ago I did something similar where there was 24 hours of hourly rsync "backups" and daily runs of Bacula to tape Cheers -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
I see your point. Interesting. So I can make it so that each new differential backup that gets created writes itself in a totally new volume for that particular job ? Is there an option for this I need to add to the pool or ? +-- |This was sent by larryb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Think about it this way: If you're backing up to tape, jobs smaller than a full tape waste tape, because you can't make the tape smaller. But if you're backing up to disk, putting more than one batch of jobs into a volume wastes disk. Because you can't compact a disk volume, and you can't delete it until you no longer need the *LAST* job written to it. But you *can* make a disk volume as small as you like. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: 603.293.8485 -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On 05/23/16 04:20, Larrybwoy wrote: > The only problem is the filesystem that I back up has 91 gigs, and > the backup keeps getting bigger and bigger with all the differential > jobs. So far the max vol size is set to 300GB, and with the backups > running all weekend it now created a second volume for the same job > since it reached its max size on the first volume. It's why I need to > find out how to make Bacula remove older differential jobs from the > first volume. The short version is, you can't. You can *mark jobs as deleted* in a volume, but that won't free any space. You can delete only the entire volume at one time. You can't compact it except by copying the jobs you don't want to delete to a new volume, then deleting the original volume. And if your disk volumes are hitting 300GB, then each of those copy operations is going to be tying up half a terabyte while it runs. In my opinion, you're going about it all wrong. On disk, you don't have a minimum efficient volume size, and that means you want your volumes SMALLER, not bigger. Your plan should be based on using each volume *only once*. If you're running hourly differential backups, then start a new autolabelled volume for each batch of jobs, with a volume use duration of, say, 59 minutes. When the use duration expires, the next batch of jobs will automatically create a new volume. As soon as a batch of jobs "ages out", you can purge them and delete the volume that contains them. Plus, now you're deleting 5GB, 10GB, 15GB volumes at a time instead of 300GB volumes. I've been running this model for years for my incremental and differential backups, and it works. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: 603.293.8485 -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On 5/23/2016 5:44 AM, Larrybwoy wrote:
> Hey guys,
>
> Thanks for the replies and good advice. The reason I thought of this backup
> plan is because what I need to back up are multiple dynamic file systems from
> abut 20 servers. These file systems contain data that is always changing
> since they contain various dynamic application files. I do not need to have
> backups that are old; in case of a disaster, I need to be able to bring back
> the data that was lost during the past hour at most, so that the people
> working with the applications only lose 1 hour of work in the worst case
> scenario. I have this set up and running pretty well on 2 hosts so far.
I would consider what you are protecting against to be hardware failure,
not a disaster. A disaster is a flood or something that takes out both
servers and the Bacula machine, along with the on site backup media.
Bacula is not meant to be a high availability solution. I suggest using
a Pacemaker/Corosync cluster with server filesystems on DRBD. This will
protect against a single machine failure with zero downtime. Bacula
backups can then be daily incrementals to protect against accidental
deletion, virus, etc., with full backups being rotated to an off-site
location on some schedule to protect against disaster.
>
> The only problem is the filesystem that I back up has 91 gigs, and the
> backup keeps getting bigger and bigger with all the differential jobs. So far
> the max vol size is set to 300GB, and with the backups running all weekend it
> now created a second volume for the same job since it reached its max size on
> the first volume. It's why I need to find out how to make Bacula remove older
> differential jobs from the first volume. As far as I know, during a restore
> on a backup that uses differential backups, only the last differential backup
> (which should contain all other changes made to the backed up filesystem in
> the previous diff jobs) and the full backup should be required.
>
> Here is my setup. (If you guys have a better ideea on how I should set this
> up considering my needs please feel free to advise)
>
> Thanks a lot for the help !
>
>
>
> The CLIENT config :
>
> Client {
> Name = server1-fd
> Address = server1.com
> FDPort = 9102
> Catalog = MyCatalog
> Password = "NDdkODYyMDM4NTZmNjYzNjYwZmE5MzIwZ" # password
> for Remote FileDaemon
> File Retention = 30 days# 30 days
> Job Retention = 6 months# six months
> AutoPrune = yes # Prune expired Jobs/Files
> }
>
> Pool {
>Name = RemoteFile
>Pool Type = Backup
>Label Format = Server1-
>Recycle = yes# Bacula can automatically recycle
> Volumes
>AutoPrune = yes # Prune expired volumes
>Volume Retention = 7 days# one year
>Maximum Volume Bytes = 300G # Limit Volume size to something
> reasonable
>Maximum Volumes = 10 # Limit number of Volumes in Pool
> }
>
>
> Job {
>Name = "Sever1-DIFF"
>JobDefs = "DifferentialJob"
>Client = Server1-fd
>Pool = RemoteFile
>FileSet="Xtm-Diff"
>Write Bootstrap = "/var/spool/bacula/%c.bsr"
>Accurate = yes
> }
>
> FileSet {
>Name = "Xtm-Diff"
>Include {
> Options {
> sparse=yes
> }
> File = /xtm
>}
>Exclude {
>}
> }
>
> The Schedule :
> -
>
> Schedule {
>Name = "HourlyCycle"
>Run = Differential hourly at 00:55
> }
>
> The JobDef
> --
>
> JobDefs {
>Name = "DifferentialJob"
>Type = Backup
>Level = Differential
>Client = Server1-fd
>Schedule = "HourlyCycle"
>Storage = File
>Messages = Standard
>Pool = RemoteFile
>Priority = 10
>Write Bootstrap = "/var/spool/bacula/%c.bsr"
>Accurate = yes
> }
>
> Laest Finished Jobs
> ---
> Terminated Jobs:
> JobId LevelFiles Bytes Status FinishedName
>
> 290 Diff 2840.27 M OK 23-May-16 05:57 Server1-DIFF
> 292 Diff 2840.51 M OK 23-May-16 06:57 Server1-DIFF
> 294 Diff 2940.70 M OK 23-May-16 07:57 Server1-DIFF
> 296 Diff 2940.94 M OK 23-May-16 08:57 Server1-DIFF
> 298 Diff 2941.19 M OK 23-May-16 09:58 Server1-DIFF
>
> THE FILESYSTEMS
> --
> To be backed up >
> /dev/md2 440G 92G 326G 22% /xtm
>
> The actual backups:
>
> -rw-r- 1 bacula tape 300G May 21 04:06 Server1-0035
> -rw-r- 1 bacula tape 223G May 23 09:58 Server1-0036
>
> (I have substituted the server hostnames with fiction
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hey guys,
Thanks for the replies and good advice. The reason I thought of this backup
plan is because what I need to back up are multiple dynamic file systems from
abut 20 servers. These file systems contain data that is always changing since
they contain various dynamic application files. I do not need to have backups
that are old; in case of a disaster, I need to be able to bring back the data
that was lost during the past hour at most, so that the people working with the
applications only lose 1 hour of work in the worst case scenario. I have this
set up and running pretty well on 2 hosts so far.
The only problem is the filesystem that I back up has 91 gigs, and the backup
keeps getting bigger and bigger with all the differential jobs. So far the max
vol size is set to 300GB, and with the backups running all weekend it now
created a second volume for the same job since it reached its max size on the
first volume. It's why I need to find out how to make Bacula remove older
differential jobs from the first volume. As far as I know, during a restore on
a backup that uses differential backups, only the last differential backup
(which should contain all other changes made to the backed up filesystem in the
previous diff jobs) and the full backup should be required.
Here is my setup. (If you guys have a better ideea on how I should set this up
considering my needs please feel free to advise)
Thanks a lot for the help !
The CLIENT config :
Client {
Name = server1-fd
Address = server1.com
FDPort = 9102
Catalog = MyCatalog
Password = "NDdkODYyMDM4NTZmNjYzNjYwZmE5MzIwZ" # password for
Remote FileDaemon
File Retention = 30 days# 30 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Pool {
Name = RemoteFile
Pool Type = Backup
Label Format = Server1-
Recycle = yes# Bacula can automatically recycle
Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 7 days# one year
Maximum Volume Bytes = 300G # Limit Volume size to something
reasonable
Maximum Volumes = 10 # Limit number of Volumes in Pool
}
Job {
Name = "Sever1-DIFF"
JobDefs = "DifferentialJob"
Client = Server1-fd
Pool = RemoteFile
FileSet="Xtm-Diff"
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
FileSet {
Name = "Xtm-Diff"
Include {
Options {
sparse=yes
}
File = /xtm
}
Exclude {
}
}
The Schedule :
-
Schedule {
Name = "HourlyCycle"
Run = Differential hourly at 00:55
}
The JobDef
--
JobDefs {
Name = "DifferentialJob"
Type = Backup
Level = Differential
Client = Server1-fd
Schedule = "HourlyCycle"
Storage = File
Messages = Standard
Pool = RemoteFile
Priority = 10
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
Laest Finished Jobs
---
Terminated Jobs:
JobId LevelFiles Bytes Status FinishedName
290 Diff 2840.27 M OK 23-May-16 05:57 Server1-DIFF
292 Diff 2840.51 M OK 23-May-16 06:57 Server1-DIFF
294 Diff 2940.70 M OK 23-May-16 07:57 Server1-DIFF
296 Diff 2940.94 M OK 23-May-16 08:57 Server1-DIFF
298 Diff 2941.19 M OK 23-May-16 09:58 Server1-DIFF
THE FILESYSTEMS
--
To be backed up >
/dev/md2 440G 92G 326G 22% /xtm
The actual backups:
-rw-r- 1 bacula tape 300G May 21 04:06 Server1-0035
-rw-r- 1 bacula tape 223G May 23 09:58 Server1-0036
(I have substituted the server hostnames with fictional stuff, for security
reasons)
+--
|This was sent by larryb...@gmail.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hey guys,
Thanks for the replies and good advice. The reason I thought of this backup
plan is because what I need to back up are multiple dynamic file systems from
abut 20 servers. These file systems contain data that is always changing since
they contain various dynamic application files. I do not need to have backups
that are old; in case of a disaster, I need to be able to bring back the data
that was lost during the past hour at most, so that the people working with the
applications only lose 1 hour of work in the worst case scenario. I have this
set up and running pretty well on 2 hosts so far.
The only problem is the filesystem that I back up has 91 gigs, and the backup
keeps getting bigger and bigger with all the differential jobs. So far the max
vol size is set to 300GB, and with the backups running all weekend it now
created a second volume for the same job since it reached its max size on the
first volume. It's why I need to find out how to make Bacula remove older
differential jobs from the first volume. As far as I know, during a restore on
a backup that uses differential backups, only the last differential backup
(which should contain all other changes made to the backed up filesystem in the
previous diff jobs) and the full backup should be required.
Here is my setup. (If you guys have a better ideea on how I should set this up
considering my needs please feel free to advise)
Thanks a lot for the help !
The CLIENT config :
Client {
Name = server1-fd
Address = server1.com
FDPort = 9102
Catalog = MyCatalog
Password = "NDdkODYyMDM4NTZmNjYzNjYwZmE5MzIwZ" # password for
Remote FileDaemon
File Retention = 30 days# 30 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Pool {
Name = RemoteFile
Pool Type = Backup
Label Format = Server1-
Recycle = yes# Bacula can automatically recycle
Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 7 days# one year
Maximum Volume Bytes = 300G # Limit Volume size to something
reasonable
Maximum Volumes = 10 # Limit number of Volumes in Pool
}
Job {
Name = "Sever1-DIFF"
JobDefs = "DifferentialJob"
Client = Server1-fd
Pool = RemoteFile
FileSet="Xtm-Diff"
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
FileSet {
Name = "Xtm-Diff"
Include {
Options {
sparse=yes
}
File = /xtm
}
Exclude {
}
}
The Schedule :
-
Schedule {
Name = "HourlyCycle"
Run = Differential hourly at 00:55
}
The JobDef
--
JobDefs {
Name = "DifferentialJob"
Type = Backup
Level = Differential
Client = Server1-fd
Schedule = "HourlyCycle"
Storage = File
Messages = Standard
Pool = RemoteFile
Priority = 10
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
Laest Finished Jobs
---
Terminated Jobs:
JobId LevelFiles Bytes Status FinishedName
290 Diff 2840.27 M OK 23-May-16 05:57 Server1-DIFF
292 Diff 2840.51 M OK 23-May-16 06:57 Server1-DIFF
294 Diff 2940.70 M OK 23-May-16 07:57 Server1-DIFF
296 Diff 2940.94 M OK 23-May-16 08:57 Server1-DIFF
298 Diff 2941.19 M OK 23-May-16 09:58 Server1-DIFF
THE FILESYSTEMS
--
To be backed up >
/dev/md2 440G 92G 326G 22% /xtm
The actual backups:
-rw-r- 1 bacula tape 300G May 21 04:06 Server1-0035
-rw-r- 1 bacula tape 223G May 23 09:58 Server1-0036
(I have substituted the server hostnames with fictional stuff, for security
reasons)
+--
|This was sent by larryb...@gmail.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
The CLIENT config :
Client {
Name = server1-fd
Address = server1.com
FDPort = 9102
Catalog = MyCatalog
Password = "NDdkODYyMDM4NTZmNjYzNjYwZmE5MzIwZ" # password for
Remote FileDaemon
File Retention = 30 days# 30 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Pool {
Name = RemoteFile
Pool Type = Backup
Label Format = Server1-
Recycle = yes# Bacula can automatically recycle
Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 7 days# one year
Maximum Volume Bytes = 300G # Limit Volume size to something
reasonable
Maximum Volumes = 10 # Limit number of Volumes in Pool
}
Job {
Name = "Sever1-DIFF"
JobDefs = "DifferentialJob"
Client = Server1-fd
Pool = RemoteFile
FileSet="Xtm-Diff"
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
FileSet {
Name = "Xtm-Diff"
Include {
Options {
sparse=yes
}
File = /xtm
}
Exclude {
}
}
The Schedule :
-
Schedule {
Name = "HourlyCycle"
Run = Differential hourly at 00:55
}
The JobDef
--
JobDefs {
Name = "DifferentialJob"
Type = Backup
Level = Differential
Client = Server1-fd
Schedule = "HourlyCycle"
Storage = File
Messages = Standard
Pool = RemoteFile
Priority = 10
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
Laest Finished Jobs
---
Terminated Jobs:
JobId LevelFiles Bytes Status FinishedName
290 Diff 2840.27 M OK 23-May-16 05:57 Server1-DIFF
292 Diff 2840.51 M OK 23-May-16 06:57 Server1-DIFF
294 Diff 2940.70 M OK 23-May-16 07:57 Server1-DIFF
296 Diff 2940.94 M OK 23-May-16 08:57 Server1-DIFF
298 Diff 2941.19 M OK 23-May-16 09:58 Server1-DIFF
THE FILESYSTEMS
--
To be backed up >
/dev/md2 440G 92G 326G 22% /xtm
The actual backups:
-rw-r- 1 bacula tape 300G May 21 04:06 Server1-0035
-rw-r- 1 bacula tape 223G May 23 09:58 Server1-0036
(I have substituted the server hostnames with fictional stuff, for security
reasons)
+--
|This was sent by larryb...@gmail.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hey guys,
Thanks for the replies and good advice. The reason I thought of this backup
plan is because what I need to back up are multiple dynamic file systems from
abut 20 servers. These file systems contain data that is always changing since
they contain various dynamic application files. I do not need to have backups
that are old; in case of a disaster, I need to be able to bring back the data
that was lost during the past hour at most, so that the people working with the
applications only lose 1 hour of work in the worst case scenario. I have this
set up and running pretty well on 2 hosts so far.
The only problem is the filesystem that I back up has 91 gigs, and the backup
keeps getting bigger and bigger with all the differential jobs. So far the max
vol size is set to 300GB, and with the backups running all weekend it now
created a second volume for the same job since it reached its max size on the
first volume. It's why I need to find out how to make Bacula remove older
differential jobs from the first volume. As far as I know, during a restore on
a backup that uses differential backups, only the last differential backup
(which should contain all other changes made to the backed up filesystem in the
previous diff jobs) and the full backup should be required.
Here is my setup. (If you guys have a better ideea on how I should set this up
considering my needs please feel free to advise)
Thanks a lot for the help !
The CLIENT config :
Client {
Name = server1-fd
Address = server1.com
FDPort = 9102
Catalog = MyCatalog
Password = "NDdkODYyMDM4NTZmNjYzNjYwZmE5MzIwZ" # password for
Remote FileDaemon
File Retention = 30 days# 30 days
Job Retention = 6 months# six months
AutoPrune = yes # Prune expired Jobs/Files
}
Pool {
Name = RemoteFile
Pool Type = Backup
Label Format = Server1-
Recycle = yes# Bacula can automatically recycle
Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 7 days# one year
Maximum Volume Bytes = 300G # Limit Volume size to something
reasonable
Maximum Volumes = 10 # Limit number of Volumes in Pool
}
FileSet {
Name = "Xtm-Full"
Include {
Options {
sparse=yes
}
File = /xtm
}
Exclude {
}
}
Job {
Name = "Sever1-DIFF"
JobDefs = "DifferentialJob"
Client = Server1-fd
Pool = RemoteFile
FileSet="Xtm-Diff"
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
FileSet {
Name = "Xtm-Diff"
Include {
Options {
sparse=yes
}
File = /xtm
}
Exclude {
}
}
The Schedule :
-
Schedule {
Name = "HourlyCycle"
Run = Differential hourly at 00:55
}
The JobDef
--
JobDefs {
Name = "DifferentialJob"
Type = Backup
Level = Differential
Client = Server1-fd
Schedule = "HourlyCycle"
Storage = File
Messages = Standard
Pool = RemoteFile
Priority = 10
Write Bootstrap = "/var/spool/bacula/%c.bsr"
Accurate = yes
}
Laest Finished Jobs
---
Terminated Jobs:
JobId LevelFiles Bytes Status FinishedName
290 Diff 2840.27 M OK 23-May-16 05:57 Server1-DIFF
292 Diff 2840.51 M OK 23-May-16 06:57 Server1-DIFF
294 Diff 2940.70 M OK 23-May-16 07:57 Server1-DIFF
296 Diff 2940.94 M OK 23-May-16 08:57 Server1-DIFF
298 Diff 2941.19 M OK 23-May-16 09:58 Server1-DIFF
THE FILESYSTEMS
--
To be backed up >
/dev/md2 440G 92G 326G 22% /xtm
The actual backups:
-rw-r- 1 bacula tape 300G May 21 04:06 Server1-0035
-rw-r- 1 bacula tape 223G May 23 09:58 Server1-0036
(I have substituted the server hostnames with fictional stuff, for security
reasons)
+--
|This was sent by larryb...@gmail.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
> On 19.05.2016 14:20, Larrybwoy wrote: >> Hello dear community, >> I could use some advice on the following scenario: >> The Bacula job that I have set up is ran on an HOURLY basis, the first backup >> being FULL and the rest that follow are all differential. What I need to know >> is how do I set up my job so that it deletes the previous differential jobs >> from the volume, after it completes the latest differential job. I need to do >> this in order to preserve disk space, since I do not need older differential >> jobs, I am only interested in the latest one, maybe latest 2 jobs even. >> If anyone could shed some light on this, it would be highly appreciated ! :) Hello, Larry: you might try one or more of this Pool Directives for this Job (if it is a single one): Maximum Volume Jobs=1 Volume Retention=90 minutes or the VUD approach if you have multiple hourly jobs: Volume Use Duration = 50 minutes Volume Retention = 60 minutes Regards, -- === Heitor Medrado de Faria | Bacula do Brasil Próximas aulas telepresencial ao-vivo - 02 de maio: http://www.bacula.com.br/agenda/ Ministro treinamento e implementação in-company Bacula: http://www.bacula.com.br/in-company/ 61 8268-4220 Site: www.bacula.com.br -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
On 19.05.2016 14:20, Larrybwoy wrote: > Hello dear community, > > I could use some advice on the following scenario: > > The Bacula job that I have set up is ran on an HOURLY basis, the first backup > being FULL and the rest that follow are all differential. What I need to know > is how do I set up my job so that it deletes the previous differential jobs > from the volume, after it completes the latest differential job. I need to do > this in order to preserve disk space, since I do not need older differential > jobs, I am only interested in the latest one, maybe latest 2 jobs even. > > If anyone could shed some light on this, it would be highly appreciated ! :) > Hello! I don't think you can delete files from volume and it shouldn't be done else you are in danger of losing data integrity of the whole volume. Better consider your schedule - is it really necessary to do diffs on hourly basis? What are your retention times? How long you must keep the data? Send your schedule, retention times and volume number and size, and let's try work something out. Regards, -- Igor Blažević -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deleting DIFFERENTIAL Jobs from Volumes
Hello dear community, I could use some advice on the following scenario: The Bacula job that I have set up is ran on an HOURLY basis, the first backup being FULL and the rest that follow are all differential. What I need to know is how do I set up my job so that it deletes the previous differential jobs from the volume, after it completes the latest differential job. I need to do this in order to preserve disk space, since I do not need older differential jobs, I am only interested in the latest one, maybe latest 2 jobs even. If anyone could shed some light on this, it would be highly appreciated ! :) Many thanks, Larry +-- |This was sent by larryb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
