Re: [Bacula-users] Bacula, Offsite, and Restores
2011/9/16 Eric Pratt eric.pr...@etouchpoint.com Thank you for your feedback, Rodrigo. I looked up the copy job information as you suggested. From what I can tell, you have to purge the original job before you can use a copy. This means to me that to do a restore, we have to: 1) identify all the jobs associated with all the files being restored. 2) purge those jobs from the database (which promoted their copies to a restorable state) 3) perform the restore Well, in order to restore from the Copy tapes, yes, you have to purge the original tapes... But you don't have to purge the original tapes if you want to do a normal routine restore. Even if you pull out the Copy tapes from your library right after a Copy Job (which is the hole idea), your restores will work normally like there never was a Copy Job ran before... The original tapes will stay intact on your on-site location and you'll use these original tapes to do your restores. You will only use your copy jobs tapes in case of a disaster... -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula, Offsite, and Restores
2011/9/17 Rodrigo Renie Braga rodrigore...@gmail.com: Well, in order to restore from the Copy tapes, yes, you have to purge the original tapes... But you don't have to purge the original tapes if you want to do a normal routine restore. Even if you pull out the Copy tapes from your library right after a Copy Job (which is the hole idea), your restores will work normally like there never was a Copy Job ran before... The original tapes will stay intact on your on-site location and you'll use these original tapes to do your restores. You will only use your copy jobs tapes in case of a disaster... Yes, but what I am looking for is the easiest method for my team to perform restores in case I'm not there, even if it is a restoration from offsite volumes. Having them go through the process of finding the right volumes to purge and then purging them is something I don't think they should have to learn how to do. It's nice that the ability is there in Bacula, but I think it's far too complicated for a robust solution. The solution needs to be easy and intuitive and merely running a second client makes it ridiculously easy and intuitive. I think it's the right way to go for now. Eric -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How to use retention to limit disk usage
On Fri, Sep 16, 2011 at 7:20 PM, Eric Sisolak haldir.j...@gmail.com wrote: Hello, I am looking for a way to limit the amount of space taken up by backups by truncating/deleting volumes whose files and jobs have been pruned according to retention periods with Bacula 5.0.3. ActionOnPurge=Truncate looks like it could do what I want, but it seemed like there many people had issues with it. Has anyone else implemented this? How did you do it? --Eric I am doing something like this, but not using ActionOnPurge. I'm using vchanger as a VTL and initializing X number of volumes. Lets say X = 80. I tell Bacula to limit the volume size to 5GB meaning I'm now only using 400GB of disk space for backups. I also tell Bacula to use a volume retention period of one day and to recycle the oldest volume. Bacula will not use these settings unless it runs out of volumes so your data will be retained in older volumes until you run out of space in the pool. But, once you've filled up your 80th volume in this scenario, it will look at the volume with the oldest last written timedate stamp in the pool and see if the volume is past its retention period. If it is, it will purge the jobs and files and re-use the volume. The net result is that Bacula is now set to use 400GB of storage space and never exceed it. It will automatically cannibalize the oldest volumes and purge records associated with those volumes as needed. If you are a little queasy about a 1 day volume retention, you can set this to something higher like one month to insure you always have at least one month's worth of backups. Just be aware that any jobs attempting to use storage when all 400GB are allocated will hang waiting for volumes if there are no volumes past their volume retention period. You must make sure that you have enough storage to handle the actual retention period you want. This also makes the names of the volumes generic, so your volume names will no longer be indicative of their contents. I find that using volume names tied to their jobs and pools or whatnot to be useless for me. So using generic names for the volumes results in no loss, but I gain the ease of use of vchanger. Here are the relevant config sections I'm using to accomplish this: -- bacula-sd.conf -- Device { Name = PrimaryVTLDevice DriveIndex = 0 Autochanger = yes Media Type = File Device Type = File Archive Device = /var/lib/bacula/PrimaryVTL/0/drive0 Random Access = yes RemovableMedia = yes LabelMedia = yes } Autochanger { Name = PrimaryVTLAutoChanger Device = PrimaryVTLDevice ChangerDevice = /etc/bacula/PrimaryVTL.conf ChangerCommand = /opt/bacula/bin/vchanger %c %o %S %a %d } --- bacula-dir.conf --- Storage { Name = PrimaryVTLStorage Address = enter.your.bacula-sd.hostname.here SDPort = 9103 Password = Device = PrimaryVTLAutoChanger Media Type = File Autochanger = yes } Pool { Name = PrimaryVTLPool PoolType = Backup Storage = PrimaryVTLStorage AutoPrune = yes VolumeRetention = 1 day MaximumVolumes = 80 MaximumVolumeBytes = 5368709120 RecycleOldestVolume = yes } --- PrimaryVTL.conf --- changer_name = PrimaryVTL work_dir = /var/lib/bacula/PrimaryVTL virtual_drives = 1 slots_per_magazine = 80 magazine_bays = 1 magazine = /var/backups/PrimaryVTL -- Hope that helps! Eric -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] How to use retention to limit disk usage
On Sat, Sep 17, 2011 at 4:39 PM, Eric Pratt eric.pr...@etouchpoint.comwrote: On Fri, Sep 16, 2011 at 7:20 PM, Eric Sisolak haldir.j...@gmail.com wrote: Hello, I am looking for a way to limit the amount of space taken up by backups by truncating/deleting volumes whose files and jobs have been pruned according to retention periods with Bacula 5.0.3. ActionOnPurge=Truncate looks like it could do what I want, but it seemed like there many people had issues with it. Has anyone else implemented this? How did you do it? --Eric I am doing something like this, but not using ActionOnPurge. I'm using vchanger as a VTL and initializing X number of volumes. Lets say X = 80. I tell Bacula to limit the volume size to 5GB meaning I'm now only using 400GB of disk space for backups. I also tell Bacula to use a volume retention period of one day and to recycle the oldest volume. Bacula will not use these settings unless it runs out of volumes so your data will be retained in older volumes until you run out of space in the pool. But, once you've filled up your 80th volume in this scenario, it will look at the volume with the oldest last written timedate stamp in the pool and see if the volume is past its retention period. If it is, it will purge the jobs and files and re-use the volume. The net result is that Bacula is now set to use 400GB of storage space and never exceed it. It will automatically cannibalize the oldest volumes and purge records associated with those volumes as needed. If you are a little queasy about a 1 day volume retention, you can set this to something higher like one month to insure you always have at least one month's worth of backups. Just be aware that any jobs attempting to use storage when all 400GB are allocated will hang waiting for volumes if there are no volumes past their volume retention period. You must make sure that you have enough storage to handle the actual retention period you want. This also makes the names of the volumes generic, so your volume names will no longer be indicative of their contents. I find that using volume names tied to their jobs and pools or whatnot to be useless for me. So using generic names for the volumes results in no loss, but I gain the ease of use of vchanger. Here are the relevant config sections I'm using to accomplish this: -- bacula-sd.conf -- Device { Name = PrimaryVTLDevice DriveIndex = 0 Autochanger = yes Media Type = File Device Type = File Archive Device = /var/lib/bacula/PrimaryVTL/0/drive0 Random Access = yes RemovableMedia = yes LabelMedia = yes } Autochanger { Name = PrimaryVTLAutoChanger Device = PrimaryVTLDevice ChangerDevice = /etc/bacula/PrimaryVTL.conf ChangerCommand = /opt/bacula/bin/vchanger %c %o %S %a %d } --- bacula-dir.conf --- Storage { Name = PrimaryVTLStorage Address = enter.your.bacula-sd.hostname.here SDPort = 9103 Password = Device = PrimaryVTLAutoChanger Media Type = File Autochanger = yes } Pool { Name = PrimaryVTLPool PoolType = Backup Storage = PrimaryVTLStorage AutoPrune = yes VolumeRetention = 1 day MaximumVolumes = 80 MaximumVolumeBytes = 5368709120 RecycleOldestVolume = yes } --- PrimaryVTL.conf --- changer_name = PrimaryVTL work_dir = /var/lib/bacula/PrimaryVTL virtual_drives = 1 slots_per_magazine = 80 magazine_bays = 1 magazine = /var/backups/PrimaryVTL -- Hope that helps! Eric Thank you. This is essentially they way we are doing it now (for servers backing up one to a handful of clients), and it works nice. The down side is that 400GB is always used, even if the retention I want would only take up 200GB. We would like to backup many more clients using one machine and want to make the most efficient use of storage that we can. Instead of allocating a static amount of space for backups, I would like to have it use the minimum needed for the desired retention. Sort of like thin provisioning the storage so it can be oversubscribed (and closely monitored). -Eric -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users