Re: [Bacula-users] Prefer disks over tape in restoring

[Ana Emília M . Arruda]

Hello Andrea,

It seems you have only two jobs ("DBackupData" and "DBackupCatalog")
with a "MonthlyWeeklyDaily"
schedule. The job DBackupCatalog is configured to use the "Full" Pool for
your full backups. So I suppose this "Full" Pool is your disk pool for your
full backups and that you use the schedule resource to change the Pool for
the weekly full backups. If I am not misunderstanding you and considering
the backup of FileSet "Data", you only have one job defined for the
full/diff/incr backups in disk and tapes.

The copy job solution is one possible solution for this. But you can
accomplish this with two job definitions. For example, you can run your
full montly backups to disk on first friday of the month and your full
backup to tapes from 2nd to 5th friday of the month:

Job {
  Name="DiskBackupData"
  JobDefs = "DefaultJob"
  FileSet="Data"
  Schedule="MonthlyWeeklyDaily"
  Client Run Before Job = /usr/local/netfence/local/bin/
bacula/preparebackup_Data.sh
  Client Run After Job = /usr/local/netfence/local/bin/
bacula/afterbackup_Data.sh
  Priority=1
  Full Backup Pool=Full
  Incremental Backup Pool=Inc
  Differential Backup Pool=Diff
}

Schedule {
   Name = MonthlyDailyDisk
   Run = Level=Full Pool=FullDiskPool 1st Friday at TIME
   Run = Level=Diff Pool=DiffDiskPool DAYS TIME # do not include fridays
here
   Run = Level=Incr Pool=IncrDiskPool DAYS TIME # do not include fridays
here
}

Job {
  Name = "TapeBackupData"
  JobDefs = "DefaultJob"
  FileSet="Data"
  Schedule="WeeklyTape"
  Client Run Before Job = /usr/local/netfence/local/bin/
bacula/preparebackup_Data.sh
  Client Run After Job = /usr/local/netfence/local/bin/
bacula/afterbackup_Data.sh
  Priority=1
  Full Backup Pool=Full
 }

Schedule {
  Name = WeeklyTape
  Run = Level=Full Pool=Tapes 2nd-5th Friday at TIME
}

Best regards,
Ana

On Fri, Jan 22, 2016 at 12:16 PM, Andrea Venturoli  wrote:

> On 01/22/16 11:30, Heitor Faria wrote:
>
> > Tape libraries are great and have huge throughput.
>
> And cost too much, unfortunately.
> The customer won't be able to afford one and I agree it would be overkill.
>
> Until a few months ago their full backup took only a single tape; now it
> spans two, but a more modern unit would be more than enough.
>
> In any case someone still needs to put the tape in and get it out
> (that's what I meant with "uncomfortable").
>
> Don't get me wrong, I'm very fond of tapes :)
>
>
>
>
>
> >> Is there a way I can tell bacula to almost ignore tape backups, so:
> >> a) always use the last disk full as a reference for subsequent
> diff/incr;
> >> a) always restore from disk, unless I really ask for the tape (e.g. by
> >> job number)?
> >
> > This looks like the copy jobs are the must suitable for your situation.
>
> You mean the following?
>
> b) do a Copy: this should work, but I'd need to copy several
> full/diff/incr jobs (not having a complete up to date full);
>
> So I should "copy" multiple jobs, not having a ready up to date "full"?
>
> In case of disaster, would "bextract"ing data from such tapes be as easy?
>
>
>
>   bye & Thanks
> av.
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Prefer disks over tape in restoring

[Andrea Venturoli]

Hello.

I'm loosing myself in what I think should be a pretty easy thing, 
probabily a FAQ, but I could not find a clear answer...



My situation is as follows:
_ I do monthly full and daily diff/incr backup to a NAS (this is fast 
and easy);
_ I also do weekly full backups to tapes (slow, unconfortable, but safer).

The Fileset is the same for both jobs.

So, in case of a restore, bacula selects the last tape backup (since 
it's usually more recent than the last full disk backup) and subsequent 
diff and incr from disk.
This is coherent with the fact that diff/incr backups are made with 
reference to the last full (on tape) backup.



What I'd like to achieve, instead, is to always use restore from disk 
and leave tapes as a last resort, in case something really went bad.

Is there a way I can tell bacula to almost ignore tape backups, so:
a) always use the last disk full as a reference for subsequent diff/incr;
a) always restore from disk, unless I really ask for the tape (e.g. by 
job number)?

Is there a flag I can mark the tape pool with?



I thought:
a) do a VirtualFull from disk to tape, but that would then again be used 
as a reference full, wouldn't it?
b) do a Copy: this should work, but I'd need to copy several 
full/diff/incr jobs (not having a complete up to date full);
c) purge the tape job just after doing it, but then I would only be able 
to bextract from it and I would have no track of it in the DB;
d) do a VirtualFull on disk, copy it to tape, delete it (since it would 
take too much space on the NAS), but should it really be this difficult?
e) is that what archive pools (currently unimplemented AFAIK) are for?



Any hint appreciated.

  bye & Thanks
av.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Prefer disks over tape in restoring

[Ana Emília M . Arruda]

Hello Andrea,

Do you have 2 different jobs to accomplish this? I do not understand what
do you mean when you say "The Fileset is the same for both jobs". If you
use different jobs, one for doing the disk backups (monthly full and daily
diff/incr) and another one for the tape backups (weekly full), both with
the same FileSet, the daily diff/incr disk backups should reference the
monthly full disk backups and not the weekly full tape backups.

Best regards,
Ana

On Fri, Jan 22, 2016 at 10:43 AM, Andrea Venturoli  wrote:

> Hello.
>
> I'm loosing myself in what I think should be a pretty easy thing,
> probabily a FAQ, but I could not find a clear answer...
>
>
>
> My situation is as follows:
> _ I do monthly full and daily diff/incr backup to a NAS (this is fast
> and easy);
> _ I also do weekly full backups to tapes (slow, unconfortable, but safer).
>
> The Fileset is the same for both jobs.
>
> So, in case of a restore, bacula selects the last tape backup (since
> it's usually more recent than the last full disk backup) and subsequent
> diff and incr from disk.
> This is coherent with the fact that diff/incr backups are made with
> reference to the last full (on tape) backup.
>
>
>
> What I'd like to achieve, instead, is to always use restore from disk
> and leave tapes as a last resort, in case something really went bad.
>
> Is there a way I can tell bacula to almost ignore tape backups, so:
> a) always use the last disk full as a reference for subsequent diff/incr;
> a) always restore from disk, unless I really ask for the tape (e.g. by
> job number)?
>
> Is there a flag I can mark the tape pool with?
>
>
>
> I thought:
> a) do a VirtualFull from disk to tape, but that would then again be used
> as a reference full, wouldn't it?
> b) do a Copy: this should work, but I'd need to copy several
> full/diff/incr jobs (not having a complete up to date full);
> c) purge the tape job just after doing it, but then I would only be able
> to bextract from it and I would have no track of it in the DB;
> d) do a VirtualFull on disk, copy it to tape, delete it (since it would
> take too much space on the NAS), but should it really be this difficult?
> e) is that what archive pools (currently unimplemented AFAIK) are for?
>
>
>
> Any hint appreciated.
>
>   bye & Thanks
> av.
>
>
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Prefer disks over tape in restoring

[Andrea Venturoli]

On 01/22/16 11:01, Ana Emília M. Arruda wrote:
> Hello Andrea,

Hello and thanks for your answer.





> Do you have 2 different jobs to accomplish this? I do not understand
> what do you mean when you say "The Fileset is the same for both jobs".

Yes.
See the extract of my bacula-dir.conf:

> Job {
>   Name="BackupData"
>   JobDefs = "DefaultJob"
>   FileSet="Data"
>   Client Run Before Job = 
> /usr/local/netfence/local/bin/bacula/preparebackup_Data.sh
>   Client Run After Job = 
> /usr/local/netfence/local/bin/bacula/afterbackup_Data.sh
>   Priority=1
> }
> Job {
>   Name="BackupCatalog"
>   JobDefs = "DefaultJob"
>   FileSet="Catalog"
>   RunBeforeJob = "/usr/local/share/bacula/make_catalog_backup bacula bacula"
>   RunAfterJob  = "/usr/local/share/bacula/delete_catalog_backup"
>   Write Bootstrap = "/var/db/bacula/BackupCatalog.bsr"
>   Priority = 11   # run after main backup
> }
> Job {
>   Name="DBackupData"
>   JobDefs = "DefaultJob"
>   FileSet="Data"
>   Schedule="MonthlyWeeklyDaily"
>   Client Run Before Job = 
> /usr/local/netfence/local/bin/bacula/preparebackup_Data.sh
>   Client Run After Job = 
> /usr/local/netfence/local/bin/bacula/afterbackup_Data.sh
>   Priority=1
>   Full Backup Pool=Full
>   Incremental Backup Pool=Inc
>   Differential Backup Pool=Diff
> }
> Job {
>   Name="DBackupCatalog"
>   JobDefs = "DefaultJob"
>   FileSet="Catalog"
>   Schedule="MonthlyWeeklyDaily"
>   RunBeforeJob = "/usr/local/share/bacula/make_catalog_backup bacula bacula"
>   RunAfterJob  = "/usr/local/share/bacula/delete_catalog_backup"
>   Priority = 11   # run after main backup
>   Pool=Full
> }
> Job {
>   Name="RestoreFiles"
>   Type = Restore
>   Client=Data-fd
>   FileSet="Data"
>   Storage = DAT160
>   Pool=Tapes
>   Messages = Standard
>   Where = /tmp/bacula-restores
> }







> If you use different jobs, one for doing the disk backups (monthly full
> and daily diff/incr) and another one for the tape backups (weekly full),
> both with the same FileSet, the daily diff/incr disk backups should
> reference the monthly full disk backups and not the weekly full tape
> backups.

That's what I thought too, but it doesn't seem to happen.
If I "restore" and "Select the most recent backup for a client", I'm 
presented to choose the FileSet (either "Data" or "Catalog"), not the 
storage or job.
Choosing "Data" I get:

> +---+---+--+-+-++
> | jobid | level | jobfiles | jobbytes| starttime   | 
> volumename |
> +---+---+--+-+-++
> | 1,174 | F |  700,309 | 127,903,911,237 | 2016-01-12 20:01:58 | Tape009  
>   |
> | 1,174 | F |  700,309 | 127,903,911,237 | 2016-01-12 20:01:58 | Tape010  
>   |
> | 1,186 | D |   55,614 |   6,397,616,958 | 2016-01-17 22:46:48 | Diff0085 
>   |
> | 1,188 | I |   20,508 |   3,672,664,590 | 2016-01-18 22:46:48 | Inc0078  
>   |
> | 1,197 | I |   38,643 |   5,146,141,599 | 2016-01-20 22:46:37 | Inc0079  
>   |
> | 1,205 | I |   24,620 |   4,515,300,651 | 2016-01-21 22:46:42 | Inc0082  
>   |
> +---+---+--+-+-++
 > You have selected the following JobIds: 1174,1186,1188,1197,1205



  bye & Thanks
av.

P.S. I'm using Bacula 7.2.0.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Prefer disks over tape in restoring

[Heitor Faria]

Hello Andrea:

> My situation is as follows:
> _ I do monthly full and daily diff/incr backup to a NAS (this is fast
> and easy);
> _ I also do weekly full backups to tapes (slow, unconfortable, but safer).

Tape libraries are great and have huge throughput. Perhaps you are not feeding 
it with enough multiplexed jobs / data.

> Is there a way I can tell bacula to almost ignore tape backups, so:
> a) always use the last disk full as a reference for subsequent diff/incr;
> a) always restore from disk, unless I really ask for the tape (e.g. by
> job number)?

This looks like the copy jobs are the must suitable for your situation. While 
you still have original jobs in disk Bacula will prefer to restore from them. 
If they got recycled you have a copy in the "evil" tapes. =)

>   av.

Regards,
-- 
=== 
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified 
Administrator II 
Próximas aulas telepresencial ao-vivo - 15 de fevereiro: 
http://www.bacula.com.br/agenda/ 
Ministro treinamento e implementação in-company Bacula: 
http://www.bacula.com.br/in-company/ 
Ou assista minhas vídeo aulas on-line: 
http://www.bacula.com.br/treinamento-bacula-ed/ 
61 8268-4220 
Site: www.bacula.com.br | Facebook: heitor.faria 


--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Prefer disks over tape in restoring

[J. Echter]

Hi,

you could use copy jobs (copying backups to tape), bacula will only ask
for tapes if there is no local backup.

greetings

Am 22.01.2016 um 11:01 schrieb Ana Emília M. Arruda:
> Hello Andrea,
> 
> Do you have 2 different jobs to accomplish this? I do not understand
> what do you mean when you say "The Fileset is the same for both jobs".
> If you use different jobs, one for doing the disk backups (monthly full
> and daily diff/incr) and another one for the tape backups (weekly full),
> both with the same FileSet, the daily diff/incr disk backups should
> reference the monthly full disk backups and not the weekly full tape
> backups.
> 
> Best regards,
> Ana
> 
> On Fri, Jan 22, 2016 at 10:43 AM, Andrea Venturoli  > wrote:
> 
> Hello.
> 
> I'm loosing myself in what I think should be a pretty easy thing,
> probabily a FAQ, but I could not find a clear answer...
> 
> 
> 
> My situation is as follows:
> _ I do monthly full and daily diff/incr backup to a NAS (this is fast
> and easy);
> _ I also do weekly full backups to tapes (slow, unconfortable, but
> safer).
> 
> The Fileset is the same for both jobs.
> 
> So, in case of a restore, bacula selects the last tape backup (since
> it's usually more recent than the last full disk backup) and subsequent
> diff and incr from disk.
> This is coherent with the fact that diff/incr backups are made with
> reference to the last full (on tape) backup.
> 
> 
> 
> What I'd like to achieve, instead, is to always use restore from disk
> and leave tapes as a last resort, in case something really went bad.
> 
> Is there a way I can tell bacula to almost ignore tape backups, so:
> a) always use the last disk full as a reference for subsequent
> diff/incr;
> a) always restore from disk, unless I really ask for the tape (e.g. by
> job number)?
> 
> Is there a flag I can mark the tape pool with?
> 
> 
> 
> I thought:
> a) do a VirtualFull from disk to tape, but that would then again be used
> as a reference full, wouldn't it?
> b) do a Copy: this should work, but I'd need to copy several
> full/diff/incr jobs (not having a complete up to date full);
> c) purge the tape job just after doing it, but then I would only be able
> to bextract from it and I would have no track of it in the DB;
> d) do a VirtualFull on disk, copy it to tape, delete it (since it would
> take too much space on the NAS), but should it really be this difficult?
> e) is that what archive pools (currently unimplemented AFAIK) are for?
> 
> 
> 
> Any hint appreciated.
> 
>   bye & Thanks
> av.
> 
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> 
> 
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
> 
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Prefer disks over tape in restoring

[Andrea Venturoli]

On 01/22/16 11:30, Heitor Faria wrote:

> Tape libraries are great and have huge throughput.

And cost too much, unfortunately.
The customer won't be able to afford one and I agree it would be overkill.

Until a few months ago their full backup took only a single tape; now it 
spans two, but a more modern unit would be more than enough.

In any case someone still needs to put the tape in and get it out 
(that's what I meant with "uncomfortable").

Don't get me wrong, I'm very fond of tapes :)





>> Is there a way I can tell bacula to almost ignore tape backups, so:
>> a) always use the last disk full as a reference for subsequent diff/incr;
>> a) always restore from disk, unless I really ask for the tape (e.g. by
>> job number)?
>
> This looks like the copy jobs are the must suitable for your situation.

You mean the following?

b) do a Copy: this should work, but I'd need to copy several
full/diff/incr jobs (not having a complete up to date full);

So I should "copy" multiple jobs, not having a ready up to date "full"?

In case of disaster, would "bextract"ing data from such tapes be as easy?



  bye & Thanks
av.

--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311=/4140
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users