[Bacula-users] Separating Full backups from incremental in the volumes.

[ccspro]

You can also do that per Job OR JobDefs too, If you the same schedule for more 
than 1 client this may be more appropriate for you:
Job or JobDefs {
  Pool = "thisjob-inc-pool" # default 'level' pool *REQUIRED even if not used*
  Full Backup Pool = "thisjob-full-pool"
  Incremental Backup Pool = "thisjob-inc-pool"
  Differential Backup Pool = "thisjob-diff-pool"
}

You also need to think about volume/job/file retention periods, volume 
expiration or how many clients per *time* or per job you allow per volume, 
volumes are tricky to grasp due to how many ways you can set retention or 
storage periods for.

Each pool (in bacula 5.2) will allow you to over ride the Client retention 
periods:
Pool { # 1 months of Full's, 1 per week - assume 5 weeks giving extra lag time.
  FileRetention=1 months
  JobRetention =1 months
  #Volume Use Duration = 23h   # Jobs placed in this pool within *time* 
go here, after *time* will be the next volume
  Volume Retention = 1 months  # how long we keep the volume for before 
reuse.
  Maximum Volume Jobs = 1  # 1 jobs per volume max. (careful mixing 
this with Volume Use Duration!)
  Maximum Volumes = 5# Force recycling per volume after 
Maximum volume reached (5 allows 1 week lag period)
}
If file retention expires before volume or job you may only export everything 
(no selection in bconsole) on restore.
If Job retention expires it might take longer to find the linked jobs and again 
it may be an export everything situation on restore

I use a series of Base, Full, Inc and Diff for every client but the retention 
periods of INC are set low (1 month tops) to keep daily changes clean, thats 
what fulls and diffs are for, I keep 3-6 months of fulls, at least 1 year of 
base and 1 month of inc's, diff's vary if they are even used.

Once you break away from the defaults everything requires a lot of planning and 
"what if" scenario's come into play!

+--
|This was sent by ccs...@hotmail.com via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--



--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula stops writing to disk volume after 2GB

[m.list]

lst_hoe02 wrote
> If this is really NFS double-check that both sides agree to use at  
> least NFSv3 as older NFS versions have a 2GB limit and Bacula clearly  
> is not able to write more data to the file in question. 

Spot on, in /etc/autofs.master the file system was mounted with the option
-nfsvers=2. After changing this to -fstype=nfs4 eveything works fine.





--
View this message in context: 
http://bacula.10910.n7.nabble.com/Bacula-stops-writing-to-disk-volume-after-2GB-tp54213p73634.html
Sent from the Bacula - Users mailing list archive at Nabble.com.

--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Separating Full backups from incremental in the volumes.

[Luis H. Forchesatto]

Woah, that's exactly whath I need.

I'm reorganizing my backup system an I need to restructure the volume
usage, so the best solution is separating full and incremental backups.

Thanks Bill and Silas for teaching me how to do this =D

Now let's implement.

Good bye and thanks.

2012/12/10 Bill Arlofski 

> On 12/10/12 11:46, Bill Arlofski wrote:
> > Hi Louis,
> >
> > Yes, you can do this withing a Schedule:
> >
> > An example is below, but it kind of funny.
>
> heh   Should have been "but it wrapped kind of funny"
>
>
> --
> Bill Arlofski
> Reverse Polarity, LLC
>
>
> --
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>



-- 
Att.*
***
Luis H. Forchesatto
Mail: luis_forchesa...@hotmail.com
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Separating Full backups from incremental in the volumes.

[Bill Arlofski]

On 12/10/12 11:46, Bill Arlofski wrote:
> Hi Louis,
> 
> Yes, you can do this withing a Schedule:
> 
> An example is below, but it kind of funny.

heh   Should have been "but it wrapped kind of funny"


--
Bill Arlofski
Reverse Polarity, LLC

--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Separating Full backups from incremental in the volumes.

[Bill Arlofski]

On 12/10/12 11:19, Luis H. Forchesatto wrote:
> Its possible to separate full and incremental backups from the same client
> in separated volumes.
> 
> The client has one full backup weekly and for the other 6 days it has an
> incremental backup. I would like to keep the full backup separated and all
> the incremental backups that is run during the week be stored in a single
> volume.

Hi Louis,

Yes, you can do this withing a Schedule:

An example is below, but it kind of funny.

Basically it says Run a full on the 1st Friday at 9:30pm and write to the
File-Full pool.

Then, run differentials all the other possible Fridays at 9:30pm and write to
the File-Diff pool.  However, if the job gets upgraded to a full backup by
Bacula, then writ tot he File-Full pool

Finally, for all other days do an incremental backup, and write to the
File-Inc pool. However, if the job is upgraded by Bacula to Differential then
write to the File-Diff pool, and if the job is upgraded to a full backup by
Bacula then write to the File-Full pool.

This way, a full backup will be on 1 volume (unless of course it spans more
than one), your diffs will be on one volume, and your incs will be on another
volume.

You can do more, (or less) with this type of setup, but I hope this helps get
the point across.

(each line starting with the word "Run" should be 1 line but it wraps in my
email client.)

Schedule {
  Name = Client_X_Weekly
  Run = Full pool=File-Full
 1st fri at 21:30

  Run = Differential pool=File-Dill FullPool=File-Full
 2nd-5th fri at 21:30

  Run = Incremental  pool=File-Inc  FullPool=File-Full
 DifferentialPool=File-Diff  mon-thu,sat,sun at 21:30
}



-- 
Bill Arlofski
Reverse Polarity, LLC

I do not speak for, nor endorse the product or service below that will be
added to this email.

--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Separating Full backups from incremental in the volumes.

[Silas Moeckel]

On 12/10/2012 11:19 AM, Luis H. Forchesatto wrote:
Its possible to separate full and incremental backups from the same 
client in separated volumes.


The client has one full backup weekly and for the other 6 days it has 
an incremental backup. I would like to keep the full backup separated 
and all the incremental backups that is run during the week be stored 
in a single volume.


--
Att.*
*
Luis H. Forchesatto
Mail: luis_forchesa...@hotmail.com 


Yes you specify a different pool for the full backups (or any other 
level for that matter) inside the schedule resource.


Silas
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Separating Full backups from incremental in the volumes.

[Luis H. Forchesatto]

Its possible to separate full and incremental backups from the same client
in separated volumes.

The client has one full backup weekly and for the other 6 days it has an
incremental backup. I would like to keep the full backup separated and all
the incremental backups that is run during the week be stored in a single
volume.

-- 
Att.*
***
Luis H. Forchesatto
Mail: luis_forchesa...@hotmail.com
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Let copy job use available tapes?

[Chris Adams]

I am trying to use a tape library for a copy job (for off-site backups).
I have 3 magazines of tapes, and I planned to just rotate between them
for 3 weeks of off-site backups.  The first week went fine, but I can't
get the second week to work.

The problem appears to be that Bacula wants to use specific tapes,
rather than just using tapes that are in the magazine currently loaded.
For example, the first copy job used BVQ900-BVQ905 (bar code labels),
out of a magazine with 900-914.  I loaded the second magazine, with
915-929, but Bacula just keeps requesting BVQ906.  How can I get it to
see what is available and just use that?

-- 
Chris Adams 
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Full job terminates claiming no space left but tape gets more data

[Marco van Wieringen]

Robert Wirth  dfki.de> writes:
> ...
> Fseek on attributes file failed: ERR=No space left on device
> ...
> Last Volume Bytes:  1,283,646,163,968 (1.283 TB)
> 
> Really, the tape isn't full!  The other 5 jobs (and this one 
> after being rescheduled) continue to write that tape AD0016L5
> which by now has 1,633,895,424,000 Bytes written and is still 
> in Append state.
> 
> Question
> 
> What may be the cause for that "ERR=No space left on device"
> 
Older versions of Bacula (at least the old ones you talk about)
never did so called attribute spooling. The problem is probably the
amount of storage on the storage daemon see commit_attribute_spool()
function that emits the error "Fseek on attributes file failed".

You have a couple of options:

- disable attribute spooling (see spoolattributes setting on director)
- make sure enough space is available on disk for spooling the
  attributes first to disk before they are batch inserted into
  the database. Keep in mind that disabling attribute spooling
  and batch insert will slow down the overall backup probably.)
  The attribute spool files are created in either the spool directory
  or the working directory.

Marco


--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Full job terminates claiming no space left but tape gets more data

[Robert Wirth]

Hi,

Situation:
--
The backup strategy is doing a yearly Full, monthly Diff and daily Incr.
This worked well over years with Bacula 1.x and 2.x.  

Some months ago, I upgraded SD and DIR from Bacula 2.2.8 to 5.0.1
The Diff/Incr jobs continued to do well.

But now, while doing a Full for the first time under 5.0.1,
I got errors (and can reproduce them)

Hardware is two LTO-5 drives with a changer and a lot of tapes.

MaximumSpoolSize = 1024 GB
MaximumJobSpoolSize = 32 GB



What's up?
--
I've 6 bigger jobs (TB size each) in the same pool and start them 
at the same time.  They're spooling and despooling alternately
to the same tape(s) as expected.

Tape AD0018L5 is written by the 6 jobs and, when full, 
automatically replaced by next free tape AD0016L5.  The 
latter one is also written by the 6 jobs...  

Then, one of the 6 jobs fails (and is rescheduled):

Committing spooled data to Volume "AD0016L5". Despooling 22,147,812,023 bytes 
...
Fseek on attributes file failed: ERR=No space left on device
...
Last Volume Bytes:  1,283,646,163,968 (1.283 TB)

Really, the tape isn't full!  The other 5 jobs (and this one 
after being rescheduled) continue to write that tape AD0016L5
which by now has 1,633,895,424,000 Bytes written and is still 
in Append state.


Question

What may be the cause for that "ERR=No space left on device"

Same configuration worked well with Bacula <= 2.2.8 and LTO-3
(with lots more full tapes and changes during a Full session
than today when filling LTO-5)


More info:
--

08-Dez 05:46 bup-serv-dir JobId 130936: Start Backup JobId 130936, 
Job=lnv-91163.2012-12-08_04.46.11_20
08-Dez 05:46 bup-serv-dir JobId 130936: Using Device "Tape1A"
08-Dez 05:46 bup-serv-sd JobId 130936: Spooling data ...
...
09-Dez 20:17 bup-serv-sd JobId 130936: Committing spooled data to Volume 
"AD0016L5". Despooling 22,147,812,023 bytes ...
10-Dez 00:52 bup-serv-sd JobId 130936: Despooling elapsed time = 00:34:51, 
Transfer rate = 10.59 M Bytes/second
10-Dez 00:52 bup-serv-sd JobId 130936: Fatal error: Fseek on attributes file 
failed: ERR=No space left on device
10-Dez 00:52 bup-serv-dir JobId 130936: Error: Bacula bup-serv-dir 5.0.1 
(24Feb10): 10-Dez-2012 00:52:50
...
  Elapsed time:   1 day 19 hours 6 mins 36 secs
  Priority:   11
  FD Files Written:   445,124
  SD Files Written:   445,124
  FD Bytes Written:   277,792,113,411 (277.7 GB)
  SD Bytes Written:   277,875,653,641 (277.8 GB)
...
  Volume name(s): AD0018L5|AD0016L5
  Volume Session Id:  202
  Volume Session Time:1354723843
  Last Volume Bytes:  1,283,646,163,968 (1.283 TB)
  Non-fatal FD errors:0
  SD Errors:  0
  FD termination status:  OK
  SD termination status:  Error
  Termination:*** Backup Error ***


list volumes pool=ServerFull
...
|   1,012 | AD0016L5   | Append|   1 | 1,633,895,424,000 |1,634 |   
63,072,000 |   1 |   56 | 1 | LTO-5 | 2012-12-10 10:24:20 |
...
|   1,014 | AD0018L5   | Full  |   1 | 1,896,767,502,336 |1,897 |   
63,072,000 |   0 |   58 | 1 | LTO-5 | 2012-12-08 14:45:09 |


Regards, 

Robert

---
+++   German Research Center for Artificial Intelligence+++
---
Robert Wirth,Campus D32,Stuhlsatzenhausweg 3,66123 Saarbruecken
@phone:+49-681-85775-5078  Germany  mailto:robert.wi...@dfki.de
---



--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] mbuffer as tool for concurrent spool/despool

[Radosław Korzeniewski]

Hello,


2012/12/7 Stieneke, Dan 

>  Reading the mail archive, it seems I’m not alone in wanting to see
> concurrent spooling/despooling.
>
>
>
> Could we hand off the job to a standalone spooler like mbuffer (
> http://www.maier-komor.de/mbuffer.html)? For a simple tar to LTO-4 job it
> makes a tremendous improvement.
>
>
>

The main problem with concurrent spooling/despolling in Bacula Storage
Daemon is not a buffer/fifo (which is already implemented because we have a
spooling functionality already) but single job thread. It is not easy to
implement concurrency with a current code. In this case mbuffer does not
simplify things.


> The Bacuala developers would need to modify the storage daemon to allow
> the datastream to be diverted out and then back in through a pair of FIFOs
> before it gets [properly re-blocked? and] written to tape.
>

Simply speaking developers have to implement a multiple threads for single
job which write data to tape. This is a main missing code for build in
concurrent spooling/despooling. :)

The user would need to tie the FIFOs together with an external spooler. It
> might look like this:
>
>
>
> --/etc/bacula-sd.conf--
>
> 
>
> Device {
>
>   …
>
>   PushToFIFOSpooler = /path/to/bufreceive# bufreceive is a FIFO on
> this machine
>

First thread to read data from client and write data to the buffer -
implemented already.


>  PullFromFIFOSpooler  = /path/to/bufsend # bufsend is also a FIFO
>

Second thread to read data from buffer and write to bacula device - missing.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users