subject:"\[Bacula\-users\] Recycling issue"

[Bacula-users] Recycling issue

2014-08-05 Thread Gael Guilmin

Hello,
I'm facing an issue with Bacula (5.2.12) on a Ubuntu server (12.04.4 LTS): I 
don't know why, but the server, after finishing writing on a tape, tries to use 
a tape that is not in the autochanger, despite the fact that there are plenty 
of online and usable tape in the Scratch pool...

Any idea?

Best regards,
Gael
--
System Administrator / IT
[PDM_Logo_RGB - registered]
78, Avenue du XXeme Corps
54000 Nancy
France
Phone(Paris): +33 (0)1 49 49 05 25
Phone(Nancy): +33 (0)3 83 67 66 39
Cell: +33 (0)6 10 18 48 43
Web: www.pdgm.com

[Description: Description: Description: 
http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/linkedin.jpg]
 [Description: Description: Description: 
http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/twitter.jpg]
   [Description: Description: Description: 
http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/facebook.jpg]
   [Description: Description: Description: 
http://www.pdgm.com/_media/newsletters/USA/USA_Edition3_June11_files/youtube.jpg]
 

--- This e-mail, including any attached files, may contain 
confidential and privileged information for the sole use of the intended 
recipient. Any review, use, distribution, or disclosure by others is strictly 
prohibited. If you are not the intended recipient (or authorized to receive 
information for the intended recipient), please contact the sender by reply 
e-mail and delete all copies of this message.
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-05 Thread Joe Rhodes

I’m also running into this issue.  

In my case, I’ve setup a disk-based auto-changer (vchanger) with two pools:  
Full and Weekly.  Each pool has multiple drives with 8 volumes on each drive.  
The script that mounts the disk also issues an “update slots” command in 
bconsole, and I can verify that bacula sees the correct volumes as “InChanger”.

When I swap out disks (weekly), Bacula continues to look for volumes that were 
append-able, but on the previous disk (and not currently marked as “InChanger”) 
and refuses to write to the append-able volumes on the new disk that are marked 
as “InChanger”.  It marks all the previous volumes it cannot access as “Error”, 
and only then will it begin writing to the currently loaded volumes.

This  line from the “Recycling Algorithm” led me to believe that it would use a 
volume that was InChanger first:  "If the request is for an Autochanger device, 
look only for Volumes in the Autochanger (i.e. with InChanger set and that have 
the correct Storage device).”

But it seems I’m missing something.  I’ve included relevant bits from my 
bacula-dir.conf file if anyone can see what I’m doing wrong.  I’m getting this 
on two systems, one running Bacula 5.2.12 and one running 7.0.4.

My other option is to split these all into separate pools, but that’s going to 
make scheduling much harder.

Thanks in advance!
-Joe Rhodes


# Definition of file Virtual Autochanger device

Storage {
  Name = removeable-drives
  Address = 10.11.1.2
  SDPort = 9103
  Password = “password"
  Device = ExternalDrives
  Media Type = File
  Maximum Concurrent Jobs = 1
}

# Pools
Pool {
  Name = FullSet
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Use Duration = 1 week
  #We're on an 2 month rotation
  VolumeRetention = 58 days
  Maximum Volumes = 32  # 4 drives with 8 volumes on each
  Storage = removeable-drives
}


Pool {
  Name = Weekly
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Use Duration = 1 week
  VolumeRetention =  3 weeks
  Maximum Volumes = 40  # 5 drives with 8 volumes on each
  Storage = removeable-drives
}


And my bacula-sd.conf:

# Define a Virtual autochanger

Autochanger {
  Name = ExternalDrives
  Device = ExternalDrives-Drive1
  Changer Command = "/usr/local/bin/vchanger -u root  %c %o %S %a %d"
  Changer Device = "/etc/bacula/vchanger1.conf"
}

Device {
  Name = ExternalDrives-Drive1
  DriveIndex = 0
  Autochanger = yes
  DeviceType = File
  Media Type = File
  Archive Device = /var/lib/bacula/removeable-drives/0/drive0
  Random Access = Yes;
  AutomaticMount = yes;  
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 5;
  Maximum Volume Size = 476250 M;  # with 8 volumes per drive, this should 
leave about 
# 4,800 MB free for other things, or about 
0.125%
}




> From: Gael Guilmin 
> Subject: [Bacula-users] Recycling issue
> To: "bacula-users@lists.sourceforge.net"
>   
> Message-ID:
>   
> <05520d1d14f2408f8d048f1e1dfe6...@blupr07mb836.namprd07.prod.outlook.com>
>   
> Content-Type: text/plain; charset="us-ascii"
> 
> Hello,
> I'm facing an issue with Bacula (5.2.12) on a Ubuntu server (12.04.4 LTS): I 
> don't know why, but the server, after finishing writing on a tape, tries to 
> use a tape that is not in the autochanger, despite the fact that there are 
> plenty of online and usable tape in the Scratch pool...
> 
> Any idea?
> 
> Best regards,
> Gael
> --
> System Administrator / IT
> [PDM_Logo_RGB - registered]
> 78, Avenue du XXeme Corps
> 54000 Nancy
> France
> Phone(Paris): +33 (0)1 49 49 05 25
> Phone(Nancy): +33 (0)3 83 67 66 39
> Cell: +33 (0)6 10 18 48 43
> Web: www.pdgm.com<http://www.pdgm.com/>
--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-05 Thread Kern Sibbald

th 8 volumes per drive,
this should leave about 
                                # 4,800 MB free for other
things, or about 0.125%
}

  
  
  
  
  
  
  
  
From: Gael Guilmin <gael.guil...@pdgm.com>
  Subject: [Bacula-users] Recycling issue
  To: "bacula-users@lists.sourceforge.net"
   <bacula-users@lists.sourceforge.net>
  Message-ID:
   <05520d1d14f2408f8d048f1e1dfe6...@blupr07mb836.namprd07.prod.outlook.com>
   
  Content-Type: text/plain; charset="us-ascii"
  
  Hello,
  I'm facing an issue with Bacula (5.2.12) on a Ubuntu server
  (12.04.4 LTS): I don't know why, but the server, after
  finishing writing on a tape, tries to use a tape that is not
  in the autochanger, despite the fact that there are plenty of
  online and usable tape in the Scratch pool...
  
  Any idea?
  
  Best regards,
  Gael
  --
  System Administrator / IT
  [PDM_Logo_RGB - registered]
  78, Avenue du XXeme Corps
  54000 Nancy
  France
  Phone(Paris): +33 (0)1 49 49 05 25
  Phone(Nancy): +33 (0)3 83 67 66 39
  Cell: +33 (0)6 10 18 48 43
  Web: www.pdgm.com<http://www.pdgm.com/>
  
  
  
  
  --
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-05 Thread Gael Guilmin

Hello Kern,
Thanks for your answer.
But I don’t think what’s you’re pointing is related to my case as I‘ve only one 
autochanger.

To give more detail on my configuration:

· I make a daily incremental backup on different servers (Pool Default)

· Every weekend I make a full backup of the servers (Pool_Full_Week)

· I make a monthly full backup of servers (Pool_Full_Month)

Here are my pools definitions in bacula-dir.conf:
# Default pool definition
Pool {
  Name = Default
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 30 days  # one month
  Cleaning Prefix = "CLN"
  RecyclePool = Scratch
}

# Full pool Month definition
Pool {
  Name = Pool_Full_Month
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 7 years # one year
  Cleaning Prefix = "CLN"
  RecyclePool = Scratch
}

# Full pool Week definition
Pool {
  Name = Pool_Full_Week
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 1 years # one year
  Cleaning Prefix = "CLN"
  RecyclePool = Scratch
}

# Scratch pool definition
Pool {
  Name = Scratch
  Pool Type = Backup
  Cleaning Prefix = "CLN"
}

Here is my Storage definition in bacula-dir.conf:
Storage {
Name = Autochanger
Address = 192.168.27.7 # name or IP address of the server that hosts the daemon 
storage
SDPort = 9103
Password = ""
Device = "Autochanger" # the same as defined in bacula-sd.conf
Media Type = LTO-5
}

Here is my bacula-sd.conf:
Autochanger {
  Name = Autochanger
  Device = Drive-1
  Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"
  Changer Device = /dev/sg2
}

Device {
  Name = Drive-1  #
  Drive Index = 0
  Media Type = LTO-5
  Archive Device = /dev/nst0
  AutomaticMount = yes;   # when device opened, read it
  AlwaysOpen = yes;
  RemovableMedia = yes;
  RandomAccess = no;
  Autochanger = yes;
  LabelMedia = no;
}


I don’t get why bacula tries to use tape that are not in the autochanger, 
beside the fact that there are a lot of usable tapes in it that are in the 
Scratch pool…

Thanks for your help,
Gael Guilmin


From: Kern Sibbald [mailto:k...@sibbald.com]
Sent: Wednesday, August 06, 2014 7:42 AM
To: Joe Rhodes; bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Recycling issue

Hello,

Probably 95% of all such problems are due to configurations where the MediaType 
is not properly set.

- If you are using any form of autochanger, each autochanger must have a 
*unique* MediaType.

- Each Device configuration in the SD must have a unique MediaType for each 
different ArchiveDevice definition.  Said the other way around, each 
ArchiveDevice definition must have its own unique MediaType.

The .conf files you list are incomplete, so it is not possible to analyze them, 
but when I see:  MediaType = File, that raises a red flag that indicates that 
there may be other Storage devices with the same MediaType, and if that is the 
case, sooner or later, you will find that Bacula is very confused.

In the next Bacula version, I will probably modify the SD and possibly the Dir 
to require unique MediaTypes, then this problem will be resolved.

I don't usually supply "support"  because I don't have the time, but this 
particular problem interests me from the standpoint that I would like to 
prevent it, but with no console/job output showing the failures and incomplete 
DIR and SD confs I cannot do any more.

Best regards,
Kern

On 08/05/2014 03:08 PM, Joe Rhodes wrote:
I’m also running into this issue.

In my case, I’ve setup a disk-based auto-changer (vchanger) with two pools:  
Full and Weekly.  Each pool has multiple drives with 8 volumes on each drive.  
The script that mounts the disk also issues an “update slots” command in 
bconsole, and I can verify that bacula sees the correct volumes as “InChanger”.

When I swap out disks (weekly), Bacula continues to look for volumes that were 
append-able, but on the previous disk (and not currently marked as “InChanger”) 
and refuses to write to the append-able volumes on the new disk that are marked 
as “InChanger”.  It marks all the previous volumes it cannot access as “Error”, 
and only then will it begin writing to the currently loaded volumes.

This  line from the “Recycling Algorithm” led me to believe that it would use a 
volume that was InChanger first:  "If the request is for an Autochanger device, 
look only for Volumes in the Autochanger (i.e. with InChanger set and that have 
the correct Storage device).”

But it seems I’m missing s

Re: [Bacula-users] Recycling issue

2014-08-06 Thread Martin Simmons

> On Wed, 06 Aug 2014 07:41:59 +0200, Kern Sibbald said:
> 
> In the next Bacula version, I will probably modify the SD and
> possibly the Dir to require unique MediaTypes, then this problem
> will be resolved.

I hope that forcing users to change the configured MediaType won't cause a
problem with old media (because the label contains the media type and won't
match anymore).

__Martin

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-06 Thread Martin Simmons

> On Wed, 6 Aug 2014 06:52:06 +, Gael Guilmin said:
> 
> Here is my Storage definition in bacula-dir.conf:
> Storage {
> Name = Autochanger
> Address = 192.168.27.7 # name or IP address of the server that hosts the 
> daemon storage
> SDPort = 9103
> Password = ""
> Device = "Autochanger" # the same as defined in bacula-sd.conf
> Media Type = LTO-5
> }

You need to add

Autochanger = yes

to this bacula-dir definition, otherwise Bacula will not apply the InChanger
logic.  I think this is missing from most of the examples.

__Martin

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-06 Thread Joe Rhodes

 I swap out disks (weekly), Bacula continues to look for volumes that 
>> were append-able, but on the previous disk (and not currently marked as 
>> “InChanger”) and refuses to write to the append-able volumes on the new disk 
>> that are marked as “InChanger”.  It marks all the previous volumes it cannot 
>> access as “Error”, and only then will it begin writing to the currently 
>> loaded volumes.
>> 
>> This  line from the “Recycling Algorithm” led me to believe that it would 
>> use a volume that was InChanger first:  "If the request is for an 
>> Autochanger device, look only for Volumes in the Autochanger (i.e. with 
>> InChanger set and that have the correct Storage device).”
>> 
>> But it seems I’m missing something.  I’ve included relevant bits from my 
>> bacula-dir.conf file if anyone can see what I’m doing wrong.  I’m getting 
>> this on two systems, one running Bacula 5.2.12 and one running 7.0.4.
>> 
>> My other option is to split these all into separate pools, but that’s going 
>> to make scheduling much harder.
>> 
>> Thanks in advance!
>> -Joe Rhodes
>> 
>> 
>> # Definition of file Virtual Autochanger device
>> 
>> Storage {
>>   Name = removeable-drives
>>   Address = 10.11.1.2
>>   SDPort = 9103
>>   Password = “password"
>>   Device = ExternalDrives
>>   Media Type = File
>>   Maximum Concurrent Jobs = 1
>> }
>> 
>> # Pools
>> Pool {
>>   Name = FullSet
>>   Pool Type = Backup
>>   Recycle = yes
>>   AutoPrune = yes
>>   Volume Use Duration = 1 week
>>   #We're on an 2 month rotation
>>   VolumeRetention = 58 days
>>   Maximum Volumes = 32  # 4 drives with 8 volumes on each
>>   Storage = removeable-drives
>> }
>> 
>> 
>> Pool {
>>   Name = Weekly
>>   Pool Type = Backup
>>   Recycle = yes
>>   AutoPrune = yes
>>   Volume Use Duration = 1 week
>>   VolumeRetention =  3 weeks
>>   Maximum Volumes = 40  # 5 drives with 8 volumes on each
>>   Storage = removeable-drives
>> }
>> 
>> 
>> And my bacula-sd.conf:
>> 
>> # Define a Virtual autochanger
>> 
>> Autochanger {
>>   Name = ExternalDrives
>>   Device = ExternalDrives-Drive1
>>   Changer Command = "/usr/local/bin/vchanger -u root  %c %o %S %a %d"
>>   Changer Device = "/etc/bacula/vchanger1.conf"
>> }
>> 
>> Device {
>>   Name = ExternalDrives-Drive1
>>   DriveIndex = 0
>>   Autochanger = yes
>>   DeviceType = File
>>   Media Type = File
>>   Archive Device = /var/lib/bacula/removeable-drives/0/drive0
>>   Random Access = Yes;
>>   AutomaticMount = yes;  
>>   RemovableMedia = no;
>>   AlwaysOpen = no;
>>   Maximum Concurrent Jobs = 5;
>>   Maximum Volume Size = 476250 M;  # with 8 volumes per drive, this should 
>> leave about 
>> # 4,800 MB free for other things, or about 
>> 0.125%
>> }
>> 
>> 
>> 
>> 
>>> From: Gael Guilmin 
>>> Subject: [Bacula-users] Recycling issue
>>> To: "bacula-users@lists.sourceforge.net"
>>>  
>>> Message-ID:
>>>  <05520d1d14f2408f8d048f1e1dfe6...@blupr07mb836.namprd07.prod.outlook.com>
>>>  
>>> Content-Type: text/plain; charset="us-ascii"
>>> 
>>> Hello,
>>> I'm facing an issue with Bacula (5.2.12) on a Ubuntu server (12.04.4 LTS): 
>>> I don't know why, but the server, after finishing writing on a tape, tries 
>>> to use a tape that is not in the autochanger, despite the fact that there 
>>> are plenty of online and usable tape in the Scratch pool...
>>> 
>>> Any idea?
>>> 
>>> Best regards,
>>> Gael
>>> --
>>> System Administrator / IT
>>> [PDM_Logo_RGB - registered]
>>> 78, Avenue du XXeme Corps
>>> 54000 Nancy
>>> France
>>> Phone(Paris): +33 (0)1 49 49 05 25
>>> Phone(Nancy): +33 (0)3 83 67 66 39
>>> Cell: +33 (0)6 10 18 48 43
>>> Web: www.pdgm.com<http://www.pdgm.com/>
>> 
>> 
>> --
>> Infragistics Professional
>> Build stunning WinForms apps today!
>> Reboot your WinForms applications with our WinForms controls. 
>> Build a bridge from your legacy apps to the future.
>> http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-06 Thread Kern Sibbald

Hello Martin,

Bacula never checks the Media Type vs what is in the label.

The problem for users who have not understood the importance of making
the Media Types unique will be to modify their catalog database.  Before
making this change, I will provide some reasonable solution.

Best regards,
Kern

On 08/06/2014 12:44 PM, Martin Simmons wrote:
>> On Wed, 06 Aug 2014 07:41:59 +0200, Kern Sibbald said:
>> In the next Bacula version, I will probably modify the SD and
>> possibly the Dir to require unique MediaTypes, then this problem
>> will be resolved.
> I hope that forcing users to change the configured MediaType won't cause a
> problem with old media (because the label contains the media type and won't
> match anymore).
>
> __Martin
>


--
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls. 
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Recycling issue

2014-08-10 Thread Joe Rhodes

Adding the line:

Autochanger = yes

to my storage definition in my bacula-dir.conf file did indeed solve the issue. 
 Bacula is now considering volumes only from the ones that are listed as 
“InChanger”.  Thanks for the pointer.

Cheers!
-Joe

On Aug 6, 2014, at 6:52 AM, Martin Simmons  wrote:

>> On Wed, 6 Aug 2014 06:52:06 +, Gael Guilmin said:
>> 
>> Here is my Storage definition in bacula-dir.conf:
>> Storage {
>> Name = Autochanger
>> Address = 192.168.27.7 # name or IP address of the server that hosts the 
>> daemon storage
>> SDPort = 9103
>> Password = ""
>> Device = "Autochanger" # the same as defined in bacula-sd.conf
>> Media Type = LTO-5
>> }
> 
> You need to add
> 
> Autochanger = yes
> 
> to this bacula-dir definition, otherwise Bacula will not apply the InChanger
> logic.  I think this is missing from most of the examples.
> 
> __Martin


--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

Re: [Bacula-users] Recycling issue

9 matches

Site Navigation

Mail list logo

Footer information