Re: [Bacula-users] Autochanger Configuration Help
Indeed it looks like my 2x 1tb mirror is bottle necking. I looked back at an older job I ran when I had data spooling off and it saved 930gb at a rate of 67.0 mb/s and then the same job ran again later with spooling on at a rate of 46.0 mb/s. Both these rates are much lower than the theoretical 160 mb/s max so should I assume even the fast server on 10gbe is a bottle neck? I guess I will keep the max jobs per client at 1 and look into setting up a ram disk for spooling. On Wed, Feb 6, 2019 at 10:06 AM Nate K wrote: > Thanks Martin, I will add the max clients jobs directive. That is a good > question regarding the mirror throughput, I’ll look into testing it. I > wonder if I could spool on a ramdisk (the bacula server has 32gb) since the > other server which is backed up is faster (raidz2 of 8x4tb 7200rpm drives > connected over 10gbe) or change to spool attributes only or leave spooling > off altogether. Is there a way to check if the drives are being > bottlenecked and causing “shoe shining”? > > On Feb 6, 2019, at 9:48 AM, Martin Simmons wrote: > > >>>>>> On Wed, 6 Feb 2019 00:05:21 -0500, Nate K said: > >> > >> I've tried to figure this out on my own with searches and going through > the > >> manual and I need some clarification. I've included the relevant > section > >> of the bacula-sd.conf file below. I'm confused because I think this > should > >> work properly but I am getting the message "is waiting on max Client > jobs" > >> for all additional jobs that are running after the first. Every other > >> daemon's config has maxes of 20 jobs. > > > > You need to increase "Maximum Concurrent Jobs" in the Client resource in > > bacula-dir.conf to prevent "is waiting on max Client jobs". It defaults > to 1. > > > > > >> I also am confused about the spool directive. The server running bacula > >> has 2x 1tb drives in a mirror zfs pool. I wonder how large I could make > >> the spool directives. It isn't clear to me when I set the spool > directives > >> in each device section, if they all share "Maximum Spool Size = 100g" > or if > >> each of the 5 drives will allocate 100gb then using 500gb total of my > disk > >> space. if I want to never exceed 80% used space on the zpool and I also > >> need 150gb for VMs and also need space for the catalog backing up > 12-15tb > >> of files, how high should I set the max and job spools? > > > > The "Maximum Spool Size" is the size per spool file, so you will use up > to > > 500GB. > > > > Does your 2 way mirror have enough throughput to feed 5 LTO3 drives > > simultaneously (or even 1 drive with 4 other jobs simultaneously writing > to > > their spool files)? > > > > __Martin > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Autochanger Configuration Help
Thanks Martin, I will add the max clients jobs directive. That is a good question regarding the mirror throughput, I’ll look into testing it. I wonder if I could spool on a ramdisk (the bacula server has 32gb) since the other server which is backed up is faster (raidz2 of 8x4tb 7200rpm drives connected over 10gbe) or change to spool attributes only or leave spooling off altogether. Is there a way to check if the drives are being bottlenecked and causing “shoe shining”? On Feb 6, 2019, at 9:48 AM, Martin Simmons wrote: >>>>>> On Wed, 6 Feb 2019 00:05:21 -0500, Nate K said: >> >> I've tried to figure this out on my own with searches and going through the >> manual and I need some clarification. I've included the relevant section >> of the bacula-sd.conf file below. I'm confused because I think this should >> work properly but I am getting the message "is waiting on max Client jobs" >> for all additional jobs that are running after the first. Every other >> daemon's config has maxes of 20 jobs. > > You need to increase "Maximum Concurrent Jobs" in the Client resource in > bacula-dir.conf to prevent "is waiting on max Client jobs". It defaults to 1. > > >> I also am confused about the spool directive. The server running bacula >> has 2x 1tb drives in a mirror zfs pool. I wonder how large I could make >> the spool directives. It isn't clear to me when I set the spool directives >> in each device section, if they all share "Maximum Spool Size = 100g" or if >> each of the 5 drives will allocate 100gb then using 500gb total of my disk >> space. if I want to never exceed 80% used space on the zpool and I also >> need 150gb for VMs and also need space for the catalog backing up 12-15tb >> of files, how high should I set the max and job spools? > > The "Maximum Spool Size" is the size per spool file, so you will use up to > 500GB. > > Does your 2 way mirror have enough throughput to feed 5 LTO3 drives > simultaneously (or even 1 drive with 4 other jobs simultaneously writing to > their spool files)? > > __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Autochanger Configuration Help
I've tried to figure this out on my own with searches and going through the
manual and I need some clarification. I've included the relevant section
of the bacula-sd.conf file below. I'm confused because I think this should
work properly but I am getting the message "is waiting on max Client jobs"
for all additional jobs that are running after the first. Every other
daemon's config has maxes of 20 jobs. Since I have 5 devices in my
autochanger, should I add a line in the Autochanger section for "Maximum
Concurrent Jobs = 5", or should I set them to 5 in the device sections?
I also am confused about the spool directive. The server running bacula
has 2x 1tb drives in a mirror zfs pool. I wonder how large I could make
the spool directives. It isn't clear to me when I set the spool directives
in each device section, if they all share "Maximum Spool Size = 100g" or if
each of the 5 drives will allocate 100gb then using 500gb total of my disk
space. if I want to never exceed 80% used space on the zpool and I also
need 150gb for VMs and also need space for the catalog backing up 12-15tb
of files, how high should I set the max and job spools?
Thanks,
Nate
Autochanger {
Name = ML6000
Device = ML6000-LTO3-0
Device = ML6000-LTO3-1
Device = ML6000-LTO3-2
Device = ML6000-LTO3-3
Device = ML6000-LTO3-4
Changer Device = /dev/tape/by-id/scsi-1ADIC_A0C0036905_LLA
Changer Command = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"
# %c = changer device
# %o = command (unload|load|loaded|list|slots)
# %S = slot index (1-based)
# %a = archive device (i.e., /dev/sd* name for tape
drive)
# %d = drive index (0-based)
}
Device {
Name = ML6000-LTO3-0
Archive Device = /dev/tape/by-id/scsi-3500308c099903094-nst
Device Type = Tape
Media Type = LTO3
Autochanger = Yes
Drive Index = 0
Autoselect = yes
RemovableMedia = yes
Random Access = no
AutomaticMount = yes
AlwaysOpen = yes
Maximum Spool Size = 100gb
Maximum Job Spool Size = 50gb
Spool Directory = /var/lib/bacula
Maximum Concurrent Jobs = 1
Control Device = /dev/tape/by-id/scsi-3500308c099903094-sg # must be SCSI
ctl for /dev/nst0
Alert Command = "/etc/bacula/scripts/tapealert %l"
}
Device {
Name = ML6000-LTO3-1
Archive Device = /dev/tape/by-id/scsi-3500308c099903098-nst
Device Type = Tape
Media Type = LTO3
Autochanger = Yes
Drive Index = 1
Autoselect = yes
RemovableMedia = yes
Random Access = no
AutomaticMount = yes
AlwaysOpen = yes
Maximum Spool Size = 100gb
Maximum Job Spool Size = 50gb
Spool Directory = /var/lib/bacula
Maximum Concurrent Jobs = 1
Control Device = /dev/tape/by-id/scsi-3500308c099903098-sg
Alert Command = "/etc/bacula/scripts/tapealert %l"
}
Device {
Name = ML6000-LTO3-2
Archive Device = /dev/tape/by-id/scsi-3500308c09990309c-nst
Device Type = Tape
Media Type = LTO3
Autochanger = Yes
Drive Index = 2
Autoselect = yes
RemovableMedia = yes
Random Access = no
AutomaticMount = yes
AlwaysOpen = yes
Maximum Spool Size = 100gb
Maximum Job Spool Size = 50gb
Spool Directory = /var/lib/bacula
Maximum Concurrent Jobs = 1
Control Device = /dev/tape/by-id/scsi-3500308c09990309c-sg
Alert Command = "/etc/bacula/scripts/tapealert %l"
}
Device {
Name = ML6000-LTO3-3
Archive Device = /dev/tape/by-id/scsi-3500308c099903000-nst
Device Type = Tape
Media Type = LTO3
Autochanger = Yes
Drive Index = 3
Autoselect = yes
RemovableMedia = yes
Random Access = no
AutomaticMount = yes
AlwaysOpen = yes
Maximum Spool Size = 100gb
Maximum Job Spool Size = 50gb
Spool Directory = /var/lib/bacula
Maximum Concurrent Jobs = 1
Control Device = /dev/tape/by-id/scsi-3500308c099903000-sg
Alert Command = "/etc/bacula/scripts/tapealert %l"
}
Device {
Name = ML6000-LTO3-4
Archive Device = /dev/tape/by-id/scsi-3500308c099903004-nst
Device Type = Tape
Media Type = LTO3
Autochanger = Yes
Drive Index = 4
Autoselect = yes
RemovableMedia = yes
Random Access = no
AutomaticMount = yes
AlwaysOpen = yes
Maximum Spool Size = 100gb
Maximum Job Spool Size = 50gb
Spool Directory = /var/lib/bacula
Maximum Concurrent Jobs = 1
Control Device = /dev/tape/by-id/scsi-3500308c099903004-sg
Alert Command = "/etc/bacula/scripts/tapealert %l"
}
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Make catalog backup switch pools
I thought about this some more and I figured out it could be done with two seperate jobs each scheduled to follow after the corresponding backup. I also realized I don't really need separate pools for full and diff backup. I think it would make pulling tapes out of the autochanger easier as they'd be listed separately but I think it's also an extra complication I can do without. On Sat, Feb 2, 2019 at 1:15 PM Nate K wrote: > Hi, > > I have four pools set up, library_a (the local pool), offsite_full, > offsite_diff, and Scratch. The way I planned this, when a full or > differential backup finishes for the offsite pools I would look at the list > of which tapes were written to and take them out of the autochanger and > move them to another location. > > I also would like a catalog backup written after each full or differential > backup at the end of whatever the last tape was used being from either the > full or differential pool. I'm not sure how I can configure the catalog > backup so it will switch pools from offsite_full to offsite_diff depending > on the last type of backup that ran. I want to avoid a tape in the > offsite_full pool being used solely for a single catalog backup after a > differential backup ran. > > I hope this makes sense. I have a feeling this isn't possible, I guess I > might just have to make catalog backups to another type of media manually > along with the config files and bootstrap files so I keep all that offsite > for possible disaster recovery. > > Thanks, > Nate > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Make catalog backup switch pools
Hi, I have four pools set up, library_a (the local pool), offsite_full, offsite_diff, and Scratch. The way I planned this, when a full or differential backup finishes for the offsite pools I would look at the list of which tapes were written to and take them out of the autochanger and move them to another location. I also would like a catalog backup written after each full or differential backup at the end of whatever the last tape was used being from either the full or differential pool. I'm not sure how I can configure the catalog backup so it will switch pools from offsite_full to offsite_diff depending on the last type of backup that ran. I want to avoid a tape in the offsite_full pool being used solely for a single catalog backup after a differential backup ran. I hope this makes sense. I have a feeling this isn't possible, I guess I might just have to make catalog backups to another type of media manually along with the config files and bootstrap files so I keep all that offsite for possible disaster recovery. Thanks, Nate ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Lack Persistent Device Naming to use with Alert Command
Thanks for explaining how to check the ATTRS value! The rules file works
well now. Awesome!
On Tue, Jan 29, 2019 at 6:50 PM Adam Nielsen wrote:
> > I ended up figuring out on accident my udev rules file did work, I had
> just
> > been trying to test it by restarting the service but it took a full
> reboot
> > for the new rules to get loaded.
>
> Normally you do something like this[1]:
>
> # udevadm control --reload-rules && udevadm trigger
>
> > *SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}!="8",
>
> This will match all devices that are not type 8 (so everything except
> autochangers).
>
> > This ends up creating symlinks for every sg* device, not just the tape
> > drives, so ideally it would be nice to figure out how to filter it.
>
> If you want to restrict it to a particular type, you can view udev
> attributes for specific devices like this:
>
> # udevadm info -a -n /dev/sch0 | grep type
> ATTRS{type}=="8"
>
> # udevadm info -a -n /dev/nst0 | grep type
> ATTRS{type}=="1"
>
> Here it suggests that changing the rule from type 8 to type 1 will only
> match tape drives. On my system a DVD drive is 5 and a hard drive is
> 0, so I guess using type 1 should work for tapes only.
>
> Cheers,
> Adam.
>
> [1]:
> https://unix.stackexchange.com/questions/39370/how-to-reload-udev-rules-without-reboot
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Lack Persistent Device Naming to use with Alert Command
I ended up figuring out on accident my udev rules file did work, I had just
been trying to test it by restarting the service but it took a full reboot
for the new rules to get loaded. This was what I did:
*> cat /etc/udev/rules.d/70-scsictrldev.rules*
*SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}!="8",
IMPORT{program}="scsi_id --sg-version=3 --export --whitelisted -d
$devnode", \*
* SYMLINK+="tape/by-id/scsi-$env{ID_SERIAL}-sg"*
This ends up creating symlinks for every sg* device, not just the tape
drives, so ideally it would be nice to figure out how to filter it. The
extra symlinks are just clutter though and don't harm anything so I'll call
this good for now.
Thanks,
Nate
On Mon, Jan 28, 2019 at 7:07 PM Nate K wrote:
> I tried creating a new rules file based on the existing tape rules but I
> didn't have any luck making it work and I didn't know how to debug what I'd
> done. I ended up creating this bash script to create the symlinks and it
> worked so I think I'll call it good:
>
> *#!/bin/bash*
>
> *#create persistent device paths for tape drive control devices*
>
> *n=($(ls /dev | grep sg[0123456789]))*
>
> *for i in "${n[@]}"*
> *do*
> *model=($(/lib/udev/scsi_id --page=0x80 --whitelisted /dev/$i))*
> *if [ "$model" == "SIBM" ]; then*
> *serial=($(/lib/udev/scsi_id --page=0x83 --whitelisted /dev/$i))*
> *ln -s /dev/$i /dev/tape/by-id/scsi-$serial-sg*
> *fi*
> *done*
>
> A udev rule would be nice because it could work for a lot of users using
> different hardware but I've run out of patience. This script can be
> adapted for other hardware simply by changing the "SIBM" model to the
> output of the" scsi_id --page=0x80" command for a different model drive.
>
> Thanks,
> Nate
>
>
> On Sun, Jan 27, 2019 at 9:14 PM Nate K wrote:
>
>> This is the rules file, I will have to spend some time going through it.
>> Thanks.
>>
>> *> cat /lib/udev/rules.d/60-persistent-storage-tape.rules*
>> *# do not edit this file, it will be overwritten on update*
>>
>> *# persistent storage links: /dev/tape/{by-id,by-path}*
>>
>> *ACTION=="remove", GOTO="persistent_storage_tape_end"*
>> *ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}=="1",
>> GOTO="persistent_storage_tape_end"*
>>
>> *# type 8 devices are "Medium Changers"*
>> *SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}=="8",
>> IMPORT{program}="scsi_id --sg-version=3 --export --whitelisted -d
>> $devnode", \*
>> * SYMLINK+="tape/by-id/scsi-$env{ID_SERIAL}"*
>>
>> *SUBSYSTEM!="scsi_tape", GOTO="persistent_storage_tape_end"*
>>
>> *KERNEL=="st*[0-9]|nst*[0-9]", ATTRS{ieee1394_id}=="?*",
>> ENV{ID_SERIAL}="$attr{ieee1394_id}", ENV{ID_BUS}="ieee1394"*
>> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="usb",
>> IMPORT{builtin}="usb_id"*
>> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="scsi",
>> KERNELS=="[0-9]*:*[0-9]", ENV{.BSG_DEV}="$root/bsg/$id"*
>> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*",
>> IMPORT{program}="scsi_id --whitelisted --export --device=$env{.BSG_DEV}",
>> ENV{ID_BUS}="scsi"*
>> *KERNEL=="st*[0-9]", ENV{ID_SERIAL}=="?*",
>> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}"*
>> *KERNEL=="nst*[0-9]", ENV{ID_SERIAL}=="?*",
>> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}-nst"*
>>
>> *# by-path (parent device path)*
>> *KERNEL=="st*[0-9]|nst*[0-9]", IMPORT{builtin}="path_id"*
>> *KERNEL=="st*[0-9]", ENV{ID_PATH}=="?*",
>> SYMLINK+="tape/by-path/$env{ID_PATH}"*
>> *KERNEL=="nst*[0-9]", ENV{ID_PATH}=="?*",
>> SYMLINK+="tape/by-path/$env{ID_PATH}-nst"*
>>
>> *LABEL="persistent_storage_tape_end"*
>>
>>
>> On Sun, Jan 27, 2019 at 3:14 AM Adam Nielsen
>> wrote:
>>
>>> > There are symlinks for the archive devices but not the tape drive
>>> control
>>> > devices which is what the tape alert commands need.
>>>
>>> Ah I see, you need the /dev/sg* equivalent for the /dev/st* in use.
>>>
>>> Can you copy the udev rules that produce symlinks for the autochangers
>>> (see link in my last post) and tweak them so that they also work for the
>>> drives themselves? It looks like they pick up anything that's a
>>> "type 8" device so if you duplicate the rule but change it to whatever
>>> type number that tape drives are, presumably you'd end up with the same
>>> result.
>>>
>>> > My only other idea is that maybe you could create a script that
>>> rewrites
>>> > the bacula-sd.conf every boot by parsing the output of this command as
>>> it
>>> > shows which sg* devices correspond to the st* ones:
>>>
>>> If you wanted to do that it might be easier to have the script
>>> create/update symlinks in /dev so you don't need to rewrite the Bacula
>>> config file. But this is pretty much what udev does already so better
>>> to get that working if you can.
>>>
>>> Cheers,
>>> Adam.
>>>
>>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Lack Persistent Device Naming to use with Alert Command
I tried creating a new rules file based on the existing tape rules but I
didn't have any luck making it work and I didn't know how to debug what I'd
done. I ended up creating this bash script to create the symlinks and it
worked so I think I'll call it good:
*#!/bin/bash*
*#create persistent device paths for tape drive control devices*
*n=($(ls /dev | grep sg[0123456789]))*
*for i in "${n[@]}"*
*do*
*model=($(/lib/udev/scsi_id --page=0x80 --whitelisted /dev/$i))*
*if [ "$model" == "SIBM" ]; then*
*serial=($(/lib/udev/scsi_id --page=0x83 --whitelisted /dev/$i))*
*ln -s /dev/$i /dev/tape/by-id/scsi-$serial-sg*
*fi*
*done*
A udev rule would be nice because it could work for a lot of users using
different hardware but I've run out of patience. This script can be
adapted for other hardware simply by changing the "SIBM" model to the
output of the" scsi_id --page=0x80" command for a different model drive.
Thanks,
Nate
On Sun, Jan 27, 2019 at 9:14 PM Nate K wrote:
> This is the rules file, I will have to spend some time going through it.
> Thanks.
>
> *> cat /lib/udev/rules.d/60-persistent-storage-tape.rules*
> *# do not edit this file, it will be overwritten on update*
>
> *# persistent storage links: /dev/tape/{by-id,by-path}*
>
> *ACTION=="remove", GOTO="persistent_storage_tape_end"*
> *ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}=="1",
> GOTO="persistent_storage_tape_end"*
>
> *# type 8 devices are "Medium Changers"*
> *SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}=="8",
> IMPORT{program}="scsi_id --sg-version=3 --export --whitelisted -d
> $devnode", \*
> * SYMLINK+="tape/by-id/scsi-$env{ID_SERIAL}"*
>
> *SUBSYSTEM!="scsi_tape", GOTO="persistent_storage_tape_end"*
>
> *KERNEL=="st*[0-9]|nst*[0-9]", ATTRS{ieee1394_id}=="?*",
> ENV{ID_SERIAL}="$attr{ieee1394_id}", ENV{ID_BUS}="ieee1394"*
> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="usb",
> IMPORT{builtin}="usb_id"*
> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="scsi",
> KERNELS=="[0-9]*:*[0-9]", ENV{.BSG_DEV}="$root/bsg/$id"*
> *KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*",
> IMPORT{program}="scsi_id --whitelisted --export --device=$env{.BSG_DEV}",
> ENV{ID_BUS}="scsi"*
> *KERNEL=="st*[0-9]", ENV{ID_SERIAL}=="?*",
> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}"*
> *KERNEL=="nst*[0-9]", ENV{ID_SERIAL}=="?*",
> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}-nst"*
>
> *# by-path (parent device path)*
> *KERNEL=="st*[0-9]|nst*[0-9]", IMPORT{builtin}="path_id"*
> *KERNEL=="st*[0-9]", ENV{ID_PATH}=="?*",
> SYMLINK+="tape/by-path/$env{ID_PATH}"*
> *KERNEL=="nst*[0-9]", ENV{ID_PATH}=="?*",
> SYMLINK+="tape/by-path/$env{ID_PATH}-nst"*
>
> *LABEL="persistent_storage_tape_end"*
>
>
> On Sun, Jan 27, 2019 at 3:14 AM Adam Nielsen
> wrote:
>
>> > There are symlinks for the archive devices but not the tape drive
>> control
>> > devices which is what the tape alert commands need.
>>
>> Ah I see, you need the /dev/sg* equivalent for the /dev/st* in use.
>>
>> Can you copy the udev rules that produce symlinks for the autochangers
>> (see link in my last post) and tweak them so that they also work for the
>> drives themselves? It looks like they pick up anything that's a
>> "type 8" device so if you duplicate the rule but change it to whatever
>> type number that tape drives are, presumably you'd end up with the same
>> result.
>>
>> > My only other idea is that maybe you could create a script that rewrites
>> > the bacula-sd.conf every boot by parsing the output of this command as
>> it
>> > shows which sg* devices correspond to the st* ones:
>>
>> If you wanted to do that it might be easier to have the script
>> create/update symlinks in /dev so you don't need to rewrite the Bacula
>> config file. But this is pretty much what udev does already so better
>> to get that working if you can.
>>
>> Cheers,
>> Adam.
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Lack Persistent Device Naming to use with Alert Command
This is the rules file, I will have to spend some time going through it.
Thanks.
*> cat /lib/udev/rules.d/60-persistent-storage-tape.rules*
*# do not edit this file, it will be overwritten on update*
*# persistent storage links: /dev/tape/{by-id,by-path}*
*ACTION=="remove", GOTO="persistent_storage_tape_end"*
*ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}=="1",
GOTO="persistent_storage_tape_end"*
*# type 8 devices are "Medium Changers"*
*SUBSYSTEM=="scsi_generic", SUBSYSTEMS=="scsi", ATTRS{type}=="8",
IMPORT{program}="scsi_id --sg-version=3 --export --whitelisted -d
$devnode", \*
* SYMLINK+="tape/by-id/scsi-$env{ID_SERIAL}"*
*SUBSYSTEM!="scsi_tape", GOTO="persistent_storage_tape_end"*
*KERNEL=="st*[0-9]|nst*[0-9]", ATTRS{ieee1394_id}=="?*",
ENV{ID_SERIAL}="$attr{ieee1394_id}", ENV{ID_BUS}="ieee1394"*
*KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="usb",
IMPORT{builtin}="usb_id"*
*KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="scsi",
KERNELS=="[0-9]*:*[0-9]", ENV{.BSG_DEV}="$root/bsg/$id"*
*KERNEL=="st*[0-9]|nst*[0-9]", ENV{ID_SERIAL}!="?*",
IMPORT{program}="scsi_id --whitelisted --export --device=$env{.BSG_DEV}",
ENV{ID_BUS}="scsi"*
*KERNEL=="st*[0-9]", ENV{ID_SERIAL}=="?*",
SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}"*
*KERNEL=="nst*[0-9]", ENV{ID_SERIAL}=="?*",
SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}-nst"*
*# by-path (parent device path)*
*KERNEL=="st*[0-9]|nst*[0-9]", IMPORT{builtin}="path_id"*
*KERNEL=="st*[0-9]", ENV{ID_PATH}=="?*",
SYMLINK+="tape/by-path/$env{ID_PATH}"*
*KERNEL=="nst*[0-9]", ENV{ID_PATH}=="?*",
SYMLINK+="tape/by-path/$env{ID_PATH}-nst"*
*LABEL="persistent_storage_tape_end"*
On Sun, Jan 27, 2019 at 3:14 AM Adam Nielsen wrote:
> > There are symlinks for the archive devices but not the tape drive control
> > devices which is what the tape alert commands need.
>
> Ah I see, you need the /dev/sg* equivalent for the /dev/st* in use.
>
> Can you copy the udev rules that produce symlinks for the autochangers
> (see link in my last post) and tweak them so that they also work for the
> drives themselves? It looks like they pick up anything that's a
> "type 8" device so if you duplicate the rule but change it to whatever
> type number that tape drives are, presumably you'd end up with the same
> result.
>
> > My only other idea is that maybe you could create a script that rewrites
> > the bacula-sd.conf every boot by parsing the output of this command as it
> > shows which sg* devices correspond to the st* ones:
>
> If you wanted to do that it might be easier to have the script
> create/update symlinks in /dev so you don't need to rewrite the Bacula
> config file. But this is pretty much what udev does already so better
> to get that working if you can.
>
> Cheers,
> Adam.
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Lack Persistent Device Naming to use with Alert Command
There are symlinks for the archive devices but not the tape drive control devices which is what the tape alert commands need. I did already have it set to use the symlink for the autochanger at least so that won't jump around. *> ls -l /dev/tape/by-id* *total 0lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-1ADIC_A0C0036905_LLA -> ../../sg8lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-3500308c099903000 -> ../../st4lrwxrwxrwx 1 root root 10 Jan 26 23:35 scsi-3500308c099903000-nst -> ../../nst4lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-3500308c099903004 -> ../../st1lrwxrwxrwx 1 root root 10 Jan 26 23:35 scsi-3500308c099903004-nst -> ../../nst1lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-3500308c099903094 -> ../../st3lrwxrwxrwx 1 root root 10 Jan 26 23:35 scsi-3500308c099903094-nst -> ../../nst3lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-3500308c099903098 -> ../../st0lrwxrwxrwx 1 root root 10 Jan 26 23:35 scsi-3500308c099903098-nst -> ../../nst0lrwxrwxrwx 1 root root 9 Jan 26 23:35 scsi-3500308c09990309c -> ../../st2* *lrwxrwxrwx 1 root root 10 Jan 26 23:35 scsi-3500308c09990309c-nst -> ../../nst2* My only other idea is that maybe you could create a script that rewrites the bacula-sd.conf every boot by parsing the output of this command as it shows which sg* devices correspond to the st* ones: *> lsscsi -g[0:0:0:0]cd/dvd HL-DT-ST DVD-ROM GDR-T10N 1.02 /dev/sr0 /dev/sg0[2:0:0:0]tapeIBM ULTRIUM-TD3 93GM /dev/st0 /dev/sg1[4:0:0:0]tapeIBM ULTRIUM-TD3 93GM /dev/st1 /dev/sg2[5:0:0:0]tapeIBM ULTRIUM-TD3 93GM /dev/st4 /dev/sg6[5:0:0:1]mediumx ADIC Scalar i500 630G /dev/sch0 /dev/sg8[6:0:0:0]tapeIBM ULTRIUM-TD3 93GM /dev/st2 /dev/sg3[7:0:0:0]tapeIBM ULTRIUM-TD3 93GM /dev/st3 /dev/sg4[9:0:0:0]diskATA ST1000VN002-2EY1 SC60 /dev/sda /dev/sg5[9:0:1:0]diskATA ST1000VN002-2EY1 SC60 /dev/sdb /dev/sg7* Thanks, Nate On Sun, Jan 27, 2019 at 12:42 AM Adam Nielsen wrote: > > I'm using Ubuntu 18.04.1 with Bacula 9.0.6 from the main repo. I'd like > to > > enable the above alerts from the sample bacula-sd.conf but can't because > my > > system is assigning random device names after each reboot so that > > /dev/nst0, /dev/nst1, etc. can be any /dev/sg* number. I looked into > > creating a udev rule for the drives but there's no output from the > control > > device that can be uniquely identified (DEVPATH can't be used in a udev > > rule I've found out): > > On my Arch Linux system, udev already has persistent rules for tapes > in /dev/tape: > > $ ls /dev/tape/by-id/ > total 0 > drwxr-xr-x 2 root root 100 Dec 18 20:45 . > drwxr-xr-x 4 root root 80 Dec 18 18:22 .. > lrwxrwxrwx 1 root root 9 Dec 19 23:21 scsi-350014380125815e0 -> ../../sg6 > lrwxrwxrwx 1 root root 9 Dec 19 23:21 scsi-350014380125815e1 -> ../../st0 > lrwxrwxrwx 1 root root 10 Dec 19 23:21 scsi-350014380125815e1-nst -> > ../../nst0 > > sg6 is the autochanger on my system. > > If your system doesn't have these, and no update to udev is available, > maybe you can copy the rules across from a newer version of udev[1]? > > > Is there another way do make this work? There's a more recent firmware > for > > my tape library, a Dell ML6000, that I've held off installing. Do you > > think a new firmware might allow the sg driver to provide more output? > > Nope, this is all part of the udev rule system. > > Cheers, > Adam. > > [1]: > https://github.com/systemd/systemd/blob/master/rules/60-persistent-storage-tape.rules > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Lack Persistent Device Naming to use with Alert Command
*# ## # New alert command in Bacula 9.0.0# # Note: you must have the sg3_utils (rpms) or the# #sg3-utils (deb) installed on your system.# #and you must set the correct control device that# #corresponds to the Archive Device# Control Device = /dev/sg?? # must be SCSI ctl for /dev/nst0# Alert Command = "/etc/bacula/scripts/tapealert %l"## ## # Enable the Alert command only if you have the mtx package loaded# # Note, apparently on some systems, tapeinfo resets the SCSI controller# # thus if you turn this on, make sure it does not reset your SCSI# # controller. I have never had any problems, and smartctl does# # not seem to cause such problems.# ## Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"# If you have smartctl, enable this, it has more info than tapeinfo# Alert Command = "sh -c 'smartctl -H -l error %c'"* *#}* I'm using Ubuntu 18.04.1 with Bacula 9.0.6 from the main repo. I'd like to enable the above alerts from the sample bacula-sd.conf but can't because my system is assigning random device names after each reboot so that /dev/nst0, /dev/nst1, etc. can be any /dev/sg* number. I looked into creating a udev rule for the drives but there's no output from the control device that can be uniquely identified (DEVPATH can't be used in a udev rule I've found out): *> udevadm info -n /dev/sg1 --query=allP: /devices/pci:00/:00:02.0/:05:00.0/:06:01.0/:09:00.0/:0a:01.0/:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/scsi_generic/sg1N: sg1E: DEVNAME=/dev/sg1E: DEVPATH=/devices/pci:00/:00:02.0/:05:00.0/:06:01.0/:09:00.0/:0a:01.0/:0b:00.0/host2/rport-2:0-0/target2:0:0/2:0:0:0/scsi_generic/sg1E: MAJOR=21E: MINOR=1E: SUBSYSTEM=scsi_generic* Is there another way do make this work? There's a more recent firmware for my tape library, a Dell ML6000, that I've held off installing. Do you think a new firmware might allow the sg driver to provide more output? Should I quit worrying about getting alerts working since the tape library itself has a similar capability or is it important for Bacula to function correctly? Thanks, Nate ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
