Hello dear all,
Abstractly spoken I need a concept for my Tier 2 to be able to write file
volumes to more than just 1 (HDD) Device (or MediaType), filling the HDDs up
one after the other. By this I mean that the individual HDDs are -NOT-
abstracted away by a logical layer device such as some RAID level (in such a
case we would just have 1 Device (or MediaType)).
(How) can this be done with Bacula?
To give you a different perspective on the problem that I am trying to solve:
If I use physical tape drives and autochangers I can always buy more tapes to
expand my storage. If the changer would only work with a limited set of tapes
(lets say tapes in a magazine that the autochanger can mechanically change
without human/manual help) it is more like the situation with volumes in files
instead of tapes. The autochanger can write to volumes until the HDD is full,
then the auto changing is limited to the set of files held by this HDD. New
backups can only be written if volumes get recycled.
And then comes the day when the HDD is full and the retention does not yet
allow for recycling.
This is where I am today.
What I tried so far: (- and it does not work for me-) (at the bottom of
this email please find attached all my directives for storage for my Director
und my SD (both on the same machine, same container).
I have structured my storage into tiers, where Tier 1 is held on internal SATA
HDDs and Tier 2 is held on external HDDs connected via USB3.
I would like to focus on Tier 2 storage for this posting (Tier 1 is not a
problem as it is held on a HDD array that can be expanded by replacing smaller
HDDs with larger HDDs without impacting existing file volumes.)
Tier 2 storage is declared as follows in the director:
Storage {
Name = "tier2-storage"
SdPort = 9103
Address = "redacted"
Password = "redacted"
Device = "tier2-ext-usb-changer"
MediaType = "tier2-ext-usb1-folder1-file"
Autochanger = yes
MaximumConcurrentJobs = 10
}
The device tier2-ext-usb1-folder1-file is an autochanger that has access to two
file volume changer devices, where /mnt/tier2-01 is on USB HDD 1 and
/mnt/tier2-02 is on USB HDD 2. These HDDs 1 and 2 are held in the same USB
case, but are addressed individually, they are explicitly not combined in a
RAID and (I would like to keep it this way).
Autochanger {
Name = "tier2-ext-usb-changer"
Device = "Tier2-FileChgr1-Dev1"
Device = "Tier2-FileChgr1-Dev2"
ChangerDevice = "/dev/null"
ChangerCommand = "/dev/null”
}
Device {
Name = "Tier2-FileChgr1-Dev1"
MediaType = "tier2-ext-usb1-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-01"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}
Device {
Name = "Tier2-FileChgr1-Dev2"
MediaType = "tier2-ext-usb2-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-02"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}
As in the Director Storage directive declared due to MediaType =
"tier2-ext-usb1-folder1-file” merely HDD 1 is used (and the MediaType directive
is mandatory for Storage in the Director).
Is there a way to allow the autochanger tier2-ext-usb-changer to also create
und use volumes on HDD2? This autochanger actually comprises of 2 “File”
devices, but now can only use one of them due to the different media types (and
I understand that the changing is meant regarding the volumes within each
“File" device, meaning for file volumes, that it can create/access the file
volumes within a given folder. What I imagine is that it could use file volumes
from any of the 2 Devices Tier2-FileChgr1-Dev1 and Tier2-FileChgr1-Dev2).
If there is no way to get this to work as I imagine it?
If not so, what is the correct way to handle my current situation: HDD 1 is
full now, no further volumes can be written to HDD1, only existing volumes on
HDD 1 could be recycled, but the retention time does not yet allow this.
How can I expand my USB-based Tier 2 storage to hold further volumes?
Note: I already have HDD 2 there in the same USB case waiting to be filled.
Adding further HDDs in the same USB case is not a problem, but each HDD must be
addressed individually (the USB case intentionally is not using a RAID
controller).
Coming back to the tape magazine analogy, I now have an autochanger where 1
magazine is full and it has a second magazine with empty tapes, but both
magazines have a different MediaType. Is there a chance that the second
magazine gets used?
All the best,
J/C
For completeness sake all relevant directives for the described situation:
========================================
bacula-sd.conf
========================================
Storage {
Name = "bacula-sd"
WorkingDirectory = "/opt/bacula/working"
PidDirectory = "/opt/bacula/working"
PluginDirectory = "/opt/bacula/plugins"
MaximumConcurrentJobs = 20
}
Device {
Name = "Restore-FileChgr1-Dev99"
MediaType = "restore-int-user0-folder1"
DeviceType = "File"
ArchiveDevice = "/mnt/restore"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
Autochanger = no
AutoSelect = no
DriveIndex = 99
}
Device {
Name = "Tier1-FileChgr1-Dev1"
MediaType = "tier1-int-user0-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier1-01"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}
Device {
Name = "Tier1-FileChgr1-Dev2"
MediaType = "tier1-int-user0-folder2-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier1-02"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
DriveIndex = 1
}
Device {
Name = "Tier2-FileChgr1-Dev1"
MediaType = "tier2-ext-usb1-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-01"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}
Device {
Name = "Tier2-FileChgr1-Dev2"
MediaType = "tier2-ext-usb2-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-02"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}
Autochanger {
Name = "tier1-int-user0-folder-changer"
Device = "Restore-FileChgr1-Dev99"
Device = "Tier1-FileChgr1-Dev1"
Device = "Tier1-FileChgr1-Dev2"
ChangerDevice = "/dev/null"
ChangerCommand = "/dev/null"
}
Autochanger {
Name = "tier2-ext-usb-changer"
Device = "Tier2-FileChgr1-Dev1"
Device = "Tier2-FileChgr1-Dev2"
ChangerDevice = "/dev/null"
ChangerCommand = "/dev/null”
}
========================================
bacula-dir.conf
========================================
Storage {
Name = "unraid-tier1-restore"
SdPort = 9103
Address = "redacted"
Password = "redacted"
Device = "Restore-FileChgr1-Dev99"
MediaType = "tier1-int-user0-folder1-file"
Autochanger = yes
MaximumConcurrentJobs = 10
}
Storage {
Name = "unraid-tier1-storage"
SdPort = 9103
Address = "redacted"
Password = "redacted"
Device = "tier1-int-user0-folder-changer"
MediaType = "tier1-int-user0-folder1-file"
Autochanger = yes
MaximumConcurrentJobs = 10
}
Storage {
Name = "unraid-tier2-storage"
SdPort = 9103
Address = "redacted"
Password = "redacted"
Device = "tier2-ext-usb-changer"
MediaType = "tier2-ext-usb1-folder1-file"
Autochanger = yes
MaximumConcurrentJobs = 10
}
Pool {
Name = "Default"
PoolType = "Backup"
MaximumVolumes = 10
MaximumVolumeBytes = 1073741824
VolumeRetention = 86400
AutoPrune = yes
Recycle = yes
}
Pool {
Name = "Scratch"
PoolType = "Backup"
}
Pool {
Name = "unraid-appdata-full-tier1"
PoolType = "Backup"
LabelFormat = "unraid-appdata-full-tier1-vol-"
ActionOnPurge = "Truncate"
MaximumVolumes = 100
MaximumVolumeJobs = 1
MaximumVolumeBytes = 50000000000
VolumeRetention = 6048000
AutoPrune = no
Catalog = "MyCatalog"
}
Pool {
Name = "unraid-appdata-full-tier2"
PoolType = "Backup"
LabelFormat = "unraid-appdata-full-tier2-vol-"
ActionOnPurge = "Truncate"
MaximumVolumes = 100
MaximumVolumeJobs = 1
MaximumVolumeBytes = 50000000000
VolumeRetention = 6048000
AutoPrune = no
Catalog = "MyCatalog"
}
Pool {
Name = "unraid-appdata-incr-tier1"
PoolType = "Backup"
LabelFormat = "unraid-appdata-incr-tier1-vol-"
ActionOnPurge = "Truncate"
MaximumVolumes = 175
MaximumVolumeJobs = 6
MaximumVolumeBytes = 50000000000
VolumeRetention = 6048000
AutoPrune = no
Catalog = "MyCatalog"
}
Pool {
Name = "unraid-appdata-incr-tier2"
PoolType = "Backup"
LabelFormat = "unraid-appdata-incr-tier2-vol-"
ActionOnPurge = "Truncate"
MaximumVolumes = 175
MaximumVolumeJobs = 6
MaximumVolumeBytes = 50000000000
VolumeRetention = 6048000
AutoPrune = no
Catalog = "MyCatalog”
}
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
