Re: [Bacula-users] Virtual autochanger
On 2016-10-02 11:46, Dimitri Maziuk wrote: > Bacula's calim to support backup to disk is, let's say "optimistic", ^ claim See also: lysdexia -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual autochanger
On 2016-10-01 15:37, Timo Neuvonen wrote: > Will there be any problem in this case, if I try to simplify the conf files > and drop away the "autochanger" and one of the two "storage devices" it > refers to, and just had a single "storage device" per each media type? Bacula's calim to support backup to disk is, let's say "optimistic", without vchanger: https://sourceforge.net/projects/vchanger/ Either that or you have to define each filesystem as holding its own "media type" and all that. Note that you can still run concurrent backups and only serialize writes: use a fast disk (SSD) for a spool device. And that if you back up to commodity HDDs, you'll need to deal with disk failures: something neither bacula nor vchanger will help you with. Or you can rethink your requirements and back up to a single zfs. Then bacula will work without any of those hoops and you can raidz your disks against failures. Dima -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual autochanger
> As you said, the Autochangers/Devices definitions on the initial
> bacula-sd.conf file are suggestions on how to use a virtual disk
> autochanger with Bacula.
> There is a lot of ways to work with disk volumes others that using disk
> autochangers, yes.
> It will mostly depend on your environment, needs, expectations of what you
> want/need.
>
> Could you please explain us better this "disks will be located in
> physically different locations"?
Maybe I'd better have omitted that last sentence (in parenthesis), it now
mislead the point into not-so-essential area. So I'll try to start over:
My primary interest was if I can drop the "virtual autochanger" and one of
the "devices" away from the example (derived from 7.0.5 example confs)
below, without a significant functionality loss in a small environment:
==
# Define a Virtual autochanger
#
Autochanger {
Name = FileChgr1
Device = FileChgr1-Dev1, FileChgr1-Dev2
Changer Command = ""
Changer Device = /dev/null
}
Device {
Name = FileChgr1-Dev1
Media Type = File1
Archive Device = /bdisk1
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
Device {
Name = FileChgr1-Dev2
Media Type = File1
Archive Device = /bdisk1
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
==
Which would essentially result in something like this:
(example copied from a very old sd.conf example of Bacula 2.4.0)
#Device {
# Name = FileStorage
# Media Type = File
# Archive Device = /tmp
# LabelMedia = yes; # lets Bacula label unlabeled media
# Random Access = Yes;
# AutomaticMount = yes; # when device opened, read it
# RemovableMedia = no;
# AlwaysOpen = no;
#}
In other words: if the old short (without virtual autochanger) way to
describe the file storage still works as I believe, what is the significant
benefit gained if I define the autochanger?
So I've used a tape autochanger for years and I have a kind of understanding
how to configure it, but what is the idea why to add an autochanger resource
to file storage, without no obvious functionality?
Regards,
Timo
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual autochanger
Hi Timo, As you said, the Autochangers/Devices definitions on the initial bacula-sd.conf file are suggestions on how to use a virtual disk autochanger with Bacula. There is a lot of ways to work with disk volumes others that using disk autochangers, yes. It will mostly depend on your environment, needs, expectations of what you want/need. Could you please explain us better this "disks will be located in physically different locations"? I mean, do you want to have all your backups spreaded in all these disks? Yes, you can use individual storage devices for individual disk devices defined on your Storage Daemon side. However, you can take advantage of disk autochangers if you're going to back your 5 servers in one of your disks: 1) have one autochanger with 5 drives (all your 5 drives pointing to the same disk volume directory, for example, /backup/volumes) 2) have each device configured with Maximum Concurrent Jobs = 1 3) have your 5 backup jobs running concurrently, each one will use one of the 5 drives 4) you can have one pool for each server, so you would have all your backups from one specific server in one specific pool (this could help you with management) 5) but you can also have all your servers backups running concurrently into one specific device/volume. In both cases (one drive/volume per job or your 5 jobs using one device/volume) will increase your backup speed. If your idea is to spread all your backups into all your disks in different locations (of course if you have enough space for that), you can use copy/Virtual Full jobs to accomplish that. Hope this helps. Best regards, Ana On Sat, Oct 1, 2016 at 10:37 PM, Timo Neuvonen wrote: > After using Bacula for close to a decade with a tape autochanger, I'm > slightly lost with ideas related to disk-based backup I'm now trying to > implement. > > Now I have a fersh test install of Bacula 7.0.5, on CentOS 7. Bacula comes > from EPEL repo. > > > The supplied example conf files define a "virtual autochanger", that refers > to two "storage devices" that both actually write to same directory (/tmp > in > the example). > > While wondering the need for this arrangement, I've figured out that this > may be to help simultaneous backup/restore jobs run smoothly. However, in > my > relatively small environment it makes things look complicated if I define > every storage this way. > > Is the suggested way of using a virtual autochanger a must to make things > work at all, or is it a way how to avoid problems in a big environment > where > may be a lot of simultaneous backup and restore jobs? I have about 5 > servers > to back up, I'll have the backups running in the nighttime, probably not > concurrently at all. If and when there will be a need for a restore job, it > will be a single restore run in the daytime. So no more than a single job > at > a time. > > Will there be any problem in this case, if I try to simplify the conf files > and drop away the "autochanger" and one of the two "storage devices" it > refers to, and just had a single "storage device" per each media type? > > > (My goal is to use 2-3 media types, all disks, but disks will be located in > physically different locations to increase fire/vandalism safety, besides > disk faults. Since every media type will require a separate storage > definition, the number of virtual autochanger definitions would multiply > correspondingly...) > > > Regards, > > Timo > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Virtual autochanger
After using Bacula for close to a decade with a tape autochanger, I'm slightly lost with ideas related to disk-based backup I'm now trying to implement. Now I have a fersh test install of Bacula 7.0.5, on CentOS 7. Bacula comes from EPEL repo. The supplied example conf files define a "virtual autochanger", that refers to two "storage devices" that both actually write to same directory (/tmp in the example). While wondering the need for this arrangement, I've figured out that this may be to help simultaneous backup/restore jobs run smoothly. However, in my relatively small environment it makes things look complicated if I define every storage this way. Is the suggested way of using a virtual autochanger a must to make things work at all, or is it a way how to avoid problems in a big environment where may be a lot of simultaneous backup and restore jobs? I have about 5 servers to back up, I'll have the backups running in the nighttime, probably not concurrently at all. If and when there will be a need for a restore job, it will be a single restore run in the daytime. So no more than a single job at a time. Will there be any problem in this case, if I try to simplify the conf files and drop away the "autochanger" and one of the two "storage devices" it refers to, and just had a single "storage device" per each media type? (My goal is to use 2-3 media types, all disks, but disks will be located in physically different locations to increase fire/vandalism safety, besides disk faults. Since every media type will require a separate storage definition, the number of virtual autochanger definitions would multiply correspondingly...) Regards, Timo -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual Autochanger
Hello Heitor, hello list, Am 2015-06-05 um 15:07 schrieb Heitor Faria: > Here: http://sourceforge.net/p/bacula/mailman/message/17071747/ I had to make some minor adjustments to your scripts, mostly because I guess bacula output you are using has changed over the years. So far it looks really nice, doing exactly what we wanted! Cheers, -- j.hofmüller aka Thesix >-<#!&$@@@? http://thesix.mur.at/ signature.asc Description: OpenPGP digital signature -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual Autochanger
On 6/5/2015 6:17 AM, Jogi Hofmüller wrote:
Dear all,
We recently bought two identical HP Ultrium 448i tape drives. We want
to use them with Bacula and found the white paper "Best Practices for
Disk Backup" [1] that describes how to configure a virtual autochanger.
We applied the necessary changes in order to model our two drives
combined in one virtual autochanger and tried it with a backup job that
needs more than one tape.
No. That is not possible. When a job starts up, it is assigned a Device
resource (from bacula-sd.conf) to write to. It is not assigned to the
Autochanger resource, but rather one of its Device resources (ie. one of
the tape drives). This assignment happens only once at job start. Once
the device is assigned, Bacula then begins selecting volumes to use in
THAT device.But it will not change devices in the middle of a job.
Using two tape drives in a virtual autochanger will work, though. It is
fine for running concurrent jobs without having to manually configure
jobs to spread the load, but it will not allow big jobs to span tapes
without operator intervention.
Bacula used the first tape as expected but when in came to use the
second tape it requested the second tape to be mounted in the first
drive although the tape was correctly mounted in the second drive.
Now I would like to ask the experts if it is possible to make a setup
like I described work and if so, what we did wrong so far. We are
currently using bacula 5.2.6 from Debian stable (8.0 aka jessie). So
far we have been using bacula with one drive for the past 10 years or so
and did not have any problems.
Here are the relevant portions of our configs:
Storage {
Name = TapeChanger
Address = 127.0.0.1
Password = "03292ab58a34fc3895a70a0d6581581c6105a5fd"
Device = Virtual
Media Type = Ultrium2
Autochanger = yes
}
JobDefs {
Name = "DefaultJob"
Type = Backup
Client = hamster-file
FileSet = HomeBackupFileSet
Messages = Standard
Storage = TapeChanger
Priority = 10
}
Job {
Name = "BackupFiles"
JobDefs = DefaultJob
Schedule = DailySchedule
Pool = WeeklyPool
Write Bootstrap = "/var/lib/bacula/BackupFiles.bsr"
}
Autochanger {
Name = Virtual
Changer Device = /dev/null
Changer Command = ""
Device = virtual-0, virtual-1
}
Device {
Name = virtual-0
Media Type = Ultrium2
Archive Device = /dev/nst0
Device Type = Tape
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 0
Volume Poll Interval = 15
Autoselect = yes
}
Device {
Name = virtual-1
Media Type = Ultrium2
Archive Device = /dev/nst1
Device Type = Tape
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 1
Volume Poll Interval = 15
Autoselect = yes
}
Thanks for any input on this matter!
[1] http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf
Cheers,
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual Autochanger
Hi Heitor, Am 2015-06-05 um 15:07 schrieb Heitor Faria: > Here: http://sourceforge.net/p/bacula/mailman/message/17071747/ Thanks, I will check it out! > Btw: reading again nowadays I don't know why I did parts of this stuff. lol :) sounds familiar :) Cheers, -- j.hofmüller mur.sat -- a space art projecthttp://sat.mur.at/ signature.asc Description: OpenPGP digital signature -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Virtual Autochanger
Here: http://sourceforge.net/p/bacula/mailman/message/17071747/
Btw: reading again nowadays I don't know why I did parts of this stuff. lol
===
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
Administrator II
15 a 26 de junho: Treinamento Telepresencial Bacula:
http://www.bacula.com.br/?p=2174
61 8268-4220
Site: www.bacula.com.br | Facebook: heitor.faria
- Mensagem original -
> De: "Heitor Faria"
> Para: "Jogi Hofmüller"
> Cc: bacula-users@lists.sourceforge.net
> Enviadas: Sexta-feira, 5 de junho de 2015 10:02:58
> Assunto: Re: [Bacula-users] Virtual Autochanger
> Hello Jogi,
>
> Perhaps you can try to use different Media Types for each device, like it is
> being discussed lately on this list and also recommended by the same
> whitepaper.
> Once I managed to deploy a solution like yours, but with Shell Script. I run
> my
> backups all at a single tape a time, and had a script that grep(ed) status
> storage output every 180s looking for blocked devices waiting for media; a
> single bacula-sd tape where the Archive Device was a symlink that was changed
> between nst0 and nst1 by the prior script. I've posted at this list (I think
> 2008) and it worked like a charm.
>
> Regards,
> ===
> Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
> Administrator II
> 15 a 26 de junho: Treinamento Telepresencial Bacula:
> http://www.bacula.com.br/?p=2174
> 61 8268-4220
> Site: www.bacula.com.br | Facebook: heitor.faria
>
>
> - Mensagem original -
>> De: "Jogi Hofmüller"
>> Para: bacula-users@lists.sourceforge.net
>> Enviadas: Sexta-feira, 5 de junho de 2015 7:17:13
>> Assunto: [Bacula-users] Virtual Autochanger
>
>> Dear all,
>>
>> We recently bought two identical HP Ultrium 448i tape drives. We want
>> to use them with Bacula and found the white paper "Best Practices for
>> Disk Backup" [1] that describes how to configure a virtual autochanger.
>> We applied the necessary changes in order to model our two drives
>> combined in one virtual autochanger and tried it with a backup job that
>> needs more than one tape.
>>
>> Bacula used the first tape as expected but when in came to use the
>> second tape it requested the second tape to be mounted in the first
>> drive although the tape was correctly mounted in the second drive.
>>
>> Now I would like to ask the experts if it is possible to make a setup
>> like I described work and if so, what we did wrong so far. We are
>> currently using bacula 5.2.6 from Debian stable (8.0 aka jessie). So
>> far we have been using bacula with one drive for the past 10 years or so
>> and did not have any problems.
>>
>> Here are the relevant portions of our configs:
>>
>>
>>
>> Storage {
>> Name = TapeChanger
>> Address = 127.0.0.1
>> Password = "03292ab58a34fc3895a70a0d6581581c6105a5fd"
>> Device = Virtual
>> Media Type = Ultrium2
>> Autochanger = yes
>> }
>>
>> JobDefs {
>> Name = "DefaultJob"
>> Type = Backup
>> Client = hamster-file
>> FileSet = HomeBackupFileSet
>> Messages = Standard
>> Storage = TapeChanger
>> Priority = 10
>> }
>>
>> Job {
>> Name = "BackupFiles"
>> JobDefs = DefaultJob
>> Schedule = DailySchedule
>> Pool = WeeklyPool
>> Write Bootstrap = "/var/lib/bacula/BackupFiles.bsr"
>> }
>>
>>
>>
>>
>>
>> Autochanger {
>> Name = Virtual
>> Changer Device = /dev/null
>> Changer Command = ""
>> Device = virtual-0, virtual-1
>> }
>>
>> Device {
>> Name = virtual-0
>> Media Type = Ultrium2
>> Archive Device = /dev/nst0
>> Device Type = Tape
>> AutomaticMount = yes
>> AlwaysOpen = yes
>> RemovableMedia = yes
>> Autochanger = yes
>> Drive Index = 0
>> Volume Poll Interval = 15
>> Autoselect = yes
>> }
>>
>> Device {
>> Name = virtual-1
>> Media Type = Ultrium2
>> Archive Device = /dev/nst1
>> Device Type = Tape
>> AutomaticMount = yes
>> AlwaysOpen = yes
>> RemovableMedia = yes
>> Autochanger = yes
>> Drive Index = 1
>> Volume Poll Interval = 15
&g
Re: [Bacula-users] Virtual Autochanger
Hello Jogi,
Perhaps you can try to use different Media Types for each device, like it is
being discussed lately on this list and also recommended by the same whitepaper.
Once I managed to deploy a solution like yours, but with Shell Script. I run my
backups all at a single tape a time, and had a script that grep(ed) status
storage output every 180s looking for blocked devices waiting for media; a
single bacula-sd tape where the Archive Device was a symlink that was changed
between nst0 and nst1 by the prior script. I've posted at this list (I think
2008) and it worked like a charm.
Regards,
===
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
Administrator II
15 a 26 de junho: Treinamento Telepresencial Bacula:
http://www.bacula.com.br/?p=2174
61 8268-4220
Site: www.bacula.com.br | Facebook: heitor.faria
- Mensagem original -
> De: "Jogi Hofmüller"
> Para: bacula-users@lists.sourceforge.net
> Enviadas: Sexta-feira, 5 de junho de 2015 7:17:13
> Assunto: [Bacula-users] Virtual Autochanger
> Dear all,
>
> We recently bought two identical HP Ultrium 448i tape drives. We want
> to use them with Bacula and found the white paper "Best Practices for
> Disk Backup" [1] that describes how to configure a virtual autochanger.
> We applied the necessary changes in order to model our two drives
> combined in one virtual autochanger and tried it with a backup job that
> needs more than one tape.
>
> Bacula used the first tape as expected but when in came to use the
> second tape it requested the second tape to be mounted in the first
> drive although the tape was correctly mounted in the second drive.
>
> Now I would like to ask the experts if it is possible to make a setup
> like I described work and if so, what we did wrong so far. We are
> currently using bacula 5.2.6 from Debian stable (8.0 aka jessie). So
> far we have been using bacula with one drive for the past 10 years or so
> and did not have any problems.
>
> Here are the relevant portions of our configs:
>
>
>
> Storage {
> Name = TapeChanger
> Address = 127.0.0.1
> Password = "03292ab58a34fc3895a70a0d6581581c6105a5fd"
> Device = Virtual
> Media Type = Ultrium2
> Autochanger = yes
> }
>
> JobDefs {
> Name = "DefaultJob"
> Type = Backup
> Client = hamster-file
> FileSet = HomeBackupFileSet
> Messages = Standard
> Storage = TapeChanger
> Priority = 10
> }
>
> Job {
> Name = "BackupFiles"
> JobDefs = DefaultJob
> Schedule = DailySchedule
> Pool = WeeklyPool
> Write Bootstrap = "/var/lib/bacula/BackupFiles.bsr"
> }
>
>
>
>
>
> Autochanger {
> Name = Virtual
> Changer Device = /dev/null
> Changer Command = ""
> Device = virtual-0, virtual-1
> }
>
> Device {
> Name = virtual-0
> Media Type = Ultrium2
> Archive Device = /dev/nst0
> Device Type = Tape
> AutomaticMount = yes
> AlwaysOpen = yes
> RemovableMedia = yes
> Autochanger = yes
> Drive Index = 0
> Volume Poll Interval = 15
> Autoselect = yes
> }
>
> Device {
> Name = virtual-1
> Media Type = Ultrium2
> Archive Device = /dev/nst1
> Device Type = Tape
> AutomaticMount = yes
> AlwaysOpen = yes
> RemovableMedia = yes
> Autochanger = yes
> Drive Index = 1
> Volume Poll Interval = 15
> Autoselect = yes
> }
>
>
>
>
> Thanks for any input on this matter!
>
> [1] http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf
>
> Cheers,
> --
> J.Hofmüller
>
> ich zitiere wie Espenlaub.
>
> https://twitter.com/TheGurkenkaiser/status/463444397678690304
>
>
> --
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Virtual Autochanger
Dear all,
We recently bought two identical HP Ultrium 448i tape drives. We want
to use them with Bacula and found the white paper "Best Practices for
Disk Backup" [1] that describes how to configure a virtual autochanger.
We applied the necessary changes in order to model our two drives
combined in one virtual autochanger and tried it with a backup job that
needs more than one tape.
Bacula used the first tape as expected but when in came to use the
second tape it requested the second tape to be mounted in the first
drive although the tape was correctly mounted in the second drive.
Now I would like to ask the experts if it is possible to make a setup
like I described work and if so, what we did wrong so far. We are
currently using bacula 5.2.6 from Debian stable (8.0 aka jessie). So
far we have been using bacula with one drive for the past 10 years or so
and did not have any problems.
Here are the relevant portions of our configs:
Storage {
Name = TapeChanger
Address = 127.0.0.1
Password = "03292ab58a34fc3895a70a0d6581581c6105a5fd"
Device = Virtual
Media Type = Ultrium2
Autochanger = yes
}
JobDefs {
Name = "DefaultJob"
Type = Backup
Client = hamster-file
FileSet = HomeBackupFileSet
Messages = Standard
Storage = TapeChanger
Priority = 10
}
Job {
Name = "BackupFiles"
JobDefs = DefaultJob
Schedule = DailySchedule
Pool = WeeklyPool
Write Bootstrap = "/var/lib/bacula/BackupFiles.bsr"
}
Autochanger {
Name = Virtual
Changer Device = /dev/null
Changer Command = ""
Device = virtual-0, virtual-1
}
Device {
Name = virtual-0
Media Type = Ultrium2
Archive Device = /dev/nst0
Device Type = Tape
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 0
Volume Poll Interval = 15
Autoselect = yes
}
Device {
Name = virtual-1
Media Type = Ultrium2
Archive Device = /dev/nst1
Device Type = Tape
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 1
Volume Poll Interval = 15
Autoselect = yes
}
Thanks for any input on this matter!
[1] http://blog.bacula.org/whitepapers/CommunityDiskBackup.pdf
Cheers,
--
J.Hofmüller
ich zitiere wie Espenlaub.
https://twitter.com/TheGurkenkaiser/status/463444397678690304
signature.asc
Description: OpenPGP digital signature
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
