Hello Kern,
Thank you for all info. I needed to install cloud plugin (yum install
bacula-cloud-storage) in order to use this configuration. Relevant
part of bacula-sd.conf is pasted bellow. Files did get uploaded to
Oracle object storage.
I intend to use lifecycle policy on the cloud bucket. It will archive
60days old files to a 10times cheaper storage. For bacula that means
that some fileparts will not be available until I manually restore
them (after 60 days status will change from available to archived).
Do you expect that this will represent any kind of problem for backup
job? Perhaps fileparts of the last full backup need to be available at
local folder in order to run incremental backup?
What about restore procedure (eg. from 90 days old incremental
backup)? Will bacula notify me which fileparts are missing at local
folder in order to complete the restore?
I will continue with my tests and hopefully move to production in a
month or two. I intend to publish on the bacula-users list, if
everything worked for me. If I forget, do not hesitate to ask me about
that.
Kind regards,
Ziga
Relevant part of bacula-sd.conf:
Device {
Name = FSOciCloudStandard
Device type = Cloud
Cloud = OracleViaStorageGateway
# Maximum Part Size = 100 MB
Media Type = File1
Archive Device = /mnt/backup_bacula/backup
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
Cloud {
Name = OracleViaStorageGateway
Driver = "File"
HostName = "/mnt/baculatest_standard/backup"
BucketName = "DummyBucket"
AccessKey = "DummyAccessKey"
SecretKey = "DummySecretKey"
Protocol = HTTPS
UriStyle = VirtualHost
}
On 08/07/2020 16:32, Kern Sibbald wrote:
Hello Ziga,
Yes, you might be able to do what you want using a "debug" feature of
the Bacula Cloud driver. It is not very well documented, but there
is one section "3 File Driver for the Cloud" in the "Bacula Cloud
Backup" that mentions it.
Basically instead of using the "S3" driver in the Cloud resource of
your Storage Daemon, you use "File" and the HostName becomes the path
where the Cloud volumes (directories + parts) will be written. For
example, I use the following for writing to disk instead of an S3 cloud.
Cloud {
Name = DummyCloud
Driver = "File"
HostName = "/home/kern/bacula/k/regress/tmp/cloud"
BucketName = "DummyBucket"
AccessKey = "DummyAccessKey"
SecretKey = "DummySecretKey"
Protocol = HTTPS
UriStyle = VirtualHost
}
The Device resource looks like:
Device {
Name = FileStorage1
Media Type = File1
Archive Device = /home/kern/bacula/k/regress/tmp
LabelMedia = yes; # lets Bacula label unlabelled
media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Device Type = Cloud
Cloud = DummyCloud
}
I know the code runs and produces correct output, but I am not sure
how it will work in your environment. If it works, great. If it
doesn't work, for the near future, unfortunately I cannot provide
support, but at some point (probably 3-6 months) the project may
support this feature.
Note: the next version of Bacula coming in a few months will have a
good number of new features and improvements to the Cloud driver
(more bconsole commands for example).
Good luck, and best regards,
Kern
PS: If it does work for you, I would appreciate it if you would
document it and publish it on the bacula-users list so others can use
the Oracle cloud.
On 7/8/20 3:41 PM, Žiga Žvan wrote:
Hi Mr. Kern,
My question was a bit different. I have noticed that Oracle S3 is
not compatible, therefore I have implemented Oracle Storage gateway
(a docker image that uses local filesystem as a cache and moves the
data automatically to oracle cloud). I have this filesystem mounted
(nfsv4) on bacula server and I am able to backup data to this
storage (and hence in cloud).
I have around 1 TB data daily and I'm a bit concerned about the
bandwidth. It will take app. 4 hours to sync to the cloud and I need
to count in the future growth. As long as bacula writes data to one
file/volume, where it stores full and incremental backups, this is
not optimal for the cloud (the file will change and all the data
will upload each day). I have noticed that bacula stores data
differently in the cloud configuration. Volume is not a file, but a
folder with fileparts. This would be better for me, because only
some fileparts would change and move to the cloud via Storage
gateway. So the question is:
Can I configure bacula-sd to store data in fileparts, without actual
cloud sync? Is this possible? I have tried several configurations of
a bacula-sd device with no luck. Should I configure some dummy
cloud resource?
Kind regards,
Ziga Zvan
On 07/07/2020 14:40, Kern Sibbald wrote:
Hello,
Oracle S3 is not compatible with Amazon S3 or at least with the
libs3 that we use to interface to AWS and other compatible S3 cloud
offerings.
Yes, Bacula Enterprise has a separate Oracle cloud driver that they
wrote. There are no plans at the moment to backport it to the
community version.
Best regards,
Kern
On 7/7/20 8:43 AM, Žiga Žvan wrote:
Dear all,
I'm testing communty version of bacula in order to change backup
sw for app. 100 virtual and physical hosts. I would like to move
all the data to local storage and then move them to public cloud
(Oracle Object storage).
I believe that community version of the software suites our
needs. I have installed:
-version 9.6.5 of bacula on centos 7 computer
-oracle storage gateway (similar to aws SG - it moves data to
object storage and exposes it localy as nfsv4; for bacula this is
backup destination).
I have read this two documents regarding bacula and cloud
https://blog.bacula.org/whitepapers/CloudBackup.pdf
https://blog.bacula.org/whitepapers/ObjectStorage.pdf
It is mentioned it the document above, that Oracle Object storage
is not supported at the moment.
Is it possible to *configure* bacula Storage device in a way that
uses *Cloud media format* (directory with file parts as a volume,
instead of a single file as a volume) *without actual cloud sync*
(Storage Gateway does this in my case)? I am experimenting with
variations of the definition bellow, but I am unable to solve
this issue for now (it tries to initialize cloud plugin or it
writes to a file, instead of a directory).
Device {
Name = FSOciCloudStandard
# Device type = Cloud
Device type = File
# Cloud = OracleViaStorageGateway
Maximum Part Size = 100 MB
# Media Type = File
Media Type = CloudType
Archive Device = /mnt/baculatest_standard/backup
LabelMedia = yes; # lets Bacula label
unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
Is there any plan to support oracle object storage in near future?
It has S3 compatible API and bacula enterprise supports it...
Kind regards,
Ziga Zvan
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
