Re: [Bacula-users] bacula-sd - file driver - cloud resource
Hello Ziga,
If you are using the Orwcle Gateway there is no reason on Earth to use the S3
Driver. It will only mess your system, since I believe even in the SD startup
it tries to connect to the bucket.
If you want smaller volumes, just tune the Maximum Volume Bytes and maybe Jobs
(e.g. one).
Regards,
--
MSc Heitor Faria
CEO Bacula LatAm
mobile1: + 1 909 655-8971
mobile2: + 55 61 98268-4220
América Latina
[ http://bacula.lat/]
Original Message
From: Žiga Žvan
Sent: Monday, December 28, 2020 04:07 PM
To: Heitor Faria ,bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource
>Hello Heitor,
>
>I am aware of that. I'm using Oracle Storage Gateway to upload data to
>cloud (to S3 bucket). For bacula-sd, backup destination is ordinary
>filesystem mounted over nfs. However I'm using file driver with cloud
>resource (in order to have volume broken down in file parts and
>therefore optimize upload amount).
>
>Everything worked until bacula decided to reuse old volume. I'm getting
>strange error (cannot download Volume). In my opinion this is some sort
>of bug, because data is available on nfs destination (see example output
>from cloudcache and backup folder bellow).
>
>Could you check my configuration (bellow)? Could you confirm that this
>is a bug and I did not misconfigure something?
>Regards,
>Ziga
>
>
>On 28.12.2020 14:46, Heitor Faria wrote:
>>
>> Hello Ziga,
>>
>> The Bacula Community S3 Plugin was built for Amazon S3 protocol.
>> I read that Oracle built some sort of S3 emulator for its cloud, but
>> since Bacula Community is a free software, it comes without warranties
>> or support.
>> You could consider running the bacula-sd in debug mode to troubleshoot
>> the problem. The Driver is not working properly.
>> Bacula Systems already developed a specific plugin for Oracle Object
>> Storages, available only in the Enterprise edition.
>>
>> Regards,
>> --
>> MSc Heitor Faria
>> CEO Bacula LatAm
>> mobile1: + 1 909 655-8971
>> mobile2: + 55 61 98268-4220
>>
>> América Latina
>> [ http://bacula.lat/]
>>
>>
>>
>> Original Message
>> From: Žiga Žvan
>> Sent: Monday, December 28, 2020 08:22 AM
>> To: bacula-users@lists.sourceforge.net
>> Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource
>>
>> Hi,
>> What am I doing wrong? I would be grateful for a hint, why there is no
>> reply to my emails.
>> I believe that my problems are a bit complex for an email. Is there any
>> way to discuss this in different way?
>> Who should I contact in case that I would like to support development of
>> oracle cloud plugin?
>>
>> Regards,
>> Ziga
>>
>> On 21.12.2020 20:38, Žiga Žvan wrote:
>> > Hello,
>> > I'm using file driver with cloud resource. Bacula was able to backup
>> > data in this way until it wrote data to new volumes. Now, after
>> > retention period, I'm getting error: Fatal error: cloud_dev.c:983
>> > Unable to download Volume (see output below). Data on cloud path looks
>> > ok but data in local cache contains only part.1 without any data.
>> >
>> > Is this expected?
>> > Has anybody tested this scenario?
>> > Should I avoid file driver in production environment?
>> >
>> > Regards,
>> > Ziga
>> >
>> >
>> > [root@bacula db-01-weekly-vol-0365]# ls -la
>> > /mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
>> > total 0
>> > drwxr-. 2 bacula disk 0 Oct 24 07:45 .
>> > drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 ..
>> > -rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1
>> > -rw-r--r--. 1 bacula disk 35992 Oct 24 07:44 part.2
>> > -rw-r--r--. 1 bacula disk 35993 Oct 24 07:44 part.3
>> > -rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4
>> >
>> > [root@bacula db-01-weekly-vol-0365]# ls -la
>> > /storage/bacula/cloudcache/db-01-weekly-vol-0365
>> > total 20
>> > drwxr-. 2 bacula disk 28 Dec 11 23:10 .
>> > drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
>> > -rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1
>> >
>> > SD config (autochanger)
>> >
>> > Device {
>> > Name = FSOciCloudStandard2
>> > Device type = Cloud
>> > Cloud = OracleViaStorageGateway
>> > Maximum Part Size = 1000 MB
>> > Media Type = File1
>> > Archive Device = /storage/bacula/cloudcache
>> > LabelMedia = yes; # lets Bacula label unlabeled
>> media
>> > Random Access = Yes;
>> > AutomaticMount = yes; # when device opened, read it
>> > RemovableMedia = no;
>> > AlwaysOpen = no;
>> > Autochanger = yes;
>> > }
>> > ...
>> > Device {
>> > Name = FSOciCloudStandard4
>> > Device type = Cloud
>> > Cloud = OracleViaStorageGateway
>> > Maximum Part Size = 1000 MB
>> > Media Type = File1
>> > Archive Device = /storage/bacula/cloudcache
>> > LabelMedia = yes; # lets Bacula label unlabeled
>> media
>> > Random Access = Yes;
>> > AutomaticMount = yes; # when device opened,
Re: [Bacula-users] bacula-sd - file driver - cloud resource
Hello Heitor,
I am aware of that. I'm using Oracle Storage Gateway to upload data to
cloud (to S3 bucket). For bacula-sd, backup destination is ordinary
filesystem mounted over nfs. However I'm using file driver with cloud
resource (in order to have volume broken down in file parts and
therefore optimize upload amount).
Everything worked until bacula decided to reuse old volume. I'm getting
strange error (cannot download Volume). In my opinion this is some sort
of bug, because data is available on nfs destination (see example output
from cloudcache and backup folder bellow).
Could you check my configuration (bellow)? Could you confirm that this
is a bug and I did not misconfigure something?
Regards,
Ziga
On 28.12.2020 14:46, Heitor Faria wrote:
Hello Ziga,
The Bacula Community S3 Plugin was built for Amazon S3 protocol.
I read that Oracle built some sort of S3 emulator for its cloud, but
since Bacula Community is a free software, it comes without warranties
or support.
You could consider running the bacula-sd in debug mode to troubleshoot
the problem. The Driver is not working properly.
Bacula Systems already developed a specific plugin for Oracle Object
Storages, available only in the Enterprise edition.
Regards,
--
MSc Heitor Faria
CEO Bacula LatAm
mobile1: + 1 909 655-8971
mobile2: + 55 61 98268-4220
América Latina
[ http://bacula.lat/]
Original Message
From: Žiga Žvan
Sent: Monday, December 28, 2020 08:22 AM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource
Hi,
What am I doing wrong? I would be grateful for a hint, why there is no
reply to my emails.
I believe that my problems are a bit complex for an email. Is there any
way to discuss this in different way?
Who should I contact in case that I would like to support development of
oracle cloud plugin?
Regards,
Ziga
On 21.12.2020 20:38, Žiga Žvan wrote:
> Hello,
> I'm using file driver with cloud resource. Bacula was able to backup
> data in this way until it wrote data to new volumes. Now, after
> retention period, I'm getting error: Fatal error: cloud_dev.c:983
> Unable to download Volume (see output below). Data on cloud path looks
> ok but data in local cache contains only part.1 without any data.
>
> Is this expected?
> Has anybody tested this scenario?
> Should I avoid file driver in production environment?
>
> Regards,
> Ziga
>
>
> [root@bacula db-01-weekly-vol-0365]# ls -la
> /mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
> total 0
> drwxr-. 2 bacula disk 0 Oct 24 07:45 .
> drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 ..
> -rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1
> -rw-r--r--. 1 bacula disk 35992 Oct 24 07:44 part.2
> -rw-r--r--. 1 bacula disk 35993 Oct 24 07:44 part.3
> -rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4
>
> [root@bacula db-01-weekly-vol-0365]# ls -la
> /storage/bacula/cloudcache/db-01-weekly-vol-0365
> total 20
> drwxr-. 2 bacula disk 28 Dec 11 23:10 .
> drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
> -rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1
>
> SD config (autochanger)
>
> Device {
> Name = FSOciCloudStandard2
> Device type = Cloud
> Cloud = OracleViaStorageGateway
> Maximum Part Size = 1000 MB
> Media Type = File1
> Archive Device = /storage/bacula/cloudcache
> LabelMedia = yes; # lets Bacula label unlabeled
media
> Random Access = Yes;
> AutomaticMount = yes; # when device opened, read it
> RemovableMedia = no;
> AlwaysOpen = no;
> Autochanger = yes;
> }
> ...
> Device {
> Name = FSOciCloudStandard4
> Device type = Cloud
> Cloud = OracleViaStorageGateway
> Maximum Part Size = 1000 MB
> Media Type = File1
> Archive Device = /storage/bacula/cloudcache
> LabelMedia = yes; # lets Bacula label unlabeled
media
> Random Access = Yes;
> AutomaticMount = yes; # when device opened, read it
> RemovableMedia = no;
> AlwaysOpen = no;
> Autochanger = yes;
> }
>
> Cloud {
> Name = OracleViaStorageGateway
> Driver = "File"
> HostName = "/mnt/ocisg/bacula/backup"
> BucketName = "DummyBucket"
> AccessKey = "DummyAccessKey"
> SecretKey = "DummySecretKey"
> Protocol = HTTPS
> UriStyle = VirtualHost
> Truncate Cache = AtEndOfJob
> }
>
>
> 21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073,
> Job=db-01-backup.2020-12-21_19.14.14_48
> 21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2"
> to write.
> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
> to download Volume="db-01-weekly-vol-0365" label.
> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
> to download Volume="db-01-weekly-vol-0365" label.
> 21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error:
> job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK
> data
> ,
Re: [Bacula-users] bacula-sd - file driver - cloud resource
Hello Ziga,
The Bacula Community S3 Plugin was built for Amazon S3 protocol.
I read that Oracle built some sort of S3 emulator for its cloud, but since
Bacula Community is a free software, it comes without warranties or support.
You could consider running the bacula-sd in debug mode to troubleshoot the
problem. The Driver is not working properly.
Bacula Systems already developed a specific plugin for Oracle Object Storages,
available only in the Enterprise edition.
Regards,
--
MSc Heitor Faria
CEO Bacula LatAm
mobile1: + 1 909 655-8971
mobile2: + 55 61 98268-4220
América Latina
[ http://bacula.lat/]
Original Message
From: Žiga Žvan
Sent: Monday, December 28, 2020 08:22 AM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource
>Hi,
>What am I doing wrong? I would be grateful for a hint, why there is no
>reply to my emails.
>I believe that my problems are a bit complex for an email. Is there any
>way to discuss this in different way?
>Who should I contact in case that I would like to support development of
>oracle cloud plugin?
>
>Regards,
>Ziga
>
>On 21.12.2020 20:38, Žiga Žvan wrote:
>> Hello,
>> I'm using file driver with cloud resource. Bacula was able to backup
>> data in this way until it wrote data to new volumes. Now, after
>> retention period, I'm getting error: Fatal error: cloud_dev.c:983
>> Unable to download Volume (see output below). Data on cloud path looks
>> ok but data in local cache contains only part.1 without any data.
>>
>> Is this expected?
>> Has anybody tested this scenario?
>> Should I avoid file driver in production environment?
>>
>> Regards,
>> Ziga
>>
>>
>> [root@bacula db-01-weekly-vol-0365]# ls -la
>> /mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
>> total 0
>> drwxr-. 2 bacula disk 0 Oct 24 07:45 .
>> drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 ..
>> -rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1
>> -rw-r--r--. 1 bacula disk 35992 Oct 24 07:44 part.2
>> -rw-r--r--. 1 bacula disk 35993 Oct 24 07:44 part.3
>> -rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4
>>
>> [root@bacula db-01-weekly-vol-0365]# ls -la
>> /storage/bacula/cloudcache/db-01-weekly-vol-0365
>> total 20
>> drwxr-. 2 bacula disk 28 Dec 11 23:10 .
>> drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
>> -rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1
>>
>> SD config (autochanger)
>>
>> Device {
>> Name = FSOciCloudStandard2
>> Device type = Cloud
>> Cloud = OracleViaStorageGateway
>> Maximum Part Size = 1000 MB
>> Media Type = File1
>> Archive Device = /storage/bacula/cloudcache
>> LabelMedia = yes; # lets Bacula label unlabeled media
>> Random Access = Yes;
>> AutomaticMount = yes; # when device opened, read it
>> RemovableMedia = no;
>> AlwaysOpen = no;
>> Autochanger = yes;
>> }
>> ...
>> Device {
>> Name = FSOciCloudStandard4
>> Device type = Cloud
>> Cloud = OracleViaStorageGateway
>> Maximum Part Size = 1000 MB
>> Media Type = File1
>> Archive Device = /storage/bacula/cloudcache
>> LabelMedia = yes; # lets Bacula label unlabeled media
>> Random Access = Yes;
>> AutomaticMount = yes; # when device opened, read it
>> RemovableMedia = no;
>> AlwaysOpen = no;
>> Autochanger = yes;
>> }
>>
>> Cloud {
>> Name = OracleViaStorageGateway
>> Driver = "File"
>> HostName = "/mnt/ocisg/bacula/backup"
>> BucketName = "DummyBucket"
>> AccessKey = "DummyAccessKey"
>> SecretKey = "DummySecretKey"
>> Protocol = HTTPS
>> UriStyle = VirtualHost
>> Truncate Cache = AtEndOfJob
>> }
>>
>>
>> 21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073,
>> Job=db-01-backup.2020-12-21_19.14.14_48
>> 21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2"
>> to write.
>> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
>> to download Volume="db-01-weekly-vol-0365" label.
>> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
>> to download Volume="db-01-weekly-vol-0365" label.
>> 21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error:
>> job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK
>> data
>> , got len=25 msg="3903 Error append data: "
>> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
>> to download Volume="db-01-weekly-vol-0365" label.
>> 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud
>> device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume
>> "db-01-weekly-vol-0365" failed: ERR=
>> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
>> to download Volume="db-01-weekly-vol-0365" label.
>> 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
>> to download Volume="db-01-weekly-vol-0365" label.
>> 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398
Re: [Bacula-users] bacula-sd - file driver - cloud resource
Hi,
What am I doing wrong? I would be grateful for a hint, why there is no
reply to my emails.
I believe that my problems are a bit complex for an email. Is there any
way to discuss this in different way?
Who should I contact in case that I would like to support development of
oracle cloud plugin?
Regards,
Ziga
On 21.12.2020 20:38, Žiga Žvan wrote:
Hello,
I'm using file driver with cloud resource. Bacula was able to backup
data in this way until it wrote data to new volumes. Now, after
retention period, I'm getting error: Fatal error: cloud_dev.c:983
Unable to download Volume (see output below). Data on cloud path looks
ok but data in local cache contains only part.1 without any data.
Is this expected?
Has anybody tested this scenario?
Should I avoid file driver in production environment?
Regards,
Ziga
[root@bacula db-01-weekly-vol-0365]# ls -la
/mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
total 0
drwxr-. 2 bacula disk 0 Oct 24 07:45 .
drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 ..
-rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1
-rw-r--r--. 1 bacula disk 35992 Oct 24 07:44 part.2
-rw-r--r--. 1 bacula disk 35993 Oct 24 07:44 part.3
-rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4
[root@bacula db-01-weekly-vol-0365]# ls -la
/storage/bacula/cloudcache/db-01-weekly-vol-0365
total 20
drwxr-. 2 bacula disk 28 Dec 11 23:10 .
drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
-rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1
SD config (autochanger)
Device {
Name = FSOciCloudStandard2
Device type = Cloud
Cloud = OracleViaStorageGateway
Maximum Part Size = 1000 MB
Media Type = File1
Archive Device = /storage/bacula/cloudcache
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Autochanger = yes;
}
...
Device {
Name = FSOciCloudStandard4
Device type = Cloud
Cloud = OracleViaStorageGateway
Maximum Part Size = 1000 MB
Media Type = File1
Archive Device = /storage/bacula/cloudcache
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Autochanger = yes;
}
Cloud {
Name = OracleViaStorageGateway
Driver = "File"
HostName = "/mnt/ocisg/bacula/backup"
BucketName = "DummyBucket"
AccessKey = "DummyAccessKey"
SecretKey = "DummySecretKey"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = AtEndOfJob
}
21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073,
Job=db-01-backup.2020-12-21_19.14.14_48
21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2"
to write.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error:
job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK
data
, got len=25 msg="3903 Error append data: "
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud
device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume
"db-01-weekly-vol-0365" failed: ERR=
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud
device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume
"db-01-weekly-vol-0365" failed: ERR=
21-Dec 19:14 bacula-sd JobId 2073: Marking Volume
"db-01-weekly-vol-0365" in Error in Catalog.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: Job 2073 canceled.
On 06.12.2020 20:52, Žiga Žvan wrote:
Dear all,
I'm using bacula 9.6.5 in a production for a month now. I'm
experiencing random backup failures from my clients. Specific hosts
report errors like the outputs attached. The same host is able to
perform backup at some other time. The error is more often at large
backups (more errors at full backups than incremental, more errors at
hosts with large data sets).
I have tried to implement heartbeat interval
(https://www.bacula.org/9.6.x-manuals/en/main/Client_File_daemon_Configur.html#SECTION00221)
but there is no improvement.
The error occures also on hosts in the same zone as bacula server (no
router/firewall in between).
Storage deamon is installed on the same server as bacula director.
I'm using File cloud driver (backup to local disk via cloud
