Hi,
What am I doing wrong? I would be grateful for a hint, why there is no
reply to my emails.
I believe that my problems are a bit complex for an email. Is there any
way to discuss this in different way?
Who should I contact in case that I would like to support development of
oracle cloud plugin?
Regards,
Ziga
On 21.12.2020 20:38, Žiga Žvan wrote:
Hello,
I'm using file driver with cloud resource. Bacula was able to backup
data in this way until it wrote data to new volumes. Now, after
retention period, I'm getting error: Fatal error: cloud_dev.c:983
Unable to download Volume (see output below). Data on cloud path looks
ok but data in local cache contains only part.1 without any data.
Is this expected?
Has anybody tested this scenario?
Should I avoid file driver in production environment?
Regards,
Ziga
[root@bacula db-01-weekly-vol-0365]# ls -la
/mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
total 0
drwxr-----. 2 bacula disk 0 Oct 24 07:45 .
drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 ..
-rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1
-rw-r--r--. 1 bacula disk 999935992 Oct 24 07:44 part.2
-rw-r--r--. 1 bacula disk 999935993 Oct 24 07:44 part.3
-rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4
[root@bacula db-01-weekly-vol-0365]# ls -la
/storage/bacula/cloudcache/db-01-weekly-vol-0365
total 20
drwxr-----. 2 bacula disk 28 Dec 11 23:10 .
drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
-rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1
SD config (autochanger)
Device {
Name = FSOciCloudStandard2
Device type = Cloud
Cloud = OracleViaStorageGateway
Maximum Part Size = 1000 MB
Media Type = File1
Archive Device = /storage/bacula/cloudcache
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Autochanger = yes;
}
...
Device {
Name = FSOciCloudStandard4
Device type = Cloud
Cloud = OracleViaStorageGateway
Maximum Part Size = 1000 MB
Media Type = File1
Archive Device = /storage/bacula/cloudcache
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Autochanger = yes;
}
Cloud {
Name = OracleViaStorageGateway
Driver = "File"
HostName = "/mnt/ocisg/bacula/backup"
BucketName = "DummyBucket"
AccessKey = "DummyAccessKey"
SecretKey = "DummySecretKey"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = AtEndOfJob
}
21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073,
Job=db-01-backup.2020-12-21_19.14.14_48
21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2"
to write.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error:
job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK
data
, got len=25 msg="3903 Error append data: "
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud
device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume
"db-01-weekly-vol-0365" failed: ERR=
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable
to download Volume="db-01-weekly-vol-0365" label.
21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud
device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume
"db-01-weekly-vol-0365" failed: ERR=
21-Dec 19:14 bacula-sd JobId 2073: Marking Volume
"db-01-weekly-vol-0365" in Error in Catalog.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: Job 2073 canceled.
On 06.12.2020 20:52, Žiga Žvan wrote:
Dear all,
I'm using bacula 9.6.5 in a production for a month now. I'm
experiencing random backup failures from my clients. Specific hosts
report errors like the outputs attached. The same host is able to
perform backup at some other time. The error is more often at large
backups (more errors at full backups than incremental, more errors at
hosts with large data sets).
I have tried to implement heartbeat interval
(https://www.bacula.org/9.6.x-manuals/en/main/Client_File_daemon_Configur.html#SECTION002210000000000000000)
but there is no improvement.
The error occures also on hosts in the same zone as bacula server (no
router/firewall in between).
Storage deamon is installed on the same server as bacula director.
I'm using File cloud driver (backup to local disk via cloud resource).
Could you please suggest a solution or a way to troubleshoot this
further?
Thx!
Regards,Ziga Zvan
Backup from linux hosts (on 05-dec 3 hosts failed, 20 hosts completed
without error):
05-Dec 03:26 bacula-dir JobId 1721: Fatal error: Network error with
FD during Backup: ERR=Connection reset by peer
05-Dec 03:27 bacula-dir JobId 1721: Fatal error: No Job status
returned from FD.
05-Dec 03:27 bacula-dir JobId 1721: Error: Bacula bacula-dir 9.6.5
(11Jun20):
Backup from windows hosts (on 05-dec 2 hosts failed, 5 hosts
completed without error):
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Error:
lib/bsock.c:383 Write error sending 57172 bytes to Storage
daemon:192.168.66.35:9103: ERR=Input/output error
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Fatal error:
filed/backup.c:848 Network send error to SD. ERR=Input/output error
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "Task Scheduler Writer", State: 0x1 (VSS_WS_STABLE)
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "VSS Metadata Store Writer", State: 0x1
(VSS_WS_STABLE)
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "Performance Counters Writer", State: 0x1
(VSS_WS_STABLE)
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "System Writer", State: 0x1 (VSS_WS_STABLE)
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "ASR Writer", State: 0x1 (VSS_WS_STABLE)
05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer
(BackupComplete): "Shadow Copy Optimization Writer", State: 0x1
(VSS_WS_STABLE)
05-Dec 01:01 bacula-dir JobId 1726: Error: bsock.c:551 Read error
from Client:
iwhost01.kranj.cetrtapot.si-fd:iwhost01.kranj.cetrtapot.si:9102:
ERR=Connection timed out
05-Dec 01:01 bacula-dir JobId 1726: Fatal error: Network error with
FD during Backup: ERR=Connection timed out
05-Dec 01:02 bacula-dir JobId 1726: Fatal error: No Job status
returned from FD.
05-Dec 01:02 bacula-dir JobId 1726: Error: Bacula bacula-dir 9.6.5
(11Jun20):
Similar output from 21-Nov:
21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error:
lib/bsock.c:383 Write error sending 4 bytes to Storage
daemon:192.168.66.35:9103: ERR=Input/output error
21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error:
lib/bsock.c:271 Socket has errors=1 on call to Storage
daemon:192.168.66.35:9103
21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error:
lib/bsock.c:271 Socket has errors=1 on call to Storage
daemon:192.168.66.35:9103
21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Fatal error:
filed/backup.c:607 Network send error to SD. ERR=Input/output error
21-Nov 05:49 bacula-dir JobId 1393: Fatal error: Network error with
FD during Backup: ERR=Connection timed out
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
