Hi,
What am I doing wrong? I would be grateful for a hint, why there is no reply to my emails. I believe that my problems are a bit complex for an email. Is there any way to discuss this in different way? Who should I contact in case that I would like to support development of oracle cloud plugin?

Regards,
Ziga

On 21.12.2020 20:38, Žiga Žvan wrote:
Hello,
I'm using file driver with cloud resource. Bacula was able to backup data in this way until it wrote data to new volumes. Now, after retention period,  I'm getting error: Fatal error: cloud_dev.c:983 Unable to download Volume (see output below). Data on cloud path looks ok but data in local cache contains only part.1 without any data.

Is this expected?
Has anybody tested this scenario?
Should I avoid file driver in production environment?

Regards,
Ziga


[root@bacula db-01-weekly-vol-0365]# ls -la /mnt/ocisg/bacula/backup/db-01-weekly-vol-0365
total 0
drwxr-----. 2 bacula disk           0 Oct 24 07:45 .
drwxr-xr-x. 2 bacula bacula         0 Dec 18 23:38 ..
-rw-r--r--. 1 bacula disk         256 Oct 24 07:43 part.1
-rw-r--r--. 1 bacula disk   999935992 Oct 24 07:44 part.2
-rw-r--r--. 1 bacula disk   999935993 Oct 24 07:44 part.3
-rw-r--r--. 1 bacula disk   381771773 Oct 24 07:45 part.4

[root@bacula db-01-weekly-vol-0365]# ls -la /storage/bacula/cloudcache/db-01-weekly-vol-0365
total 20
drwxr-----.   2 bacula disk      28 Dec 11 23:10 .
drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 ..
-rw-r--r--.   1 bacula disk       0 Dec 11 23:10 part.1

SD config (autochanger)

Device {
  Name = FSOciCloudStandard2
  Device type = Cloud
  Cloud = OracleViaStorageGateway
  Maximum Part Size = 1000 MB
  Media Type = File1
  Archive Device = /storage/bacula/cloudcache
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Autochanger = yes;
}
...
Device {
  Name = FSOciCloudStandard4
  Device type = Cloud
  Cloud = OracleViaStorageGateway
  Maximum Part Size = 1000 MB
  Media Type = File1
  Archive Device = /storage/bacula/cloudcache
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
  Autochanger = yes;
}

Cloud {
  Name = OracleViaStorageGateway
  Driver = "File"
  HostName = "/mnt/ocisg/bacula/backup"
  BucketName = "DummyBucket"
  AccessKey = "DummyAccessKey"
  SecretKey = "DummySecretKey"
  Protocol = HTTPS
  UriStyle = VirtualHost
  Truncate Cache = AtEndOfJob
}


21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073, Job=db-01-backup.2020-12-21_19.14.14_48 21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2" to write. 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable to download Volume="db-01-weekly-vol-0365" label. 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable to download Volume="db-01-weekly-vol-0365" label. 21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error: job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK data
, got len=25 msg="3903 Error append data:  "
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable to download Volume="db-01-weekly-vol-0365" label. 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume "db-01-weekly-vol-0365" failed: ERR= 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable to download Volume="db-01-weekly-vol-0365" label. 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable to download Volume="db-01-weekly-vol-0365" label. 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume "db-01-weekly-vol-0365" failed: ERR= 21-Dec 19:14 bacula-sd JobId 2073: Marking Volume "db-01-weekly-vol-0365" in Error in Catalog.
21-Dec 19:14 bacula-sd JobId 2073: Fatal error: Job 2073 canceled.


On 06.12.2020 20:52, Žiga Žvan wrote:
Dear all,
I'm using bacula 9.6.5 in a production for a month now. I'm experiencing random backup failures from my clients. Specific hosts report errors like the outputs attached. The same host is able to perform backup at some other time. The error is more often at large backups (more errors at full backups than incremental, more errors at hosts with large data sets).

I have tried to implement heartbeat interval (https://www.bacula.org/9.6.x-manuals/en/main/Client_File_daemon_Configur.html#SECTION002210000000000000000) but there is no improvement. The error occures also on hosts in the same zone as bacula server (no router/firewall in between). Storage deamon is installed on the same server as bacula director. I'm using File cloud driver (backup to local disk via cloud resource).

Could you please suggest a solution or a way to troubleshoot this further?
Thx!

Regards,Ziga Zvan

Backup from linux hosts (on 05-dec 3 hosts failed, 20 hosts completed without error): 05-Dec 03:26 bacula-dir JobId 1721: Fatal error: Network error with FD during Backup: ERR=Connection reset by peer 05-Dec 03:27 bacula-dir JobId 1721: Fatal error: No Job status returned from FD. 05-Dec 03:27 bacula-dir JobId 1721: Error: Bacula bacula-dir 9.6.5 (11Jun20):

Backup from windows hosts (on 05-dec 2 hosts failed, 5 hosts completed without error): 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Error: lib/bsock.c:383 Write error sending 57172 bytes to Storage daemon:192.168.66.35:9103: ERR=Input/output error 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Fatal error: filed/backup.c:848 Network send error to SD. ERR=Input/output error 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "Task Scheduler Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "VSS Metadata Store Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "Performance Counters Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "System Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "ASR Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer (BackupComplete): "Shadow Copy Optimization Writer", State: 0x1 (VSS_WS_STABLE) 05-Dec 01:01 bacula-dir JobId 1726: Error: bsock.c:551 Read error from Client: iwhost01.kranj.cetrtapot.si-fd:iwhost01.kranj.cetrtapot.si:9102: ERR=Connection timed out 05-Dec 01:01 bacula-dir JobId 1726: Fatal error: Network error with FD during Backup: ERR=Connection timed out 05-Dec 01:02 bacula-dir JobId 1726: Fatal error: No Job status returned from FD. 05-Dec 01:02 bacula-dir JobId 1726: Error: Bacula bacula-dir 9.6.5 (11Jun20):

Similar output from 21-Nov:
21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: lib/bsock.c:383 Write error sending 4 bytes to Storage daemon:192.168.66.35:9103: ERR=Input/output error 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: lib/bsock.c:271 Socket has errors=1 on call to Storage daemon:192.168.66.35:9103 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: lib/bsock.c:271 Socket has errors=1 on call to Storage daemon:192.168.66.35:9103 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Fatal error: filed/backup.c:607 Network send error to SD. ERR=Input/output error 21-Nov 05:49 bacula-dir JobId 1393: Fatal error: Network error with FD during Backup: ERR=Connection timed out




_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Reply via email to