Re: [Bacula-users] Perl 5.20 upgrade breaks make_catalog_backup.pl?
On 06/02/15 21:01, Doug Sampson wrote: Recently FreeBSD announced that its default Perl version is now 5.20 and urged all users to upgrade to that version. I did so. Now make_catalog_backup.pl doesn’t run. The first indication was as follows: Hello. I've had some troubles with 5.20, but I'm not sure they are the same you are seeing. I even wrote about this on questions@... In my case the problem was that: _ older PERLs used to symlink /usr/local/bin/perl to /usr/bin/perl, but 5.20 does not; _ make_catalog_backup.pl starts with #!/usr/bin/env perl, which finds /usr/local/bin/perl when run as root, but does not when run from Bacula, since /usr/local/bin/ is not in path. So I could either: _ try and make Bacula see /usr/local/bin/ in path; _ make the symlink myself; _ modify the scripts to start with #!/usr/local/bin/perl. I choose the third option and, after that, everything is running fine. From your mail I think something actually runs on your system; are you sure, however, the correct perl binary has been picked up? Could you try modifying the scripts as above? bye av. -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use multiple HDDs to backup files to
On 6/2/2015 1:42 PM, Heitor Faria wrote: I particularly do not like the idea of having separate volumes of the same media type into two different mount points, but I know people that are configuring this way. So I decided to do some tests and see what really was happening. No issues. Instead I prefer the LVM solution. The point is to make sure severall disks are being write at the same time (load balance) and more resiliency since a dead LVM is a terrible situation. But I kind of agree with you: I don't know why to use different Media Types for compatible media. Because, using the previous example config, volumes in directory /sdb1/bacula/backup cannot be loaded into vDrive1 and volumes in directory /bacula/backup cannot be loaded into vDrive2. Bacula uses the MediaType to decide which volumes can be loaded into which Devices. A backup job will run without error, writing to either one directory or the other. But unless tvDrive1 and vDrive2 have different Media Types, a subsequent restore job will assume it can load any of those volumes into whichever Device it happens to get assigned to read from. Regards, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II 15 a 26 de junho: Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174 61 tel:%2B55%2061%202021-82608268-4220 tel:%2B55%2061%208268-4220 Site: www.bacula.com.br http://www.bacula.com.br/ | Facebook: heitor.faria http://www.facebook.com/heitor.faria | Gtalk: heitorfa...@gmail.com mailto:heitorfa...@gmail.com On Tue, Jun 2, 2015 at 2:12 PM, Bryn Hughes li...@nashira.ca mailto:li...@nashira.ca wrote: Do you have some specific reason to need to have these two volumes separate? You might be best off using LVM to create a single logical drive. That would certainly be a more flexible solution long term. Bryn On 2015-05-20 09:32 AM, SPQR wrote: I would like to set up bacula - well, my first steps have been successful. All my backups are written to /bacula/backup. But / has only 900GB. So I want bacula to write to /sdb1/bacula/backup, too. /sdb1 has another 950GB of space. The sum of available backup-space should be 1850GB HDD: Well, I got the following config of /etc/bacula/bacula-sd.conf: Storage { # definition of myself Name = backup-sd SDPort = 9103 # Director's port WorkingDirectory = /var/lib/bacula Pid Directory = /var/run/bacula Maximum Concurrent Jobs = 20 SDAddress = backup.example.com http://backup.example.com } Director { Name = backup-dir Password = doyoureallywanttoknow } Device { Name = FileStorage Media Type = File Archive Device = /bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } Okay - now I would like to add another device. I guess I could just add a second entry like this: Device { Name = FileStorage Media Type = File Archive Device = /sdb1/bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } ? Do I have to change something else so that bacula is able to write more than 900GB - if the first 900GB are full it should switch to the second hdd. Thank you very much :-) +-- |This was sent by r...@ehrenwert.it mailto:r...@ehrenwert.it via Backup Central. |Forward SPAM to ab...@backupcentral.com mailto:ab...@backupcentral.com. +-- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net mailto:Bacula-users@lists.sourceforge.net
Re: [Bacula-users] Use multiple HDDs to backup files to
On 6/2/2015 12:43 PM, Ana Emília M. Arruda wrote: Hello Josh, On Tue, Jun 2, 2015 at 8:12 AM, Josh Fisher jfis...@pvct.com mailto:jfis...@pvct.com wrote: On 6/1/2015 5:21 PM, Ana Emília M. Arruda wrote: Hello, Maybe you could use the virtual autochanger resource: This will not work as expected. If the ArchiveDevice in the two Device resources specifies different directories (mountpoints), then the two Device resources cannot have the same MediaType. This is because Bacula expects to be able to load any volume that has MediaType=File into any Device having that same MediaType, and a volume in /sdb1/bacula/backup cannot be loaded into Device vDrive-1, not can a volume in /bacula/backup be loaded into vDrive-2. No, volumes in /sdb1/bacula/backup will not be loaded into vDrive-1 and volumes in /bacula/backup will not be loaded into vDrive-2. I read about this in the white papers from blog.bacula.org http://blog.bacula.org. I decided to do some tests with this. Lots of people are working with virtual autochangers this way. When you submit concurrent jobs, the volumes are mounted into their respective devices. If a requested volume is in /sdb1/bacula/backup , it will be loaded into vDrive-1 and if a requested volume is in /bacula/backup, it will be loaded into vDrive-2. This works this way and gives no error. But I agree with you that the option using LVM is a lot better. Yes. For a backup job using automatic labeling, it will indeed work. But that is misleading. Say on 1 June a full job ran using vDrive1 and wrote to volume full_1 in directory /bacula/backup. On 2 June an incremental job is run, only this time it was assigned vDrive2 and wrote to volume inc_1 in directory /sdb1/bacula/backup. Both of these jobs will run without error. However, now a restore is impossible, as is a subsequent Virtual Full job. Best regards, Ana To use multiple HDDs as a single autochanger, you must: Define each Device resource to have a unique MediaType, if using the native virtual autochanger or Use vchanger, which is designed for using multiple removable HDDs (See http://sourceforge.net/projects/vchanger/) or Combine the HDD partitions using LVM2 or md so that there is only a single mountpoint . bacula-sd.conf: Autochanger { Name = VChanger Changer Device = /dev/null Changer Command = # versão 5.2.6. For 7.0.5 version, use /dev/null Device = vDrive-1, vDrive-2 } Device { Name = vDrive-1 Drive Index = 0 Device Type = File Media Type = File Archive Device = /bacula/backup ... } Device { Name = vDrive-2 Drive Index = 1 Device Type = File Media Type = File Archive Device = /sdb1/bacula/backup # is this your mount point? ... } In your bacula-dir.conf (or wherever you have your storage definiton for director): Storage { Name = MyVirtualAutochanger Address = X.X.X.X SDPort = 9103 Password = xx Device = VChanger Media Type = File Maximum Concurrent Jobs = XX Autochanger=yes } Best regards, Ana On Wed, May 20, 2015 at 1:32 PM, SPQR bacula-fo...@backupcentral.com mailto:bacula-fo...@backupcentral.com wrote: I would like to set up bacula - well, my first steps have been successful. All my backups are written to /bacula/backup. But / has only 900GB. So I want bacula to write to /sdb1/bacula/backup, too. /sdb1 has another 950GB of space. The sum of available backup-space should be 1850GB HDD: Well, I got the following config of /etc/bacula/bacula-sd.conf: Storage { # definition of myself Name = backup-sd SDPort = 9103 # Director's port WorkingDirectory = /var/lib/bacula Pid Directory = /var/run/bacula Maximum Concurrent Jobs = 20 SDAddress = backup.example.com http://backup.example.com } Director { Name = backup-dir Password = doyoureallywanttoknow } Device { Name = FileStorage Media Type = File Archive Device = /bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } Okay - now I would like to add another device. I guess I could just add a second entry like this: Device { Name = FileStorage Media Type = File Archive Device = /sdb1/bacula/backup
Re: [Bacula-users] Bacula not recycling all volumes
Hi Marc, Yes. This is for preserving your data as long as possible. In your pool definition, you have configured: Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 100 # Limit number of Volumes in Pool This way, Bacula will only recycle a volume if the limit of 100 volumes is reached and also if it could not find an appendable volume available in the pool. You have less then 50 volumes up to now. Best regards, Ana On Wed, Jun 3, 2015 at 2:53 AM, Marc van Houtum m...@dutchsecurity.net wrote: Hi Ana, Does this mean that Bacula prefers to create new volumes over recycling old ones? Here's the output, Vol-0001 for example isn't used by any backup jobs looking at the bacula database, as are 15 others: *list media Pool: File +-++---+-++--+--+-+--+---+---+-+ | MediaId | VolumeName | VolStatus | Enabled | VolBytes | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-++---+-++--+--+-+--+---+---+-+ | 72 | Vol-0001 | Full | 1 | 53,687,079,572 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 16:34:38 | | 73 | Vol-0073 | Full | 1 | 53,687,079,571 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 23:48:02 | | 74 | Vol-0074 | Full | 1 | 53,687,051,357 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-22 01:09:10 | | 83 | Vol-0083 | Full | 1 | 53,687,079,336 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-29 02:07:13 | | 84 | Vol-0084 | Full | 1 | 53,687,079,650 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-29 04:02:07 | | 85 | Vol-0085 | Full | 1 | 53,687,038,466 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-30 00:20:19 | | 86 | Vol-0086 | Full | 1 | 53,687,072,062 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 00:43:51 | | 87 | Vol-0087 | Full | 1 | 53,687,078,955 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 03:31:16 | | 88 | Vol-0088 | Full | 1 | 53,687,079,760 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 05:46:51 | | 89 | Vol-0089 | Full | 1 | 53,687,079,537 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 07:12:04 | | 99 | Vol-0099 | Full | 1 | 53,687,035,899 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-14 00:56:20 | | 100 | Vol-0100 | Full | 1 | 53,687,079,400 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 08:30:19 | | 101 | Vol-0101 | Full | 1 | 53,687,079,499 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 09:50:03 | | 102 | Vol-0102 | Full | 1 | 53,687,079,535 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 11:24:41 | | 103 | Vol-0103 | Full | 1 | 53,687,079,562 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-31 13:12:54 | | 106 | Vol-0104 | Full | 1 | 53,687,079,466 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-10 19:35:51 | | 107 | Vol-0107 | Full | 1 | 53,687,061,994 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 00:28:01 | | 108 | Vol-0108 | Full | 1 | 53,687,033,748 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 03:09:56 | | 109 | Vol-0109 | Full | 1 | 53,687,079,058 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 06:36:13 | | 110 | Vol-0110 | Full | 1 | 53,687,079,593 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 07:44:40 | | 111 | Vol-0111 | Full | 1 | 53,687,079,599 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 08:53:21 | | 112 | Vol-0112 | Full | 1 | 53,687,079,515 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 10:07:45 | | 113 | Vol-0113 | Full | 1 | 53,687,079,470 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 11:35:35 | | 114 | Vol-0114 | Full | 1 | 53,687,079,758 | 12 |1,209,600 | 1 |0 | 0 | Disk | 2015-05-17 13:34:05 | | 115
Re: [Bacula-users] Use multiple HDDs to backup files to
Ana, You are right. Now I understand this workaround. Regards, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II 15 a 26 de junho: Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174 61 8268-4220 Site: www.bacula.com.br | Facebook: heitor.faria De: Ana Emília M. Arruda emiliaarr...@gmail.com Para: Josh Fisher jfis...@pvct.com Cc: Bacula-users@lists.sourceforge.net bacula-users@lists.sourceforge.net Enviadas: Quarta-feira, 3 de junho de 2015 15:19:23 Assunto: Re: [Bacula-users] Use multiple HDDs to backup files to Thank you Josh. I read in the Best Practices for Disk Based Backup white paper this recommendation (different media types for different mount points). Lots of people were reporting that this was working fine. So I had been testing the backup with this configuration, not the restores. Now I see that in the case of a restore, this will only work for volumes that are in the first device (mount point). Best regards, Ana On Wed, Jun 3, 2015 at 10:21 AM, Josh Fisher jfis...@pvct.com wrote: On 6/2/2015 1:42 PM, Heitor Faria wrote: I particularly do not like the idea of having separate volumes of the same media type into two different mount points, but I know people that are configuring this way. So I decided to do some tests and see what really was happening. No issues. Instead I prefer the LVM solution. The point is to make sure severall disks are being write at the same time (load balance) and more resiliency since a dead LVM is a terrible situation. But I kind of agree with you: I don't know why to use different Media Types for compatible media. Because, using the previous example config, volumes in directory /sdb1/bacula/backup cannot be loaded into vDrive1 and volumes in directory /bacula/backup cannot be loaded into vDrive2. Bacula uses the MediaType to decide which volumes can be loaded into which Devices. A backup job will run without error, writing to either one directory or the other. But unless tvDrive1 and vDrive2 have different Media Types, a subsequent restore job will assume it can load any of those volumes into whichever Device it happens to get assigned to read from. Regards, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II 15 a 26 de junho: Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174 61 8268-4220 Site: www.bacula.com.br | Facebook: heitor.faria | Gtalk: heitorfa...@gmail.com On Tue, Jun 2, 2015 at 2:12 PM, Bryn Hughes li...@nashira.ca wrote: Do you have some specific reason to need to have these two volumes separate? You might be best off using LVM to create a single logical drive. That would certainly be a more flexible solution long term. Bryn On 2015-05-20 09:32 AM, SPQR wrote: I would like to set up bacula - well, my first steps have been successful. All my backups are written to /bacula/backup. But / has only 900GB. So I want bacula to write to /sdb1/bacula/backup, too. /sdb1 has another 950GB of space. The sum of available backup-space should be 1850GB HDD: Well, I got the following config of /etc/bacula/bacula-sd.conf: Storage { # definition of myself Name = backup-sd SDPort = 9103 # Director's port WorkingDirectory = /var/lib/bacula Pid Directory = /var/run/bacula Maximum Concurrent Jobs = 20 SDAddress = backup.example.com } Director { Name = backup-dir Password = doyoureallywanttoknow } Device { Name = FileStorage Media Type = File Archive Device = /bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } Okay - now I would like to add another device. I guess I could just add a second entry like this: Device { Name = FileStorage Media Type = File Archive Device = /sdb1/bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } ? Do I have to change something else so that bacula is able to write more than 900GB - if the first 900GB are full it should switch to the second hdd. Thank you very much :-) +-- |This was sent by r...@ehrenwert.it via Backup Central. |Forward SPAM to ab...@backupcentral.com . +-- --
Re: [Bacula-users] Bacula not recycling all volumes
Thanks for your help Ana, I will our maximum volumes and maximum volume bytes settings. Best regards, Marc On 3-6-2015 20:29, Ana Emília M. Arruda wrote: Hello Marc, Your autoprune configurations seems ok for me. Bacula will delete from catalog jobs/files older than 14 days and purge volumes older than 14 days. In the case of volumes, they will not be reused if Bacula finds an available volume for writing (status = append) or if it can create a new one. I don´t know a solution with automatic label configuration where you have space for new volumes and Bacula recycle/reuse an existing one instead of creating a new one. Your problem is This is causing our disk based storage to fill up with unused volume files. If you know exactly the amount of space you have for your backups, you can limit the number/size of your volumes to fit in this amount of space. In the case of your pool configuration, you will use 5 TB (50G x 100 volumes). This will be the total amount of space that will be used by your pool. Pool { Name = File Pool Type = Backup Recycle = yes # Bacula can automatically recycle Volumes AutoPrune = yes # Prune expired volumes Action On Purge = Truncate Volume Retention = 14 days # Max retention Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 100 # Limit number of Volumes in Pool Label Format = Vol- # Auto label } Best regards, Ana -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula not recycling all volumes
Hi Ana, Thanks for the explanation. Is this something that can be configured? Ideally I'd like Bacula to recycle/purge every volume that no longer has any jobs related to it. I was under the impression that AutoPrune would do this seeing as it does mark volumes are purged if run manually: *prune yes volume=Vol-0104 The current Volume retention period is: 14 days There are no more Jobs associated with Volume Vol-0104. Marking it purged. Have I configured AutoPrune incorrectly? Best regards, Marc On 3-6-2015 17:53, Ana Emília M. Arruda wrote: Hi Marc, Yes. This is for preserving your data as long as possible. In your pool definition, you have configured: Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 100 # Limit number of Volumes in Pool This way, Bacula will only recycle a volume if the limit of 100 volumes is reached and also if it could not find an appendable volume available in the pool. You have less then 50 volumes up to now. Best regards, Ana -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use multiple HDDs to backup files to
Thank you Josh. I read in the Best Practices for Disk Based Backup white paper this recommendation (different media types for different mount points). Lots of people were reporting that this was working fine. So I had been testing the backup with this configuration, not the restores. Now I see that in the case of a restore, this will only work for volumes that are in the first device (mount point). Best regards, Ana On Wed, Jun 3, 2015 at 10:21 AM, Josh Fisher jfis...@pvct.com wrote: On 6/2/2015 1:42 PM, Heitor Faria wrote: I particularly do not like the idea of having separate volumes of the same media type into two different mount points, but I know people that are configuring this way. So I decided to do some tests and see what really was happening. No issues. Instead I prefer the LVM solution. The point is to make sure severall disks are being write at the same time (load balance) and more resiliency since a dead LVM is a terrible situation. But I kind of agree with you: I don't know why to use different Media Types for compatible media. Because, using the previous example config, volumes in directory /sdb1/bacula/backup cannot be loaded into vDrive1 and volumes in directory /bacula/backup cannot be loaded into vDrive2. Bacula uses the MediaType to decide which volumes can be loaded into which Devices. A backup job will run without error, writing to either one directory or the other. But unless tvDrive1 and vDrive2 have different Media Types, a subsequent restore job will assume it can load any of those volumes into whichever Device it happens to get assigned to read from. Regards, === Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified Administrator II 15 a 26 de junho: Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174 61 %2B55%2061%202021-82608268-4220 %2B55%2061%208268-4220 Site: www.bacula.com.br | Facebook: heitor.faria http://www.facebook.com/heitor.faria | Gtalk: heitorfa...@gmail.com On Tue, Jun 2, 2015 at 2:12 PM, Bryn Hughes li...@nashira.ca wrote: Do you have some specific reason to need to have these two volumes separate? You might be best off using LVM to create a single logical drive. That would certainly be a more flexible solution long term. Bryn On 2015-05-20 09:32 AM, SPQR wrote: I would like to set up bacula - well, my first steps have been successful. All my backups are written to /bacula/backup. But / has only 900GB. So I want bacula to write to /sdb1/bacula/backup, too. /sdb1 has another 950GB of space. The sum of available backup-space should be 1850GB HDD: Well, I got the following config of /etc/bacula/bacula-sd.conf: Storage { # definition of myself Name = backup-sd SDPort = 9103 # Director's port WorkingDirectory = /var/lib/bacula Pid Directory = /var/run/bacula Maximum Concurrent Jobs = 20 SDAddress = backup.example.com } Director { Name = backup-dir Password = doyoureallywanttoknow } Device { Name = FileStorage Media Type = File Archive Device = /bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } Okay - now I would like to add another device. I guess I could just add a second entry like this: Device { Name = FileStorage Media Type = File Archive Device = /sdb1/bacula/backup LabelMedia = yes; # lets Bacula label unlabeled media Random Access = Yes; AutomaticMount = yes; # when device opened, read it RemovableMedia = no; AlwaysOpen = no; } ? Do I have to change something else so that bacula is able to write more than 900GB - if the first 900GB are full it should switch to the second hdd. Thank you very much :-) +-- |This was sent by r...@ehrenwert.it via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users