Re: [Bacula-users] restore raid array

[Jerry Lowry]

Thank you Kern and fellow Bacula users,

I had a suspicion that was the case, but thought I would ask. I appreciate
the help and commend the community for its endeavors to work with other
users.

thank you again,

jerry

On Wed, Dec 12, 2018 at 12:25 AM Kern Sibbald  wrote:

> Hello,
>
> Bacula has no way to reconstruct the data in an original backup Volume.
> If you lose the Volume, it is gone.  About the only mitigating factors
> after you delete the original volumes are: as is your case, switch to using
> the Copy volume in place of the original Backup volumes;  begin re-creating
> new volumes (obviously the history for prior backups will not be available
> in those volumes).
>
> There is one other possibility that might work.  After deleting the
> original Volumes, your Copy Volumes will be promoted to being the Backup
> Volumes.  You could possibly then do a sort of reverse Copy of your Copy
> volumes (now promoted to Backup) to your on-site location.  Then by some
> SQL magic, you might be able to swap your Backup volumes and Copy volumes
> so that you will be back to the original configuration, except that your
> original Backups will be reconstructed from copies of your Copy volumes.
> To the best of my knowledge no one has ever done this, and without a lot of
> technical knowledge of the catalog formats, it is not possible.
>
> I have never been fully happy with how Copy volumes are handled, or more
> precisely the lack of control the user has from bconsole to manipulate Copy
> volumes.  Perhaps some additional future code could make situations like
> yours easier to manage.
>
> Perhaps someone else has a good idea to solve this problem.
>
> Best regards,
> Kern
>
>
>
> On 12/11/18 6:13 PM, Jerry Lowry wrote:
>
> Josh,
> Yes, I understand how the copy jobs works when the original job is
> deleted.  What I need to do is rebuild the client directories with the
> backup database using the latest offsite backup, if possible.  I don't want
> to restore the backup for the client, I want to rebuild the data in the
> directories so that a restore can be done from there.  Hope that makes
> since.  I know that I can use the offsite backups to restore the client
> data. I want to know if I can rebuild what would be the initial backup of
> the volume.
>
> thanks,
> jerry
>
> On Tue, Dec 11, 2018 at 6:18 AM Josh Fisher  wrote:
>
>>
>> On 12/11/2018 1:09 AM, Jerry Lowry wrote:
>>
>> Well,  The raid was (8) 6TB disks attached to an ATTO Tech raid
>> controller in a supermicro cabinet. It was setup as a raid-5 disk array.
>> The ATTO support group know how it was configured as they have been helping
>> me since Thursday.  The raid setup is not the problem, that can be rebuilt
>> to duplicate the on disk file structure. Bacula was using this raid array
>> as storage for different clients in my network.  Each client had a
>> directory on the array and with in each directory there were anywhere from
>> 3 bacula volumes to 8 volumes. Each volume held any where from 250 GB to
>> 320 GB. Two of the clients would have an offsite backup done each week. A
>> Bacula copy job would run each week and copy that weeks backups on to a hot
>> swap raid disk running on the same system. The system was the Director and
>> Storage director combined.
>> What I want to do is to restore the daily volumes into the new raid array
>> from the offsite disks from the most recent offsite backups. Will any of
>> the bacula utilities enable me to do this?
>>
>>
>> No special tools are needed. If the original volume files no longer
>> exist, then the volumes (and their jobs) can be deleted from the catalog
>> using 'delete volume'. When bacula finds a Copy of a job when a Job is
>> deleted from the catalog, then it will automatically promote the Copy as
>> the real backup for that job so that subsequent restores use the promoted
>> copy rather than the original. The Copy literally replaces the original and
>> the original ceases to exist. At that point, a normal restore will
>> automatically use those promoted volumes. You should ensure that those
>> promoted volumes are marked as Used so that no jobs will attempt to write
>> to them.
>>
>> If auto-labeling is being used, then jobs should create new volumes as
>> needed when they run. If not, then you will manually create new empty
>> volume files in the client directories and label them using the Label
>> command from bconsole.
>>
>>
>>
>> Thanks,
>> jerry
>>
>> On Mon, Dec 10, 2018 at 6:01 PM Phil Stracchino 
>> wrote:
>>
>>> On 12/10/18 7:44 PM, Jerry Lowry wrote:
>>> > Hi,
>>> > Last thursday I was adding a disk to the bacula raid array when the
>>> > system decide to fail. When it rebooted my raid array was gone. This
>>> > raid was where all of my daily backups were held, which is where I do
>>> my
>>> > offsite backups from.  The database is fine, catalog is in working
>>> > order.  I rebuilt the server and per Raid Support I have all new disks.
>>> >
>>> > I need to recreate the physica

Re: [Bacula-users] [External] Re: "Full" tapes with zero bytes?

[mark . bergman]

In the message dated: Wed, 12 Dec 2018 13:40:04 +,
The pithy ruminations from Martin Simmons on 
[[External] Re: [Bacula-users] "Full" tapes with zero bytes?] were:
=> > On Tue, 11 Dec 2018 16:02:19 -0500, mark bergman said:
=> > 
=> > I'm running Bacula 9.0.8 under CentOS 6, and I'm seeing something odd with 
some tapes. Backups were written to the tapes, but a query of the status of 
volumes in the tape changer shows that the media is
=> > "Full" with zero GB:
=> > 
=> > 
=> > 
=> >Choose a query (1-50): 15
=> >
+-++---+-+--+-+---+---+
=> >| mediaid | volumename | gb| storage | slot | pool| 
mediatype | volstatus |
=> >
+-++---+-+--+-+---+---+
=> >| 264 | 006273L6   | 0 | neoxl80 |   19 | Full| LTO6
  | Full  |
=> >| 220 | 006200L6   | 0 | neoxl80 |   20 | Incremental | LTO6
  | Full  |
=> 
=> The output of "llist volume=006273L6" would be useful, to see the full info
=> about the volume (yes, that's llist with two ll's).

Sure:

-
*llist volume=006273L6 
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
  mediaid: 264
   volumename: 006273L6
 slot: 19
   poolid: 2
mediatype: LTO6
  mediatypeid: 0
 firstwritten: 2017-12-22 03:45:03
  lastwritten: 2018-11-17 01:50:55
labeldate: 2017-06-05 02:45:04
  voljobs: 7
 volfiles: 178
volblocks: 2
 volparts: 0
volcloudparts: 0
   cacheretention: 0
volmounts: 6
 volbytes: 322,561
volabytes: 0
  volapadding: 0
 volholebytes: 0
 volholes: 0
lastpartbytes: 0
volerrors: 0
volwrites: 50,901,328
 volcapacitybytes: 0
volstatus: Full
  enabled: 1
  recycle: 1
 volretention: 46,656,000
   voluseduration: 0
   maxvoljobs: 0
  maxvolfiles: 0
  maxvolbytes: 0
inchanger: 1
  endfile: 178
 endblock: 0
  voltype: 2
labeltype: 0
storageid: 2
 deviceid: 0
  mediaaddressing: 0
  volreadtime: 0
 volwritetime: 46,066,122,018
   locationid: 0
 recyclecount: 1
 initialwrite: 
scratchpoolid: 0
recyclepoolid: 4
actiononpurge: 0
expiresin: 44,460,471
  comment: 



*llist volume=006200L6
  mediaid: 220
   volumename: 006200L6
 slot: 20
   poolid: 1
mediatype: LTO6
  mediatypeid: 0
 firstwritten: 2018-11-06 11:53:17
  lastwritten: 2018-11-17 01:48:07
labeldate: 2017-04-04 03:50:05
  voljobs: 14
 volfiles: 147
volblocks: 1
 volparts: 0
volcloudparts: 0
   cacheretention: 0
volmounts: 3
 volbytes: 129,025
volabytes: 0
  volapadding: 0
 volholebytes: 0
 volholes: 0
lastpartbytes: 0
volerrors: 0
volwrites: 45,040,851
 volcapacitybytes: 0
volstatus: Full
  enabled: 1
  recycle: 1
 volretention: 8,467,200
   voluseduration: 0
   maxvoljobs: 0
  maxvolfiles: 0
  maxvolbytes: 0
inchanger: 1
  endfile: 147
 endblock: 0
  voltype: 2
labeltype: 0
storageid: 2
 deviceid: 0
  mediaaddressing: 0
  volreadtime: 0
 volwritetime: 18,106,715,487
   locationid: 0
 recyclecount: 1
 initialwrite: 
scratchpoolid: 0
recyclepoolid: 4
actiononpurge: 0
expiresin: 6,275,051
  comment: 
-



Hmmm. volbytes looks "wrong" for both volumes, if they really contain 
backups.

I'm going to run some diagnostics with "btape" and "bls" to try to determine 
whether there is actually data on the tapes.

Mark

=> 
=> __Martin
=> 

-- 
Mark Bergman   voice: 215-746-4061  
 
mark.berg...@uphs.upenn.edu  fax: 215-614-0266
http://www.med.upenn.edu/cbica/
IT Technical Director, Center for Biomedical Image Computing and Analytics
Department of Radiology University of Pennsylvania


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] "Full" tapes with zero bytes?

[Martin Simmons]

> On Tue, 11 Dec 2018 16:02:19 -0500, mark bergman said:
> 
> I'm running Bacula 9.0.8 under CentOS 6, and I'm seeing something odd with 
> some tapes. Backups were written to the tapes, but a query of the status of 
> volumes in the tape changer shows that the media is
> "Full" with zero GB:
> 
> 
> 
>   Choose a query (1-50): 15
>   
> +-++---+-+--+-+---+---+
>   | mediaid | volumename | gb| storage | slot | pool| 
> mediatype | volstatus |
>   
> +-++---+-+--+-+---+---+
>   | 264 | 006273L6   | 0 | neoxl80 |   19 | Full| LTO6
>   | Full  |
>   | 220 | 006200L6   | 0 | neoxl80 |   20 | Incremental | LTO6
>   | Full  |

The output of "llist volume=006273L6" would be useful, to see the full info
about the volume (yes, that's llist with two ll's).

__Martin


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore raid array

[Kern Sibbald]

  
  
Hello,

Bacula has no way to reconstruct the data in an original backup
Volume.  If you lose the Volume, it is gone.  About the only
mitigating factors after you delete the original volumes are: as is
your case, switch to using the Copy volume in place of the original
Backup volumes;  begin re-creating new volumes (obviously the
history for prior backups will not be available in those volumes).  


There is one other possibility that might work.  After deleting the
original Volumes, your Copy Volumes will be promoted to being the
Backup Volumes.  You could possibly then do a sort of reverse Copy
of your Copy volumes (now promoted to Backup) to your on-site
location.  Then by some SQL magic, you might be able to swap your
Backup volumes and Copy volumes so that you will be back to the
original configuration, except that your original Backups will be
reconstructed from copies of your Copy volumes.  To the best of my
knowledge no one has ever done this, and without a lot of technical
knowledge of the catalog formats, it is not possible.

I have never been fully happy with how Copy volumes are handled, or
more precisely the lack of control the user has from bconsole to
manipulate Copy volumes.  Perhaps some additional future code could
make situations like yours easier to manage.

Perhaps someone else has a good idea to solve this problem.

Best regards,
Kern



On 12/11/18 6:13 PM, Jerry Lowry wrote:


  
  
Josh,
Yes, I understand how the copy jobs works when the original
  job is deleted.  What I need to do is rebuild the client
  directories with the backup database using the latest offsite
  backup, if possible.  I don't want to restore the backup for
  the client, I want to rebuild the data in the directories so
  that a restore can be done from there.  Hope that makes
  since.  I know that I can use the offsite backups to restore
  the client data. I want to know if I can rebuild what would be
  the initial backup of the volume.


thanks,
jerry

  
  
  
On Tue, Dec 11, 2018 at 6:18 AM Josh Fisher 
  wrote:


  


On
  12/11/2018 1:09 AM, Jerry Lowry wrote:


  
Well,  The raid was (8) 6TB disks attached to an
  ATTO Tech raid controller in a supermicro cabinet. It
  was setup as a raid-5 disk array.  The ATTO support
  group know how it was configured as they have been
  helping me since Thursday.  The raid setup is not the
  problem, that can be rebuilt to duplicate the on disk
  file structure. Bacula was using this raid array as
  storage for different clients in my network.  Each
  client had a directory on the array and with in each
  directory there were anywhere from 3 bacula volumes to
  8 volumes. Each volume held any where from 250 GB to
  320 GB. Two of the clients would have an offsite
  backup done each week. A Bacula copy job would run
  each week and copy that weeks backups on to a hot swap
  raid disk running on the same system. The system was
  the Director and Storage director combined.
What I want to do is to restore the daily volumes
  into the new raid array from the offsite disks from
  the most recent offsite backups. Will any of the
  bacula utilities enable me to do this?
  



No special tools are needed. If the original volume files
  no longer exist, then the volumes (and their jobs) can be
  deleted from the catalog using 'delete volume'. When
  bacula finds a Copy of a job when a Job is deleted from
  the catalog, then it will automatically promote the Copy
  as the real backup for that job so that subsequent
  restores use the promoted copy rather than the original.
  The Copy literally replaces the original and the original
  ceases to exist. At that point, a normal restore will
  automatically use those promoted volumes. You should
  ensure that those promoted volumes are marked as Used so
  that no jobs will attempt to write to them.  

If auto-labeling is being used, then jobs should create
  new volumes as needed when they run. If not, then you will
  manually create new empty volume files in