Re: [Bacula-users] restore raid array

[Jerry Lowry]

Thank you Kern and fellow Bacula users,

I had a suspicion that was the case, but thought I would ask. I appreciate
the help and commend the community for its endeavors to work with other
users.

thank you again,

jerry

On Wed, Dec 12, 2018 at 12:25 AM Kern Sibbald  wrote:

> Hello,
>
> Bacula has no way to reconstruct the data in an original backup Volume.
> If you lose the Volume, it is gone.  About the only mitigating factors
> after you delete the original volumes are: as is your case, switch to using
> the Copy volume in place of the original Backup volumes;  begin re-creating
> new volumes (obviously the history for prior backups will not be available
> in those volumes).
>
> There is one other possibility that might work.  After deleting the
> original Volumes, your Copy Volumes will be promoted to being the Backup
> Volumes.  You could possibly then do a sort of reverse Copy of your Copy
> volumes (now promoted to Backup) to your on-site location.  Then by some
> SQL magic, you might be able to swap your Backup volumes and Copy volumes
> so that you will be back to the original configuration, except that your
> original Backups will be reconstructed from copies of your Copy volumes.
> To the best of my knowledge no one has ever done this, and without a lot of
> technical knowledge of the catalog formats, it is not possible.
>
> I have never been fully happy with how Copy volumes are handled, or more
> precisely the lack of control the user has from bconsole to manipulate Copy
> volumes.  Perhaps some additional future code could make situations like
> yours easier to manage.
>
> Perhaps someone else has a good idea to solve this problem.
>
> Best regards,
> Kern
>
>
>
> On 12/11/18 6:13 PM, Jerry Lowry wrote:
>
> Josh,
> Yes, I understand how the copy jobs works when the original job is
> deleted.  What I need to do is rebuild the client directories with the
> backup database using the latest offsite backup, if possible.  I don't want
> to restore the backup for the client, I want to rebuild the data in the
> directories so that a restore can be done from there.  Hope that makes
> since.  I know that I can use the offsite backups to restore the client
> data. I want to know if I can rebuild what would be the initial backup of
> the volume.
>
> thanks,
> jerry
>
> On Tue, Dec 11, 2018 at 6:18 AM Josh Fisher  wrote:
>
>>
>> On 12/11/2018 1:09 AM, Jerry Lowry wrote:
>>
>> Well,  The raid was (8) 6TB disks attached to an ATTO Tech raid
>> controller in a supermicro cabinet. It was setup as a raid-5 disk array.
>> The ATTO support group know how it was configured as they have been helping
>> me since Thursday.  The raid setup is not the problem, that can be rebuilt
>> to duplicate the on disk file structure. Bacula was using this raid array
>> as storage for different clients in my network.  Each client had a
>> directory on the array and with in each directory there were anywhere from
>> 3 bacula volumes to 8 volumes. Each volume held any where from 250 GB to
>> 320 GB. Two of the clients would have an offsite backup done each week. A
>> Bacula copy job would run each week and copy that weeks backups on to a hot
>> swap raid disk running on the same system. The system was the Director and
>> Storage director combined.
>> What I want to do is to restore the daily volumes into the new raid array
>> from the offsite disks from the most recent offsite backups. Will any of
>> the bacula utilities enable me to do this?
>>
>>
>> No special tools are needed. If the original volume files no longer
>> exist, then the volumes (and their jobs) can be deleted from the catalog
>> using 'delete volume'. When bacula finds a Copy of a job when a Job is
>> deleted from the catalog, then it will automatically promote the Copy as
>> the real backup for that job so that subsequent restores use the promoted
>> copy rather than the original. The Copy literally replaces the original and
>> the original ceases to exist. At that point, a normal restore will
>> automatically use those promoted volumes. You should ensure that those
>> promoted volumes are marked as Used so that no jobs will attempt to write
>> to them.
>>
>> If auto-labeling is being used, then jobs should create new volumes as
>> needed when they run. If not, then you will manually create new empty
>> volume files in the client directories and label them using the Label
>> command from bconsole.
>>
>>
>>
>> Thanks,
>> jerry
>>
>> On Mon, Dec 10, 2018 at 6:01 PM Phil Stracchino 
>> wrote:
>>
>>> On 12/10/18 7:44 PM, Jerry Lowry wrote:
>>> > Hi,
>>> > Last thursday I was adding a disk to the bacula raid array when the
>>> > system decide to fail. When it rebooted my raid array was gone. This
>>> > raid was where all of my daily backups were held, which is where I do
>>> my
>>> > offsite backups from.  The database is fine, catalog is in working
>>> > order.  I rebuilt the server and per Raid Support I have all new disks.
>>> >
>>> > I need to recreate the physica

Re: [Bacula-users] restore raid array

[Kern Sibbald]

  
  
Hello,

Bacula has no way to reconstruct the data in an original backup
Volume.  If you lose the Volume, it is gone.  About the only
mitigating factors after you delete the original volumes are: as is
your case, switch to using the Copy volume in place of the original
Backup volumes;  begin re-creating new volumes (obviously the
history for prior backups will not be available in those volumes).  


There is one other possibility that might work.  After deleting the
original Volumes, your Copy Volumes will be promoted to being the
Backup Volumes.  You could possibly then do a sort of reverse Copy
of your Copy volumes (now promoted to Backup) to your on-site
location.  Then by some SQL magic, you might be able to swap your
Backup volumes and Copy volumes so that you will be back to the
original configuration, except that your original Backups will be
reconstructed from copies of your Copy volumes.  To the best of my
knowledge no one has ever done this, and without a lot of technical
knowledge of the catalog formats, it is not possible.

I have never been fully happy with how Copy volumes are handled, or
more precisely the lack of control the user has from bconsole to
manipulate Copy volumes.  Perhaps some additional future code could
make situations like yours easier to manage.

Perhaps someone else has a good idea to solve this problem.

Best regards,
Kern



On 12/11/18 6:13 PM, Jerry Lowry wrote:


  
  
Josh,
Yes, I understand how the copy jobs works when the original
  job is deleted.  What I need to do is rebuild the client
  directories with the backup database using the latest offsite
  backup, if possible.  I don't want to restore the backup for
  the client, I want to rebuild the data in the directories so
  that a restore can be done from there.  Hope that makes
  since.  I know that I can use the offsite backups to restore
  the client data. I want to know if I can rebuild what would be
  the initial backup of the volume.


thanks,
jerry

  
  
  
On Tue, Dec 11, 2018 at 6:18 AM Josh Fisher 
  wrote:


  


On
  12/11/2018 1:09 AM, Jerry Lowry wrote:


  
Well,  The raid was (8) 6TB disks attached to an
  ATTO Tech raid controller in a supermicro cabinet. It
  was setup as a raid-5 disk array.  The ATTO support
  group know how it was configured as they have been
  helping me since Thursday.  The raid setup is not the
  problem, that can be rebuilt to duplicate the on disk
  file structure. Bacula was using this raid array as
  storage for different clients in my network.  Each
  client had a directory on the array and with in each
  directory there were anywhere from 3 bacula volumes to
  8 volumes. Each volume held any where from 250 GB to
  320 GB. Two of the clients would have an offsite
  backup done each week. A Bacula copy job would run
  each week and copy that weeks backups on to a hot swap
  raid disk running on the same system. The system was
  the Director and Storage director combined.
What I want to do is to restore the daily volumes
  into the new raid array from the offsite disks from
  the most recent offsite backups. Will any of the
  bacula utilities enable me to do this?
  



No special tools are needed. If the original volume files
  no longer exist, then the volumes (and their jobs) can be
  deleted from the catalog using 'delete volume'. When
  bacula finds a Copy of a job when a Job is deleted from
  the catalog, then it will automatically promote the Copy
  as the real backup for that job so that subsequent
  restores use the promoted copy rather than the original.
  The Copy literally replaces the original and the original
  ceases to exist. At that point, a normal restore will
  automatically use those promoted volumes. You should
  ensure that those promoted volumes are marked as Used so
  that no jobs will attempt to write to them.  

If auto-labeling is being used, then jobs should create
  new volumes as needed when they run. If not, then you will
  manually create new empty volume files in 

Re: [Bacula-users] restore raid array

[Jerry Lowry]

Josh,
Yes, I understand how the copy jobs works when the original job is
deleted.  What I need to do is rebuild the client directories with the
backup database using the latest offsite backup, if possible.  I don't want
to restore the backup for the client, I want to rebuild the data in the
directories so that a restore can be done from there.  Hope that makes
since.  I know that I can use the offsite backups to restore the client
data. I want to know if I can rebuild what would be the initial backup of
the volume.

thanks,
jerry

On Tue, Dec 11, 2018 at 6:18 AM Josh Fisher  wrote:

>
> On 12/11/2018 1:09 AM, Jerry Lowry wrote:
>
> Well,  The raid was (8) 6TB disks attached to an ATTO Tech raid controller
> in a supermicro cabinet. It was setup as a raid-5 disk array.  The ATTO
> support group know how it was configured as they have been helping me since
> Thursday.  The raid setup is not the problem, that can be rebuilt to
> duplicate the on disk file structure. Bacula was using this raid array as
> storage for different clients in my network.  Each client had a directory
> on the array and with in each directory there were anywhere from 3 bacula
> volumes to 8 volumes. Each volume held any where from 250 GB to 320 GB. Two
> of the clients would have an offsite backup done each week. A Bacula copy
> job would run each week and copy that weeks backups on to a hot swap raid
> disk running on the same system. The system was the Director and Storage
> director combined.
> What I want to do is to restore the daily volumes into the new raid array
> from the offsite disks from the most recent offsite backups. Will any of
> the bacula utilities enable me to do this?
>
>
> No special tools are needed. If the original volume files no longer exist,
> then the volumes (and their jobs) can be deleted from the catalog using
> 'delete volume'. When bacula finds a Copy of a job when a Job is deleted
> from the catalog, then it will automatically promote the Copy as the real
> backup for that job so that subsequent restores use the promoted copy
> rather than the original. The Copy literally replaces the original and the
> original ceases to exist. At that point, a normal restore will
> automatically use those promoted volumes. You should ensure that those
> promoted volumes are marked as Used so that no jobs will attempt to write
> to them.
>
> If auto-labeling is being used, then jobs should create new volumes as
> needed when they run. If not, then you will manually create new empty
> volume files in the client directories and label them using the Label
> command from bconsole.
>
>
>
> Thanks,
> jerry
>
> On Mon, Dec 10, 2018 at 6:01 PM Phil Stracchino 
> wrote:
>
>> On 12/10/18 7:44 PM, Jerry Lowry wrote:
>> > Hi,
>> > Last thursday I was adding a disk to the bacula raid array when the
>> > system decide to fail. When it rebooted my raid array was gone. This
>> > raid was where all of my daily backups were held, which is where I do my
>> > offsite backups from.  The database is fine, catalog is in working
>> > order.  I rebuilt the server and per Raid Support I have all new disks.
>> >
>> > I need to recreate the physical backup volumes for each of the clients
>> > back on the raid array. I have looked at the utility document. Is bcopy
>> > what I need to use?  I want recreate for example the file structure on
>> > disk call /engineering/tools with the volumes "tool-3, tools-4,
>> tools5...".
>>
>>
>> In ordert to be able to answer this question, anyone here would need to
>> know a lot more about how your RAID was set up.  But you're probably a
>> lot better off asking for help from whoever made the tools you used to
>> build it, or community forums for them.  They'll need to know what you
>> built it with too.
>>
>>
>>
>> --
>>   Phil Stracchino
>>   Babylon Communications
>>   ph...@caerllewys.net
>>   p...@co.ordinate.org
>>   Landline: +1.603.293.8485
>>   Mobile:   +1.603.998.6958
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
>
> ___
> Bacula-users mailing 
> listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore raid array

[Josh Fisher]

On 12/11/2018 1:09 AM, Jerry Lowry wrote:
Well,  The raid was (8) 6TB disks attached to an ATTO Tech raid 
controller in a supermicro cabinet. It was setup as a raid-5 disk 
array.  The ATTO support group know how it was configured as they have 
been helping me since Thursday.  The raid setup is not the problem, 
that can be rebuilt to duplicate the on disk file structure. Bacula 
was using this raid array as storage for different clients in my 
network. Each client had a directory on the array and with in each 
directory there were anywhere from 3 bacula volumes to 8 volumes. Each 
volume held any where from 250 GB to 320 GB. Two of the clients would 
have an offsite backup done each week. A Bacula copy job would run 
each week and copy that weeks backups on to a hot swap raid disk 
running on the same system. The system was the Director and Storage 
director combined.
What I want to do is to restore the daily volumes into the new raid 
array from the offsite disks from the most recent offsite backups. 
Will any of the bacula utilities enable me to do this?



No special tools are needed. If the original volume files no longer 
exist, then the volumes (and their jobs) can be deleted from the catalog 
using 'delete volume'. When bacula finds a Copy of a job when a Job is 
deleted from the catalog, then it will automatically promote the Copy as 
the real backup for that job so that subsequent restores use the 
promoted copy rather than the original. The Copy literally replaces the 
original and the original ceases to exist. At that point, a normal 
restore will automatically use those promoted volumes. You should ensure 
that those promoted volumes are marked as Used so that no jobs will 
attempt to write to them.


If auto-labeling is being used, then jobs should create new volumes as 
needed when they run. If not, then you will manually create new empty 
volume files in the client directories and label them using the Label 
command from bconsole.





Thanks,
jerry

On Mon, Dec 10, 2018 at 6:01 PM Phil Stracchino > wrote:


On 12/10/18 7:44 PM, Jerry Lowry wrote:
> Hi,
> Last thursday I was adding a disk to the bacula raid array when the
> system decide to fail. When it rebooted my raid array was gone. This
> raid was where all of my daily backups were held, which is where
I do my
> offsite backups from.  The database is fine, catalog is in working
> order.  I rebuilt the server and per Raid Support I have all new
disks.
>
> I need to recreate the physical backup volumes for each of the
clients
> back on the raid array. I have looked at the utility document.
Is bcopy
> what I need to use?  I want recreate for example the file
structure on
> disk call /engineering/tools with the volumes "tool-3, tools-4,
tools5...".


In ordert to be able to answer this question, anyone here would
need to
know a lot more about how your RAID was set up.  But you're probably a
lot better off asking for help from whoever made the tools you used to
build it, or community forums for them.  They'll need to know what you
built it with too.



-- 
  Phil Stracchino

  Babylon Communications
ph...@caerllewys.net 
p...@co.ordinate.org 
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore raid array

[Radosław Korzeniewski]

Hello,

wt., 11 gru 2018 o 07:11 Jerry Lowry  napisał(a):

> (...)
> What I want to do is to restore the daily volumes into the new raid array
> from the offsite disks from the most recent offsite backups. Will any of
> the bacula utilities enable me to do this?
>

How did you make your "offsite backups"? As the possibilities are unlimited
there are almost unlimited answers to your question.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore raid array

[Jerry Lowry]

Well,  The raid was (8) 6TB disks attached to an ATTO Tech raid controller
in a supermicro cabinet. It was setup as a raid-5 disk array.  The ATTO
support group know how it was configured as they have been helping me since
Thursday.  The raid setup is not the problem, that can be rebuilt to
duplicate the on disk file structure. Bacula was using this raid array as
storage for different clients in my network.  Each client had a directory
on the array and with in each directory there were anywhere from 3 bacula
volumes to 8 volumes. Each volume held any where from 250 GB to 320 GB. Two
of the clients would have an offsite backup done each week. A Bacula copy
job would run each week and copy that weeks backups on to a hot swap raid
disk running on the same system. The system was the Director and Storage
director combined.
What I want to do is to restore the daily volumes into the new raid array
from the offsite disks from the most recent offsite backups. Will any of
the bacula utilities enable me to do this?

Thanks,
jerry

On Mon, Dec 10, 2018 at 6:01 PM Phil Stracchino 
wrote:

> On 12/10/18 7:44 PM, Jerry Lowry wrote:
> > Hi,
> > Last thursday I was adding a disk to the bacula raid array when the
> > system decide to fail. When it rebooted my raid array was gone. This
> > raid was where all of my daily backups were held, which is where I do my
> > offsite backups from.  The database is fine, catalog is in working
> > order.  I rebuilt the server and per Raid Support I have all new disks.
> >
> > I need to recreate the physical backup volumes for each of the clients
> > back on the raid array. I have looked at the utility document. Is bcopy
> > what I need to use?  I want recreate for example the file structure on
> > disk call /engineering/tools with the volumes "tool-3, tools-4,
> tools5...".
>
>
> In ordert to be able to answer this question, anyone here would need to
> know a lot more about how your RAID was set up.  But you're probably a
> lot better off asking for help from whoever made the tools you used to
> build it, or community forums for them.  They'll need to know what you
> built it with too.
>
>
>
> --
>   Phil Stracchino
>   Babylon Communications
>   ph...@caerllewys.net
>   p...@co.ordinate.org
>   Landline: +1.603.293.8485
>   Mobile:   +1.603.998.6958
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] restore raid array

[Phil Stracchino]

On 12/10/18 7:44 PM, Jerry Lowry wrote:
> Hi,
> Last thursday I was adding a disk to the bacula raid array when the
> system decide to fail. When it rebooted my raid array was gone. This
> raid was where all of my daily backups were held, which is where I do my
> offsite backups from.  The database is fine, catalog is in working
> order.  I rebuilt the server and per Raid Support I have all new disks.
> 
> I need to recreate the physical backup volumes for each of the clients
> back on the raid array. I have looked at the utility document. Is bcopy
> what I need to use?  I want recreate for example the file structure on
> disk call /engineering/tools with the volumes "tool-3, tools-4, tools5...".


In ordert to be able to answer this question, anyone here would need to
know a lot more about how your RAID was set up.  But you're probably a
lot better off asking for help from whoever made the tools you used to
build it, or community forums for them.  They'll need to know what you
built it with too.



-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] restore raid array

[Jerry Lowry]

Hi,
Last thursday I was adding a disk to the bacula raid array when the system
decide to fail. When it rebooted my raid array was gone. This raid was
where all of my daily backups were held, which is where I do my offsite
backups from.  The database is fine, catalog is in working order.  I
rebuilt the server and per Raid Support I have all new disks.

I need to recreate the physical backup volumes for each of the clients back
on the raid array. I have looked at the utility document. Is bcopy what I
need to use?  I want recreate for example the file structure on disk call
/engineering/tools with the volumes "tool-3, tools-4, tools5...".

thanks,

jerry
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users