Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/03/2010 06:30 PM, Phil Stracchino wrote: On 09/03/10 03:36, Bruno Friedmann wrote: nothing is wrong, but as I've said, the challenge here is not to get the old Pool, Device etc in the new server. Bscan work nicely in this case, no doubt on that. Test prove that. But we have to re-import them in new defined pool, perharps also new name. Perharps with some details it would be better. I've old data like this for entA client was serverA-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 31,536,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ and for entC exactly the same pool, same media name client was serverB-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 8,644,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ Now I have to consolidate those 2 medias in new structure, I'm pretty sure I can't have two media with same name in same pool Or can I without difficulties to select the correct one during restore operation. Pool_serverA_Week WEEK-04 Pool_serverB_Week WEEK-04 Same for months (total of 26), and years total 12 media for the last 5 years Month 6 is always made twice, same for Month 12. Now time will pass, and each time a week or a month is done in the new pool/media scheme, we can delete the old record. So after a certain amount of time, all old media will be deleted and pools can be removed. OK, now I have a better understanding of the problem. I really don't grasp the details of your configuration, but it appears to me that what you need to do is to TEMPORARILY create Storage devices on the new system with the same names and media types as those on the old server, controlled the same Storage daemon that controls the new system's devices, then bscan the volumes in, THEN you can set up a migration job to move all the jobs from the old media onto volumes in your new server's pools. Once you've done that, you should be able to remove the temporary device and pool. Yes that's a way to do it. I also made another test yesterday and finally found something that could do it. Prepare (by label) the new medium in the new pools, and after that use bcopy from old medium to new one. (So I don't need to recreate all old stuff config). And do a bscan on the new medium to get record in database. In this case everything goes at the right place, with the right name. and operators can work normally. -- Bruno Friedmann br...@ioda-net.ch Ioda-Net Sàrl www.ioda-net.ch openSUSE Member User www.ioda.net/r/osu Blog www.ioda.net/r/blog GPG KEY : D5C9B751C4653227 -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/04/10 04:59, Bruno Friedmann wrote: Yes that's a way to do it. I also made another test yesterday and finally found something that could do it. Prepare (by label) the new medium in the new pools, and after that use bcopy from old medium to new one. (So I don't need to recreate all old stuff config). And do a bscan on the new medium to get record in database. Good solution. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/02/2010 10:20 PM, Phil Stracchino wrote: On 09/02/10 15:21, Bruno Friedmann wrote: On 09/02/2010 08:33 PM, Phil Stracchino wrote: I meant to say, bscan will do part of what you want. Depending on the media, you should be able to either copy the disk volumes to the new datacenter, or send tapes and install a drive that can read them if necessary; at that point you should be able to bscan their metadata into your new Catalog, at which point you should be set. I theory, I've think of it, but in reality, as it's not tape it seems I have to rebuild all the devices and the file system where they have been + give bscan the old bacula-sd.conf otherwise : nothing work ... I don't see why that should be necessary. A disk volume is just a file. It's not tied to any specific filesystem. You should be able to put it anywhere you want, and make the sd.conf file that you feed to bscan match that location and a valid disk Pool. If you can't do that and have it work, something is wrong. Dear Phil, nothing is wrong, but as I've said, the challenge here is not to get the old Pool, Device etc in the new server. Bscan work nicely in this case, no doubt on that. Test prove that. But we have to re-import them in new defined pool, perharps also new name. Perharps with some details it would be better. I've old data like this for entA client was serverA-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 31,536,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ and for entC exactly the same pool, same media name client was serverB-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 8,644,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ Now I have to consolidate those 2 medias in new structure, I'm pretty sure I can't have two media with same name in same pool Or can I without difficulties to select the correct one during restore operation. Pool_serverA_Week WEEK-04 Pool_serverB_Week WEEK-04 Same for months (total of 26), and years total 12 media for the last 5 years Month 6 is always made twice, same for Month 12. Now time will pass, and each time a week or a month is done in the new pool/media scheme, we can delete the old record. So after a certain amount of time, all old media will be deleted and pools can be removed. The trouble are located here : If I didn't put all old config during a restore test After having choose some files to restore Unable to find Storage resource for MediaType File_FSWEEK, needed by the Jobs you selected. I can force the new storage where I put the old media. but 03-Sep 09:24 orville-dir JobId 7: Start Restore Job Restore_Dumbo-ACL.2010-09-03_09.24.53_08 *m 03-Sep 09:24 orville-dir JobId 7: Using Device SDev_TRANSFERT 03-Sep 09:24 orville-sd JobId 7: acquire.c:117 Changing read device. Want Media Type=File_FSWEEK have=TRANSFERT device=SDev_TRANSFERT (/media/bacula.storage/TRANSFERT) 03-Sep 09:24 orville-fd JobId 7: Fatal error: job.c:2031 Bad response to Read Data command. Wanted 3000 OK data , got 3000 error 03-Sep 09:24 orville-sd JobId 7: Fatal error: acquire.c:166 No suitable device found to read Volume FS_SAMEDI-0004 03-Sep 09:24 orville-dir JobId 7: Error: Bacula orville-dir 5.0.3 (04Aug10): 03-Sep-2010 09:24:55 So this clearly explain, my first asking for advise. I need not only reload content of media inside db, but need to drive it to specific Pool, and also change the Media type concerned. Can I do that, in a pseudo automatic way, or will I finally have to trick manually the database ? ps : why we have so many file device type is due to the concurrency jobs. (One store, One specific Device, One specific Device Type ) -- Bruno Friedmann br...@ioda-net.ch
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/03/10 03:36, Bruno Friedmann wrote: nothing is wrong, but as I've said, the challenge here is not to get the old Pool, Device etc in the new server. Bscan work nicely in this case, no doubt on that. Test prove that. But we have to re-import them in new defined pool, perharps also new name. Perharps with some details it would be better. I've old data like this for entA client was serverA-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 31,536,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ and for entC exactly the same pool, same media name client was serverB-fd Pool: Pool_FileWeekly +-++---+-+--+--+--+-+--+---+-+-+ | mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | +-++---+-+--+--+--+-+--+---+-+-+ | 4 | FS_SAMEDI-0004 | Archive | 1 |0 |0 | 8,644,000 | 0 |0 | 0 | File_FSWEEK | | +-++---+-+--+--+--+-+--+---+-+-+ Now I have to consolidate those 2 medias in new structure, I'm pretty sure I can't have two media with same name in same pool Or can I without difficulties to select the correct one during restore operation. Pool_serverA_Week WEEK-04 Pool_serverB_Week WEEK-04 Same for months (total of 26), and years total 12 media for the last 5 years Month 6 is always made twice, same for Month 12. Now time will pass, and each time a week or a month is done in the new pool/media scheme, we can delete the old record. So after a certain amount of time, all old media will be deleted and pools can be removed. OK, now I have a better understanding of the problem. I really don't grasp the details of your configuration, but it appears to me that what you need to do is to TEMPORARILY create Storage devices on the new system with the same names and media types as those on the old server, controlled the same Storage daemon that controls the new system's devices, then bscan the volumes in, THEN you can set up a migration job to move all the jobs from the old media onto volumes in your new server's pools. Once you've done that, you should be able to remove the temporary device and pool. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Advise for a bit complicated migration/consolidation
Hi all, I need some advises to resolve a complex case. Global situation : 2 enterprises each running bacula entA : run 1.38 with sqlite DB entB : run 2.28 with mysql 5.0 Now they merge into a new entity. entC I discuss with the new DataManager about the new needs about backups We prepare devices (mainly files), pools etc. Now bacula is reading to run : bacula 5.0.3 + postgresql (the new db standard in entC ) The tests are conclusive. Bacula is on one server which has it's own pg db and all the needed disk place. Master server contain now all consolidated data from entA and entB. Now the dilemma : We need to have the ability to restore (in a qas most as possible quite simple way) any data contained in the old volumes. What would be the better approach to that. It's frequent users ask for restoring data (2 to 6 times per week) Is there a magic way to read old media and pipe data in them to a new media which will be stored on a new location, new name, and have it's file path recorded in DB ? Remember the new bacula server doesn't know anything about the old media, pool, device names, so usual case like dump db - import on new, or bscan old media seems not possible (Of course I've kept the configuration and can retrieve them) Any inputs are welcome ! -- Bruno Friedmann br...@ioda-net.ch Ioda-Net Sàrl www.ioda-net.ch openSUSE Member User www.ioda.net/r/osu Blog www.ioda.net/r/blog fsfe fellowship www.fsfe.org tigerfoot on irc GPG KEY : D5C9B751C4653227 -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/02/10 13:56, Bruno Friedmann wrote: Is there a magic way to read old media and pipe data in them to a new media which will be stored on a new location, new name, and have it's file path recorded in DB ? Have you looked at bscan? -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/02/10 14:17, Phil Stracchino wrote: On 09/02/10 13:56, Bruno Friedmann wrote: Is there a magic way to read old media and pipe data in them to a new media which will be stored on a new location, new name, and have it's file path recorded in DB ? Have you looked at bscan? Er, I sent that rather prematurely. I meant to say, bscan will do part of what you want. Depending on the media, you should be able to either copy the disk volumes to the new datacenter, or send tapes and install a drive that can read them if necessary; at that point you should be able to bscan their metadata into your new Catalog, at which point you should be set. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/02/2010 08:33 PM, Phil Stracchino wrote: On 09/02/10 14:17, Phil Stracchino wrote: On 09/02/10 13:56, Bruno Friedmann wrote: Is there a magic way to read old media and pipe data in them to a new media which will be stored on a new location, new name, and have it's file path recorded in DB ? Have you looked at bscan? Er, I sent that rather prematurely. I meant to say, bscan will do part of what you want. Depending on the media, you should be able to either copy the disk volumes to the new datacenter, or send tapes and install a drive that can read them if necessary; at that point you should be able to bscan their metadata into your new Catalog, at which point you should be set. I theory, I've think of it, but in reality, as it's not tape it seems I have to rebuild all the devices and the file system where they have been + give bscan the old bacula-sd.conf otherwise : nothing work ... That's why I've said, not so simple todo. -- Bruno Friedmann br...@ioda-net.ch Ioda-Net Sàrl www.ioda-net.ch openSUSE Member User www.ioda.net/r/osu Blog www.ioda.net/r/blog fsfe fellowship www.fsfe.org (bruno.friedmann (at) fsfe.org ) tigerfoot on irc GPG KEY : D5C9B751C4653227 -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Advise for a bit complicated migration/consolidation
On 09/02/10 15:21, Bruno Friedmann wrote: On 09/02/2010 08:33 PM, Phil Stracchino wrote: I meant to say, bscan will do part of what you want. Depending on the media, you should be able to either copy the disk volumes to the new datacenter, or send tapes and install a drive that can read them if necessary; at that point you should be able to bscan their metadata into your new Catalog, at which point you should be set. I theory, I've think of it, but in reality, as it's not tape it seems I have to rebuild all the devices and the file system where they have been + give bscan the old bacula-sd.conf otherwise : nothing work ... I don't see why that should be necessary. A disk volume is just a file. It's not tied to any specific filesystem. You should be able to put it anywhere you want, and make the sd.conf file that you feed to bscan match that location and a valid disk Pool. If you can't do that and have it work, something is wrong. -- Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355 ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org Renaissance Man, Unix ronin, Perl hacker, Free Stater It's not the years, it's the mileage. -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
