Thanks Arno, you always seem to have very helpful insights. These features would definitely represent a step up in complexity, and functionality.
Honestly, I'd love the restore-from-one-of-several-copies feature even if I had to choose which volume to use for the restore from a list of candidate presented on a menu, and wait for the more complex prioritization code to come out as a separate module/code block/whatever... I'm guessing there are shops that fit the scenario of copying between SDs, but I'm guessing the majority would be pleased to be able to do it within a single SD and, also, wait for the multi-SD functionality as a separate project. Again, thanks for your many contributions to the list; I've benefited from your knowledge several times. If I had the skills or the $$ I'd be part of the solution in a New York minute (instantly). On Wed, 2007-02-07 at 10:46 +0100, Arno Lehmann wrote: > Hi, > > I'm following your discussion for a while now... > > On 2/7/2007 3:54 AM, Don MacArthur wrote: > > I think some of the code contains what might be a good solution. > > What you want seems to be he ability to copy volumes,possibly to another > SD, and keep complete catalog inform- Deciding which volumes / jobs to > restore from: A concept of a 'volume > cost' representing the efort needed to access volumes wil be helpful. > For example, define the cost for disk based volumes to be lower than > that of tapes stored locally, and the cost of off-site tapes would be > highes.ation about both copies. > > That part is probably rather simple to code (says another non-programmer > :-) because a big part of what is needed already exists with the > migration jobs. > > > Right now, when a scheduled backup issues a mount request for a tape, it > > doesn't matter what tape I load as long as it meets the requirements to > > be used. If it asks me for volume123 from pool xyz, and I load > > volume456 from pool xyz the job will run (again, assuming the status of > > the tape is correct and there aren't any other problems like file number > > mismatch). > > > > I'm haven't coded for ... years, but I wonder if the same logic that > > allows the director or storage daemon to determine that it didn't get > > what it asked for (volume123), but did get what it wanted (a "qualified" > > volume), could be applied to this situation. That is, read the label, > > query for the file and version, and if the mounted volume has the file, > > rock and roll. If not, explain it to the operator and iterate the mount > > request. > > > > Yes? > > As far as I understand, the restore part is what will be most difficult > to solve; currently, Bacula is not able to choose from different copies > of a job. This would have to be thought through first. > > The user interface to such a function would be quite straightforward as > you outline, but the key problem lies behind the scenes. > > I assume that job copies will be implemented some day, but we'd need a > capable programmer for that... There is a feature request dealing with > copies already, by the way. > > Many details need to be worked out before implementation can begin, I > assume. Three main areas might be: > - Deciding which volumes / jobs to restore from: A concept of a 'volume > cost' representing the efort needed to access volumes wil be helpful. > For example, define the cost for disk based volumes to be lower than > that of tapes stored locally, and the cost of off-site tapes would be > highes. > First, the DIR has to learn that there can be multiple copies of one > job, though - this is not something it understands today, I think. > - Moving data without expiring the catalog information about the source > job. This should be rather simple. > - Allowing the transfer of data from one SD to another one. Today, IIRC, > migration only works inside one SD. That is ok for migration from disk > to tape, but fore real copies one will want the ability to create > off-site volumes directly, i.e. copy from a local disk volume to a > remote tape. > > I imagine that the first and the last item will require most of the > implemetation time, and I'm qute sure that I overlooked many interesting > things ;-) > > Anyway, that discussion probably belongs to the -devel list. > > Today, if you want copies of jobs or volumes, you have only two choices: > Run the jobs twice, to different storage devices, and be prepared for > differences between jobs and manual decision which one to use for > restores. Or copy the volumes outside of Baculas scope, using simple > file copies for disk volumes, tape-to-tape dumps for tape volumes, or > use bcopy. In any case, you'll have to manage the copies manually, too, > but you will have identical copies. > > Anyway, this mainly repeats what has been written in this thread before. > What I find most interesting, though: Is one you intersted enough in the > job copy feature to sponsor it or to implement it himself? I'm quite > sure you'll get support from Kern and the other developers, and I'm also > quite sure that many users of Bacula will be very grateful... > > Arno > > > > > On Tue, 2007-02-06 at 18:31 -0800, Tauren Mills wrote: > > > >>This sounds like a reasonable approach. If I request to restore a > >>particular object and the exact object version exists on multiple > >>volumes, it really doesn't matter which volume it is restored from. > >>As long as that object gets restored, I will be happy. Of course, it > >>could make my life easier if bacula would automatically choose a > >>volume that was mounted, such as a disk volume over a tape volume. Or > >>if I'm doing a manual restore, it could ask me interactively which > >>volume to restore from, suggesting one that is mounted if available. > >> > >>The bottom line is that it would have information about both copies of > >>the same object, and then logic could be added to choose which one to > >>restore. That logic could be very simple, such as "restore from the > >>first volume in the list that contains the object" or more > >>sophisticated such as "restore from a volume that contains the object, > >>give preference to disk volumes over tape volumes". > >> > >>Of course, I haven't looked at any of the bacula code, so these are > >>just perceptions from the outside as a user. Those with more > >>experience with the internals may see issues that I do not at this > >>time. > >> > >>On 2/6/07, Don MacArthur <[EMAIL PROTECTED]> wrote: > >> > >>>I was one of the more recent posters on this topic. > >>> > >>>The tools I have used in the past used whichever volume was loaded as > >>>long as it was one of the ones it asked for. So, if the original backup > >>>was on volume123, and copied to volume456, a list of *all* the volumes > >>>that contain the version of the object would be displayed, the label > >>>read, and any volume (assuming it is on the list displayed) used. > >>> > >>>FWIW. > >>> > >>>On Tue, 2007-02-06 at 20:01 -0500, Ryan Novosielski wrote: > >>> > >>>>-----BEGIN PGP SIGNED MESSAGE----- > >>>>Hash: SHA1 > >>>> > >>>>Tauren Mills wrote: > >>>> > >>>>>>I agree. It would be nice if there was a 'clone volume' command that > >>>>>>would create a copy of the tape (or backup file) and create new records > >>>>>>for the associated catalog data with pointers to the second volume. I > >>>>>>used to work with a Legato system and I'm pretty sure that it had that > >>>>>>feature. Or maybe Bacula does have that and I just haven't found it > >>>>>>yet. > >>>>> > >>>>>That's exactly why I was posting! I would think that a copy or clone > >>>>>command would be a built in feature and that I just hadn't found it. > >>>>>I'll have to figure out where to post feature requests I guess. > >>>> > >>>>Searched the list archives -- this has been discussed before. The > >>>>condensed version of the problem is exactly what has been described > >>>>prior: how would Bacula know from which volume you wish to restore? I > >>>>don't have a good answer for that. I suspect Kern has thought about it a > >>>>lot, but last I checked he wasn't sure how this would be done either. A > >>>>lot of the work is done as a result of migration being possible now, but > >>>>there are still pieces left. > >>>> > >>>>Also, check the current projects list; I believe this is on it > >>>>(therefore, no reason for a feature request). > >>>> > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job > > easier. > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Bacula-users mailing list > > Bacula-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bacula-users > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
