Hello, Below, you will find a few notes on my ideas ...
On Saturday 01 September 2007 13:55, James Harper wrote: > Here's some thoughts on the way I'd like to see a plugin API implemented > in the bacula file daemon, based on the work I've done on the Exchange > Agent for Bacula. > > Agent loading a 'bacula library', or bacula loading an 'agent library'? I think the most logical solution is a "plugin". > > As long as the agent and bacula both support the same version of the API > (and it would be good if they could negotiate a version, although this > may be a heap more work), it would be nice to use dynamically loadable > libraries rather than something compiled in. I think my preference would > be for the bacula filed to load a .dll/.so based on a configuration > option. So if 'plugin exchange.dll' was specified, bacula would use that > rather than its own internal backup/restore code. Or maybe the normal > filesystem backup itself would be a plugin? The most likely mode of working would be to add a directive that defines a plugin directory, from which Bacula would load all plugins. This seems to be rather standard. > > What bacula would provide? > > Bacula would provide all of the network code, and make calls to the > agent as required (eg 'prepare this file to start restoring'), and > provide some helper functions (eg 'is this file in the fileset?') It > would be responsible for handling compression (it would take a FILE_DATA > stream and turn it into a GZIP_DATA (or other) stream as directed by the > options), and would also add on any signature/digest information > required, and would also provide data encryption if required. > > What the agent would provide? > > The way I've set up the Exchange agent, there is a base class which > provides roughly what the bacula filed would, and then the actual > Exchange code, which does the talking to exchange. > > For a backup, the following is roughly what I see that needs to happen: > > 1. bacula makes a 'prepare backup' call with the list of fileset > options, and the type of backup being done (full, differential, etc) > 2. bacula makes a 'next file' call to get a handle on the next file to > be backed up, and a list of the stream types to be backed up > 3. agent finds the next file to be backed up, making calls to a 'is this > file in the fileset' function provided by bacula, and when it finds > something, returns that file to bacula. > 4. bacula makes a 'start backup of stream x' call to initiate the backup > of a given stream type > 5. bacula makes a 'read data' call to read the data for the current > stream > 6. loop to 4 until end of file > 7. loop to 3 until all streams are done > 8. loop to 2 until all files are done This is one part that I have not totally worked out. As you describe it, it is not really complete in that Bacula needs to have a core backup functionality that can be *extended* or *overridden* by plugins. Thus we need some way in the beginning that the plugin can more or less register to receive control for certain files, or it could even receive control for all files and either decide to handle them or not. IMO the plugin shouldn't really have to deal with a lot of options or the FileSets -- that is base Bacula responsibility. One of the big problems I have not yet figured out is: if you backup a particular file with a plugin, then it should really be restored by the same plugin. How do you assure that? What do you do if the plugin is not there? Do you save the name of the plugin on the Volume? ... > > Very early on I did some work in trying to actually implement this sort > of functionality into the bacula filed, and while it wasn't that hard to > do a basic 'next file' and 'read data' call, most of that stuff is too > tightly integrated into the bacula filed itself for the rest of it to be > particularly easy. I don't think this will be a big problem. Mostly it is a matter (I think) of modifying the bfile read and the write routines. > > One of the things I haven't gotten my head around yet is the difference > between how an incremental or differential backup is handled in the > bacula filed and Exchange. If I understand it correctly, the director > tells the filed when the last backup was done, and the filed gives it > any files that have changed since then (with obvious problems for > deleted or renamed files). Exchange keeps it's own internal numbers on > when the backup was last done, and just gives you all the logfiles since > the last full backup (differential) or all the logfiles since the last > incremental. I'm pretty sure it will all still work just fine, but maybe > the director might have some misinformation about exactly what the > incremental/differential backup represents... > > On a completely unrelated note... I wonder if there is a way for any > filesystems to keep some sort of journal (like the journal in journaling > filesystems but with different requirements) around long enough to be > able to track deleted or moved files in a way that would be useful to > bacula to do accurate incremental/differential backups... > The problem with such things is that if there are several backup programs, the scheme will fail. If you want real security, Bacula has to do this itself. Regards, Kern ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
