[Bacula-users] question about backup levels
I'm a new Bacula user, having just set up a system for backing up the machines in my house (the storage server is a Raspberry Pi with a 4TB external disk drive attached to it). My question concerns backup levels. This afternoon, I think while learning to use "bat", I accidentally scheduled two consecutive incremental backups on my desktop. The first one ran as expected, but the second one looks like it basically turned into a full backup, even though it still appears in the job listing as an incremental: |13 | anathem | 2013-08-02 20:37:18 | B| F | 800,022 | 137,247,853,895 | T | |43 | anathem | 2013-08-07 14:32:19 | B| I | 27,916 | 2,345,766,355 | T | |44 | anathem | 2013-08-07 14:41:45 | B| I | 773,678 | 136,134,397,714 | T | I was under the impression that an incremental would only back up files modified since the last backup of any kind, so what made the second incremental act like a full? --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
On Thu, 2013-08-08 at 11:08 +0100, Martin Simmons wrote: > >>>>> On Wed, 07 Aug 2013 19:11:33 -0600, Greg Woods said: > > |13 | anathem | 2013-08-02 20:37:18 | B| F | 800,022 > > | 137,247,853,895 | T | > > |43 | anathem | 2013-08-07 14:32:19 | B| I | 27,916 > > | 2,345,766,355 | T | > > |44 | anathem | 2013-08-07 14:41:45 | B| I | 773,678 > > | 136,134,397,714 | T | > > > > I was under the impression that an incremental would only back up files > > modified since the last backup of any kind, so what made the second > > incremental act like a full? > > Look strange to me too -- it would be interesting to see the logs of these > backups. Thanks for taking an interest. This has now become a more-or-less fatal problem for me as all my Incrementals are getting translated into Full. At least one of my servers has 735GB of stuff (music, videos, photos, RPM repositories) that takes over 21 hours to do a full backup on, so it's important to get Incrementals working the way they should (most of that stuff never changes, so once backed up once, I shouldn't have to write it again until the next Full). I have a feeling that this is related to my attempts to modularize the director configuration, which I have just done in the past couple of days. I think the time of that change corresponds with when the problem started, to it's likely related somehow. That is, I have a /etc/bacula/clients subdirectory in which I have a file for each client that defines the File Set and Schedule for each client (which do vary from client to client of course), and a template file for the parts of the configuration that are the same for every client except for the client name. The actual configuration gets put together with shell scripts run out of bacula-dir.conf, so it looks like this: @|"sh -c 'for client in `ls /etc/bacula/clients` ; do sed -e s/CLIENT-NAME/${client}/ /etc/bacula/bacula-client.conf; cat /etc/bacula/clients/${client} ; done'" Here's what's in bacula-client.conf: Job { Name = "CLIENT-NAME" JobDefs = "DefaultJob" Type = Backup Schedule = CLIENT-NAME Client = CLIENT-NAME File Set = CLIENT-NAME } Job { Name = "CLIENT-NAME-Restore" Type = Restore Client=CLIENT-NAME FileSet="CLIENT-NAME" Storage = BKUP Pool = File Messages = Standard Where = /tmp/bacula-restores } Client { Name = "CLIENT-NAME" Address = CLIENT-NAME.gregandeva.net Catalog = "GregAndEva" Password = "bkup$ucks" File Retention = 200 Job Retention = 200 } For completeness, the DefaultJob is defined as: JobDefs { Name = "DefaultJob" Type = Backup Level = Incremental FileSet = "Full Set" Storage = BKUP Messages = Standard Pool = File Priority = 10 Accurate = yes Write Bootstrap = "/var/spool/bacula/%c.bsr" } And a sample client file: FileSet { Name = "anathem" Include { Options { signature = MD5 } File = / File = /xp File = /local File = /games File = /home } # # If you backup the root directory, the following two excluded # files can be useful # Exclude { File = /var/spool/bacula File = /tmp File = /proc File = /tmp File = /.journal File = /.fsck } } Schedule { Name = anathem } Maybe another pair of eyes can spot where I messed this up. (If I run "bacula-dir -t -c /etc/bacula/bacula-dir.conf", it completes successfully. I can't seem to find any way to have it run the "preprocessor" and show me what the actual generated bacula-dir.conf file would look like, although I have manually run the shell command and the output looks good). Thanks, --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
> Password = Well, that was stupid of me. I am now going to have to change my password in all of the &*#()@! bacula-fd.conf files. Boot to the head :-) --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
On Thu, 2013-08-08 at 12:12 -0600, Greg Woods wrote: > On Thu, 2013-08-08 at 11:08 +0100, Martin Simmons wrote: > > >>>>> On Wed, 07 Aug 2013 19:11:33 -0600, Greg Woods said: > > > > |13 | anathem | 2013-08-02 20:37:18 | B| F | 800,022 > > > | 137,247,853,895 | T | > > > |43 | anathem | 2013-08-07 14:32:19 | B| I | 27,916 > > > | 2,345,766,355 | T | > > > |44 | anathem | 2013-08-07 14:41:45 | B| I | 773,678 > > > | 136,134,397,714 | T | > > > I left out one important detail. I can see that my Job specifications are all Full backups by default. The last job in the above list was invoked from bconsole by typing "run Level=Incremental anathem". That results in a job that lists itself as an Incremental but acts like a Full. Similarly, my Schedule stanzas look like: Schedule { Name = "confusion" Run = Level=Incremental on 5-31 at 00:05 } The relevant output from "list jobs" is: |27 | confusion | 2013-08-04 06:16:04 | B| F | 726,854 | 27,626,867,742 | T | |38 | confusion | 2013-08-07 00:05:58 | B| I | 684 | 1,786,383,078 | T | |47 | confusion | 2013-08-08 00:11:09 | B| I | 726,862 | 28,140,663,999 | T | Again, after the first incremental, subsequent incrementals list as incrementals but act like Fulls. --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
On Thu, 2013-08-08 at 14:39 -0400, John Drescher wrote: > Make sure the times of the folders (modified and ctime) also any > attributes are not changing between backups. I suppose it's good to check the obvious and the stupid first, but I don't think that's it. I don't know how to check to see if "attributes" are changing, but it seems unlikely that would be happening on all my desktops and servers. Here's one example: |27 | confusion | 2013-08-04 06:16:04 | B| F | 726,854 | 27,626,867,742 | T | |38 | confusion | 2013-08-07 00:05:58 | B| I | 684 | 1,786,383,078 | T | |47 | confusion | 2013-08-08 00:11:09 | B| I | 726,862 | 28,140,663,999 | T | FileSet { Name = "confusion" Include { Options { signature = MD5 } File = / File = /old } # # If you backup the root directory, the following two excluded # files can be useful # Exclude { File = /var/spool/bacula File = /tmp File = /proc File = /tmp File = /.journal File = /.fsck } } [root@confusion /]# find / -xdev -mtime -2 -print | wc -l 409 [root@confusion /]# find /old -xdev -mtime -2 -print | wc -l 0 [root@confusion /]# find /old -xdev -ctime -2 -print | wc -l 0 [root@confusion /]# find / -xdev -ctime -2 -print | wc -l 719 Some of the ctime and mtime changes are likely for the same files, but in any case, it doesn't look like there should have been more than 1100 files or so on today's incremental, instead there were 726,000. Anything else I can use besides "find" to see what ought to be in an incremental? I'm sure it's either something I've done wrong or something I don't understand about how Bacula is supposed to work. Just for grins, I tried explicitly entering a Job stanza for the incremental backups, like this: Job { Name = "CLIENT-NAME-Incremental" JobDefs = "DefaultJob" Type = Backup Level = Incremental Schedule = CLIENT-NAME Client = CLIENT-NAME File Set = CLIENT-NAME } If after this change, I do "estimate" and select confusion-Incremental, it still says 726,000 files will be backed up. --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
OK, I think I have it figured out, and it's not pretty. I suspect that most of the backups I have already done are useless and I will have to repeat all of the Full backups and start all over again. The answer came when I tried to do a restore. I thought, OK, if I messed up the director configuration, why not restore the version of bacula-dir.conf prior to my most recent set of changes and see what happens if I use that? The problem was that I was unable to find this file on the Full backup. It's simply not there. And here's the problem, in my template client: Client { Name = "CLIENT-NAME" Address = CLIENT-NAME.gregandeva.net Catalog = "GregAndEva" Password = "bkup$ucks" File Retention = 200 Job Retention = 200 } I thought I was specifying retention times of 200 days, but I'll bet I'm not, and in fact, my Full backups have been pruned and everything removed from the Catalog. This is particularly bad because I've already got my backup storage drive about 25% full, and it's full of stuff that is useless because without the catalog entries, it can never be accessed. So that means I'll just have to wipe the backup drive clean, purge all of the current records in the catalog, and redo all the full backups (which literally took days to complete). AUUUGH! Lucy pulled the football away again :-) So what's the easiest way to clear everything out so I can start over? Drop the database and reload it from the bacula.sql script? --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about backup levels
On Thu, 2013-08-08 at 17:36 -0400, John Drescher wrote: > You will have to also clean up your volumes. I'm planning on deleting all the jobs, then purging all the volumes. If my understanding of what a purge does is correct, then the data that is currently written to the volumes (which in my case are just disk files) will not actually be deleted, but the same space will be reused once all of the unused volumes have been filled. --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] modifying FileSet
Last night I modified the FileSet for my main house server. I got the Windows version of Bacula running inside the Windows 7 virtual machine, so that I don't have to copy the entire 12GB virtual disk image on every incremental backup of the server, plus I get the ability to restore individual files. So what I changed in the FileSet was only to add an exclusion for the virtual disk image file. This resulted in the next incremental backup turning into a Full. Is that expected behavior? Does any modification of the FileSet invalidate all the previously-stored catalog entries for that client? --Greg -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Post starting bacula-dir does not succeed error on line 29
On Mon, 2013-08-26 at 13:34 -0700, Benny_pixelweb wrote: > Hi Ana, > > i have set the root "/" also as you can see in the screenshot now its 2.69Gb > backup. > > But that could not be all my data anny suggestions? Each file system has to be declared separately. If all you have in your FileSet resource Include section is "File = /", then you are just going to get the root partition. You need to also have File declarations for any other mounted partitions that you want backed up. You might notice messages in the console that say things like "/home is a separate file system, will not descend into it" or something like that. In that example, if you wanted /home backed up, you would need "File = /home" in your Include section as well. --Greg -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Post starting bacula-dir does not succeed error on line 29
On Wed, 2013-08-28 at 06:24 -0700, Benny_pixelweb wrote: > Also when i run a backup > > there are manny errors now. > > "17: Error: Read error on file > /sys/devices/pci:00/:00:1e.0/:01:04.2/resource1. ERR=Input/output > error > 28-Aug 15:06 bacula-fd JobId 17: Error: Read error on file > /sys/devices/pci:00/:00:1e.0/:01:04.2/resource2. ERR=Input/output > error /sys is not a real file system, it's full of entries into the kernel, so you don't need or want to back up /sys. It's completely recreated every time the system reboots. You can use "df" to see where things are mounted. As a general rule, you don't need/want to back up things that are mounted as tmpfs, or things that are not "mounted" at all (that don't show up in the "df" output). You only want to back up file systems that are mounted on physical devices (/dev/something). Whether a given file system really needs to be backed up depends a lot on what you have stored there; you will have to decide that for yourself based on your own use of the system. For instance, I have one system that has a lot (several TB) of videos stored on it. These take up a huge amount of room and are not that important, so I do not back them up (I do not include those file systems in my FileSet). If the drive they are stored on fails, I know I will lose them, but I accept that. This type of decision, nobody can advise you on. --Greg -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] backing up NTFS file systems
This mostly seems to work, but with one annoying oddity. When I run an incremental backup, it seems to want to back up just about the entire NTFS file system, even though commands like "ls -lt" and "ls -lc" don't show the files or their metadata as having been modified. I expect this isn't an issue with Bacula specifically, but rather an artifact of how Linux deals with M$ file systems. Has anyone else seen this? Have a workaround? Thanks, --Greg -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backing up NTFS file systems
On Sun, 2013-09-08 at 09:32 -0600, compdoc wrote: > But since you mention the commands "ls -lt" and "ls -lc", which are not > windows or dos commands, I'm assuming that your NTFS volumes are attached to > a nix OS? That may have more to do with the problem than being NTFS > volumes. Well, yes, I did say: > I expect this isn't an issue with Bacula specifically, but rather an > artifact of how Linux deals with M$ file systems. I am just wondering what exactly the issue is, and whether or not there is a workaround. This is a home setup. At first, I thought using something like Bacula for my house might be overkill, but it turns out that I have three Linux laptops (work, personal, and wife's) two desktops (mine and wife's) plus two other desktops that act as servers. One of the servers has a Windows 7 VM that is up all the time. All three laptops also have a dual boot Windows 7 system, and two of the laptops also have Windows 7 virtual machines. So that's a total of (I have to count them :-) 7 Linux OS's and 8 Windows OS's to back up. So I used a Raspberry Pi with a 4TB "green" drive (that shuts itself down when idle) as a storage server (oops, make that 8 Linux OS's to back up). Except for the one Windows VM that is up all the time (on which I did install bacula-fd), all the other Windows systems only get used occasionally, as needed, so I would really rather back them up via the Linux OS that gets used all the time. So the dual-boot systems have their file systems mounted under Linux, and I mount the virtual machine file systems also (one has to use an offset to mount the partitions within the virtual disk image). Their files are included in the backups of Linux OS, and can be restored that way. This works well except for the fact that some of the incrementals turn into full backups of the NTFS file systems, and I would like to find out why that happens. I suppose I could work around the problem by installing bacula-fd on 7 more Windows clients, but that's a lot more hassle than just mounting the NTFS file systems under Linux and backing the files up that way. Hopefully this explains what I'm trying to do and why. --Greg -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] USB tape drives?
Pardon me if this is a dumb or frequently-asked question, but I have not found a definitive answer searching the list archives, Google, or the Bacula documentation. My question is whether there is any such thing as a USB tape drive that is known to work with Bacula. When reading about tape drives, I see phrases such as "DLT" and "LTO", but the only USB tape drives I have been able to find available are "DAT 160". I know next to nothing about tape technology, but I'd like to have a tape device so that I can migrate my backups from an attached hard drive to tape, so that I have copies of my backups that are not on a device that is always connected. Thanks, --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] USB tape drives?
On Sat, 2013-09-14 at 14:02 -0600, Greg Woods wrote: > My question is whether there is any such thing as a USB tape drive that > is known to work with Bacula. It's clear from the responses I got that I left out an important detail, since all the responses were telling me why I should use something other than a USB tape drive. This is a cheap home setup. My storage server is a Raspberry Pi. So I do not have SATA bays, SCSI interfaces, or eSATA interfaces available. The only connection for peripherals is USB. My current storage device is a 4TB USB drive, of the "green" type that shuts itself down automatically when inactive. But it is still connected, because I want to be able to fire off an incremental backup for a laptop or desktop whenever I want, without having to fiddle with hardware connections. I want a secondary backup device that can be used for migration and/or copy jobs. For this one, I don't want the media always online, because of course I want to insulate myself against software going nuts, power surges, etc. (yes I have a UPS and those are very good but they are not perfect; ideally, I'd like to copy my full backups to media that I can store offsite). At this point, I can see three options: getting a USB tape drive, using a second hard drive that I manually connect and mount as needed, or using another machine in the house as a secondary storage server (more complicated to set up and maintain) so that other hardware options become available. --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] USB tape drives?
On Mon, 2013-09-16 at 18:12 +0300, Guy wrote: > I use SATA drives and pretend they are tape drives by using vchanger. I will look into doing something like this with one of those USB-to-SATA drive docks that was recommended. I presume I will be able to find some way to pretend that the drives are 2TB capacity "tapes", so that I can label the drives and Bacula will keep track of which archives are on which drive, and tell me which one I need to insert should I ever need to do a restore from an archive. I figure once every few weeks, I'll manually run some copy jobs to copy backup jobs to the archive drives. On the 4TB active drive, I just created via the "label" command a bunch of files and made them 50GB capacity, which does cause Bacula to write 50GB to each file, and it will automatically choose the next file to write to when one is full. I have done some restores (including one that was actually not just a test) from these backups successfully. For me, it doesn't matter that USB is slow. I know that of course. I knew I wasn't going to get great performance from the storage server as soon as I decided to try using a Raspberry Pi. At that time, I was looking for a way to avoid having to have yet another always-up server in the house, and the always-up servers I already have are already running enough stuff that I didn't want to add to them. So getting a server for 50 bucks and very low power use seemed like a good thing to try, and it has worked very well so far. The amount of data I write is low enough that performance is not a major issue. Slow but reliable is fine for my purposes. Thanks to everyone who chimed in on this. --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] USB tape drives?
On Fri, 2013-09-20 at 12:17 -0600, Greg Woods wrote: > On Tue, 2013-09-17 at 13:13 +0300, Guy wrote: > > Yes I do this with vchanger... > > I've run into a wall trying to get vchanger to compile on Raspbian (a > limited version of Debian for the Raspberry Pi). I did solve the compile issue. With a lot more Googling (I really wish I were better at searching for things), I did find the -dev packages. For some unknown reason, they are not in the repository, I had to download them manually and install them. When I did that and re-ran configure and make, it does compile. Now on to see if I can get it to work. --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] USB tape drives?
On Tue, 2013-09-17 at 13:13 +0300, Guy wrote: > Yes I do this with vchanger... I've run into a wall trying to get vchanger to compile on Raspbian (a limited version of Debian for the Raspberry Pi). When I run configure, it notes that I do not have libuuid. I believe this is correct; there is a shared library (.so) version, but there is no development (.a) version. However, it still tries to use -luuid anyway, and I get linker errors: libtool: link: g++ -g -O2 -o vchanger vchanger-vchanger_common.o vchanger-compat_util.o vchanger-junctions.o vchanger-uuidlookup.o vchanger-util.o vchanger-vconf.o vchanger-loghandler.o vchanger-diskchanger.o vchanger-changerstate.o vchanger-vchanger.o -L/lib/arm-linux-gnueabihf -luuid -lblkid /usr/bin/ld: cannot find -luuid /usr/bin/ld: cannot find -lblkid collect2: ld returned 1 exit status make: *** [vchanger] Error 1 I tried manually editing the Makefile in the src directory to comment out this line: #vchanger_LDADD = -L/usr/local/lib -luuid -lblkid I didn't really expect that to work, since manually editing Makefiles produced by configure seldom does, and it didn't: vchanger-uuidlookup.o: In function `GetMountpointFromUUID': /usr/local/src/vchanger/src/uuidlookup.c:236: undefined reference to `blkid_get_devname' collect2: ld returned 1 exit status Is there any hope of getting vchanger to compile? A set of configure options that will prevent the linker errors, perhaps? Thanks, --Greg P.S. It doesn't appear that -dev packages are available for these libraries: r# dpkg -l '*uuid*' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==---= ii libuuid1:armhf 2.20.1-5.3 armhfUniversally Unique ID library ii uuid-runtime 2.20.1-5.3 armhfruntime components for the Univer root@bstore:/usr/local/src/vchanger# dkpg -l '*blkid*" > ^C root@bstore:/usr/local/src/vchanger# ^"^' dkpg -l '*blkid*' bash: dkpg: command not found root@bstore:/usr/local/src/vchanger# dpkg -l '*blkid*' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==---= ii libblkid1:armh 2.20.1-5.3 armhfblock device id library -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] vchanger
I'm having some trouble getting vchanger to work, partly because the documentation that comes with it seems not to be up to date. For instance, the keywords described in the howto HTML file that comes with the source are invalid keywords. The example config file has the right keywords, but I'm having difficulty getting vchanger to work. When I get to testing vchanger, the list command works; it shows there are 34 slots. But "label barcodes" appears not to work correctly: *label barcodes Automatically selected Catalog: GregAndEva Using Catalog "GregAndEva" The defined Storage resources are: 1: BKUP 2: ARCH Select Storage resource (1-2): 2 Enter autochanger drive[0]: Connecting to Storage daemon ARCH at bstore.gregandeva.net:9103 ... 3306 Issuing autochanger "slots" command. Device "ARCH" has 34 slots. Connecting to Storage daemon ARCH at bstore.gregandeva.net:9103 ... 3306 Issuing autochanger "list" command. No Volumes found to label, or no barcodes. * The inserted magazine disk does automount correctly, but there isn't anything there except the lost+found directory. Nothing in the documentation tells me how the index file is created or the one-file-per-slot files or what name those files should have or if I am supposed to create them myself or if vchanger is supposed to create them and if so, how. Is there any up-to-date documentation that explains how to set up and configure vchanger? Is there something obvious that I have done wrong? I can post the relevant parts of the config files if necessary. Thanks, --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] vchanger
On Fri, 2013-09-20 at 19:20 -0600, Greg Woods wrote: > I'm having some trouble getting vchanger to work, partly because the > documentation that comes with it seems not to be up to date. Ugh, never mind. I've been spending so much time on this and doing so many searches that I got my versions of the Howto confused. It looks like the one that actually comes with the source code is not the one I was actually reading. Now that I have the right Howto, I'll see how it goes from here. Sorry for the noise. --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] vchanger (one more time)
I apologize for yet another posting, but I am just having a HELL of a time trying to get vchanger to work. I have been following the instructions in the Howto that came with the source code. I have the automounting working, and the permissions are all correct (the "bacula" user owns the work_dir and the automount_dir, as well as the root directory of my first magazine drive: # ls -ld /var/spool/bacula/mag /wd/mag /wd/mag/* drwxr-xr-x 2 bacula bacula 4096 Sep 21 11:06 /var/spool/bacula/mag drwxr-xr-x 3 bacula bacula0 Sep 21 11:06 /wd/mag drwxrwxrwx 3 bacula bacula 4096 Sep 20 19:06 /wd/mag/fea7e396-427b-49c4-af1e-195a6e6036dd Unless I forgot something else, this isn't a permissions problem. The problem is that "initmag" does not work: # vchanger -u bacula -g bacula /etc/bacula/ARCH.conf initmag 1 magazine bay 1 has no magazine mounted The "initmag" command does trigger a successful automount before the error occurs. Obviously I cannot get past this. Another oddity is, if I run the "listmags" command, even with no disk drive inserted, I get this: # vchanger -u bacula -g bacula /etc/bacula/ARCH.conf listmags 1::0: According to the Howto, the "0" should be a "-1" when there is no disk inserted. I also end up with this in the work_dir: # ls -l /var/spool/bacula/mag total 4 -rw-r--r-- 1 bacula bacula 1 Sep 21 11:15 bay1 -rw-r--r-- 1 bacula bacula 0 Sep 21 10:23 state0 (the "bay1" file contains only a single newline). I must have made some stupid mistake in configuring vchanger, but I'm damned if I can see it. Here is my ARCH.conf file: changer_name = ARCH slots_per_magazine = 34 virtual_drives = 1 magazine_bays = 1 automount_dir = /wd/mag work_dir = /var/spool/bacula/mag log_level = LOG_DEBUG logfile = /var/log/ARCH.log magazine = UUID:fea7e396-427b-49c4-af1e-195a6e6036dd magazine = UUID:c15c828b-0237-4b05-aa85-a9e3102ffe42 magazine = UUID:fe3c5faf-7ea2-4051-8a92-bc4c6c94b0ab magazine = UUID:6c7279d5-f287-4e3b-8a33-51e398a3c932 Here's what's in the log file after the "initmag" command: Sat Sep 21 11:20:43 2013 Bay 1 was previously unloaded Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID returned -3 Sat Sep 21 11:20:44 2013 Init: magazine UUID:fea7e396-427b-49c4-af1e-195a6e6036dd not mounted Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID returned -3 Sat Sep 21 11:20:44 2013 Init: magazine UUID:c15c828b-0237-4b05-aa85-a9e3102ffe42 not mounted Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID returned -3 Sat Sep 21 11:20:44 2013 Init: magazine UUID:fe3c5faf-7ea2-4051-8a92-bc4c6c94b0ab not mounted Sat Sep 21 11:20:45 2013 MagazineState::set() - GetMountpointFromUUID returned -3 Sat Sep 21 11:20:45 2013 Init: magazine UUID:6c7279d5-f287-4e3b-8a33-51e398a3c932 not mounted Sat Sep 21 11:20:45 2013 confirm bay 1 loaded with mag 0 = Sat Sep 21 11:20:45 2013 Init: drive 0 last state was unloaded Sat Sep 21 11:20:45 2013 Info: drive 0 is unloaded Sat Sep 21 11:20:45 2013 magazine bay 1 has no magazine mounted I have been struggling with this for two days now, so any ideas would be gratefully accepted. Thanks, --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] vchanger (one more time) SOLVED
On Sat, 2013-09-21 at 11:23 -0600, Greg Woods wrote: > Unless I forgot something else, this isn't a permissions problem. The > problem is that "initmag" does not work: > > # vchanger -u bacula -g bacula /etc/bacula/ARCH.conf initmag 1 > magazine bay 1 has no magazine mounted [...] > Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID > returned -3 I thought about this, and it looked as though perhaps the uuid library package I had installed was not actually working. It links OK, but it doesn't work when called. So I tried Josh Fisher's suggested fix for the linking issue: > The quick and dirty way is to comment out the vchanger_LDADD line in the Makefile.am in the src directory and replace the uuidlookup.c file with a version having a single function as follows: A GetMountpointFromUUID function that does nothing and always returns failure (look for the code and complete instructions in the "USB tape drives?" thread if you need it). This disables the ability to declare magazines by UUID, but since I was already automounting by UUID anyway, it was merely a syntactic change to switch to defining magazines by their mount points instead. So in ARCH.conf, instead of: magazine = UUID:fea7e396-427b-49c4-af1e-195a6e6036dd I now have: magazine = /wd/mag/fea7e396-427b-49c4-af1e-195a6e6036dd ...and now initmag worked, and I was able to move on. I was able to successfully create and run Copy jobs, and I am a happy camper. The Copy jobs are horribly slow on the Raspberry Pi (copying from one USB disk to another), but I can live with that. Thanks to everyone who offered advice. --Greg -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] copy, delete, purge question
I just want to make sure I understand how this is supposed to work before I do something that might cause me to lose backups. Suppose I take a full backup for a couple of clients to a backup pool. Then I successfully run a copy job which copies these to an archive pool. Am I correct in assuming that, if I now delete the original backup jobs, any volumes in the backup pool that contain only files from the deleted jobs will be marked Purged and be available for re-use? And that the copy jobs will now be marked as backups and I would be able to restore files from those jobs from the volumes in the archive pool? That's what I want to happen; I want to be able to delete jobs that I have archive copies of, to free up space in the backup pool, but still be able to restore files from the archived jobs. (The backup pool is always online so that I can schedule backups 24x7, the archive pool is much larger, and uses removable drives that are only connected while in use, so they can't be taken out by a power surge etc. and can be stored off site to guard against total disaster). --Greg -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] ERROR: Msg delivery error: Unable to store data in database
On the surface this is obvious, but I don't see why this should suddenly be happening. I haven't modified the configurations for either bacula-dir or mysql in some time, yet suddenly now the director can't access the database. Even weirder, this is only happening for Copy jobs, normal backups work fine. Manually accessing the database from the bacula account command line also works fine. Anybody ever seen anything like this or have any tips for debugging this? Thanks, --Greg Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 Message delivery ERROR: Msg delivery error: Unable to store data in database. Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 Message delivery ERROR: Msg delivery error: Unable to store data in database. Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05 Message delivery ERROR: Msg delivery error: Unable to store data in database. Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05 Message delivery ERROR: Msg delivery error: Unable to store data in database. Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 bdir: ABORTING due to ERROR in mem_pool.c:167 Nov 21 11:05:33 confusion bacula-dir[14226]: obuf is NULL Nov 21 11:05:33 confusion bacula-dir[14226]: bacula-dir: message.c:1270: void e_msg(const char*, int, int, int, const char*, ...): Assertion `p! =__null' failed. Nov 21 11:05:33 confusion bacula-dir[14226]: Bacula interrupted by signal 6: IOT trap Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! bacula-dir, bdir got signal 6 - IOT trap. Attempting traceback. Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! exepath=/usr/sbin/ Nov 21 11:05:33 confusion bacula-dir: bdir: ABORTING due to ERROR in mem_pool.c:167 Nov 21 11:05:33 confusion bacula-dir: Bacula interrupted by signal 6: IOT trap Nov 21 11:05:33 confusion bacula-dir[14226]: Calling: /usr/sbin/btraceback /usr/sbin/bacula-dir 14226 /var/spool/bacula Nov 21 11:05:33 confusion bacula-dir[14226]: It looks like the traceback worked... -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database
Isn't there some way I can debug this? I have spent my entire Saturday trying to fix this unsuccessfully. My backup jobs work, but copy jobs fail. I really haven't changed anything in the Bacula or Mysql configurations before today in quite a while (that I know of). Today I played games trying to give Mysql more resources in my.cnf, but I don't think that's really where the problem is. It looks like an authorization/grants issue but I'm damned if I can figure out what grant is needed or why this has suddenly started happening when it has been working fine for months. I'll put in the time, I'll read the docs, if somebody can just point me to somewhere that will help me make some progress. I have to give up for today because I am totally frustrated. I will post relevant config files if that will help. Today I tried updating Bacula and Mysql on the director server (now at 5.2.13-17 and 5.5.34-1 respectively) but it didn't make any difference. I am running 64-bit Fedora 18 if that matters (and yes, I know it will be end-of-life soon but as of now it's still supported). --Greg On Thu, 2013-11-21 at 11:13 -0700, Greg Woods wrote: > On the surface this is obvious, but I don't see why this should suddenly > be happening. I haven't modified the configurations for either > bacula-dir or mysql in some time, yet suddenly now the director can't > access the database. Even weirder, this is only happening for Copy jobs, > normal backups work fine. Manually accessing the database from the > bacula account command line also works fine. > > Anybody ever seen anything like this or have any tips for debugging > this? > > Thanks, > --Greg > > Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 Message > delivery ERROR: Msg delivery error: Unable to store data in database. > Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 Message > delivery ERROR: Msg delivery error: Unable to store data in database. > Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05 Message delivery > ERROR: Msg delivery error: Unable to store data in database. > Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05 Message delivery > ERROR: Msg delivery error: Unable to store data in database. > Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 bdir: ABORTING > due to ERROR in mem_pool.c:167 > Nov 21 11:05:33 confusion bacula-dir[14226]: obuf is NULL > Nov 21 11:05:33 confusion bacula-dir[14226]: bacula-dir: message.c:1270: > void e_msg(const char*, int, int, int, const char*, ...): Assertion `p! > =__null' failed. > Nov 21 11:05:33 confusion bacula-dir[14226]: Bacula interrupted by > signal 6: IOT trap > Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! bacula-dir, bdir > got signal 6 - IOT trap. Attempting traceback. > Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! exepath=/usr/sbin/ > Nov 21 11:05:33 confusion bacula-dir: bdir: ABORTING due to ERROR in > mem_pool.c:167 > Nov 21 11:05:33 confusion bacula-dir: Bacula interrupted by signal 6: > IOT trap > Nov 21 11:05:33 confusion bacula-dir[14226]: > Calling: /usr/sbin/btraceback /usr/sbin/bacula-dir > 14226 /var/spool/bacula > Nov 21 11:05:33 confusion bacula-dir[14226]: It looks like the traceback > worked... -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database
On Sat, 2013-11-23 at 15:18 -0700, Greg Woods wrote: > I > played games trying to give Mysql more resources in my.cnf, but I don't > think that's really where the problem is. It turns out that *is* where a problem is, but there were absolutely no error messages anywhere to indicate this, other than the generic "cannot connect to MySQL server" error. I did check the bacula logs, the system log files and the mysqld log file. I think the problem was that I was using the "Selection Type = PoolUncopiedJobs", and the number of jobs to be copied had gotten larger than previously and caused some limit to be exceeded. I increased a number of resource limits in my.cnf, but it still doesn't work. I don't see any smoking gun in the bacula or system logs to tell me what the problem is. The only error message I see anywhere is "Cannot connect to database, either the server is not running or your authorization is incorrect". However, I know the authorization is correct and the server is running, so it's got to be some kind of resource exhaustion. I did try running a copy for a single client. That also works, also pointing to resource exhaustion. The problem with that is that it also copies jobs that have already been copied. What I'd really like is to be able to copy all jobs for a single client that have not already been copied, so I can do all of them a few jobs at a time, but there doesn't seem to be any way to do that. Maybe with some kind of SQL query? I could maybe develop one, but it would take a lot of time to learn enough about how the database is structured to be able to do that, so I want to have someone who knows say that this is possible before I invest all that time. Another approach is to accelerate my move of Bacula and MythTV to a new machine with more RAM, which I was going to do anyway. Then I can allocate more memory to the database. But of course I don't know for sure that that will fix the issue either. --Greg -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database
On Sun, 2013-11-24 at 11:32 -0700, Greg Woods wrote: > . I increased a > number of resource limits in my.cnf, but it still doesn't work. Got a teeny bit farther. Now, with increased limits, bacula-dir no longer segfaults, and I no longer get the database connection error. The problem now is that all the copy jobs are spawned, but the first one is just hanging there doing nothing. Job log looks normal so far: Log records for job 3496 2013-11-24 11:54:20 bdir Copying using JobId=2014 Job=orolo.2013-11-02_19.00.09_28 Bootstrap records written to /var/spool/bacula/bdir.restore.1.bsr 2013-11-24 11:54:23 bdir Start Copying JobId 3496, Job=ARCHIVE-ALL.2013-11-24_11.54.20_06 That was 20 minutes ago and it's still just sitting there. The Media tab in bat shows none of the archive volumes are being written to (although they are online). I'll let it sit there for a while longer just in case, but it kind of looks like I have to find a way to limit the number of jobs spawned at once while also avoiding copying jobs that have already been copied. --Greg -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] question about how retention works
There is some stuff I still don't understand here, and I'd like to avoid learning the hard way. For instance, I never realized that "File Retention" referred to the individual file records in the database, not to the actual files on backup. So I ended up losing many of my File records (learning the hard way), when what I really wanted was for the volumes containing those files to be made available for re-use when necessary. So now I *think* I have accomplished this by setting Job Retention and File Retention to 365 days, and Volume Retention to 30 days. I would assume that, after 30 days since the last time a Volume was written, it will be available for recycling, but I won't lose the database records when that happens. In my case, I'm only using Bacula for a dozen or so systems and VMs at home as clients, so I'm not overly concerned with the size of the database. What good does it do to have database records for files that are no longer available, you may well ask? I use Copy jobs to copy my online backup media to my archive backup media. The former is just one big disk, the latter is a series of disks inserted in an external USB caddy, implemented with vchanger. I periodically run Copy jobs to copy backups from the main disk to the vchanger, so that if the main disk volume gets recycled, the files still exist on the vchanger. Since these are separate Pools, I can and do set the Volume Retention differently for each. But my impression is that the Copy jobs don't really create separate File records. File Retention seems to be set only on a per client basis, so if I want to be able to do full-feature restores from the vchanger, I need to keep the File records around even if the volume in the main backup Pool has been recycled. Am I understanding how this works correctly? Next question is about volume purging. How does Bacula determine when it is time to recycle a volume? Will it do this automatically after the Volume Retention period has expired? How exactly is the Volume Retention period measured? (Meaning, if I set it to 30 days, it's 30 days from when?) When a volume expires, does it immediately become unusable, or only when space is needed? When space is needed, will Bacula automatically use a volume that is past its retention period, or do I have to do something manually to make the expired volumes available? Sorry to be so dense here. I have spent a lot of time reading the online docs, and I've been using Bacula since August, but some of this is still not clear to me. Thanks, --Greg -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about how retention works
On Thu, 2013-12-19 at 11:02 -0500, Dan Langille wrote: > There was a lot in the above and I couldn't parse it all > just now. Sorry about that; I suppose it would help if I just explicitly stated what I really want. I have a main backup pool where all the backup jobs write. This pool is manually copied to the archive pool using Copy jobs. What I want is for volumes in the main pool to be recycled relatively quickly (because space is limited), while the files are retained much longer in the archive pool. I'd like restore attempts to automatically figure out when the files are still available in the backup pool and use it when possible, but get them from the archive pool if necessary. That's what I'm working toward. > Instead, I'll point you to something I wrote a few years ago. > If you have questions after that, please ask again. > > http://dan.langille.org/2010/02/08/bacula-retention-periods/ OK, good. It's good that we're having this discussion. If I understand any of this at all (which I may not), there are a couple of things in there that seem not to match my experience so far. For instance, in that document it says "you can’t restore a given file after the File retention passed". But that's not really true. You can't restore it in the "usual" way, by marking files, presumably because the list of available files for marking is created using those File records that aren't there any more. But you *can* restore those files by using a regular expression; I've had to do this because I have lost a lot of my File records by initially setting File Retention too low. I'm not sure how it actually does this, but there must be some information written to the backup volume itself that can be used to determine what files are there. It's much slower than a database access, but I have restored directory trees and individual files this way. The document also says "Volumes are eligible for recycling once ALL retention periods have passed." But that doesn't seem to be true either; I have apparently seen Bacula recycle some volumes even though the Job retention period has not passed. Which means, presumably, that there are Job records in the database that point to backup files that aren't there any more, since the Volume has been recycled. > Perhaps the Recycling Algorithm will help: > > http://www.bacula.org/en/dev-manual/main/main/Automatic_Volume_Recycling.html That does help, and I've looked at this before, but it's not always clear (to me) exactly how it works, even after reading that. I notice that the paragraph on Automatic Pruning says that you can't restore files once the File records are gone, but I know that's not true because I have done it. I suspect the documentation may be a few versions old and I can't necessarily trust everything it says. It also states that when volume recycling happens, "all Jobs that are older than the VolumeRetention period will be pruned", presumably regardless of what I set for Job Retention, which leads to some of my confusion about what these retention periods really mean. It's even more confusing (to me) when you throw in Copy jobs, so that the same files may be on multiple Volumes in different pools. --Greg -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about how retention works
On Thu, 2013-12-19 at 12:38 -0500, Dan Langille wrote: > I set my FILE and JOB retentions high. 3 years. Then I set my VOLUME > retention lower. Whichever retention period expires first, that's the > one which counts. <== I will refer to that as 'first one counts' later > in this email. > > Therefore, in your archive pool, set your VOLUME retention to 3 years, > for example. > > And in your main pool, set it to 5 weeks, for example. > > Does that help? Yes. This is more-or-less what I have now. I have File Retention = Job Retention = 365 days. In the main pool, Volume Retention = 30 days, and in the archive pool, Volume Retention = 365 days. I'm just trying to make sure that this is really going to accomplish what I want. The one big question remaining is how Copy jobs affect this. If I use a Copy job to make a copy of one or more jobs (usually all the jobs for a given client) from the main pool to the archive pool, what happens to the File records? Are there now two File records for every file on the backup? For me that's fine, but in an enterprise setup with thousands of clients, effectively doubling the size of the database would be a pretty big deal, so it wouldn't surprise me to learn that there are some efficiency improvements there. That's what leads me to wonder what really happens when a Volume in the main pool is recycled. The Recycle Algorithm document suggests that the File records associated with that Volume are pruned, so does that or does that not mean that the corresponding jobs in the archive pool no longer have File records? I may have to poke around in the database a little to see if I can answer that question for myself. > > I notice that the paragraph on Automatic Pruning says that you can't > > restore files once the File records are gone, but I know that's not > > true > > because I have done it. I suspect the documentation may be a few > > versions old and I can't necessarily trust everything it says. > > You've done it a different way. Not the 'usual way'. I guess that depends on how you define "the usual way", so I'll clarify. This is what I see, running "restore" out of "bconsole". I first select "most recent backup for a client", and go through the usual process of selecting the client and the File Set, and then I see: You have selected the following JobIds: 56,3603,3609,3616,3622,3710,3716,3722,3832,3839 Building directory tree for JobId(s) 56,3603,3609,3616,3622,3710,3716,3722,3832,3839 ... ++ For one or more of the JobIds selected, no files were found, so file selection is not possible. Most likely your retention policy pruned the files. Do you want to restore all the files? (yes|no): no Regexp matching files to restore? (empty to abort): At this point, I can type in a regex. Then it shows the usual parameters followed by "yes/no/mod", which I can then modify in the usual way, and then, although it takes quite a while, it will restore the files that matched the regex. > > Does what you've observed make sense when you consider my claim of > 'first one counts'? I'm not sure I can say it makes sense to me until I find some time to poke around in the database to see what is really going on. The one thing I am afraid of is having a Volume in the main pool getting recycled causing the File records for files in the archive pool getting pruned. --Greg -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] question about how retention works
Good, we're getting somewhere. On Fri, 2013-12-20 at 11:45 -0500, Dan Langille wrote: > Let's confirm this. Let's look at this job: {SQL stuff snipped} That is what I was eventually going to get around to doing, so thanks for doing that for me. So it looks like, if the same file is written to Volumes in multiple jobs, then there are multiple File records for it. > > Building directory tree for JobId(s) > > 56,3603,3609,3616,3622,3710,3716,3722,3832,3839 ... > > ++ > > > > For one or more of the JobIds selected, no files were found, > > so file selection is not possible. > > Ahh, look there: 'one or more' > > Perhaps you are able to restore by regex because you're using the jobs > which are not passed the File retention period? I'll have to check that, but I'm pretty sure I was able to restore the entire /etc/.* tree, and the Full backup is definitely more than 30 days old and got pruned. But I'll verify that. But there is also the "bscan" command, the man page for which states: The purpose of bscan is to read (scan) a Bacula Volume and to recreate or update the database contents with the information found on the Volume. This is done in a non-destructive way. This permits restoring database entries that have been lost by pruning, purging, deleting, or a database corruption problem. This implies that there is information on the Volumes that duplicates what's in the database, but it doesn't say whether or not that includes File records. So there is also the "bls" command, the man page for which implies that it can list the files from a volume. So I tried it. It's slow on my poor Raspberry Pi (the bacula-sd server), but it works: bls: butil.c:287 Using device: "BKUP" for reading. 20-Dec 10:47 bls JobId 0: Ready to read from volume "BKUP.2" on device "BKUP" (/bkup/bacula). bls JobId 0: drwxrwxr-x 2 root 499 4096 2013-12-07 11:34:48 /var/lib/VirtualBox/ bls JobId 0: -rw-r--r-- 1 989 986 3072 2013-12-08 00:32:01 /var/spool/asterisk/astdb.sqlite3 bls JobId 0: drwxr-x--- 7 989 986 4096 2013-12-08 00:32:01 /var/spool/asterisk/ bls JobId 0: drwxrwx--- 2 51 51 4096 2013-12-08 00:14:38 /var/spool/clientmqueue/ bls JobId 0: drwx--x--- 3 root lp 4096 2013-12-07 09:04:26 /var/spool/cups/ bls JobId 0: drwx-- 2 root man 4096 2013-12-08 00:14:39 /var/spool/mqueue/ bls JobId 0: -rw--- 1 root root 9 2013-12-08 01:01:01 /var/spool/anacron/cron.weekly bls JobId 0: -rw--- 1 root root 9 2013-12-08 01:01:01 /var/spool/anacron/cron.monthly bls JobId 0: -rw--- 1 root root 9 2013-12-08 01:01:01 /var/spool/anacron/cron.daily [] This does not contain information about which client those files came from though. But that information would be available from Job records if those are still present in the database. So it would appear that, in fact, there is information on the Volumes that duplicates the File records in the database, which means I may well have been restoring files for which there were no longer any File records in the database. I'll play some SQL games this weekend and see if I can find an explicit example for which there is no File record but I can still restore the file. I hope some others out there have found this to be as informative as I have. --Greg > > > Most likely your retention policy pruned the files. > > > > Do you want to restore all the files? (yes|no): no > > > > Regexp matching files to restore? (empty to abort): > > > > At this point, I can type in a regex. Then it shows the usual > > parameters > > followed by "yes/no/mod", which I can then modify in the usual way, and > > then, although it takes quite a while, it will restore the files that > > matched the regex. > > > >> > >> Does what you've observed make sense when you consider my claim of > >> 'first one counts'? > > > > I'm not sure I can say it makes sense to me until I find some time to > > poke around in the database to see what is really going on. > > > > The one thing I am afraid of is having a Volume in the main pool > > getting > > recycled causing the File records for files in the archive pool getting > > pruned. > > I think that is why I keep File and Job retention to 3 years. To avoid > this situation. I let Volume retention dictate how long I keep File and > Job records. i.e. up to 3 years. > -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___
Re: [Bacula-users] question about how retention works
On Fri, 2013-12-20 at 13:14 -0500, Dan Langille wrote: > Read http://www.bacula.org/en/dev-manual/main/main/Restore_Command.html > > 'If a Job's file records have been pruned from the catalog, the restore > command will be unable to find any files to restore. Bacula will ask if > you want to restore all of them or if you want to use a regular > expression to restore only a selection while reading media. See > FileRegex optionFileRegex and below for more details on this.' That pretty much implies that it *is* possible to restore files for which there are no more file records in the catalog. And that is a VERY good thing. I would hate to have all my backup data become useless due to a database crash (yes, I do back up my database outside of Bacula just in case, but still...) --Greg -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula suid files
On Wed, 2014-02-05 at 15:57 +0100, sca...@interia.eu wrote: > During backup verification I've noticed that files like: > srw-rw-rw- 1 ntp Debian-exim 0 Jun 29 2011 > /home/backup/_old_system/var/spool/postfix/private/discard > aren't backuped. Is there any problem with suid files? That's not an suid file, that's a socket. Since sockets are created by running programs and are not useful unless that program is running, they don't need to be backed up. --Greg -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Failure Backing Up Windows Client Machine
On Sat, 2014-02-22 at 15:09 +, Larry Kemp wrote: > > > 22-Feb 08:53 bacula1-dir JobId 3652: Fatal error: bsock.c:134 Unable > to connect to Client: bnspflu1-ts01-fd on 10.100.101.203:9102. > ERR=Connection refused > > > Very likely this means that either your Windows firewall is not allowing the connection, or else the bacula-fd process is not actually running. Use the Windows taskmgr to verify that bacula-fd is running (Accessories -> Run Program, type "taskmgr"). If it is, then try disabling the firewall temporarily; I don't remember exactly where this is, but it would be under the Control Panel somewhere (security settings). Just don't forget to turn it back on when your test is completed. --Greg -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Failure Backing Up Windows Client Machine
On Sat, 2014-02-22 at 14:16 -0500, Larry Kemp wrote: > I will check to see if fd is running at all...I assumed it was...maybe > it is not at all. At least with the version of the Windows bacula-fd that I have on my VM, installing the daemon doesn't automatically create a startup entry for it. I usually start it from a command prompt window (running as Administrator, otherwise bacula-fd doesn't have permission to read all the files). I'm sure there's a way to put it into the system startup, but I haven't figured out how to do that on Windows 7 yet. --Greg -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Failure Backing Up Windows Client Machine
On Thu, 2014-02-27 at 15:08 -0500, John Drescher wrote: > > Entered: > > sc create Bacula-FD binpath= "C:\Program Files > \Bacula>bacula-fd.exe" -c ./”C:/Program > Files/Bacula/bacula-fd.conf” type= share start= auto Shouldn't the > above be another backslash? --Greg -- Flow-based real-time traffic analytics software. Cisco certified tool. Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer Customize your own dashboards, set traffic alerts and generate reports. Network behavioral analysis & security monitoring. All-in-one tool. http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Failure Backing Up Windows Client Machine
On Thu, 2014-02-27 at 16:53 -0500, John Drescher wrote: > > How about: > > C:\Documents and Settings\lkemp>sc create Bacula-FD binpath= "C: > \Program Files\Bacula>bacula-fd.exe" -c "C:\Program Files\Bacula > \bacula-fd.conf" type= share start= auto Still has a > that should be \ -- Flow-based real-time traffic analytics software. Cisco certified tool. Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer Customize your own dashboards, set traffic alerts and generate reports. Network behavioral analysis & security monitoring. All-in-one tool. http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Removable Disk Vchanger
On Mon, 2014-03-03 at 09:50 -1000, WebDawg wrote: > I need to backup to multiple drives and use them in one slot like > tapes. > > > That is what you mean right? Yes, I believe so. I use disk drives for backups at home. One of them is a 4TB drive that is always online, and is therefore available for unattended backups. This drive doesn't need vchanger, I just create files on the drive that are bacula volumes. They get recycled automatically as needed. I also have a single-drive capacity mounter that allows inserting a drive which is then recognized and available. I've got 6 drives that get used in this. Obviously, not all the volumes on those drives are online all the time (in fact none of them are), and this is where vchanger comes in. Each drive is a "magazine" in a single mag capacity vchanger drive. Insert a drive, run "update slots", and the volumes from that drive are available. These drives are used as the target of copy jobs, and stored offline in a fireproof safe. Better would be completely off site of course, but that's overkill just for my home systems. The volumes on the 4TB drive end up getting recycled more often than I can do full backups, so I usually will need one or more of the vchanger drives when doing restores, so it's not practical to keep them offsite. If I were at work with more resources, then of course I would use a higher capacity first tier backup storage device, and keep the second tier backup storage at an offsite location when not in active use. --Greg -- Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce. With Perforce, you get hassle-free workflows. Merge that actually works. Faster operations. Version large binaries. Built-in WAN optimization and the freedom to use Git, Perforce or both. Make the move to Perforce. http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula only showing directory structure on restores
On Wed, 2014-03-19 at 16:32 +, James Lumby wrote: > Files do not seem to have expired as the file retention is 365 > days. Since I was confused by this as well, I thought I'd jump in here briefly. "File retention" really means how long the file records in the database are kept. It has nothing directly to do with how long the actual files are kept. The latter is a function of when the volumes are recycled, which is controlled by volume retention. I don't know if this is related to your issue, but I thought I'd point it out so that you don't beat your head against the same wall that I did. --Greg -- Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bsmtp problem.
On Fri, 2014-03-21 at 23:12 +, Levie, Jim wrote: > The default for all Linux and Unix was in the past to install sendmail or > postfix (limited to localhost). Dunno if that might have changed in the > latest versions of Fedora. In fact, it has. Fedora does not install an SMTP server by default any more. This was the subject of flame wars on the Fedora users list, but the bottom line is that this did change recently. One can, of course, easily install an SMTP server if that is really needed. --Greg -- Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] symbol lookup error: /usr/lib/libbaccats-7.0.2.so: undefined symbol: mysql_thread_safe
On Sun, May 4, 2014 at 6:44 AM, Kern Sibbald wrote: > MariaDB is not 100% compatible with MySQL, and thus possibly not > compatible with Bacula. Just as a data point: I am using Bacula with the director and database running on Fedora 20. This distro is using Mariadb. It works fine with Bacula, although this is only a small home installation, I can't speak to how well it would work with a large enterprise installation. --Greg -- "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available. Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula backups failing
Maybe it's removing the existing /var/bacula/.my.cnf file and then trying to write a new one? What are the permissions on /var/bacula? --Greg -- Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with schedule
On Sun, Jan 4, 2015 at 11:58 PM, Florian wrote: > Hello everyone. > > I seem to be having a strange issue with the schedule of my backups. > This is the schedule I use: > > Schedule { > Name = "MonthlyCycle" > Run = Full on 1 at 3:15 > Run = Differential sun at 3:15 > Run = Incremental mon-sat at 3:15 > } > > I do not know why, but on the first of every month, bacula tries to do > both the Full backup AND the incremental > I think you need to explicitly tell bacula that you don't want the incremental run on the 1st. For example, I have this: Schedule { Name = worldsys Run = Level=Incremental on 1-2 at 00:05 Run = Level=Incremental on 4-31 at 00:05 Run = Level=Differential on 3 at 00:05 } I am using Bacula in a small home operation, so I run my Full backups by hand, once every 6 months, but I want to run a Differential instead of an Incremental once a month. The above schedule says to run the incremental on all days except the 3rd, and run a differential on the 3rd. --Greg -- --Greg -- Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Is this an Inter-version Compatibility Issue???
On Wed, Jan 7, 2015 at 9:49 AM, John Florian wrote: > > My Fedora 21 workstation has: > > > > bacula-client-7.0.5-2.fc21.x86_64 > > bacula-common-7.0.5-2.fc21.x86_64 > > bacula-libs-7.0.5-2.fc21.x86_64 > > > > > I ran into the same problem. I solved it by removing the bacula packages, then installing the fc20 RPMs. You probably will have to download the RPMs, then installl them from the local RPM files you downloaded. My problem is that my SD server is a Raspberry Pi, and there are no Bacula 7 packages for it. I'd have to compile from source. It was easier to downgrade the f21 system than to do this. Eventually I may try it, because the system that the Bacula director runs on is going to need upgrading soon too (for reasons having nothing to do with Bacula). --Greg -- Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-users replyTo address
On Wed, Apr 8, 2015 at 5:24 AM, Robert Oschwald wrote: > > > Is it possible for the admins to set the reply-to address to the > list-address so replies are posted to the list instead of the author? > Possible? Probably. Desireable? This is one of those religious wars that comes up on every mailing list eventually. I personally hate "reply to list"; I would much rather accidentally reply to author when I meant reply to list than accidentally send a reply to the list that was intended for only one person. Others will argue just as vociferously the other way. --Greg -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] SQL query for copy jobs
I have a job called ARCHIVE-ALL that looks like this: JobDefs { Name = "ArchiveJob" Type = Copy Level = Full Selection Type = SQLQuery Messages = Standard File Set = Copy Pool = File } # Copy (to archive) all not-already-copied successful backup jobs Job { JobDefs = ArchiveJob Name = "ARCHIVE-ALL" Client = bstore Selection Pattern = "select JobId from Job j where PoolId=2 and Type='B'and Jobstatus='T' and (select count(*) from Job where PriorJobId = j.JobId) = 0" } PoolId=2 corresponds to my "File" pool, which is where all the backups are written to initially. I also have an Archive pool where the copy jobs get written. This worked well for over a year, but recently, the SQL query is matching a whole bunch of jobs that have already been copied, and it is failing to match some (like a Full backup I did on a client just today) that need to be copied. I am theorizing that the reason it stopped working correctly is that a lot of the jobs have now been expired, and a recent round of full backups required that some jobs from the File pool had to be purged. Does this reset the PriorJobId to zero for any jobs that had a PriorJobId pointing to a job that is being purged? Does anybody have an SQL query that will do what I want (which is to copy all the backup jobs from the File pool to the Archive pool that have not yet been copied? Thanks, --Greg -- Monitor Your Dynamic Infrastructure at Any Scale With Datadog! Get real-time metrics from all of your servers, apps and tools in one place. SourceForge users - Click here to start your Free Trial of Datadog now! http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] SQL query for copy jobs
Thanks for looking at this. I am a bit redfaced right now, because it looks like I have to withdraw the question. When I used "bat" to view the Jobs Run, and clicked "Filter Copy Jobs", a lot of the confusing stuff was removed, and I could see that the dates actually had a monotonically increasing progression. Which means that, in fact, I wasn't getting duplicate jobs copied and the ones I thought hadn't been copied actually were. > To check if a job was already copied you have to verify priorjobid column > in job table. The original job will have a priorjobid=0 and a copied job > will have a priorjobid= of the original. > Yes, that is exactly what the "ArchiveAll" job is counting on. > > Your SQL is very weird to me. You copy jobs from pool File to the Pool > File? > Sorry for the confusion there. Pool { Name = File Pool Type = Backup Recycle = yes # Bacula can automatically recycle Volumes AutoPrune = yes # Prune expired volumes Volume Retention = 90 days # 3 months Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 70# Limit number of Volumes in Pool Storage = BKUP Next Pool = Archive # Where to copy/migrate to } The Pool definition for the File pool lists "Next Pool = Archive", so the copy jobs really were being written to the Archive pool. Sorry for the noise everyone. --Greg -- Monitor Your Dynamic Infrastructure at Any Scale With Datadog! Get real-time metrics from all of your servers, apps and tools in one place. SourceForge users - Click here to start your Free Trial of Datadog now! http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] authorization errors after upgrade
I am trying to upgrade from 5.2.13 to 7.0.5, and I am getting authorization errors between the director daemon and storage daemon. I am using the identical config files in both versions, so I am sure that the passwords match. The director and storage run on different machines, but they are both 7.0.5 . If it matters, the director is Fedora 23, and the storage is Debian Jessie (on a Raspberry Pi). The director was installed from the release packages, the storage compiled from source (as the Pi version of Jessie still has 5.2.13). Are there changes needed in the 7.0.5 config files to make it work with 5.2.13 config files? Thanks, --Greg -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and NFS
On Wed, Dec 16, 2015 at 9:58 AM, Bryn Hughes wrote: > The 'ERR=Connection refused' message means that the storage daemon itself > isn't accepting connections. This could be due to firewall rules or due to > mismatched passwords between the storage daemon and the other components. This can also happen due to a version mismatch. The requirement is FD <= (SD == DIR) (someone will no doubt correct me if I have screwed that up :-) --Greg -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment
On Wed, Dec 23, 2015 at 5:10 AM, Daniel Bareiro wrote: > am thinking of using Debian Jessie which includes Bacula 5.2.6 in its > repository for director and storage daemon. But I remember a few cases > where I have experienced incompatibilities if the client versions are > far from the server version. Correct me if I'm wrong, please. > The requirement is FD <= (SD == DIR). The storage daemon and director must be the same version, and must be newer than or the same as any file daemons of any of the clients. I do use a 7.0.5 director and storage daemon (compiled from source on Debian Jessie in the SD case) with some clients that have 5.2.6 file daemons and that works just fine. --Greg -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment
On Thu, Dec 24, 2015 at 8:34 AM, Daniel Bareiro wrote: > Greg, you mention having compiled the SD on Jessie, but I guess you've > also compiled the Director, right? Since both must have the same > version. Though I suppose that the Director may also be on another host > that includes Bacula 7.0.5 in its repositories. > The latter is correct. Most of my machines are at Fedora 23. At release, these had 7.0.5, although 7.2.0 update packages have since been released. The one system that is not Fedora 23 is my Raspberry Pi that I use as the storage server; that runs Debian Jessie. In order to be able to use the 7.0.5 director, I had to compile 7.0.5 storage server on Jessie. The hardest part about this was creating a proper systemd unit file for bacula-sd. I finally installed the bacula-sd package, saved off the systemd unit file that it provided, then removed the package and installed the compiled storage daemon. This all works, but it is very convoluted. > About using Jessie, is it worth compile the Director and SD? > If you're going to compile one, you should probably compile both. Now I have heard that Fedora 23 can be installed on a Raspberry Pi 2, and I got one of those for Christmas, so I will be attempting that this weekend. This isn't because I have anything against Debian, it's just that maintenance will be a lot easier if I can use the distro's package management system and easily update all the various Bacula components at the same time. Running "dnf update bacula*" is way easier than compiling from source and mucking about with systemd. And now I have had to exclude bacula packages from updates on my systems to avoid having the file daemons upgraded to a newer version than my storage server, so this becomes even more of an issue for me now. --Greg -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment
On Mon, Jan 4, 2016 at 6:51 AM, Peter Keller wrote: > > > # bacula-dir -t > > If it comes back with a 0 return value, then your config file is of a good > syntax. Then try: > > # bacula-dir -v -f -d100 > > > Excellent suggestions. Also useful in debugging systemd unit files is the journal. Something like: # journalctl -xb | grep bacula (shows all bacula-related entries since the last boot). --Greg -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] trying to build a static bacula-fd
On Fri, Feb 9, 2018 at 6:26 AM, Mike Eggleston wrote: > > /bin/ld: cannot find -lz > /bin/ld: cannot find -lpthread > /bin/ld: cannot find -ldl > /bin/ld: cannot find -ldl > /bin/ld: cannot find -lssl > /bin/ld: cannot find -lcrypto > /bin/ld: cannot find -lstdc++ > /bin/ld: cannot find -lm > /bin/ld: cannot find -lc You are missing the packages that contain the libraries for static linking. For instance, on my Fedora system, "dnf whatprovides */libssl.a" shows that I would need the "openssl-static" package. You would need to do the equivalent for your distro and install the packages that provide those static libraries. --Greg -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Storage Daemon Service not working at start-up (after os update)
On Thu, Feb 21, 2019 at 11:32 AM Yılmaz Bilgili via Bacula-users < bacula-users@lists.sourceforge.net> wrote: > > > If I stop the system service and start it manually from command line > with "bacula-sd -d 200 -c /opt/bacula/etc/bacula-sd.conf" command then I > can get status of storage daemon and the backups are working as > expected. > I had the same symptom on a Fedora system. It turned out that the storage daemon was being started before the network interface was fully up. This is common with systemd-based systems when something is either compiled from source, or installed from a package that was not built by someone with a systemd-based system. For me, I fixed it by writing my own service override file. I put this in /etc/systemd/system/bacula-sd.service.d/override.conf : [Unit] After=network.target nss-lookup.target NetworkManger-wait-online.service autofs.service I put autofs.service in there because I am using autofs to mount USB-attached disks that contain my Bacula volumes; you may not need that. NetworkManager-wait-online.service is the critical piece (and of course you may need to use systemctl to enable the NetworkManager-wait-online service). There is also an easier but kludgy fix that usually works, which is to put a sleep into the startup script to get it to wait a while for the network to come up before starting bacula-sd. Not guaranteed to work every time, but in practice this may be good enough. It does introduce a few seconds delay in getting the system fully booted, but perhaps that's not a big deal for you. --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] help debugging new SD
I have a small home installation (less than a dozen clients) that has the director and MySQL on one host, and the SD on a Raspberry Pi. What I am trying to do now is replace an old Fedora 28 Pi 2 with a new Raspbian Pi 4. I'm having trouble getting the new SD to work. I've got two storage devices: BKUP (disk volumes) and ARCH (uses a vchanger). Ironically, the ARCH storage appears to be fine (although I haven't actually tried to write to it yet). The volumes in the magazine that is currently inserted show green in bat. But all the BKUP volumes show red, so it's not surprising that it's not working. The problem is when I run a normal backup job, it hangs, asking me to label a new volume because no appendable volumes were found. But the catalog does list at least one volume with Append status, and all the volumes have Recycle set to 1. The catalog listing for all the BKUP volumes looks fine (no surprise there as it's the same director and catalog as before). So I am trying to debug this running the SD with -d 999, and here's the tail of what I get: bstore: askdir.c:233-41356 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] help debugging new SD
On Thu, Dec 5, 2019 at 12:45 PM Greg Woods wrote: > > bstore: askdir.c:235-41356 get_volume_info failed: ERR=1000 OK > VolName=BKUP.65 VolJobs=5 VolFiles=7 VolBlocks=491843 VolBytes=31729597852 > VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=164 VolErrors=0 > VolWrites=7981752 MaxVolBytes=53687091200 VolCapacityBytes=0 > VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 > VolReadTime=5665051214 VolWriteTime=15363285212 EndFile=7 > EndBlock=2345686412 VolType=1 LabelType=0 MediaId=100 ScratchPoolId=0 > VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 > bstore: askdir.c:398-41356 No vol. index 1 return false. dev="BKUP" > (/bkup/bacula) > bstore: askdir.c:406-41356 unlock_volumes at askdir.c:406 precnt=1 > bstore: mount.c:363-41356 not dir_find_next > bstore: askdir.c:799-41356 enter dir_ask_sysop_to_create_appendable_volume > > I have checked the obvious and the stupid > Well, as it turns out I *thought* I had checked the obvious and the stupid, but I missed one thing: version numbers. Turns out the new SD is 9.4, and the director was 9.2 . Upgrading the director to 9.4 fixed the issue. Sorry for the noise. --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] volumes in error status
I have a small home setup (about 10 clients). I have one pool used for backups which is just a directory full of disk files (50GB each). The other pool is a vchanger used for copy jobs. The problem I have is that for some reason, the directory holding the on-disk volumes became unreachable (probably due to an automount error but I'm not sure about that). This resulted in several volumes being set to Error status. I know the data in those files is still good. Is there any way to recover it? I tried just setting the volume status to "Full", but that didn't work because bacula thinks the volume size is 1 byte. I could set the status to "purged", which might allow the volumes to be re-used(?), but of course then I would lose all the data on the volumes. Thanks, --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Cannot find any appendable volumes
I am getting this error when running copy jobs. I have an "ARCHIVE-ALL" job that searches for any client backup jobs that terminated normally and have not yet been copied from online to archive backups, and runs copy jobs for each. I have been using this system for 7 years now, and it has worked well until today. Here's the Pool definitions (File is the online pool, and Archive is a vchanger with 6 removable disks as magazines): Pool { Name = File Pool Type = Backup Recycle = yes # Bacula can automatically recycle Volumes AutoPrune = yes # Prune expired volumes Volume Retention = 365 days # 1 year Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 245# Limit number of Volumes in Pool Storage = BKUP Next Pool = Archive # Where to copy/migrate to } Pool { Name = Archive Pool Type = Backup Recycle = yes # Bacula can automatically recycle Volumes AutoPrune = yes # Prune expired volumes Volume Retention = 365 days # one year Maximum Volume Bytes = 50G # Limit Volume size to something reasonable Maximum Volumes = 136# Limit number of Volumes in Pool Storage = ARCH } As you can see, the volume retention in the Archive pool is set to 365 days., and Recycle is set to yes, which means (if I understand this correctly) that any volume that hasn't been written to in more than a year should be available to recycle. So why is it suddenly telling me that there are no appendable volumes after 7 years of running this configuration? All of my client definitions look like this: Client { Name = "myhost" Address = myhost.my.domain Catalog = "My Catalog" # there is only one Password = "big secret" File Retention = 365 days Job Retention = 365 days } Did I miss something? There are definitely some volumes in the Archive pool that have not been written to in more than a year; shouldn't those be available for recycling? Lastly, what command would I use to let Bacula know that a given volume can be made available? Eventually I have to solve my configuration issue, but for now I'd like to just mark some old volumes as available manually so that my copy jobs can continue. Thanks for any pointers, --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] how to tell 9.6 director to use mysql?
I am migrating my Bacula director and database from a Fedora 31 machine running 9.4 to a new Fedora 32 machine running 9.6. I am having what seems like a simple configuration issue but I can't figure it out after about an hour of Googling around and trying different things by trial and error. The error I get is this: [root@seveneves greg]# bacula-dir -t -c /etc/bacula/bacula-dir.conf bacula-dir: dird.c:1248-0 Dbdriver field within director config file "mysql" mismatched with the Database argument "PostgreSQL" passed during Bacula compilation. bacula-dir: dird.c:1273-0 Could not open Catalog "GregAndEva", database "bacula". bacula-dir: dird.c:1279-0 postgresql.c:330 Unable to connect to PostgreSQL server. Database=bacula User=bacula Possible causes: SQL server not running; password incorrect; max_connections exceeded. 20-Jul 16:24 bacula-dir ERROR TERMINATION Please correct configuration file: /etc/bacula/bacula-dir.conf This is my Catalog definition: Catalog { Name = GregAndEva # Uncomment the following line if you want the dbi driver dbdriver = "mysql"; dbname = "bacula"; dbuser = "bacula"; dbpassword = "MY BIG SECRET"; } Unmodified from what worked on 9.4 . I also tried "dbdriver="dbi:mysql" with the same result. Could somebody just tell me what the proper incantation is here? Thank you, --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] how to tell 9.6 director to use mysql?
On Mon, Jul 20, 2020 at 4:56 PM Gary R. Schmidt wrote: > Bacula can only be built with support for one database, if you want to use > MySQL you will either have to ask the distribution maintainers to produce > one, or build Bacula yourself from source. > Has this always been the case? If so, why is there a "dbdriver" parameter in the director configuration at all, if it cannot actually be used? I could certainly buy that the Fedora package maintainer decided to switch from MySQL (which has always worked in every previous Fedora distribution since I started using Bacula back with version 5) to PostgreSQL; this sort of thing happens in Fedora. I did verify that the Bacula 9.4 packaged for Fedora 31 seems to be set for MySQL, because if I try to specify "dbdriver = postgresql" on my old 9.4/F31 box, then I get the converse error: 20-Jul 17:05 bacula-dir JobId 0: Warning: Dbdriver field within director config file "postgresql" mismatched with the Database argument "MySQL" passed during Bacula compilation. Assuming this is all correct: I think there is nearly zero chance of getting the Fedora people to produce a different package; Fedora is notoriously short of package maintainers. So I could either compile 9.6 from source, or downgrade to the Fedora 31 version that worked with MySQL. I have the additional issue to deal with that my Storage server is on a Raspbian system which of course isn't going to get new Bacula versions until the next OS upgrade, but I have found that generally different minor versions work together just fine as long as it's the same major version. I realize it's not guaranteed unless DIR = SD >= FD. Thanks for setting me straight, one way or the other I can move forward now. --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] how to tell 9.6 director to use mysql?
On Mon, Jul 20, 2020 at 5:43 PM Dmitri Maziuk wrote: > On 7/20/2020 6:31 PM, Gary R. Schmidt wrote: > > On 21/07/2020 09:21, Greg Woods wrote: > >> > >> Has this always been the case? > > Yes. > > > > Wow. I could swear I installed a backend-specific library RPM and made a > symlink or something in the version we're using. It all must've been a > figment of my imagination I guess. > Maybe not, because I found the answer. I tried downgrading as I suggested to myself above, but it didn't work. I was running the exact same RPM packages for Bacula that I was running on my older system, and it still complained that Bacula was compiled for Postgres. It turns out that Fedora has this "alternatives" system. It defines, for example, whether "/usr/bin/sendmail" will point to "sendmail" or "postfix" for mail delivery. They have done this for Bacula too; there is a library called "libbaccats.so" which is apparently linked to by their Bacula binaries, and the alternatives system can be used to determine whether this points to a MySQL, PostgreSQL, or SQLite3 version. I finally hit upon the right Google search words to find this: https://src.fedoraproject.org/rpms/bacula . Since I have always run my Director on a Fedora machine, this explains why I didn't realize Bacula was designed to be hard-compiled for a single database backend. After reading this, I knew that PostgreSQL is the default, but it can be changed by running "alternatives --config libbaccats.so" and selecting the MySQL option. The alternatives system is quite slick as can be seen here; it's easy to use one set of Bacula packages no matter which database you are using; no separate package or compiling from source necessary. But it's not always easy to figure out which things have "alternatives" and how to use them. Worse still, it appears that I must have gone down this road before, as even on my older system, Postgres is the default but it had already been set to use MySQL. I really wish my Google search fu was better, then I could have avoided all this. --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] how to tell 9.6 director to use mysql? SOLVED
On Mon, Jul 20, 2020 at 11:04 PM Davide Franco wrote: > Hello, > > I can produce Fedora 32 rpm packages if you think that can help you. > Thank you very much for the offer. But I think the alternatives system is sufficient for my needs. It is even documented. The only problem is that my Google search fu is not good enough to have found the document I needed before posting to the list. Sorry for all the noise. --Greg ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users