Scott, On Monday 01 October 2007 23:50, Scott Ruckh wrote: > This is what you said Josh Fisher > > > Scott Ruckh wrote: > >> I am running CentOS 4.5 x64 with self compiled kernel 2.6.13.4. I have > >> installed the latest bacula 2.2.4 Director, Client, and Storage daemon > >> on > >> this server. > >> > >> Bacula run flawlessly most of the time. > >> > >> Unfortunately when I run VMWare Workstation and have a virtual machine > >> running, Bacula crashes the entire system. I am running the latest > >> version of the 5.x series of VMWare. I am also running fluxbox as my > >> Window Manager, although I don't think that has anything to do with the > >> problem. > >> > >> This problem has existed with all versions of Bacula starting with > >> 1.3.8.11 which is the first version I installed. > >> > >> My backups are to an external USB disk connected to the Host OS. > >> > >> The virtual machine is not configured with USB port because at one time > >> I > >> thought there might be contention between the Physical USB disk and the > >> USB port configured with the Virtual Machine. > >> > >> All virtual machines and host system work fine when bacula backups are > >> not > >> running. > >> > >> I have tested this with all sorts of guest Virtual Machines and the > >> results are the same; bacula will crash (completely unusable) the host > >> machine. The only recovery method it to reboot the server. > > > > All bacula daemons are running in user mode and only the FD is running > > as root. A hard lock like this means something is amiss at the kernel > > mode level. You could try backing up to a directory on SCSI/SATA/IDE > > disk as a test. If all works as expected when backing up to the non-USB > > drive, then maybe something is wrong in the way USB is being > > virtualized/configured. > > > >> Is anyone else successfully running a similar environment? As bacula is > >> the only program that appears to cause the problem I am making an > >> assumption it is a problem with bacula. > >> > >> If anyone has this type of environment working successfully I would like > >> to hear about it. > > I had reported it was believed that not backing up the VMware disk files > of a live virtual machine solved the problem. Unfortunately I was wrong > and the problem persists. > > Now that more testing has been completed, I can officially say the problem > still exists. > > I recompiled the kernel, upgraded VMWare Workstation (to version 6), > Upgraded VMWare Tools, and rebuilt all the VMWare modules using the same > gcc that the kernel was built with, but none of that helped. I have the > exact same issue. > > Unfortunately the problem is not always present as the backups do complete > some of time. The only constant is that the hard crash only happens when > bacula is running. > > Although the VM is not configured with a USB device there might possibly > be some contention with the physical USB drive I back up to and the VM > instance. As the crash does not happen every time it is hard to determine > what exactly is going on. > > As no other programs really access the USB device, and the crash only > happens when bacula is running, I am just guessing the problem might be > with that combination. > > I know this is not really a priority on anyone's list, but I thought I > would keep everyone informed who might have been following this thread.
I'm rather late to the party, but I have an idea that this might be a RAM problem or, to be specific, that you are running out of RAM. I encountered something similar a year or two ago. The good news is that the solution is simply a matter of adjusting a kernel parameter. The kernel parameter that I had to adjust was vm.min_free_kbytes, setting it to 16224 instead of the default 4060. Take a look at your kernel logs while the backup job is running and/or run vmstat. If this is the problem you will quickly see it. I'm afraid that I don't remember the details of why this worked or how I chose that particular value. I stumbled upon some emails discussing it after googling the particular kernel error that I was receiving; made the change & then forgot about it. HTH! cmr -- Debian 'Etch' - Registered Linux User #241964 -------- "More laws, less justice." -- Marcus Tullius Ciceroca, 42 BC ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
