[Bacula-users] Rif: Re: File Table entries not pruned
As far as I understand dbcheck is to be run regularly (I run it during the day, when bacula is idle), not mailly for File table, but for Filename and Path tables. If the systems that are backuped have a lot of auto generated filenames that are lately deleted. Table and Path tables do not have a use count (at least in 1.36.3 that I am still using), so they cannot be removed during pruning or purging of jobs (unless you scan tables like dbcheck, but it would be very unefficient). -- Ferdinando Pasqualetti G.T.Dati srl Tel. 0557310862 - 3356172731 - Fax 055720143 Christoph Haas <[EMAIL PROTECTED]> Inviato da: [EMAIL PROTECTED] 25/10/2006 23.39 Per bacula-users@lists.sourceforge.net CC Oggetto Re: [Bacula-users] File Table entries not pruned On Wednesday 25 October 2006 23:18, Martin Simmons wrote: > > On Wed, 25 Oct 2006 23:00:09 +0200, Christoph Haas said: > > Actually I haven't changed my config in months. And after each full > > backup I can run dbcheck and it will remove a few ten thousand File > > entries. So I suspected that some part of Bacula misses to remove old > > database rows. And it didn't crash either. The storage daemon dies > > from time to time though. But that should keep altering the database > > directly anyway. > > Yes, it sounds like a bug to me. Which database is that? Did Bacula do > any pruning during the night? MySQL 4.1.11a on Debian Sarge. I intend to switch to PostgreSQL 8.1 anyway when I upgrade that machine to Debian Etch. I'm not entirely happy with MySQL. Yes, I found these pruning entries in the logs: 25-Oct 05:02 torf-dir: Begin pruning Jobs. 25-Oct 05:02 torf-dir: Pruned 2 Jobs for client lonestar-fd from catalog. 25-Oct 05:02 torf-dir: Begin pruning Files. 25-Oct 05:02 torf-dir: No Files found to prune. 25-Oct 05:02 torf-dir: End auto prune. 25-Oct 05:01 torf-dir: Begin pruning Jobs. 25-Oct 05:01 torf-dir: Pruned 2 Jobs for client mentors-fd from catalog. 25-Oct 05:01 torf-dir: Begin pruning Files. 25-Oct 05:01 torf-dir: No Files found to prune. 25-Oct 05:01 torf-dir: End auto prune. 25-Oct 03:06 torf-dir: Begin pruning Jobs. 25-Oct 03:06 torf-dir: Pruned 2 Jobs for client torf-fd from catalog. 25-Oct 03:06 torf-dir: Begin pruning Files. 25-Oct 03:06 torf-dir: No Files found to prune. 25-Oct 03:06 torf-dir: End auto prune. 24-Oct 21:29 torf-dir: Begin pruning Jobs. 24-Oct 21:29 torf-dir: No Jobs found to prune. 24-Oct 21:29 torf-dir: Begin pruning Files. 24-Oct 21:32 torf-dir: Pruned Files from 2 Jobs for client torf-fd from catalog. 24-Oct 21:32 torf-dir: End auto prune. 24-Oct 05:00 torf-dir: Begin pruning Jobs. 24-Oct 05:00 torf-dir: No Jobs found to prune. 24-Oct 05:00 torf-dir: Begin pruning Files. 24-Oct 05:00 torf-dir: No Files found to prune. 24-Oct 05:00 torf-dir: End auto prune. 24-Oct 05:00 torf-dir: Begin pruning Jobs. 24-Oct 05:00 torf-dir: No Jobs found to prune. 24-Oct 05:00 torf-dir: Begin pruning Files. 24-Oct 05:00 torf-dir: No Files found to prune. 24-Oct 05:00 torf-dir: End auto prune. 24-Oct 03:05 torf-dir: Begin pruning Jobs. 24-Oct 03:05 torf-dir: Pruned 0 Jobs for client torf-fd from catalog. 24-Oct 03:05 torf-dir: Begin pruning Files. 24-Oct 03:05 torf-dir: No Files found to prune. 24-Oct 03:05 torf-dir: End auto prune. 23-Oct 05:32 torf-dir: Begin pruning Jobs. 23-Oct 05:32 torf-dir: No Jobs found to prune. 23-Oct 05:32 torf-dir: Begin pruning Files. 23-Oct 05:32 torf-dir: No Files found to prune. 23-Oct 05:32 torf-dir: End auto prune. Most of the days there are neither jobs nor files pruned. > It would be useful to remove the cron job one night and run dbcheck > manually to list the orphaned File records and find the JobIds. Okay. How would I do that? Doesn't dbcheck just tell me the quantity of orphaned records? I just tried it: Select function number: 8 Checking for orphaned File entries. This may take some time! Found 176 orphaned File records. That's from last night where I incrementally backed up three hosts. 6 jobs were pruned according to the logs but no files were found to prune. > Then > try to find out when the corresponding records with the same JobId were > removed from the Job table (since that's what orphaned means for a File > record). Uhm. Any hint on what to look for? Thanks, Christoph - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - Using Tomcat
[Bacula-users] Rif: Re: File Table entries not pruned
Christoph Haas <[EMAIL PROTECTED]> Inviato da: [EMAIL PROTECTED] 30/10/2006 12.35 Per bacula-users@lists.sourceforge.net CC Oggetto Re: [Bacula-users] File Table entries not pruned On Thursday 26 October 2006 11:53, Martin Simmons wrote: > > On Wed, 25 Oct 2006 23:39:16 +0200, Christoph Haas said: > > > > On Wednesday 25 October 2006 23:18, Martin Simmons wrote: > > > > On Wed, 25 Oct 2006 23:00:09 +0200, Christoph Haas said: > > > > Actually I haven't changed my config in months. And after each > > > > full backup I can run dbcheck and it will remove a few ten > > > > thousand File entries. So I suspected that some part of Bacula > > > > misses to remove old database rows. And it didn't crash either. > > > > The storage daemon dies from time to time though. But that should > > > > keep altering the database directly anyway. > > > > > > Yes, it sounds like a bug to me. Which database is that? Did > > > Bacula do any pruning during the night? > > > > MySQL 4.1.11a on Debian Sarge. I intend to switch to PostgreSQL 8.1 > > anyway when I upgrade that machine to Debian Etch. I'm not entirely > > happy with MySQL. > > OK. Hopefully it doesn't lose SQL commands though! > > I think you should upgrade to Bacula 1.38.11 before investigating any > further, because that contained a bug fix for pruning (see > http://bugs.bacula.org/bug_view_advanced_page.php?bug_id=613). It appears like the upgrade improved things at least. My nightly dbcheck: # /usr/sbin/dbcheck -b -f -c /etc/bacula/bacula-dir.conf Checking for Paths without a trailing slash Found 0 bad Path records. Checking for Filenames with a trailing slash Found 0 bad Filename records. Checking for duplicate Filename entries. Found 0 duplicate Filename records. Checking for duplicate Path entries. Found 0 duplicate Path records. Checking for orphaned JobMedia entries. Found 0 orphaned JobMedia records. Checking for orphaned File entries. This may take some time! Found 0 orphaned File records. Checking for orphaned Path entries. This may take some time! Found 12 orphaned Path records. Deleting 12 orphaned Path records. Checking for orphaned Filename entries. This may take some time! Found 642 orphaned Filename records. Deleting 642 orphaned Filename records. Checking for orphaned FileSet entries. This takes some time! Found 1 orphaned FileSet records. Deleting 1 orphaned FileSet records. Checking for orphaned Client entries. Found 0 orphaned Client records. Checking for orphaned Job entries. Found 0 orphaned Job records. Checking for Admin Job entries. Found 0 Admin Job records. Checking for Restore Job entries. Found 0 Restore Job records. No idea why there are still orphaned entries. But it's not the ridiculous high number I had before. I already tried to give an answer to orphan Filenames and Paths entries in tables. Purging of Jobs or Files cannot erase Filenames or Paths from the database, so you need to run dbcheck on a regular basis if you keep the database clean. Maybe a use count could be manteined for each filename and pathname, so it could be decreased during the purge and if it get to zero the entry could be removed, but a regular cleanup is also effective. -- Ferdinando Pasqualetti G.T.Dati srl Tel. 0557310862 - 3356172731 - Fax 055720143 Christoph - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
