Re: [Bacula-users] Bacula fails to purge volume, backup hangs with "waiting for appendable volume"
On Fri, Jan 08, 2021 at 12:36:58PM +0100, Eric Bollengier via Bacula-users wrote: > > I would recommend to check the volume content on the catalog side (JobMedia > > mostly), I think that I have seen and fixed a similar issue few months ago. > (in 11.0) It > > was a loop, the volume was not purged because some orphan jobmedia still > present > > were blocking the recycling, and the volume was always selected by the Purge > Oldest algorithm. > > Hope it helps! > > Best Regards, > > Eric Hello Eric, thanks for your comment, it's much appreciated. I've now purged the volume in question manually and the error hasn't reappeared, but I guess I'll have to wait a few weeks to ensure the problem is really "fixed" as we're running on a weekly schedule (keeping three fulls and eight incrementals in two separate pools). I've also updated the instance to version 9.6.7 (from source) which had the problem once with a different client, but so far it's been smooth sailing. All the best, Uwe -- Uwe Schürkamp | email: Arvato Systems S4M GmbH | Sitz Köln | Amtsgericht Köln HRB 27038 Geschäftsführer: Ralf Schürmann | Dr. Manfred Heinen ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula fails to purge volume, backup hangs with "waiting for appendable volume"
Hello Uwe, On 1/8/21 10:09 AM, Uwe Schuerkamp wrote: Hi folks, I'm experiencing a weird issue with one of our bacula servers (9.6.5 on ubuntu 18.04 compiled from source). Most of the backups work just fine, however for one client they fail / hang consistently as bacula fails to complete purging of a volume from the pool (I use separate storages & pools for fulls and incrementals, both disk-based). I'm seeing this message in bconsole under these circumstances: 08-Jan 07:13 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:18 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:23 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:28 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:33 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:38 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:43 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:48 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:53 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:58 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:03 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:08 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:13 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:18 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:23 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:28 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:33 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:38 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:43 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:48 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:53 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:58 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 09:03 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 09:08 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" The volume isn't too big and purging it manually takes only a few seconds. I would recommend to check the volume content on the catalog side (JobMedia mostly), I think that I have seen and fixed a similar issue few months ago. (in 11.0) It was a loop, the volume was not purged because some orphan jobmedia still present were blocking the recycling, and the volume was always selected by the Purge Oldest algorithm. Hope it helps! Best Regards, Eric ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula fails to purge volume, backup hangs with "waiting for appendable volume"
Hi folks, I'm experiencing a weird issue with one of our bacula servers (9.6.5 on ubuntu 18.04 compiled from source). Most of the backups work just fine, however for one client they fail / hang consistently as bacula fails to complete purging of a volume from the pool (I use separate storages & pools for fulls and incrementals, both disk-based). I'm seeing this message in bconsole under these circumstances: 08-Jan 07:13 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:18 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:23 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:28 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:33 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:38 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:43 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:48 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:53 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 07:58 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:03 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:08 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:13 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:18 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:23 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:28 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:33 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:38 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:43 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:48 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:53 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 08:58 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 09:03 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" 08-Jan 09:08 director-dir JobId 3988: Purging oldest volume "clientXXX-0284" The volume isn't too big and purging it manually takes only a few seconds. When looking at the pool at the time of the hang, the volume status is still shown as "used" for the volume that's being purged. Once I manually purge the volume in bconsole that bacula is waiting for and mount the storage manually, the backup job continues & finishes normally. DB backend is 10.1.47-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04. I'd be most grateful for any ideas on what could be causing this as all other backups (about 50 clients or so, all of them disk-based) work without issues; the same can be said for our other bacula instances (about 500 clients in total with 3 active directors). Thanks much in advance for your help & all the best, Uwe -- Uwe Schürkamp | email: Arvato Systems S4M GmbH | Sitz Köln | Amtsgericht Köln HRB 27038 Geschäftsführer: Ralf Schürmann | Dr. Manfred Heinen ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
