[Bacula-users] Tape shoe-shining with spooling enabled - why?
Hi all, I've been using Bacula for a number of years but recently got hold of an old tape autochanger with an LTO-4 drive, which I would like to use to have Bacula write to tape instead of my previous disk/file config. I think I have everything set up, all the tapes are labelled and Bacula switches tapes when each one fills up, however after the first tape started writing at 80MB/sec, the rest of them only write at 60-65MB/sec. Listening to the unit the drive stops, rewinds, then starts again quite a lot, so the drop in write speed appears to be due to shoe-shining. I have set up a spool file on my /tmp partition, which is a Linux tmpfs filesystem (data is stored in RAM), so there should be no speed issue reading or writing the spool file. Certainly Bacula writes to it at over 120MB/sec when the file daemon can deliver data fast enough, yet when despooling the shoe-shining occurs. The machine has no swap file (and 64GB RAM) so the data is definitely staying in memory the whole time. I am not sure why the system cannot basically write from memory to the tape at its full speed. The bacula-sd process only uses 30% of one CPU when writing, so it can't be an issue with CRC calculations (although I would've thought those would happen during spooling rather than despooling). In the manual there is some mention of setting the maximum block size to be larger to avoid shoe-shining, so I will try that when the current full backup has finished (probably another day or two at this rate) but I'm not sure why that would affect things either. Does anyone know why despooling 8GB of data from memory onto the tape drive would result in shoe-shining? Many thanks, Adam. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] "ERR=database or disk is full" while building directory tree for restore
Kern Sibbald wrote on 12/21/18 01:34: Hello Arizona EDU admin (Chandler?), Hi Kern! Yes Chandler here... (am not using the default mail client and so I have to modify "From" address to add my name each time) I don't think you mentioned what version of Bacula you are running. We are using 5.2.13 You should be aware that the project no long support SQLite as it is not suitable for backups of the size we are seeing today. We do leave the code in Bacula so that users can upgrade to MySQL or PostgreSQL at their ease rather than being forced. Thanks, I didn't realize that. I guess I should work on migrating to MySQL... our catalog is about 20GB and probably over 50 million file records. That said, perhaps some of the Bacula users can help you. Otherwise, I suggest that reading the SQLite documentation might help you find a solution. Good luck. Thanks.. Martin did come up with a work-around for now... ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] "ERR=database or disk is full" while building directory tree for restore
Martin Simmons wrote on 12/21/18 04:29: The actual work is done in bacula-dir, not bconsole, so you need to restart bacula-dir with SQLITE_TMPDIR set in its environment. Thanks Martin! That worked! I added an export for SQLITE_TMPDIR to bacula-dir init script and verified the /proc//environ has the SQLITE_TMPDIR set. When building the directory tree for the restore, the free space on / doesn't run out any longer and it is apparently using the value in SQLITE_TMPDIR, although I checked that directory and it doesn't have any contents (not even hidden 'dot' files)... but the mtime did keep changing... ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Package repos URL redirects to blog
Hello Humberto, The installation guide have been fixed, the updated documentation can be found at the same location. Best regards Davide On Thu, 20 Dec 2018 at 17:45, Humberto Fraga Ribeiro Junior < humberto.fr...@mpce.mp.br> wrote: > Hi Davide, > > Em 20/12/2018 13:33, Davide Franco escreveu: > > Hello Humberto, > > On Thu, Dec 20, 2018 at 4:27 PM Humberto Fraga Ribeiro Junior < > humberto.fr...@mpce.mp.br> wrote: > >> Hi, >> >> I'm new on the list, and I'm trying to use the community packages as >> described in >> https://blog.bacula.org/whitepapers/CommunityInstallationGuide.pdf but >> all I get is a 404 error. >> >> > failure: repodata/repomd.xml from Bacula-Community: [Errno 256] No >> more mirrors to try. >> > >> http://bacula.org/packages//rpms/9.0.8/el7/repodata/repomd.xml: >> [Errno 14] HTTPS Error 404 - Not Found >> > > The issue is now fixed, sorry again. > > Thank you, I've just tried and now I can see the packages. > > By the way, correct Bacula 9.0.8 rpm repo for rhel7-64 is here below > https://bacula.org/packages//rpms/9.0.8/el7/x86_64/ > > Looks like the whitepaper needs to be updated/fixed as it mention a wrong > url (missing architecture at the end of the url). > > Yes, the whitepaper mentions the URL without the architecture. I've > figured this out by browsing the repo. > > > Hope it helps > > Best regards > > Thank you again. > > > Davide > >> >> When I access the URL I received by mail, I get a redirection to >> https://blog.bacula.org and the 404 error. Is there a new address for >> the repos? >> >> -- >> Humberto Fraga >> >> ___ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users >> > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] "ERR=database or disk is full" while building directory tree for restore
> On Thu, 20 Dec 2018 15:55:53 -0700, admin said: > > Would anyone here happen to know how i could tell the sqlite that bacula > is using, to use a different location for temporary data? I did read > that document that Martin mentioned and tried to set > SQLITE_TMPDIR="/bacula/foo" before starting bconsole but it didn't seem > to have any effect since the / disk still filled up with temporary data > when the directory tree was being built. So I guess it is using the > PRAGMA temp_store_directory or the sqlite3_temp_directory global > variable? There isn't a /etc/sqlite.conf file or anything Anyone > know how i could update that location? Or maybe i need to ask sqlite > community? The actual work is done in bacula-dir, not bconsole, so you need to restart bacula-dir with SQLITE_TMPDIR set in its environment. __Martin ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] "ERR=database or disk is full" while building directory tree for restore
Hello Arizona EDU admin (Chandler?), I am sorry to hear you are having these problems. Although it is possible that Bacula incremental has problems, it is unlikely. There is more likely some other cause. I don't think you mentioned what version of Bacula you are running. Could you post the output from one of your successful backups. You should be aware that the project no long support SQLite as it is not suitable for backups of the size we are seeing today. We do leave the code in Bacula so that users can upgrade to MySQL or PostgreSQL at their ease rather than being forced. That said, perhaps some of the Bacula users can help you. Otherwise, I suggest that reading the SQLite documentation might help you find a solution. Good luck. Best regards, Kern On 12/20/18 11:55 PM, ad...@genome.arizona.edu wrote: Thanks all... I think this problem has also caused a problem with bacula incremental backup, perhaps the accurate information? If you look at my original post, can see the backup is quite large and a large amount of files... the incremental backup that was running (started on 12-15) was acting like a full backup... i think it was at 29 million files and over 3TB of data when i noticed something was wrong. I did free up about 1GB on the / disk and that seemed to be enough for sqlite to get by now... i am able to do a restore, and the most recent incremental backup after 12-14 was only 35GB. Would anyone here happen to know how i could tell the sqlite that bacula is using, to use a different location for temporary data? I did read that document that Martin mentioned and tried to set SQLITE_TMPDIR="/bacula/foo" before starting bconsole but it didn't seem to have any effect since the / disk still filled up with temporary data when the directory tree was being built. So I guess it is using the PRAGMA temp_store_directory or the sqlite3_temp_directory global variable? There isn't a /etc/sqlite.conf file or anything Anyone know how i could update that location? Or maybe i need to ask sqlite community? Thanks ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
