Re: [Bacula-users] Trying to restore, but can't

[Ryan Butler]

It's attempting to restore directly from a single FULL backup job.

I know that once it finishes generating the directory listing that I can mark 
the directory I want and then it will go ahead and restore everything in it 
including subfolders. However, option 11, from the online manual says "Item 11 
allows you to enter a list of JobIds from which you can select directories to 
be restored. You may then enter a full path to a directory name.. All 
files in those directories will be restored, but if the directory contains 
subdirectories, nothing will be restored in the subdirectory unless you 
explicitly enter its name."

Either way, this is fine, I just really need to know if using option 11 has it 
generating a directory listing the same as the option 3 that I am using, 
because this is the part that it's taking forever on, and I'm really just 
looking for a way to get this directory restored as quickly as possible. Once 
I've gotten the directory restored, I'm going to look into improving restore 
times. (This is a Bacula system that I inherited from a previous sysadmin)

The server that Bacula is running on has 12 cores and 16GB of RAM. This 
directory listing generation has so far been running for 28 hours. Is there any 
hope of this finishing or are there any suggestions?

Thanks!

-ryan

-Original Message-
From: Phil Stracchino  
Sent: Tuesday, June 12, 2018 3:25 PM
To: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Trying to restore, but can't

On 06/12/18 12:35, Ryan Butler wrote:

> I really just need to restore a single directory (and all of its files 
> and subdirectories), so I've been looking at option 11 (enter a list 
> of directories to restore for jobid), and it looks like I'd have to 
> manually specify all the subdirectories, which is doable. However, 
> would that option also need to "Build directory tree", and therefore 
> I'd be back at the original problem?

If you tell it to restore a directory, everything under that directory will 
automatically be included.


> Here's the details on that particular Job:
> 
> JobFiles: 2,927,789
> 
> JobBytes: 54,769,194,913 (54 GB)


How many incrementals/differentials since your last FULL?  That's almost three 
million files in that Job.  If it's got to scan a large number of
jobs ...   yeah, that could take a while.


--
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most engaging tech 
sites, Slashdot.org! http://sdm.link/slashdot 
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Trying to restore, but can't

[Dimitri Maziuk via Bacula-users]

On 06/12/2018 11:35 AM, Ryan Butler wrote:

> We're on version 7.0.5, which I know is old, and I have plans to update it 
> after this disaster.
> 
> Here's the details on that particular Job:
> 
> JobFiles: 2,927,789

Version has nothing to do with it. There are several SQL joins involved
in getting from job to file, plus as Phil said: through all incrementals
to the last full. You simply need a database and hardware that can do
that on 3M files in a tolerable timeframe.

My production machines have 16-24 cores and 64-256GB RAM. And postgres.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Trying to restore, but can't

[Phil Stracchino]

On 06/12/18 12:35, Ryan Butler wrote:

> I really just need to restore a single directory (and all of its files
> and subdirectories), so I’ve been looking at option 11 (enter a list of
> directories to restore for jobid), and it looks like I’d have to
> manually specify all the subdirectories, which is doable. However, would
> that option also need to “Build directory tree”, and therefore I’d be
> back at the original problem?

If you tell it to restore a directory, everything under that directory
will automatically be included.


> Here’s the details on that particular Job:
> 
> JobFiles: 2,927,789
> 
> JobBytes: 54,769,194,913 (54 GB)


How many incrementals/differentials since your last FULL?  That's almost
three million files in that Job.  If it's got to scan a large number of
jobs ...   yeah, that could take a while.


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Trying to restore, but can't

[Ryan Butler]

Hello all,

After a failed upgrade to one of our organization's web apps, I'm trying to 
restore the web app directory from Bacula. I went into bconsole, restore, 
option 3 (enter jobid), and have been sitting on "Building directory tree for 
JobId(s) 56810 ..." for the past 12 hours. The MySQL process is using 100% CPU 
on one core, but other than that, I have no indication that anything is 
happening. And this is something that I need to figure out why it is happening, 
but what I really need right now is to just get the web app restored so users 
don't start getting angry.

I really just need to restore a single directory (and all of its files and 
subdirectories), so I've been looking at option 11 (enter a list of directories 
to restore for jobid), and it looks like I'd have to manually specify all the 
subdirectories, which is doable. However, would that option also need to "Build 
directory tree", and therefore I'd be back at the original problem?

We're on version 7.0.5, which I know is old, and I have plans to update it 
after this disaster.

Here's the details on that particular Job:

JobFiles: 2,927,789
JobBytes: 54,769,194,913 (54 GB)

Any help is appreciated!!!

Ryan Butler
Systems Administrator

[/Users/ryan_butler/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1633926894]

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?

[Kern Sibbald]

Hello Dan,

Thank you for you kind remarks.

Just for information: In reality Bacula uses the block sizes you 
configure (or the default).  You can cause Bacula to write fixed sized 
blocks, but modern tape drives  (LTO and similar) handle variable sizes 
quite well, so by default, Bacula will attempt to block everything to 
the maximum block size set, but if there is a reason (e.g. last block of 
a job), Bacula will write smaller blocks.  This has never been a problem 
to the best of my knowledge.  I suspect that tar does not check all 
return statuses as well as it should.  Baculla's concept is to do 
everything possible (not all is practical though) to ensure that writes 
are valid so that restores will work.


Best regards,
Kern

On 06/11/2018 09:44 PM, Stieneke, Dan wrote:

Thanks all for your input & confirming it pretty much had to be a hardware 
problem.

In the interest of completeness / helping the next person who's googling for 
answers, reseating the SCSI card fixed it - it just completed a 900GB backup 
w/out any problems, onto one of the same tapes that it had rejected before 
after only a few GB.

Now that the major problem has been solved, I'm still curious about why Bacula ran into 
the (real!) hardware issue where tar did not. The tar tape was software compressed 
& then software encrypted, so the restore had to successfully decrypt & then 
decompress the data, so there couldn't have been any bit errors on that tar tape. This 
was true four months ago, with the sketchy cable, and this time, with the SCSI card 
that needed re-seated. Are fixed-size (tar) blocks just a little bit more robust than 
variable-sized (Bacula) blocks?

And thanks, Kern, for an outstanding product.

Dan Stieneke
IT Specialist
USDA - ARS - NWISRL
3793 N 3600 E
Kimberly, ID 83341
208/423-6519





  
-Original Message-

From: Kern Sibbald [mailto:k...@sibbald.com]
Sent: Saturday, June 9, 2018 4:16 AM
To: Stieneke, Dan 
Cc: bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?

Hello,

Well, Bacula does not check what was written from time to time, but when it reaches the 
end of the tape, Bacula will re-read the last block written to make sure it corresponds 
to what it wrote, then it writes a double end of file.  In your case, something is going 
wrong -- either there is a hardware error, or there is really an end of tape marker that 
is telling Bacula that the tape is full.  From what you write, it looks more like a 
hardware error, and the kernel logs that you show below indicate that something serious 
is wrong with your tape drive.  While Bacula is writing you should never see such 
messages, and when they occur, Bacula will receive a write error.  Everything is 
consistent with a hardware problem.  You may get a better idea of what is going on by 
running the "btape test" command.  Please see the manual for instructions on 
how to run it.  I recommend both the test, and the fill commands.  Note: both of these 
commands will write on the tape.  Prior to using a tape with btape, if it has been 
labeled by Bacula, you should rewind the tape and write one or two eof marks at the 
beginning so that btape will take it as a blank tape.

If both btape "test" and "fill" work, you should not have problems with failing 
Bacula backups.  If either one of those tests fail, you must fix it prior to trying to backup on 
tape with Bacula.

Best regards,
Kern

On 06/08/2018 07:48 PM, Stieneke, Dan wrote:

@ Dan Langille - yes, I think it is an issue with the tape drive, but only 
Bacula runs into it; tar does not.

@Martin Simmons - of course I should have checked/reported the log, sorry.
===BEGIN SYSLOG
==
 Jun  4 08:06:11 SRVName kernel: [410468.465702] st0: Sense
Key : Unit Attention [current] Jun  4 08:06:11 SRVName kernel:
[410468.465714] st0: Add. Sense: Power on, reset, or bus device reset
occurred Jun  4 08:10:47 SRVName kernel: [410744.629015] st0: Sense
Key : Unit Attention [current] Jun  4 08:10:47 SRVName kernel:
[410744.629026] st0: Add. Sense: Power on, reset, or bus device reset
occurred Jun  4 08:14:02 SRVName kernel: [410939.819168] st0: Sense
Key : Unit Attention [current] Jun  4 08:14:02 SRVName kernel:
[410939.819180] st0: Add. Sense: Power on, reset, or bus device reset
occurred Jun  4 08:16:57 SRVName kernel: [44.538975] st0: Sense
Key : Unit Attention [current] Jun  4 08:16:57 SRVName kernel:
[44.538988] st0: Add. Sense: Power on, reset, or bus device reset
occurred ===END SYSLOG
==


Googling for those entries I found 
http://bacula.10910.n7.nabble.com/Bacula-tapes-marked-FULL-too-early-VolBytes-too-low-td58881i20.html. 
Similar issue (but no report of tar), the thread ended with "similar problem went away with 
replaced drive" & "get your drive tested"

  From the 

Re: [Bacula-users] [Bacula-devel] Bacula Status Report -- 9 June 2018

[Davide Franco]

Hello Wanderlei,

There's nothing to worry about the Debian repository you've mentioned

If you you compare with a "standard" Debian repo, the major difference is
that Bacula version and your personal "key" has been added to the url.
Nothing much more.

Debian Stretch repository
http://ftp.ch.debian.org/debian/dists/Debian9.4/main/binary-amd64/

Bacula repository for Debian Stretch
http://bacula.org/packages/X/debs/9.0.8/
stretch/amd64/dists/stretch/main/binary-amd64/

Hope it helps.

Best regards

Davide

On Mon, Jun 11, 2018 at 1:11 AM, Wanderlei Huttel  wrote:

> Hello Kern
>
> Can you take a look in the debian repo.
> It looks that the debian directory are "duplicated" (too many folders
> comparing with centos repository)
>
> Debian:
> http://bacula.org/packages/X/debs/9.0.8/
> stretch/amd64/dists/stretch/main/binary-amd64/
>
> CentOS
> http://bacula.org/packages/ X/ rpms/9.0.8/el7/x86_64/
>
>
> Best Regards
>
> *Wanderlei Hüttel*
> http://www.huttel.com.br
>
>
> Em sáb, 9 de jun de 2018 às 10:33, Kern Sibbald 
> escreveu:
>
>> Hello,
>>
>> It has been a while since the last status report, so it is now time.
>>
>> I would like to discuss the following subjects:
>>
>> 1. Bacula Release Status
>>
>> 2. Bacula Binaries (New)
>>
>> 3. Windows Binaries (more coming)
>>
>> 4. Next Bacula Version
>>
>> ===
>>
>> 1. Bacula Release Status
>> As probably know, the latest Bacula version 9.0.8 was released recently
>> to Source Forge and will be released to www.bacula.org shortly. Mostly
>> this release corrects some copyright oversights. However, since version
>> 9.0.0, we have had several minor releases, so most of the minor
>> annoyances, including a few long time bugs are behind us. Just the same
>> there are quite a few additional fixes and many new features coming in
>> the next release (see below).
>>
>> 2. Bacula Binaries
>> We are pleased to announce that Bacula binaries have been built for a
>> few of the most popular platforms. This is thanks to the build support
>> from Bacula Systems. We will start with a small number most popular
>> binaries and over time expand it to include a broader base of OS
>> releases.  In this release is the Aligned driver plugin for the Storage
>> daemon, which permits Bacula to write an archive volume that is properly
>> aligned so that it can bee deduplicated by filesystems such as ZFS and
>> NetApp.
>>
>> Please note that the release format is with standard packaging using
>> debs rpms, ... that are specific to each system. However, unlike most
>> Linux distributions, we install the Community Binaries into the
>> directory /opt/bacula as recommended in the Bacula manual and as used by
>> the Bacula Enterprise Edition. This Bacula file release standard is not
>> uncommon and vastly simplifies installation, support, backup and
>> recovery of Bacula itself. Basically, most of the release will be
>> installed into /opt/bacula except for the few system files that must be
>> placed in certain locations such as the start/stop scripts and the man
>> text.
>>
>> One change is that we ask you to register to get access to the binaries.
>> This is very similar to signing up for the email lists. We have always
>> been interested in who is using Bacula, and provided you supply correct
>> information, this registration will help. You will be asked a few simple
>> questions and also be given an opportunity to receive the Bacula
>> Enterprise Newsletter. If you decline the newsletter, you will not be
>> bothered. The advantage of getting the newsletter is you will be
>> informed what is coming in the next Enterprise releases, which means
>> that most of those features will follow into a subsequent community
>> release.
>>
>> To register and get access to Bacula pre-built binaries please go to
>> www.bacula.org and on the main menu bar select Downloads -> Bacula
>> Binary Package Download, then fill out the registration.
>>
>> 3. Windows Binaries
>> Since version 9.0.0, we have re-integrated the Windows File daemon
>> source code, and built Windows Binaries packages.  They can be found in
>> www.bacula.org -> Downloads -> Binary Download Center. Version 9.0.8
>> binaries will be uploaded shortly.
>>
>> 4. Next Bacula Version
>> We have begun working on the next version of Bacula. It may be released
>> as version 9.2.0 or possibly 11.0.0. The major part of this new release
>> consists of backporting more Bacula Enterprise features -- for example,
>> getting the tray-monitor to build correctly as well as many performance
>> improvements.
>>
>> Thanks for using Bacula — be happy.
>>
>> Kern
>>
>>
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Bacula-devel mailing list
>> bacula-de...@lists.sourceforge.net
>> 

[Bacula-users] Differential job not performing as expected

[Chris Wilkinson]

Hello

I have a puzzling result that I hope someone might be able to throw some
light on.

Having completed a full backup successfully, I then run a differential
backup. Everything is the same except the backup level=diff. Since no files
were modified, I expected the job size to be quite small.

Estimating the diff job with Baculum shows the size to be = to the full
backup size.

The log reports that no prior full backup was found in the catalog and so
upgrades the job automatically to full. I aborted this before completion.

Best
Chris Wilkinson

11-Jun 20:44 homeserver-dir JobId 9: No prior Full backup Job record found.
11-Jun 20:44 homeserver-dir JobId 9: No prior or suitable Full backup found
in catalog. Doing FULL backup.
11-Jun 20:44 homeserver-dir JobId 9: Start Backup JobId 9,
Job=homeserver-diff.2018-06-11_20.44.40_29
11-Jun 20:44 homeserver-dir JobId 9: Using Device "homeserver-dev" to write.
11-Jun 20:44 homeserver-sd JobId 9: Volume "homeserver0002" previously
written, moving to end of data.
11-Jun 20:45 homeserver-sd JobId 9: Fatal error: append.c:184 Error reading
data header from FD. n=-2 msglen=0 ERR=Interrupted system call
11-Jun 20:45 homeserver-dir JobId 9: Bacula homeserver-dir 9.0.6 (20Nov17):
  Build OS:   x86_64-pc-linux-gnu debian 9.4
  JobId:  9
  Job:homeserver-diff.2018-06-11_20.44.40_29
  Backup Level:   Full (upgraded from Differential)
  Client: "homeserver-fd" 9.0.6 (20Nov17)
x86_64-pc-linux-gnu,debian,9.4
  FileSet:"chrisw" 2018-06-08 11:14:36
  Pool:   "file_homeserver" (From Command input)
  Catalog:"MyCatalog" (From Client resource)
  Storage:"homeserver-sd" (From Command input)
  Scheduled time: 11-Jun-2018 20:44:40
  Start time: 11-Jun-2018 20:44:43
  End time:   11-Jun-2018 20:45:45
  Elapsed time:   1 min 2 secs
  Priority:   10
  FD Files Written:   7,033
  SD Files Written:   0
  FD Bytes Written:   750,430,240 (750.4 MB)
  SD Bytes Written:   0 (0 B)
  Rate:   12103.7 KB/s
  Software Compression:   None
  Comm Line Compression:  None
  Snapshot/VSS:   no
  Encryption: no
  Accurate:   no
  Volume name(s):
  Volume Session Id:  1
  Volume Session Time:1528731294
  Last Volume Bytes:  47,769,180,975 (47.76 GB)
  Non-fatal FD errors:0
  SD Errors:  0
  FD termination status:  Canceled
  SD termination status:  Canceled
  Termination:Backup Canceled

11-Jun 20:49 homeserver-dir JobId 0: Error: bsock.c:544 Bsock send while
terminated=1 on call to client:192.168.1.126:9101
11-Jun 20:49
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Upgraded to Fedora 29 and got an unresolved dependency (libbaccats)

[David While]

Did a

dnf --enablerepo=updates-testing update bacula*

and it appeared OK except for the following:

Running scriptlet: bacula-libs-9.0.6-2.fc27.x86_64  
13/14

/sbin/ldconfig: relative path `1' used to build cache
warning: %postun(bacula-libs-9.0.6-2.fc27.x86_64) scriptlet failed, exit 
status 1

Non-fatal POSTUN scriptlet failure in rpm package bacula-libs
Non-fatal POSTUN scriptlet failure in rpm package bacula-libs


On 11/06/2018 13:05, Simone Caronni wrote:

I've created new updates, they are here:

https://bodhi.fedoraproject.org/updates/FEDORA-2018-98ccf382ee
https://bodhi.fedoraproject.org/updates/FEDORA-2018-5156ca7a45

Just wait until they reach testing and do a:

dnf --enablerepo=updates-testing update bacula*

Or just pick up stuff from here:

https://koji.fedoraproject.org/koji/packageinfo?packageID=4820

If you are registered, please leave feedback.

Thanks,
--Simone






On Mon, Jun 11, 2018 at 1:50 PM, Bill Damage > wrote:


Dammit - of course the title to all this should have read
"Upgraded to Fedora 28". F29 isn't relevant at all.

On Monday, 11 June 2018, 11:54:10 BST, Bill Damage via
Bacula-users mailto:bacula-users@lists.sourceforge.net>> wrote:


Hope this helps - from a different installation, running
perfectly, a routine "dnf update" just gave this, so there's
definitely a problem with these dependencies:

[root@sabbath ~]# dnf update
Fedora 28 - x86_64 - Updates          2.1 MB/s |  16 MB  00:07
Fedora 28 - x86_64            2.1 MB/s |  60 MB  00:28
google-chrome            57 kB/s | 3.7 kB  00:00
Opera packages             91 kB/s |  11 kB  00:00
Fedora 28 - x86_64 - VirtualBox            90 kB/s |  62 kB  00:00
vivaldi           200 kB/s | 9.8 kB  00:00
Webmin Distribution Neutral           1.6 MB/s | 8.7 MB  00:05
Last metadata expiration check: 0:00:00 ago on Mon 11 Jun 2018
11:46:22 BST.
Dependencies resolved.

 Problem 1: cannot install the best update candidate for package
bacula-director-9.0.6-8.fc28.x86_64
  - nothing provides libbaccats-9.0.7.so
()(64bit) needed by
bacula-director-9.0.7-3.fc28.x86_64
 Problem 2: cannot install the best update candidate for package
bacula-storage-9.0.6-8.fc28.x86_64
  - nothing provides libbaccats-9.0.7.so
()(64bit) needed by
bacula-storage-9.0.7-3.fc28.x86_64
 Problem 3: problem with installed package
bacula-storage-9.0.6-8.fc28.x86_64
  - package bacula-storage-9.0.6-8.fc28.x86_64 requires
bacula-common(x86-64) = 9.0.6-8.fc28, but none of the providers
can be installed
  - cannot install both bacula-common-9.0.7-3.fc28.x86_64 and
bacula-common-9.0.6-8.fc28.x86_64
  - cannot install both bacula-common-9.0.6-8.fc28.x86_64 and
bacula-common-9.0.7-3.fc28.x86_64
  - cannot install the best update candidate for package
bacula-common-9.0.6-8.fc28.x86_64
  - nothing provides libbaccats-9.0.7.so
()(64bit) needed by
bacula-storage-9.0.7-3.fc28.x86_64
 Problem 4: problem with installed package
bacula-director-9.0.6-8.fc28.x86_64
  - package bacula-director-9.0.6-8.fc28.x86_64 requires
bacula-libs(x86-64) = 9.0.6-8.fc28, but none of the providers can
be installed
  - package bacula-director-9.0.6-8.fc28.x86_64 requires
libbac-9.0.6.so ()(64bit), but none of the
providers can be installed
  - cannot install both bacula-libs-9.0.7-3.fc28.x86_64 and
bacula-libs-9.0.6-8.fc28.x86_64
  - cannot install both bacula-libs-9.0.6-8.fc28.x86_64 and
bacula-libs-9.0.7-3.fc28.x86_64
  - cannot install the best update candidate for package
bacula-libs-9.0.6-8.fc28.x86_64
  - nothing provides libbaccats-9.0.7.so
()(64bit) needed by
bacula-director-9.0.7-3.fc28.x86_64

===
 Package                         Arch Version        Repository  
        Size

===
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 bacula-common                         x86_64 9.0.7-3.fc28        
updates            36 k
 bacula-libs                         x86_64 9.0.7-3.fc28        
updates           501 k
Skipping packages with broken dependencies:
 bacula-director                         x86_64 9.0.7-3.fc28      
  updates           400 k
 bacula-storage                        x86_64 9.0.7-3.fc28        
updates           185 k

Transaction Summary