Re: [Bacula-users] Bacula in the cloud

[Uwe Schuerkamp]

Hello Jason,

On Mon, Oct 17, 2016 at 09:37:12PM -0500, Jason Voorhees wrote:
> Hello guys:
> 
> Based on your experience, what alternative do we have for backing up
> information to the cloud preferably using Bacula?
> 

I wrote a script a while ago that runs as a RunAfterJob element which
encrypts (gpg) and copies a full backup of a client (or its disk
volume rather) to an S3 bucket using the aws shell client.

It's still very rudimentary but it does the job nicely when it comes
to keeping a full backup safe (and secure) from a local disaster.

I seem to recall "cloud support" (whatever that may mean in today's
buzzword bingo) was announced for Bacula 8.

All the best,

Uwe

-- 
Uwe Schürkamp | email: 








--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] The number of files mismatch.

[Gi Dot]

Hi all,

It is quite frequent that my backup job ending up with the "The number of
files mismatch!" error. What could be the root cause of this?

Full error from log:
17-Oct 17:05 srv2-dir JobId 54: shell command: run BeforeJob
"/usr/lib64/bacula/make_catalog_backup.pl MyCatalog"
17-Oct 17:05 srv2-dir JobId 54: Start Backup JobId 54,
Job=ApplicationBackup_Server2.2016-10-17_17.05.04_04
17-Oct 17:05 srv2-dir JobId 54: Using Device "ibm-dds6"
17-Oct 17:05 srv2-sd JobId 54: Volume "srv2-set1-sun" previously written,
moving to end of data.
17-Oct 17:05 srv2-sd JobId 54: Error: Bacula cannot write on tape Volume
"srv2-set1-sun" because:
The number of files mismatch! Volume=0 Catalog=1
17-Oct 17:05 srv2-sd JobId 54: Marking Volume "srv2-set1-sun" in Error in
Catalog.
17-Oct 17:05 srv2-sd JobId 54: Please mount Volume "srv2-set2-sun" or label
a new one for:
Job:  ApplicationBackup_Server2.2016-10-17_17.05.04_04
Storage:  "ibm-dds6" (/dev/nst0)
Pool: SundayPool
Media type:   DDS6


I am using IBM DDS6 tape. My sd, pool and job config:
https://dpaste.de/QBW9

Thanks.
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula-7.4.4 in db version 14

[Ralf Brinkmann]

Am 14.10.2016 um 12:50 schrieb Personal Técnico:
> Hi,
>
> It seems in version 15, database has some news fields for SSL. If I
> configure bacula with ""--without-openssl", I suppose that new
> parameters take "null" value, but when director tries to connect to
> bacula database, I get some errors from sslkey, sslcert, etc.
>
> thanks.

I just took a look into the bacula host configuration - we have the
openssl devel pack included without making use of openssl.

-- 
Ralf Brinkmann


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] The number of files mismatch.

[Kern Sibbald]

  
  
When the volume has less files than the
  catalog, you are in a *very* bad situation.
  
  Perhaps you never ran the btape test command to ensure that your
  tape drive was
  configured correctly, because it looks like Bacula thinks there is
  nothing on your tape.
  
  Best regards,
  Kern
  
  On 10/18/2016 10:50 AM, Gi Dot wrote:


  

  
Hi all,
  

It is quite frequent that my backup job ending up with the
"The number of files mismatch!" error. What could be the
root cause of this?

  
  Full error from log:
17-Oct 17:05 srv2-dir JobId 54: shell command: run BeforeJob
"/usr/lib64/bacula/make_catalog_backup.pl
MyCatalog"
17-Oct 17:05 srv2-dir JobId 54: Start Backup JobId 54,
Job=ApplicationBackup_Server2.2016-10-17_17.05.04_04
17-Oct 17:05 srv2-dir JobId 54: Using Device "ibm-dds6"
17-Oct 17:05 srv2-sd JobId 54: Volume "srv2-set1-sun"
previously written, moving to end of data.
17-Oct 17:05 srv2-sd JobId 54: Error: Bacula cannot write on
tape Volume "srv2-set1-sun" because:
The number of files mismatch! Volume=0 Catalog=1
17-Oct 17:05 srv2-sd JobId 54: Marking Volume
"srv2-set1-sun" in Error in Catalog.
17-Oct 17:05 srv2-sd JobId 54: Please mount Volume
"srv2-set2-sun" or label a new one for:
    Job: 
ApplicationBackup_Server2.2016-10-17_17.05.04_04
    Storage:  "ibm-dds6" (/dev/nst0)
    Pool: SundayPool
    Media type:   DDS6

  
  
  
  I am using IBM DDS6 tape. My sd, pool and job config:
  https://dpaste.de/QBW9
  

Thanks.


  
  
  
  
  --
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Josh Fisher]

On 10/18/2016 3:42 AM, Uwe Schuerkamp wrote:
> Hello Jason,
>
> On Mon, Oct 17, 2016 at 09:37:12PM -0500, Jason Voorhees wrote:
>> Hello guys:
>>
>> Based on your experience, what alternative do we have for backing up
>> information to the cloud preferably using Bacula?
>>
> I wrote a script a while ago that runs as a RunAfterJob element which
> encrypts (gpg) and copies a full backup of a client (or its disk
> volume rather) to an S3 bucket using the aws shell client.
>
> It's still very rudimentary but it does the job nicely when it comes
> to keeping a full backup safe (and secure) from a local disaster.
>
> I seem to recall "cloud support" (whatever that may mean in today's
> buzzword bingo) was announced for Bacula 8.

I tend to think that will be targeting local cloud storage, for example 
ownCloud, in enterprise environments. I'm not sure something like S3 is 
very useful for direct backup storage over the Internet. A 1 TB backup 
over a 100 Mbps connection would take a minimum of 22+ hours, assuming 
maximum throughput and that S3 could actually sustain 12.5 MB/s.

For S3, copying via a script seems the best way to go.

>
> All the best,
>
> Uwe
>


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Kern Sibbald]

Hello,

Bacula Systems has a White Paper on Bacula Enterprise Edition in the 
cloud, and they have given me permission to publish it. However, as it 
is currently written for Bacula Enterprise customers it needs some 
modification, which I will make over the next week or so then release it.

It discusses a number of different ways that Bacula can work with the 
cloud, so you all might find it very interesting.  Obviously one of the 
current limitations for most people (like me) who do not have a big 
budget for high-speed fiber optic Internet connections is the upload 
speed.  I have spent a lot of time thinking about this, and I think 
there are a number of very interesting solutions that will become 
available in the near future.

Best regards,
Kern

On 10/18/2016 01:45 PM, Josh Fisher wrote:
> On 10/18/2016 3:42 AM, Uwe Schuerkamp wrote:
>> Hello Jason,
>>
>> On Mon, Oct 17, 2016 at 09:37:12PM -0500, Jason Voorhees wrote:
>>> Hello guys:
>>>
>>> Based on your experience, what alternative do we have for backing up
>>> information to the cloud preferably using Bacula?
>>>
>> I wrote a script a while ago that runs as a RunAfterJob element which
>> encrypts (gpg) and copies a full backup of a client (or its disk
>> volume rather) to an S3 bucket using the aws shell client.
>>
>> It's still very rudimentary but it does the job nicely when it comes
>> to keeping a full backup safe (and secure) from a local disaster.
>>
>> I seem to recall "cloud support" (whatever that may mean in today's
>> buzzword bingo) was announced for Bacula 8.
> I tend to think that will be targeting local cloud storage, for example
> ownCloud, in enterprise environments. I'm not sure something like S3 is
> very useful for direct backup storage over the Internet. A 1 TB backup
> over a 100 Mbps connection would take a minimum of 22+ hours, assuming
> maximum throughput and that S3 could actually sustain 12.5 MB/s.
>
> For S3, copying via a script seems the best way to go.
>
>> All the best,
>>
>> Uwe
>>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[C M Reinehr]

On 10/17/2016 09:37 PM, Jason Voorhees wrote:
> Hello guys:
>
> Based on your experience, what alternative do we have for backing up
> information to the cloud preferably using Bacula?
>
> I've been reading some posts about similar topics. Bandwidth always
> seem to be a problem because it isn't to big (Gigs per second) or
> there's to much information (several terabytes) and Bacula can't run
> jobs for so long without modifying source code and recompiling.
>
> I've been thinking something alternatives like these:
>
> 1. Backup to local disk and configure Copy jobs to make a copy to
> Amazon S3. Local backups can run always fast but Copy jobs might be
> delayed ... without issues?
>
> 2. Configure Amazon Storage Gateway as VTL so Bacula can backup
> directly to Amazon using virtual tape devices through iSCSI. What do
> you think about this?
>
> 3. For a single fileserver to be backed up (let's say a Samba server),
> I could create a replica in the cloud (i.e. Amazon EC2) which can be
> constantly synchronized (via rsync) and run Bacula locally in such EC2
> instance.
>
> What other ideas have you thought? Maybe a combination of other open
> source tools that can be combined with Bacula? or maybe a different
> open source solution that replaces Bacula to save backups in the
> cloud?
>
> I'd appreciate some ideas, pros and/or cons to be discussed.
>
> Thanks in advance for your time bats!
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>

My configuration, here, is to do daily backups at night to a disk array. 
The backup files then are mirrored to an on-line cloud storage service 
called SpiderOak. I keep several days worth of backups on the disk array 
& then rely on the SpiderOak files for long term storage. In this way, I 
completely have eliminated tape backups.

CMR
-- 
Linux distribution Debian v8.5, "Jessie"
--
"Firearms are second only to the Constitution in importance; they are 
the peoples' liberty teeth." -- George Washington


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Multiple autochanger devices

[Eddie Appel]

Hi all,

Looking for a bit of suggestion or direction here. Keep in mind, RTFM is a 
valid response so long as you provide some hints as to which M to R where. 😀

I have a pretty straight forward setup using bacula 7 on centos 6 and two tape 
libraries attached via FC to the host. Using a single client to backup several 
nfs/cifs mounts to lto5 tape.

This setup has given me some three years of mostly headache free backups. What 
i have been unable to accomplish is having both changers available for all jobs 
coming from this single client.

Couple of questions,

Should I be using multiple clients or storage daemons?

I already have the single sd configured to use both devices, how can i tell dir 
or client to pick a free tape device from ANY changer when client starts job?

Thanks all for your time and consideration!

--eddie

Cloud Services for Business www.j2.com
j2 | eFax | eVoice | FuseMail | Campaigner | KeepItSafe | Onebox


This email, its contents and attachments contain information from j2 Global, 
Inc. and/or its affiliates which may be privileged, confidential or otherwise 
protected from disclosure. The information is intended to be for the 
addressee(s) only. If you are not an addressee, any disclosure, copy, 
distribution, or use of the contents of this message is prohibited. If you have 
received this email in error please notify the sender by reply e-mail and 
delete the original message and any copies. (c) 2015 j2 Global, Inc. All rights 
reserved. eFax, eVoice, Campaigner, FuseMail, KeepItSafe, and Onebox are 
registered trademarks of j2 Global, Inc. and its affiliates.
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Jason Voorhees]

Thank you all for your responses.

I'll take a look at Bacula systems' whitepaper to see what they're
talking about. Meanwhile I'll explore some of the alternatives
discussed on this thread like copying files with scripts and making a
replica on SpiderOak or anything similar.

I hope we can have an interesting solution for this "problem" in the
near future.

Thanks again for your time :)

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Josip Deanovic]

On Tuesday 2016-10-18 12:34:08 Jason Voorhees wrote:
> Thank you all for your responses.
> 
> I'll take a look at Bacula systems' whitepaper to see what they're
> talking about. Meanwhile I'll explore some of the alternatives
> discussed on this thread like copying files with scripts and making a
> replica on SpiderOak or anything similar.
> 
> I hope we can have an interesting solution for this "problem" in the
> near future.


Hi Jason!

You have said that "Bacula can't run jobs for so long without modifying
source code and recompiling".

What did you mean by that and can you give an example of the problem
you have experienced?

I am asking because I am not aware of the bacula's job duration related
limitations.

-- 
Josip Deanovic

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Clark, Patti]

From Bacula’s main.pdf documentation:

Max Run Time =  The time specifies the maximum allowed time that a job 
may run, counted from when the job starts, (not necessarily the same as when 
the job was scheduled).
By default, the the watchdog thread will kill any Job that has run more than 6 
days. The maximum watchdog timeout is independent of MaxRunTime and cannot be 
changed.


Patti Clark
Linux System Administrator
R&D Systems Support Oak Ridge National Laboratory

From: Josip Deanovic 
Date: Tuesday, October 18, 2016 at 4:06 PM
To: "bacula-users@lists.sourceforge.net" 
Subject: Re: [Bacula-users] Bacula in the cloud

On Tuesday 2016-10-18 12:34:08 Jason Voorhees wrote:
Thank you all for your responses.
I'll take a look at Bacula systems' whitepaper to see what they're
talking about. Meanwhile I'll explore some of the alternatives
discussed on this thread like copying files with scripts and making a
replica on SpiderOak or anything similar.
I hope we can have an interesting solution for this "problem" in the
near future.


Hi Jason!

You have said that "Bacula can't run jobs for so long without modifying
source code and recompiling".

What did you mean by that and can you give an example of the problem
you have experienced?

I am asking because I am not aware of the bacula's job duration related
limitations.

--
Josip Deanovic

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Heitor Faria]

> Thank you all for your responses.
> 
> I'll take a look at Bacula systems' whitepaper to see what they're
> talking about. Meanwhile I'll explore some of the alternatives
> discussed on this thread like copying files with scripts and making a
> replica on SpiderOak or anything similar.

Hello, Jason: why don't you install a 2nd remote Bacula storage daemon on a VPS 
/ Dedicated Server?

> I hope we can have an interesting solution for this "problem" in the
> near future.
> 
> Thanks again for your time :)

Regards,
-- 
=== 
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified 
Administrator II 
• Do you need Bacula training? http://bacula.us/video-classes/ 
+55 61 8268-4220 | http://bacula.us 
===

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula in the cloud

[Josip Deanovic]

On Tuesday 2016-10-18 21:28:44 Clark, Patti wrote:
> From Bacula’s main.pdf documentation:
> 
> Max Run Time =  The time specifies the maximum allowed time that a
> job may run, counted from when the job starts, (not necessarily the
> same as when the job was scheduled). By default, the the watchdog
> thread will kill any Job that has run more than 6 days. The maximum
> watchdog timeout is independent of MaxRunTime and cannot be changed.

Thanks.

I have never stumbled on this one and I missed this part in the
documentation.
The longest running job I had was a little over five days.

-- 
Josip Deanovic

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users