Re: [Bacula-users] Bacula in the cloud

[Daniele Palumbo]

Hi Kern,

News about it?

Thanks,
Daniele

> Il giorno 18 ott 2016, alle ore 14:13, Kern Sibbald  ha 
> scritto:
> 
> Hello,
> 
> Bacula Systems has a White Paper on Bacula Enterprise Edition in the
> cloud, and they have given me permission to publish it. However, as it
> is currently written for Bacula Enterprise customers it needs some
> modification, which I will make over the next week or so then release it.
> 
> It discusses a number of different ways that Bacula can work with the
> cloud, so you all might find it very interesting.  Obviously one of the
> current limitations for most people (like me) who do not have a big
> budget for high-speed fiber optic Internet connections is the upload
> speed.  I have spent a lot of time thinking about this, and I think
> there are a number of very interesting solutions that will become
> available in the near future.
> 
> Best regards,
> Kern
> 
> On 10/18/2016 01:45 PM, Josh Fisher wrote:
>> On 10/18/2016 3:42 AM, Uwe Schuerkamp wrote:
>>> Hello Jason,
>>> 
>>> On Mon, Oct 17, 2016 at 09:37:12PM -0500, Jason Voorhees wrote:
 Hello guys:
 
 Based on your experience, what alternative do we have for backing up
 information to the cloud preferably using Bacula?
 
>>> I wrote a script a while ago that runs as a RunAfterJob element which
>>> encrypts (gpg) and copies a full backup of a client (or its disk
>>> volume rather) to an S3 bucket using the aws shell client.
>>> 
>>> It's still very rudimentary but it does the job nicely when it comes
>>> to keeping a full backup safe (and secure) from a local disaster.
>>> 
>>> I seem to recall "cloud support" (whatever that may mean in today's
>>> buzzword bingo) was announced for Bacula 8.
>> I tend to think that will be targeting local cloud storage, for example
>> ownCloud, in enterprise environments. I'm not sure something like S3 is
>> very useful for direct backup storage over the Internet. A 1 TB backup
>> over a 100 Mbps connection would take a minimum of 22+ hours, assuming
>> maximum throughput and that S3 could actually sustain 12.5 MB/s.
>> 
>> For S3, copying via a script seems the best way to go.
>> 
>>> All the best,
>>> 
>>> Uwe
>>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



signature.asc
Description: Message signed with OpenPGP using GPGMail
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Dell TL1000 IBM3850-HH7

[Jim Richardson]

Team,

Okay curiosity got the better of me, I did three things.  One, built from 
latest source on Sourceforge 7.4.7-7 binaries only - tested with same results.  
Two, using the Slaanesh repository (thank you Ben) was able to install version 
7.4.7-1 - tested with the same results.  Three, I used the configuration from 
Alan (thank you) - test with same results.  

At this point I am leaning towards the post from Mariusz Mazur (link below) 
from 2013.  He is asserting that he is able to use the IBM lin_tape driver with 
Bacula by building in support for EOD operations using FSF which end up with an 
EIO instead of an EOF.  He offers two patches.  I am not interested in the IBM 
driver if I can get the ST to work.  Bacula's code changed significantly since 
his patches and my initial debug, notably the calls in dev.c seem to have moved 
to tape_dev.c.  I may attempt to add a few more Dmsg at a higher debug to the 
code reporting on any sense data from the tape itself to dig into the issue.  I 
am fairly sure from my initial debug of btape that the EIO instead of an EOF / 
ENOSPC is the root of the issue.  

Is there anything already in the code that can give insight into the sense data 
from the drive without the need for extra code?  Or a non-coding option to get 
btape to ignore these errors and just complete its job.  Then we can look at 
the actual tape results to see if, while in error, desired functionality is 
obtained?  My use case will be one job/tape a day anyway.  The append 
functionality will only come into play for one offs - which I can do with 
mt/tar/etc. 

I am compiling a build that blanketly accepts EIO as EOF to see if that fixes 
the issue, if that works then a portion Mariusz's code may be the fix, allow 
for an option of EIO at EOF = yes, default of no as it applies this to the FSF 
function. 

Be in touch.

Referenced Link:
http://bacula.10910.n7.nabble.com/No-EIO-support-on-EOD-read-td75874.html

Jim Richardson

-Original Message-
From: Kern Sibbald [mailto:k...@sibbald.com] 
Sent: Sunday, March 19, 2017 5:13 AM
To: Jim Richardson ; bacula-users@lists.sourceforge.net
Cc: Simone Caronni 
Subject: Re: [Bacula-users] Dell TL1000 IBM3850-HH7

Hello Jim,

What Alan says is interesting.  I was aware that there was a recent bug with 
btape "fill", but I have been unaware of the kinds of problems you are seeing.  
Anyway, your problem worries me as it is possible, though unlikely, there is 
some issue with LTO-7 drives.

Yesterday, I just upgraded from LTO-4 to LTO-5, and btape worked perfectly for 
me.

If I were in your shoes, I would do it just as you say, but if you prefer not 
to build the binaries yourself, there are two alternatives:

1. Wait a week or two and the community will certainly have RedHat or CentOS 
7.x binaries available.  We are just putting the finishing touches on the 
download page.

2. Ask Simone Caronni, copied on this email, as he is the RedHat/Fedora 
packager and he probably already has a recent RedHat 7.4 packaged.  The current 
Bacula release is 7.4.7.

Best regards,

Kern


On 03/19/2017 12:42 AM, Jim Richardson wrote:
> Alan,
>
> Thank you for joining my inquiry.  I will try this config on Monday.  As for 
> the bug, no and kind of, I read a thread from 2013 that pointed to a fix for 
> the FSF calls for EOF/EOT with LTO drives.  It includes a few patches and 
> questioned their plausibility for inclusion into Bacula stating that it would 
> make Bacula compatible with the IBM lin_tape driver.  The thread died with 
> what seemed like internet trollism.  I wanted to avoid building from source 
> due to the fact that I know it will tie me to the install forever.  Never the 
> less, if that is the only way to get this moving forward I will do so Monday 
> and will report back with results.  I am using bacula-7.0.5-7 which looks 
> like it was packaged for el7 in May of 2015.  I will use the SPEC and related 
> files from it with the latest source and see where things go.
>
> Thank you again
>
> Jim Richardson
>
> -Original Message-
> From: Alan Brown [mailto:a.br...@ucl.ac.uk]
> Sent: Saturday, March 18, 2017 1:01 PM
> To: Kern Sibbald ; Jim Richardson 
> ; bacula-users@lists.sourceforge.net
> Subject: Re: [Bacula-users] Dell TL1000 IBM3850-HH7
>
> On 18/03/17 07:41, Kern Sibbald wrote:
>> Hello Jim,
>>
>> I am checking with a tape drive "expert" perhaps he has some ideas.
>> My problem is time, not money.
> I'm no expert, just someone who's had to debug things :)
>
> Thankfully all ultrium(LTO) drives behave the same no matter who 
> they're made by and I see you're using the generic st driver
>
>
>
>
> I take it you're aware there was a bug introduced into older versions of 
> btape which will result in tests failing?
> It's important to be running a bacula version from after January 2017 to 
> avoid this.
>
>
>
>
>
> This is my current config (IBM SAS LTO6 

Re: [Bacula-users] Dell TL1000 IBM3850-HH7

[Roberts, Ben]

Hi Jim,

> I am using bacula-7.0.5-7 which looks like it was packaged for el7 in May of 
> 2015. I will use the SPEC and related files from it with the latest source 
> and see where things go.

If you're looking for newer el6/7 releases to avoid building from source, take 
a look at Slaanesh's COPR repository: 
https://copr.fedorainfracloud.org/coprs/slaanesh/Bacula/. These include the 
latest releases since Jan 2017 and so will contain the btape fixes mentioned 
earlier in this thread.

Regards,
Ben Roberts

This email and any files transmitted with it contain confidential and 
proprietary information and is solely for the use of the intended recipient.
If you are not the intended recipient please return the email to the sender and 
delete it from your computer and you must not use, disclose, distribute, copy, 
print or rely on this email or its contents.
This communication is for informational purposes only.
It is not intended as an offer or solicitation for the purchase or sale of any 
financial instrument or as an official confirmation of any transaction.
Any comments or statements made herein do not necessarily reflect those of GSA 
Capital.
GSA Capital Partners LLP is authorised and regulated by the Financial Conduct 
Authority and is registered in England and Wales at Stratton House, 5 Stratton 
Street, London W1J 8LA, number OC309261.
GSA Capital Services Limited is registered in England and Wales at the same 
address, number 5320529.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Dell TL1000 IBM3850-HH7

[Kern Sibbald]

Hello Jim,

What Alan says is interesting.  I was aware that there was a recent bug 
with btape "fill", but I have been unaware of the kinds of problems you 
are seeing.  Anyway, your problem worries me as it is possible, though 
unlikely, there is some issue with LTO-7 drives.

Yesterday, I just upgraded from LTO-4 to LTO-5, and btape worked 
perfectly for me.

If I were in your shoes, I would do it just as you say, but if you 
prefer not to build the binaries yourself, there are two alternatives:

1. Wait a week or two and the community will certainly have RedHat or 
CentOS 7.x binaries available.  We are just putting the finishing 
touches on the download page.

2. Ask Simone Caronni, copied on this email, as he is the RedHat/Fedora 
packager and he probably already has a recent RedHat 7.4 packaged.  The 
current Bacula release is 7.4.7.

Best regards,

Kern


On 03/19/2017 12:42 AM, Jim Richardson wrote:
> Alan,
>
> Thank you for joining my inquiry.  I will try this config on Monday.  As for 
> the bug, no and kind of, I read a thread from 2013 that pointed to a fix for 
> the FSF calls for EOF/EOT with LTO drives.  It includes a few patches and 
> questioned their plausibility for inclusion into Bacula stating that it would 
> make Bacula compatible with the IBM lin_tape driver.  The thread died with 
> what seemed like internet trollism.  I wanted to avoid building from source 
> due to the fact that I know it will tie me to the install forever.  Never the 
> less, if that is the only way to get this moving forward I will do so Monday 
> and will report back with results.  I am using bacula-7.0.5-7 which looks 
> like it was packaged for el7 in May of 2015.  I will use the SPEC and related 
> files from it with the latest source and see where things go.
>
> Thank you again
>
> Jim Richardson
>
> -Original Message-
> From: Alan Brown [mailto:a.br...@ucl.ac.uk]
> Sent: Saturday, March 18, 2017 1:01 PM
> To: Kern Sibbald ; Jim Richardson ; 
> bacula-users@lists.sourceforge.net
> Subject: Re: [Bacula-users] Dell TL1000 IBM3850-HH7
>
> On 18/03/17 07:41, Kern Sibbald wrote:
>> Hello Jim,
>>
>> I am checking with a tape drive "expert" perhaps he has some ideas.
>> My problem is time, not money.
> I'm no expert, just someone who's had to debug things :)
>
> Thankfully all ultrium(LTO) drives behave the same no matter who they're made 
> by and I see you're using the generic st driver
>
>
>
>
> I take it you're aware there was a bug introduced into older versions of 
> btape which will result in tests failing?
> It's important to be running a bacula version from after January 2017 to 
> avoid this.
>
>
>
>
>
> This is my current config (IBM SAS LTO6 drive in a Quantum i500 robot) - as 
> you can see I just run them in as vanilla a state as possible.
>
> The configuration for the previous setup (HP FC LTO5 drives in a Neo8000
> robot) was almost identical.
>
> The /etc/bacula/DEVICE/* items are symlinks to /dev/tape/by-id/ in order to 
> make identification of the drives easier for operators.
> In a single drive setup there's no advantage to this over using /dev/nst0
>
> Device {
> ## NB: THIS IS THE PATH TO THE CHANGER. IF YOU LOSE THIS DRIVE, YOU LOSE 
> CONTROL.
> ### library physical position 0,2
>Name = MSSLYF-5
>Drive Index = 5
>Device Type = Tape
>Media Type = LTO5
>AutoChanger = yes ;
>Autoselect = yes ## change to no to prevent bacula using the drive
>Archive Device = /etc/bacula/DEVICES/MSSLYF-5
>Control Device = /etc/bacula/DEVICES/MSSLYF-5-sg # beyond this number of 
> jobs for a drive, bacula will attempt # to load another tape in the same pool 
> in another drive
>Maximum Concurrent Jobs = 40
> # allow for cleaning cycles
>Maximum Changer Wait = 20 minutes
>AutomaticMount = yes;   # when device opened, read it
>AlwaysOpen = yes;
>LabelMedia = yes;   # lets Bacula label unlabeled media
>RemovableMedia = yes;
>RandomAccess = no;
>Volume Poll Interval = 600
>#Alert Command = "sh -c '/usr/local/bin/gettapeinfo.sh 
> /etc/bacula/DEVICES/MSSLYF-5'"
>Alert Command = /opt/bacula/scripts/tapealert %l
>Spool Directory = /var/bacula/spool/TAPE
>Maximum File Size = 16G
>Maximum Network Buffer Size = 262144
>Maximum block size = 2M
>Maximum Spool Size = 220G
>Maximum Job Spool Size = 50G
> }
>
>> What OS and version are you using, and what version of Bacula are you
>> using (sorry if you already answered these questions).
>>
>> Best regards,
>>
>> Kern
>>
>>
>> On 03/17/2017 08:02 PM, Jim Richardson wrote:
>>> Kern,
>>>
>>> Unfortunately both of your options did not work.  Both failed with the 
>>> exact same results.  I would really like to get this solution running even 
>>> if it means I have to put some dollars into it.  The drive I have is new 
>>> manufactured in 12/2015 with the latest firmware from 1/2016.  I took the 
>>>