date:20161014

Re: [Bacula-users] Using bacual to backup a gluster cluster

2016-10-14 Thread Dimitri Maziuk

On 10/14/2016 10:17 AM, Josh Fisher wrote:

> I use a simpler setup. VMs run on a pacemaker/corosync cluster and use
> native filesystems on DRBD disks in active/passive mode.
...
> I have found that it is preferable to install bacula-fd on the VMs themselves 
> and backup the VMs

+1

I've migrated our storage to zfs (all you need is cache='writeback' in
/etc/libvirt/qemu/machine.xml) and added a cron job to snapshot it and
copy the snapshot to another zfs nightly. I keep a week's worth of
snapshots, I don't need any more backups than that

-- but if I did, I too would back up the actual data I need from within
the VM.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

2016-10-14 Thread Andreas Koch

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 10/14/2016 05:31 PM, Hankins, Jonathan wrote:
> I'm not really familiar with EMC's NFS, NFSv4 and automounter, but here's
> a shot in the dark - what do your export and mount options look like? In 
> particular, look into nohide or crossmnt export options. Also, have a
> look at this:
> https://utcc.utoronto.ca/~cks/space/blog/unix/MultiFilesystemNFSIssue. 
> Just guesses, may not have any bearing on your issue.

Hello Jonathan,

we had just been using something pretty standard:

stud   -rsize=32768,wsize=32768,vers=4   moria-0:/stud

The export setup is also really simple: the exported file systems themselves
have no other filesystems mounted inside, so crossmnt and nohide are not
applicable to our use-case.

At the moment, I suspect misbehavior on the part of the automounter. I have
just hardmounted the file systems (which does not hurt us on that specific
server) and will see how the next Differential on Sunday morning fares.

Best,
  Andreas


> 
> -Jonathan Hankins
> 
> On Fri, Oct 14, 2016 at 7:34 AM Andreas Koch 
>  > wrote:
> 
> Hello all,
> 
> we have observed strange behavior in our recent backups. We have a number
> of filesystems, and explicitly list those to be backed up in the fileset
> 
> One of these file systems is
> 
> /home/stud
> 
> which has subdirectories for each user. Backing these up has worked
> perfectly for the last decade by just including the file system in the
> FileSet
> 
> FileSet { Name = "fs_gundabad" Include { Options { signature = SHA1; 
> verify=pins1; accurate=pins1; xattrsupport = yes; } ... File =
> /home/stud ... }
> 
> However, a recent differential backup shows warnings for _all_ of user 
> subdirectories of the form:
> 
> 13-Oct 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak01 is a
> different filesystem. Will not descend from /home/stud into it. 13-Oct
> 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak02 is a different 
> filesystem. Will not descend from /home/stud into it. ...
> 
> Even stranger, the backup _does_ appear to contain files from these 
> directories (or more accurately, shows them in the catalog):
> 
> * list files jobid=2161 ... 
> /home/stud/aqsprak01/Templates/test01/projSRC_Master/3cuS_1_l.dat ...
> 
> Has anyone observed something like this elsewhere? As I wrote above, we
> use Bacula 7.4.4, on Scientific Linux 7.2 backing-up NFS v4.0 filesystems
> mounted via automounter from an EMC VNX 5200.
> 
> Best, Andreas
> 
> 
- --
>
> 
Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! 
> http://sdm.link/slashdot___ 
> Bacula-users mailing list Bacula-users@lists.sourceforge.net
>  
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> This e-mail is intended only for the recipient and may contain 
> confidential or  proprietary information. If you are not the intended 
> recipient, the review,  distribution, duplication or retention of this 
> message and its attachments is  prohibited. Please notify the sender of 
> this error immediately by reply  e-mail, and permanently delete this 
> message and its attachments in any form in  which they may have been 
> preserved.
> 
> 
> 
- --
>
> 
Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> 
> 
> 
> ___ Bacula-users mailing
> list Bacula-users@lists.sourceforge.net 
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 


- -- 
*** !!! Geänderte Rufnummern / Phone numbers have changed !!! **
Prof. Dr.-Ing. Andreas Koch  k...@esa.cs.tu-darmstadt.de
Technische Universitaet Darmstadt, FB20   Phone  : x49-6151-16-22420
Embedded Systems and Applications Group (ESA) FAX: x49-6151-16-22422
Hochschulstr. 10, D-64289 Darmstadt, Germany   * PGP key available *
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iEYEARECAAYFAlgBGF8ACgkQk5ta2EV7DoxXkgCgomQ1+eOamkoz8Q7AmPJKiPI6
XEkAnAq9kkl+URSgOhwja4t/xGW4Mhrm
=L4Hp
-END PGP SIGNATURE-

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

2016-10-14 Thread Hankins, Jonathan

I'm not really familiar with EMC's NFS, NFSv4 and automounter, but here's a
shot in the dark - what do your export and mount options look like? In
particular, look into nohide or crossmnt export options. Also, have a look
at this:
https://utcc.utoronto.ca/~cks/space/blog/unix/MultiFilesystemNFSIssue. Just
guesses, may not have any bearing on your issue.

-Jonathan Hankins

On Fri, Oct 14, 2016 at 7:34 AM Andreas Koch <
k...@esa.informatik.tu-darmstadt.de> wrote:

> Hello all,
>
> we have observed strange behavior in our recent backups. We have a number
> of
> filesystems, and explicitly list those to be backed up in the fileset
>
> One of these file systems is
>
> /home/stud
>
> which has subdirectories for each user. Backing these up has worked
> perfectly
> for the last decade by just including the file system in the FileSet
>
> FileSet {
>   Name = "fs_gundabad"
>   Include {
> Options {
>   signature = SHA1;
>   verify=pins1;
>   accurate=pins1;
>   xattrsupport = yes;
> }
> ...
> File = /home/stud
> ...
> }
>
> However, a recent differential backup shows warnings for _all_ of user
> subdirectories of the form:
>
> 13-Oct 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak01 is a
> different
> filesystem. Will not descend from /home/stud into it.
> 13-Oct 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak02 is a
> different
> filesystem. Will not descend from /home/stud into it.
> ...
>
> Even stranger, the backup _does_ appear to contain files from these
> directories (or more accurately, shows them in the catalog):
>
> * list files jobid=2161
> ...
> /home/stud/aqsprak01/Templates/test01/projSRC_Master/3cuS_1_l.dat
> ...
>
> Has anyone observed something like this elsewhere? As I wrote above, we use
> Bacula 7.4.4, on Scientific Linux 7.2 backing-up NFS v4.0 filesystems
> mounted
> via automounter from an EMC VNX 5200.
>
> Best,
>   Andreas
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>

-- 
This e-mail is intended only for the recipient and may contain confidential 
or proprietary information. If you are not the intended recipient, the 
review, distribution, duplication or retention of this message and its 
attachments is prohibited. Please notify the sender of this error 
immediately by reply e-mail, and permanently delete this message and its 
attachments in any form in which they may have been preserved.
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Using bacual to backup a gluster cluster

2016-10-14 Thread Josh Fisher

On 10/13/2016 11:38 PM, Thing wrote:

Hi,

I am just finishing building the Gluster cluster on 3 x Dell 990s 
running centos7 Gluster 3.8.4.   The bacula server will be a 4th Dell 
990 with a 5TB USB3 disk out the back.   It has no OS on it yet but as 
I'd like to run FreeIPA 4.0 on it my preference would be Centos 7 but 
I can virtualise that function instead if Debian or Ubuntu would be 
better for Bacula.

ie the ultimate reason for the gluster cluster is to back 3x VM hosts, 
I dont have that hardware yet though.

I use a simpler setup. VMs run on a pacemaker/corosync cluster and use 
native filesystems on DRBD disks in active/passive mode.  It is kept 
simple on purpose, as a VM restart in the event of a node failure is 
"good enough" availability for my purposes and active/passive mode 
alleviates much of the split-brain worries. That said, I have found that 
it is preferable to install bacula-fd on the VMs themselves and backup 
the VMs, rather than backing up the disk images on the host. This allows 
restoring an accidentally deleted/corrupted file directly to the VM 
without having to restore an entire disk image. This does require more 
resources for the Bacula catalog db, but is far more useful. I have had 
to restore a few files several times, whereas I have never (thankfully) 
had a catastrophic loss requiring a full disaster recovery.

On 14 October 2016 at 00:54, Josh Fisher > wrote:

On 10/12/2016 11:15 PM, Thing wrote:
> Hi,
>
> Has anyone done this?  I have a small 3 node 1TB gluster setup I'd
> like to make up to an external USB3 disk using bacula so I can take
> the disk offsite.   What I dont want to do however is impact the
> gluster performance if I can so I am wondering how best to get the
> data off and onto the external USB3 disk.
>
> Can I do this directly?  ie have a fd on one gluster node? it seems
> awfully hard to find anyone how has done this and if there would be
> problems.

Yes. You can mount the gluster on the host running bacula-sd,
(where the
USB3 drive is attached), and backup directly.

>
> Or alternatively, I am wondering maybe stream the data off
gluster to
> a "new' disk in the bacula server via nfs with rsync and then have
> bacula back that "new" disk up to the external USB3 disk?

Either way, the entire gluster, of necessity, must be read, so
performance will be impacted. The pre-copy is not necessary.
Bacula has
built-in spooling to disk that can be enabled. The spooling
essentially
does the same stream-to-disk, but via Bacula's client-server
socket i/o,
rather than NFS. Whether or not spooling would be helpful is hard to
say. It depends on many things, including network throughput, USB3
drive
sequential write performance, etc.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/bacula-users

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] More Bacula 7.4.4 differential Backup strangeness

2016-10-14 Thread Wanderlei Huttel

Hello Andreas

Yes I've noticed that too.
Even I have opened a bug report, you could read and check if the same bug
and maybe complement information.
http://bugs.bacula.org/view.php?id=2244


Best Regards

*Wanderlei Hüttel*
http://www.huttel.com.br

2016-10-14 10:24 GMT-03:00 Andreas Koch :

> Hi again,
>
> I looked at some of the files in our mystery differential backup (see last
> post) again. I am pretty sure that some of the backed-up files are really
> old
> and have not been modified since the last full backup (last Thursday,
> 2016-10-06).
>
> Here is one of these ancient files on disk:
>
> [root@gundabad ~]# stat /home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat
>   File: ‘/home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat’
>   Size: 37  Blocks: 16 IO Block: 65536  regular file
> Device: 2eh/46d Inode: 2204123 Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1033/  mb)   Gid: ( 1001/stud)
> Context: system_u:object_r:nfs_t:s0
> Access: 2016-10-09 04:19:46.0 +0200
> Modify: 2010-12-03 09:37:36.0 +0100
> Change: 2016-09-06 18:32:55.1 +0200
>  Birth: -
>
> With these stats, I would assume that the file would not be part of the
> differential, since they were also part of the full (job id 2152):
>
> * list files jobid=2152
> ...
> | /home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat
> ...
>
> We have restored the file from the full backup, and it is identical to the
> file which ended up again in the differential.
>
> I am now a bit concerned about the integrity of all of our backups. We did
> have problems on the _previous_ full Backup that the partition holding the
> Postgres database had overflowed (due to the database exceeding 13GB). We
> have
> since fixed that (increasing the size of the file system). The previous
> differential on Sunday (2016-10-9) also completed without any errors or
> warnings, so I had assumed that database problems (if any) due to the
> overflow
> had been resolved.
>
> So, any ideas what is happening here? Is there an SQL command to find out
> what
> Bacula thinks about these files (why they are considered to be different,
> thus
> ending up in differential backup)?
>
> Best,
>   Andreas
>
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] More Bacula 7.4.4 differential Backup strangeness

2016-10-14 Thread Andreas Koch

Hi again,

I looked at some of the files in our mystery differential backup (see last
post) again. I am pretty sure that some of the backed-up files are really old
and have not been modified since the last full backup (last Thursday, 
2016-10-06).

Here is one of these ancient files on disk:

[root@gundabad ~]# stat /home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat
  File: ‘/home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat’
  Size: 37  Blocks: 16 IO Block: 65536  regular file
Device: 2eh/46d Inode: 2204123 Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1033/  mb)   Gid: ( 1001/stud)
Context: system_u:object_r:nfs_t:s0
Access: 2016-10-09 04:19:46.0 +0200
Modify: 2010-12-03 09:37:36.0 +0100
Change: 2016-09-06 18:32:55.1 +0200
 Birth: -

With these stats, I would assume that the file would not be part of the
differential, since they were also part of the full (job id 2152):

* list files jobid=2152
...
| /home/stud/mx/simsel/csrc.rtl/3cuS_1_l.dat
...

We have restored the file from the full backup, and it is identical to the
file which ended up again in the differential.

I am now a bit concerned about the integrity of all of our backups. We did
have problems on the _previous_ full Backup that the partition holding the
Postgres database had overflowed (due to the database exceeding 13GB). We have
since fixed that (increasing the size of the file system). The previous
differential on Sunday (2016-10-9) also completed without any errors or
warnings, so I had assumed that database problems (if any) due to the overflow
had been resolved.

So, any ideas what is happening here? Is there an SQL command to find out what
Bacula thinks about these files (why they are considered to be different, thus
ending up in differential backup)?

Best,
  Andreas



signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

2016-10-14 Thread Andreas Koch

Hello all,

we have observed strange behavior in our recent backups. We have a number of
filesystems, and explicitly list those to be backed up in the fileset

One of these file systems is

/home/stud

which has subdirectories for each user. Backing these up has worked perfectly
for the last decade by just including the file system in the FileSet

FileSet {
  Name = "fs_gundabad"
  Include {
Options {
  signature = SHA1;
  verify=pins1;
  accurate=pins1;
  xattrsupport = yes;
}
...
File = /home/stud
...
}

However, a recent differential backup shows warnings for _all_ of user
subdirectories of the form:

13-Oct 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak01 is a different
filesystem. Will not descend from /home/stud into it.
13-Oct 03:25 gundabad-fd JobId 2161:  /home/stud/aqsprak02 is a different
filesystem. Will not descend from /home/stud into it.
...

Even stranger, the backup _does_ appear to contain files from these
directories (or more accurately, shows them in the catalog):

* list files jobid=2161
...
/home/stud/aqsprak01/Templates/test01/projSRC_Master/3cuS_1_l.dat
...

Has anyone observed something like this elsewhere? As I wrote above, we use
Bacula 7.4.4, on Scientific Linux 7.2 backing-up NFS v4.0 filesystems mounted
via automounter from an EMC VNX 5200.

Best,
  Andreas



signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula-7.4.4 in db version 14

2016-10-14 Thread Personal Técnico

Hi,

It seems in version 15, database has some news fields for SSL. If I 
configure bacula with ""--without-openssl", I suppose that new 
parameters take "null" value, but when director tries to connect to 
bacula database, I get some errors from sslkey, sslcert, etc.

thanks.

El 14/10/2016 a las 9:14, Ralf Brinkmann escribió:
> Am 06.10.2016 um 10:32 schrieb Personal Técnico:
>
>> Anybody here knows if bacula-7.4.4 could be compiled without "ssl"
>> support in db and, then, allow use database in version 14?
>
> I don't know if that helps. I added to the my.cnf configuration file the
> following line:
>
>> skip-networking
>
> I think the local communication between bacula-director and data base is
> done exclusively via localhost or sockets then.
>



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula-7.4.4 in db version 14

2016-10-14 Thread Ralf Brinkmann

Am 06.10.2016 um 10:32 schrieb Personal Técnico:

> Anybody here knows if bacula-7.4.4 could be compiled without "ssl"
> support in db and, then, allow use database in version 14?

I don't know if that helps. I added to the my.cnf configuration file the
following line:

> skip-networking

I think the local communication between bacula-director and data base is
done exclusively via localhost or sockets then.

-- 
Ralf Brinkmann


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Using bacual to backup a gluster cluster

Re: [Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

Re: [Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

Re: [Bacula-users] Using bacual to backup a gluster cluster

Re: [Bacula-users] More Bacula 7.4.4 differential Backup strangeness

[Bacula-users] More Bacula 7.4.4 differential Backup strangeness

[Bacula-users] Bacula 7.4.4: Strange warnings non-existent filesystem crossings

Re: [Bacula-users] Bacula-7.4.4 in db version 14

Re: [Bacula-users] Bacula-7.4.4 in db version 14

9 matches

Site Navigation

Mail list logo

Footer information