Re: [Gluster-users] Gluster healing VM images

[Gregor Burck]

Am Freitag, 24. Juli 2015, 21:28:01 schrieb Andrus, Brian Contractor:
> I had this as well. It was a BIG pain for me. FWIW, after upgrading to
> gluster 3.7, I have not had an issue. YMMV
Hmm, with 3.7 I've had it, but maybee it cause on my settings.
Could you past your option settings? Maybee I play to much on my 
testenviroment,...

Bye

Gregor
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster healing VM images

[Andrus, Brian Contractor]

I had this as well. It was a BIG pain for me. FWIW, after upgrading to gluster 
3.7, I have not had an issue.
YMMV


Brian Andrus
ITACS/Research Computing
Naval Postgraduate School
Monterey, California
voice: 831-656-6238


-Original Message-
From: gluster-users-boun...@gluster.org 
[mailto:gluster-users-boun...@gluster.org] On Behalf Of André Bauer
Sent: Friday, July 24, 2015 7:44 AM
To: gluster-users@gluster.org
Subject: Re: [Gluster-users] Gluster healing VM images

Same Here.
Glusterfs 3.5.5.
4 Nodes. Distributed Replicated.
Qemu uses libgfapi to access vmimages.

When one node goes down, Harddisk of VM goes ro.
Works again immediately after VM restart.


Am 21.07.2015 um 12:21 schrieb Gregor Burck:
> Am Dienstag, 21. Juli 2015, 10:04:25 schrieb Andrew Roberts:
>> The VMs using the healing image files freeze completely, also 
>> freezing Virt-Manager, and then all of the other VMs either freeze or become 
>> slow.
> This is the same as in my test enviroment.
> 
> Work in the vm isn't possible after freeze for me, cause the 
> filesystem in vm is ro,... No solution for yet, but I test somethings on.
> 
> On which OS did you run the glusterfs-server?
> 
> Bye
> 
> Gregor
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 


--
Mit freundlichen Grüßen
André Bauer

MAGIX Software GmbH
André Bauer
Administrator
August-Bebel-Straße 48
01219 Dresden
GERMANY

tel.: 0351 41884875
e-mail: aba...@magix.net
aba...@magix.net <mailto:Email>
www.magix.com <http://www.magix.com/>


Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael Keith 
Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205

Find us on:

<http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de> 
<http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
--
The information in this email is intended only for the addressee named above. 
Access to this email by anyone else is unauthorized. If you are not the 
intended recipient of this message any disclosure, copying, distribution or any 
action taken in reliance on it is prohibited and may be unlawful. MAGIX does 
not warrant that any attachments are free from viruses or other defects and 
accepts no liability for any losses resulting from infected email 
transmissions. Please note that any views expressed in this email may be those 
of the originator and do not necessarily represent the agenda of the company.
--
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster healing VM images

[André Bauer]

Same Here.
Glusterfs 3.5.5.
4 Nodes. Distributed Replicated.
Qemu uses libgfapi to access vmimages.

When one node goes down, Harddisk of VM goes ro.
Works again immediately after VM restart.


Am 21.07.2015 um 12:21 schrieb Gregor Burck:
> Am Dienstag, 21. Juli 2015, 10:04:25 schrieb Andrew Roberts:
>> The VMs using the healing image files freeze completely, also freezing
>> Virt-Manager, and then all of the other VMs either freeze or become slow.
> This is the same as in my test enviroment.
> 
> Work in the vm isn't possible after freeze for me, cause the filesystem in vm 
> is ro,... No solution for yet, but I test somethings on.
> 
> On which OS did you run the glusterfs-server?
> 
> Bye
> 
> Gregor
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 


-- 
Mit freundlichen Grüßen
André Bauer

MAGIX Software GmbH
André Bauer
Administrator
August-Bebel-Straße 48
01219 Dresden
GERMANY

tel.: 0351 41884875
e-mail: aba...@magix.net
aba...@magix.net 
www.magix.com 


Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael Keith
Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205

Find us on:

 
 
--
The information in this email is intended only for the addressee named
above. Access to this email by anyone else is unauthorized. If you are
not the intended recipient of this message any disclosure, copying,
distribution or any action taken in reliance on it is prohibited and
may be unlawful. MAGIX does not warrant that any attachments are free
from viruses or other defects and accepts no liability for any losses
resulting from infected email transmissions. Please note that any
views expressed in this email may be those of the originator and do
not necessarily represent the agenda of the company.
--
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster healing VM images

[Gregor Burck]

> Gluster Servers: CentOS 7
> VMs: A mix of CentOS 6 / 7 / Windows Server 2008
> VM Image size 20GB to 250GB
In my case debain8 for nodes and vm-guest. image is 8GB

I found a old mail-threat from 07.07.2014:
self-heal stops some vms (virtual machines)

I think this is similar to our problem.

By

Gregor
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster healing VM images

[Andrew Roberts]

> > The VMs using the healing image files freeze completely, also freezing
> > Virt-Manager, and then all of the other VMs either freeze or become
slow.
> This is the same as in my test enviroment.

> Work in the vm isn't possible after freeze for me, cause the filesystem in
vm 
is ro,... No solution for yet, but I test somethings on.

> On which OS did you run the glusterfs-server?

Gluster Servers: CentOS 7
VMs: A mix of CentOS 6 / 7 / Windows Server 2008
VM Image size 20GB to 250GB

Noticed in Windows Server 2008, when the VM unfroze, the clock ticked at
about 10 seconds per second, until it caught itself up.

Thanks again

Andy

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster healing VM images

[Gregor Burck]

Am Dienstag, 21. Juli 2015, 10:04:25 schrieb Andrew Roberts:
> The VMs using the healing image files freeze completely, also freezing
> Virt-Manager, and then all of the other VMs either freeze or become slow.
This is the same as in my test enviroment.

Work in the vm isn't possible after freeze for me, cause the filesystem in vm 
is ro,... No solution for yet, but I test somethings on.

On which OS did you run the glusterfs-server?

Bye

Gregor
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Gluster healing VM images

[Andrew Roberts]

Hello,
 
Running Gluster 3.7.2, distributed(3) / replicated(2) / 20Gb teamed network. Pretty much default options.
I am having an issue with self heal in Gluster. The second gluster node disconnected overnight, and now large files such as VM images are out of sync.
The self-heal does not use the full disc IO, nor the full network bandwidth, but it does appear to slow down the volume during a heal, and lock up the files it is healing.
A simple # df -h takes 1 - 2 minutes, where the volumes are FUSE mounted.
The VMs using the healing image files freeze completely, also freezing Virt-Manager, and then all of the other VMs either freeze or become slow.
In the interest of getting our office network operational this morning, I have removed the second node from the network - this has stopped the self heal and allowed us to work with just 1 storage node - but it is very slow and repeatedly freezes for a few seconds every minute or so.
As I understand, gluster uses granular locking during a heal, so that a VM image may not entirely lock up during the heal / diff checking. However, it appears the entire VM hangs waiting for the heal to finish, with errors such as 'Device or Resource Busy'.
The effect we are observing with the second node disconnected (few second freezing every few minutes) is was I would expect if it was healing with granular locking, where it would freeze each time it attempted to heal a block that was being accessed.
 
I have been looking at the qemu / gluster libgfapi, but have been unable to make this work. From what I understand, this will only improve performance. Will this also allow a VM to be accessible during a self heal?
Other than not using the libgfapi for the VM images, is there anything else I am doing wrong?
Is there a way to use a VM image whilst it is healing?
 
Many thanks

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users