Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
On Wed, Jun 27, 2012 at 12:23 AM, Brian Candler wrote: > On Tue, Jun 26, 2012 at 02:08:42PM -0700, Simon Blackstein wrote: > >Thanks Brian. > >Yes, got rid of the .glusterfs and .vSphereHA directory that VMware > >makes. Rebooted, so yes it was remounted and used a different mount > >point name. Also got rid of attribute I found set on the root: > >setfattr -x trusted.gfid / && setfattr -x trusted.glusterfs.dht / > >Any other tips? :) > > Not from me I'm afraid... anyone else? > Looks like Simon already found the stale attribute on the forth node and is all set now. Avati ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
On Tue, Jun 26, 2012 at 02:08:42PM -0700, Simon Blackstein wrote: >Thanks Brian. >Yes, got rid of the .glusterfs and .vSphereHA directory that VMware >makes. Rebooted, so yes it was remounted and used a different mount >point name. Also got rid of attribute I found set on the root: >setfattr -x trusted.gfid / && setfattr -x trusted.glusterfs.dht / >Any other tips? :) Not from me I'm afraid... anyone else? ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
OK, figured this one out (my word this took some time). Found an attribute set on the fourth node after running through them again. Back to work... after patching all nodes and removing/recreating the volume, this now works for me. Definitely not simple and please, if we can clean up volumes after a delete (remove .glusterfs dir and extra attributes across all nodes) that would be tremendously useful. I'm going to throw a bunch of VMware VMs on my new volume and try it out. Recommendation for you, Fernando. I had to roll my own RPMs to get close to consistency across all nodes. Basically: - Download glusterfs-3.3.0.tar.gz, untar into its directory - Replace the files from Anand's patch ( http://review.gluster.com/#change,3617) - Re-tar up the file and run 'rpmbuild -ta glusterfs-3.3.0.tar.gz' - You'll end up with RPMs in the rpmbuild/RPMS directory from which you can scp to other nodes and bring them all up on the new version You may need to do the same with the Fuse client and work through some dependencies but it should work. Many Thanks! Simon On Tue, Jun 26, 2012 at 3:37 PM, Anand Avati wrote: > Can you get the output of getfattr -d -e hex -m . /gfs > > Avati > On Jun 26, 2012 5:08 PM, "Simon Blackstein" wrote: > >> Thanks Brian. >> >> Yes, got rid of the .glusterfs and .vSphereHA directory that VMware >> makes. Rebooted, so yes it was remounted and used a different mount >> point name. Also got rid of attribute I found set on the root: >> >> setfattr -x trusted.gfid / && setfattr -x trusted.glusterfs.dht / >> >> Any other tips? :) >> >> Many Rgds, >> >> Simon >> >> On Tue, Jun 26, 2012 at 12:19 PM, Brian Candler wrote: >> >>> On Tue, Jun 26, 2012 at 11:46:52AM -0700, Simon Blackstein wrote: >>> >Basically did all of that as previously noted: >>> >>> And rm -rf .glusterfs ? >>> >>> If /gfs is the mountpoint, you could also try >>> unmount /gfs >>> rmdir /gfs >>> mkdir /gfs >>> and remount. >>> >> >> ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Can you get the output of getfattr -d -e hex -m . /gfs Avati On Jun 26, 2012 5:08 PM, "Simon Blackstein" wrote: > Thanks Brian. > > Yes, got rid of the .glusterfs and .vSphereHA directory that VMware > makes. Rebooted, so yes it was remounted and used a different mount > point name. Also got rid of attribute I found set on the root: > > setfattr -x trusted.gfid / && setfattr -x trusted.glusterfs.dht / > > Any other tips? :) > > Many Rgds, > > Simon > > On Tue, Jun 26, 2012 at 12:19 PM, Brian Candler wrote: > >> On Tue, Jun 26, 2012 at 11:46:52AM -0700, Simon Blackstein wrote: >> >Basically did all of that as previously noted: >> >> And rm -rf .glusterfs ? >> >> If /gfs is the mountpoint, you could also try >> unmount /gfs >> rmdir /gfs >> mkdir /gfs >> and remount. >> > > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Thanks Brian. Yes, got rid of the .glusterfs and .vSphereHA directory that VMware makes. Rebooted, so yes it was remounted and used a different mount point name. Also got rid of attribute I found set on the root: setfattr -x trusted.gfid / && setfattr -x trusted.glusterfs.dht / Any other tips? :) Many Rgds, Simon On Tue, Jun 26, 2012 at 12:19 PM, Brian Candler wrote: > On Tue, Jun 26, 2012 at 11:46:52AM -0700, Simon Blackstein wrote: > >Basically did all of that as previously noted: > > And rm -rf .glusterfs ? > > If /gfs is the mountpoint, you could also try > unmount /gfs > rmdir /gfs > mkdir /gfs > and remount. > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
On Tue, Jun 26, 2012 at 11:46:52AM -0700, Simon Blackstein wrote: >Basically did all of that as previously noted: And rm -rf .glusterfs ? If /gfs is the mountpoint, you could also try unmount /gfs rmdir /gfs mkdir /gfs and remount. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Basically did all of that as previously noted: setfattr -x trusted.gfid /gfs setfattr -x trusted.glusterfs.volume-id /gfs setfattr -x trusted.afr.gfs-vdi-client-0 /gfs setfattr -x trusted.afr.gfs-vdi-client-1 /gfs setfattr -x trusted.glusterfs.dht /gfs On Tue, Jun 26, 2012 at 11:38 AM, Brian Candler wrote: > On Tue, Jun 26, 2012 at 10:01:21AM -0700, Simon Blackstein wrote: > >Honestly, I've been trying to reset this volume completely to see if > >the error was transitional but now getting the '/gfs or a prefix of it > >is already part of a volume' message even after removing the > attributes > >from the directory... lots of changes in this version to watch out for > >:( > > https://bugzilla.redhat.com/show_bug.cgi?id=812214 > (esp. comment 17) > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
On Tue, Jun 26, 2012 at 10:01:21AM -0700, Simon Blackstein wrote: >Honestly, I've been trying to reset this volume completely to see if >the error was transitional but now getting the '/gfs or a prefix of it >is already part of a volume' message even after removing the attributes >from the directory... lots of changes in this version to watch out for >:( https://bugzilla.redhat.com/show_bug.cgi?id=812214 (esp. comment 17) ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
I changed the mount point here, but still the same issue. I'd rather not kill the LVM partition if possible. Anand, any tips for this? On Tue, Jun 26, 2012 at 10:13 AM, Fernando Frediani (Qube) < fernando.fredi...@qubenet.net> wrote: > Yeah, that’s quiet annoying on 3.3 > > I’ve run into the same problem when trying to re-create a volume made of > the same disks that I couldn’t get rid of even using these commands, so I > ended up replacing the physical disks and re-creating the partition. > > Even when you do a Delete on the volume it seems to run into the same > trouble. > > > Fernando > > ** ** > > *From:* Simon Blackstein [mailto:si...@blackstein.com] > *Sent:* 26 June 2012 18:01 > *To:* Anand Avati > *Cc:* Fernando Frediani (Qube); gluster-users@gluster.org > *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > ** ** > > Honestly, I've been trying to reset this volume completely to see if the > error was transitional but now getting the '/gfs or a prefix of it is > already part of a volume' message even after removing the attributes from > the directory... lots of changes in this version to watch out for :( > > ** ** > > setfattr -x trusted.gfid /gfs > > setfattr -x trusted.glusterfs.volume-id /gfs > > setfattr -x trusted.afr.gfs-vdi-client-0 /gfs > > setfattr -x trusted.afr.gfs-vdi-client-1 /gfs > > setfattr -x trusted.glusterfs.dht /gfs > > ** ** > > Many Rgds, > > ** ** > > Simon > > ** ** > > On Tue, Jun 26, 2012 at 8:28 AM, Anand Avati > wrote: > > Is this at the same 'time' as before (at the time for VM boot), or does > it actually progress a little more (i.e, "start booting") and then throw > up? It will be helpful if we move this discussion to bugzilla and you > provide trace logs. > > ** ** > > Avati > > On Tue, Jun 26, 2012 at 7:59 AM, Simon Blackstein > wrote: > > Hi Avati, > > ** ** > > Thanks. I just tried a recompile (I'd installed from RPM before) and > brought up the volume again. I now get a similar but different message:*** > * > > ** ** > > An unexpected error was received from the ESX host while powering on VM > vm-26944. > > Failed to power on VM. > > Could not power on VM : Not found. > > Failed to create swap file '/gfs/gfs-test1/./gfs-test1-d27c6ac2.vswp' : > Not found > > ** ** > > I only modified the first brick node which is where I'm pointing the ESXi > server as NFS client. Do I need to modify all nodes like this? > > ** ** > > Many Thanks! > > ** ** > > Simon > > ** ** > > On Tue, Jun 26, 2012 at 4:14 AM, Anand Avati > wrote: > > Fernando, > > Yes, to try the patch you need to install from source. We will include > the patch in the next release if you need RPMs. > > ** ** > > Avati > > On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) < > fernando.fredi...@qubenet.net> wrote: > > Hi Avati, > > > > How I suppose to apply the patch if I have installed the RPM version ? > Should I have a compiled from source installed instead ? > > > > Regards, > > > Fernando > > > > *From:* gluster-users-boun...@gluster.org [mailto: > gluster-users-boun...@gluster.org] *On Behalf Of *Anand Avati > *Sent:* 26 June 2012 04:00 > *To:* Simon > *Cc:* gluster-users@gluster.org > > > *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > > > Simon - can you please try this patch: http://review.gluster.com/3617 > > > > Thanks, > > Avati > > On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote: > > I'm having the same error deploying a green field ESXi 5.0 farm against > GlusterFS 3.3. Can provision a VM but can't start it with the identical > error: > > An unexpected error was received from the ESX host while powering on VM > vm-26941. > > Failed to power on VM. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > > Is GlusterFS supported against VMware or should I be looking somewhere > else? > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > > > ** ** > > ** ** > > ** ** > > ** ** > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Yeah, that's quiet annoying on 3.3 I've run into the same problem when trying to re-create a volume made of the same disks that I couldn't get rid of even using these commands, so I ended up replacing the physical disks and re-creating the partition. Even when you do a Delete on the volume it seems to run into the same trouble. Fernando From: Simon Blackstein [mailto:si...@blackstein.com] Sent: 26 June 2012 18:01 To: Anand Avati Cc: Fernando Frediani (Qube); gluster-users@gluster.org Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Honestly, I've been trying to reset this volume completely to see if the error was transitional but now getting the '/gfs or a prefix of it is already part of a volume' message even after removing the attributes from the directory... lots of changes in this version to watch out for :( setfattr -x trusted.gfid /gfs setfattr -x trusted.glusterfs.volume-id /gfs setfattr -x trusted.afr.gfs-vdi-client-0 /gfs setfattr -x trusted.afr.gfs-vdi-client-1 /gfs setfattr -x trusted.glusterfs.dht /gfs Many Rgds, Simon On Tue, Jun 26, 2012 at 8:28 AM, Anand Avati mailto:anand.av...@gmail.com>> wrote: Is this at the same 'time' as before (at the time for VM boot), or does it actually progress a little more (i.e, "start booting") and then throw up? It will be helpful if we move this discussion to bugzilla and you provide trace logs. Avati On Tue, Jun 26, 2012 at 7:59 AM, Simon Blackstein mailto:si...@blackstein.com>> wrote: Hi Avati, Thanks. I just tried a recompile (I'd installed from RPM before) and brought up the volume again. I now get a similar but different message: An unexpected error was received from the ESX host while powering on VM vm-26944. Failed to power on VM. Could not power on VM : Not found. Failed to create swap file '/gfs/gfs-test1/./gfs-test1-d27c6ac2.vswp' : Not found I only modified the first brick node which is where I'm pointing the ESXi server as NFS client. Do I need to modify all nodes like this? Many Thanks! Simon On Tue, Jun 26, 2012 at 4:14 AM, Anand Avati mailto:anand.av...@gmail.com>> wrote: Fernando, Yes, to try the patch you need to install from source. We will include the patch in the next release if you need RPMs. Avati On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) mailto:fernando.fredi...@qubenet.net>> wrote: Hi Avati, How I suppose to apply the patch if I have installed the RPM version ? Should I have a compiled from source installed instead ? Regards, Fernando From: gluster-users-boun...@gluster.org<mailto:gluster-users-boun...@gluster.org> [mailto:gluster-users-boun...@gluster.org<mailto:gluster-users-boun...@gluster.org>] On Behalf Of Anand Avati Sent: 26 June 2012 04:00 To: Simon Cc: gluster-users@gluster.org<mailto:gluster-users@gluster.org> Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Simon - can you please try this patch: http://review.gluster.com/3617 Thanks, Avati On Mon, Jun 25, 2012 at 7:13 PM, Simon mailto:si...@blackstein.com>> wrote: I'm having the same error deploying a green field ESXi 5.0 farm against GlusterFS 3.3. Can provision a VM but can't start it with the identical error: An unexpected error was received from the ESX host while powering on VM vm-26941. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Is GlusterFS supported against VMware or should I be looking somewhere else? ___ Gluster-users mailing list Gluster-users@gluster.org<mailto:Gluster-users@gluster.org> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Honestly, I've been trying to reset this volume completely to see if the error was transitional but now getting the '/gfs or a prefix of it is already part of a volume' message even after removing the attributes from the directory... lots of changes in this version to watch out for :( setfattr -x trusted.gfid /gfs setfattr -x trusted.glusterfs.volume-id /gfs setfattr -x trusted.afr.gfs-vdi-client-0 /gfs setfattr -x trusted.afr.gfs-vdi-client-1 /gfs setfattr -x trusted.glusterfs.dht /gfs Many Rgds, Simon On Tue, Jun 26, 2012 at 8:28 AM, Anand Avati wrote: > Is this at the same 'time' as before (at the time for VM boot), or does > it actually progress a little more (i.e, "start booting") and then throw > up? It will be helpful if we move this discussion to bugzilla and you > provide trace logs. > > Avati > > On Tue, Jun 26, 2012 at 7:59 AM, Simon Blackstein wrote: > >> Hi Avati, >> >> Thanks. I just tried a recompile (I'd installed from RPM before) and >> brought up the volume again. I now get a similar but different message: >> >> An unexpected error was received from the ESX host while powering on VM >> vm-26944. >> Failed to power on VM. >> Could not power on VM : Not found. >> Failed to create swap file '/gfs/gfs-test1/./gfs-test1-d27c6ac2.vswp' : >> Not found >> >> I only modified the first brick node which is where I'm pointing the ESXi >> server as NFS client. Do I need to modify all nodes like this? >> >> Many Thanks! >> >> Simon >> >> On Tue, Jun 26, 2012 at 4:14 AM, Anand Avati wrote: >> >>> Fernando, >>> Yes, to try the patch you need to install from source. We will include >>> the patch in the next release if you need RPMs. >>> >>> Avati >>> >>> On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) < >>> fernando.fredi...@qubenet.net> wrote: >>> >>>> Hi Avati, >>>> >>>> ** ** >>>> >>>> How I suppose to apply the patch if I have installed the RPM version ? >>>> Should I have a compiled from source installed instead ?**** >>>> >>>> ** ** >>>> >>>> Regards, >>>> >>>> >>>> Fernando >>>> >>>> ** ** >>>> >>>> *From:* gluster-users-boun...@gluster.org [mailto: >>>> gluster-users-boun...@gluster.org] *On Behalf Of *Anand Avati >>>> *Sent:* 26 June 2012 04:00 >>>> *To:* Simon >>>> *Cc:* gluster-users@gluster.org >>>> >>>> *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 >>>> >>>> ** ** >>>> >>>> Simon - can you please try this patch: http://review.gluster.com/3617** >>>> ** >>>> >>>> ** ** >>>> >>>> Thanks, >>>> >>>> Avati >>>> >>>> On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote:*** >>>> * >>>> >>>> I'm having the same error deploying a green field ESXi 5.0 farm against >>>> GlusterFS 3.3. Can provision a VM but can't start it with the identical >>>> error: >>>> >>>> An unexpected error was received from the ESX host while powering on VM >>>> vm-26941. >>>> >>>> Failed to power on VM. >>>> Unable to retrieve the current working directory: 0 (No such file or >>>> directory). >>>> Check if the directory has been deleted or unmounted. >>>> Unable to retrieve the current working directory: 0 (No such file or >>>> directory). >>>> Check if the directory has been deleted or unmounted. >>>> >>>> Is GlusterFS supported against VMware or should I be looking somewhere >>>> else? >>>> >>>> >>>> ___ >>>> Gluster-users mailing list >>>> Gluster-users@gluster.org >>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>>> >>>> ** ** >>>> >>> >>> >> > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Is this at the same 'time' as before (at the time for VM boot), or does it actually progress a little more (i.e, "start booting") and then throw up? It will be helpful if we move this discussion to bugzilla and you provide trace logs. Avati On Tue, Jun 26, 2012 at 7:59 AM, Simon Blackstein wrote: > Hi Avati, > > Thanks. I just tried a recompile (I'd installed from RPM before) and > brought up the volume again. I now get a similar but different message: > > An unexpected error was received from the ESX host while powering on VM > vm-26944. > Failed to power on VM. > Could not power on VM : Not found. > Failed to create swap file '/gfs/gfs-test1/./gfs-test1-d27c6ac2.vswp' : > Not found > > I only modified the first brick node which is where I'm pointing the ESXi > server as NFS client. Do I need to modify all nodes like this? > > Many Thanks! > > Simon > > On Tue, Jun 26, 2012 at 4:14 AM, Anand Avati wrote: > >> Fernando, >> Yes, to try the patch you need to install from source. We will include >> the patch in the next release if you need RPMs. >> >> Avati >> >> On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) < >> fernando.fredi...@qubenet.net> wrote: >> >>> Hi Avati, >>> >>> ** ** >>> >>> How I suppose to apply the patch if I have installed the RPM version ? >>> Should I have a compiled from source installed instead ? >>> >>> ** ** >>> >>> Regards, >>> >>> >>> Fernando**** >>> >>> ** ** >>> >>> *From:* gluster-users-boun...@gluster.org [mailto: >>> gluster-users-boun...@gluster.org] *On Behalf Of *Anand Avati >>> *Sent:* 26 June 2012 04:00 >>> *To:* Simon >>> *Cc:* gluster-users@gluster.org >>> >>> *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 >>> >>> ** ** >>> >>> Simon - can you please try this patch: http://review.gluster.com/3617*** >>> * >>> >>> ** ** >>> >>> Thanks, >>> >>> Avati >>> >>> On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote: >>> >>> I'm having the same error deploying a green field ESXi 5.0 farm against >>> GlusterFS 3.3. Can provision a VM but can't start it with the identical >>> error: >>> >>> An unexpected error was received from the ESX host while powering on VM >>> vm-26941. >>> >>> Failed to power on VM. >>> Unable to retrieve the current working directory: 0 (No such file or >>> directory). >>> Check if the directory has been deleted or unmounted. >>> Unable to retrieve the current working directory: 0 (No such file or >>> directory). >>> Check if the directory has been deleted or unmounted. >>> >>> Is GlusterFS supported against VMware or should I be looking somewhere >>> else? >>> >>> >>> ___ >>> Gluster-users mailing list >>> Gluster-users@gluster.org >>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>> >>> ** ** >>> >> >> > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi Avati, Thanks. I just tried a recompile (I'd installed from RPM before) and brought up the volume again. I now get a similar but different message: An unexpected error was received from the ESX host while powering on VM vm-26944. Failed to power on VM. Could not power on VM : Not found. Failed to create swap file '/gfs/gfs-test1/./gfs-test1-d27c6ac2.vswp' : Not found I only modified the first brick node which is where I'm pointing the ESXi server as NFS client. Do I need to modify all nodes like this? Many Thanks! Simon On Tue, Jun 26, 2012 at 4:14 AM, Anand Avati wrote: > Fernando, > Yes, to try the patch you need to install from source. We will include > the patch in the next release if you need RPMs. > > Avati > > On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) < > fernando.fredi...@qubenet.net> wrote: > >> Hi Avati, >> >> ** ** >> >> How I suppose to apply the patch if I have installed the RPM version ? >> Should I have a compiled from source installed instead ? >> >> ** ** >> >> Regards, >> >> >> Fernando >> >> ** ** >> >> *From:* gluster-users-boun...@gluster.org [mailto: >> gluster-users-boun...@gluster.org] *On Behalf Of *Anand Avati >> *Sent:* 26 June 2012 04:00 >> *To:* Simon >> *Cc:* gluster-users@gluster.org >> >> *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 >> >> ** ** >> >> Simon - can you please try this patch: http://review.gluster.com/3617 >> >> ** ** >> >> Thanks, >> >> Avati >> >> On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote: >> >> I'm having the same error deploying a green field ESXi 5.0 farm against >> GlusterFS 3.3. Can provision a VM but can't start it with the identical >> error: >> >> An unexpected error was received from the ESX host while powering on VM >> vm-26941. >> >> Failed to power on VM. >> Unable to retrieve the current working directory: 0 (No such file or >> directory). >> Check if the directory has been deleted or unmounted. >> Unable to retrieve the current working directory: 0 (No such file or >> directory). >> Check if the directory has been deleted or unmounted. >> >> Is GlusterFS supported against VMware or should I be looking somewhere >> else? >> >> >> ___ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> >> ** ** >> > > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Fernando, Yes, to try the patch you need to install from source. We will include the patch in the next release if you need RPMs. Avati On Tue, Jun 26, 2012 at 3:02 AM, Fernando Frediani (Qube) < fernando.fredi...@qubenet.net> wrote: > Hi Avati, > > ** ** > > How I suppose to apply the patch if I have installed the RPM version ? > Should I have a compiled from source installed instead ? > > ** ** > > Regards, > > > Fernando > > ** ** > > *From:* gluster-users-boun...@gluster.org [mailto: > gluster-users-boun...@gluster.org] *On Behalf Of *Anand Avati > *Sent:* 26 June 2012 04:00 > *To:* Simon > *Cc:* gluster-users@gluster.org > > *Subject:* Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > ** ** > > Simon - can you please try this patch: http://review.gluster.com/3617 > > ** ** > > Thanks, > > Avati > > On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote: > > I'm having the same error deploying a green field ESXi 5.0 farm against > GlusterFS 3.3. Can provision a VM but can't start it with the identical > error: > > An unexpected error was received from the ESX host while powering on VM > vm-26941. > > Failed to power on VM. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > > Is GlusterFS supported against VMware or should I be looking somewhere > else? > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > ** ** > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi Avati, How I suppose to apply the patch if I have installed the RPM version ? Should I have a compiled from source installed instead ? Regards, Fernando From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Anand Avati Sent: 26 June 2012 04:00 To: Simon Cc: gluster-users@gluster.org Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Simon - can you please try this patch: http://review.gluster.com/3617 Thanks, Avati On Mon, Jun 25, 2012 at 7:13 PM, Simon mailto:si...@blackstein.com>> wrote: I'm having the same error deploying a green field ESXi 5.0 farm against GlusterFS 3.3. Can provision a VM but can't start it with the identical error: An unexpected error was received from the ESX host while powering on VM vm-26941. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Is GlusterFS supported against VMware or should I be looking somewhere else? ___ Gluster-users mailing list Gluster-users@gluster.org<mailto:Gluster-users@gluster.org> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Simon - can you please try this patch: http://review.gluster.com/3617 Thanks, Avati On Mon, Jun 25, 2012 at 7:13 PM, Simon wrote: > I'm having the same error deploying a green field ESXi 5.0 farm against > GlusterFS 3.3. Can provision a VM but can't start it with the identical > error: > > An unexpected error was received from the ESX host while powering on VM > vm-26941. > Failed to power on VM. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > Unable to retrieve the current working directory: 0 (No such file or > directory). > Check if the directory has been deleted or unmounted. > > Is GlusterFS supported against VMware or should I be looking somewhere > else? > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
I'm having the same error deploying a green field ESXi 5.0 farm against GlusterFS 3.3. Can provision a VM but can't start it with the identical error: An unexpected error was received from the ESX host while powering on VM vm-26941. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Is GlusterFS supported against VMware or should I be looking somewhere else? ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Was there any clue on that amount of logs on why a Virtual Machine cann't be Powered On using VMware ? Is it a NFS related problem ? Fernando -Original Message- From: Fernando Frediani (Qube) Sent: 14 June 2012 10:02 To: 'Tomoaki Sato'; 'gluster-users@gluster.org' Subject: RE: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Hi, This logs way too much data too quickly, so I have cut the part of the nfs.log for during the time I tried to power on the VM. Find it attached. Regards, Fernando -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Tomoaki Sato Sent: 14 June 2012 01:13 To: gluster-users@gluster.org Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Could you try "gluster volume set diagnostics.client-log-level TRACE" ? Tomo Sato (2012/06/14 0:19), Fernando Frediani (Qube) wrote: > Hi, > > I don't see anything on the nfs log files when watching > /var/log/glusterfs/nfs.log and trying to power on the machine at the same > time. On the Glusterd logs I don't see anything as well. > Anywhere else to check that it should be logging to ? > > Fernando > > -Original Message- > From: Vijay Bellur [mailto:vbel...@redhat.com] > Sent: 11 June 2012 17:54 > To: Fernando Frediani (Qube) > Cc: 'Atha Kouroussis'; 'gluster-users@gluster.org'; Rajesh Amaravathi; > Krishna Srinivas > Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > On 06/11/2012 05:52 PM, Fernando Frediani (Qube) wrote: >> Was doing some read on RedHat website and found this URL which I wonder if >> the problem would have anything to do with this: >> http://docs.redhat.com/docs/en-US/Red_Hat_Storage_Software_Appliance/3 >> .2/html/User_Guide/ch14s04s08.html >> >> Although both servers and client are 64 I wonder if somehow this could be >> related as it seems the closest thing I could think of. >> >> The error I get when trying to power up a VM is: >> >> An unexpected error was received from the ESX host while powering on VM >> vm-21112. >> Failed to power on VM. >> Unable to retrieve the current working directory: 0 (No such file or >> directory). Check if the directory has been deleted or unmounted. >> Unable to retrieve the current working directory: 0 (No such file or >> directory). Check if the directory has been deleted or unmounted. >> Unable to retrieve the current working directory: 0 (No such file or >> directory). Check if the directory has been deleted or unmounted. >> >> > > Can you please post nfs log file from the Gluster server that you are trying > to mount from? > > Thanks, > Vijay > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Could you try "gluster volume set diagnostics.client-log-level TRACE" ? Tomo Sato (2012/06/14 0:19), Fernando Frediani (Qube) wrote: Hi, I don't see anything on the nfs log files when watching /var/log/glusterfs/nfs.log and trying to power on the machine at the same time. On the Glusterd logs I don't see anything as well. Anywhere else to check that it should be logging to ? Fernando -Original Message- From: Vijay Bellur [mailto:vbel...@redhat.com] Sent: 11 June 2012 17:54 To: Fernando Frediani (Qube) Cc: 'Atha Kouroussis'; 'gluster-users@gluster.org'; Rajesh Amaravathi; Krishna Srinivas Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 On 06/11/2012 05:52 PM, Fernando Frediani (Qube) wrote: Was doing some read on RedHat website and found this URL which I wonder if the problem would have anything to do with this: http://docs.redhat.com/docs/en-US/Red_Hat_Storage_Software_Appliance/3 .2/html/User_Guide/ch14s04s08.html Although both servers and client are 64 I wonder if somehow this could be related as it seems the closest thing I could think of. The error I get when trying to power up a VM is: An unexpected error was received from the ESX host while powering on VM vm-21112. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Can you please post nfs log file from the Gluster server that you are trying to mount from? Thanks, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi, I don't see anything on the nfs log files when watching /var/log/glusterfs/nfs.log and trying to power on the machine at the same time. On the Glusterd logs I don't see anything as well. Anywhere else to check that it should be logging to ? Fernando -Original Message- From: Vijay Bellur [mailto:vbel...@redhat.com] Sent: 11 June 2012 17:54 To: Fernando Frediani (Qube) Cc: 'Atha Kouroussis'; 'gluster-users@gluster.org'; Rajesh Amaravathi; Krishna Srinivas Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 On 06/11/2012 05:52 PM, Fernando Frediani (Qube) wrote: > Was doing some read on RedHat website and found this URL which I wonder if > the problem would have anything to do with this: > http://docs.redhat.com/docs/en-US/Red_Hat_Storage_Software_Appliance/3 > .2/html/User_Guide/ch14s04s08.html > > Although both servers and client are 64 I wonder if somehow this could be > related as it seems the closest thing I could think of. > > The error I get when trying to power up a VM is: > > An unexpected error was received from the ESX host while powering on VM > vm-21112. > Failed to power on VM. > Unable to retrieve the current working directory: 0 (No such file or > directory). Check if the directory has been deleted or unmounted. > Unable to retrieve the current working directory: 0 (No such file or > directory). Check if the directory has been deleted or unmounted. > Unable to retrieve the current working directory: 0 (No such file or > directory). Check if the directory has been deleted or unmounted. > > Can you please post nfs log file from the Gluster server that you are trying to mount from? Thanks, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
On 06/11/2012 05:52 PM, Fernando Frediani (Qube) wrote: Was doing some read on RedHat website and found this URL which I wonder if the problem would have anything to do with this: http://docs.redhat.com/docs/en-US/Red_Hat_Storage_Software_Appliance/3.2/html/User_Guide/ch14s04s08.html Although both servers and client are 64 I wonder if somehow this could be related as it seems the closest thing I could think of. The error I get when trying to power up a VM is: An unexpected error was received from the ESX host while powering on VM vm-21112. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Can you please post nfs log file from the Gluster server that you are trying to mount from? Thanks, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Was doing some read on RedHat website and found this URL which I wonder if the problem would have anything to do with this: http://docs.redhat.com/docs/en-US/Red_Hat_Storage_Software_Appliance/3.2/html/User_Guide/ch14s04s08.html Although both servers and client are 64 I wonder if somehow this could be related as it seems the closest thing I could think of. The error I get when trying to power up a VM is: An unexpected error was received from the ESX host while powering on VM vm-21112. Failed to power on VM. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Unable to retrieve the current working directory: 0 (No such file or directory). Check if the directory has been deleted or unmounted. Fernando -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Fernando Frediani (Qube) Sent: 07 June 2012 16:53 To: 'Atha Kouroussis'; 'gluster-users@gluster.org' Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Hi Atha, I have a very similar setup and behaviour here. I have two bricks with replication and I am able to mount the NFS, deploy a machine there, but when I try to Power it On it simply doesn't work and gives a different message saying that it couldn't find some files. I wonder if anyone actually got it working with VMware ESXi and can share with us their scenario setup. Here I have two CentOS 6.2 and Gluster 3.3.0. Fernando -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Atha Kouroussis Sent: 07 June 2012 15:29 To: gluster-users@gluster.org Subject: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Hi everybody, we are testing Gluster 3.3 as an alternative to our current Nexenta based storage. With the introduction of granular based locking gluster seems like a viable alternative for VM storage. Regrettably we cannot get it to work even for the most rudimentary tests. We have a two brick setup with two ESXi 5 servers. We created both distributed and replicated volumes. We can mount the volumes via NFS on the ESXi servers without any issues but that is as far as we can go. When we try to migrate a VM to the gluster backed datastore there is no activity on the bricks and eventually the operation times out on the ESXi side. The nfs.log shows messages like these (distributed volume): [2012-06-07 00:00:16.992649] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : 7d25cb9a-b9c8-440d-bbd8-973694ccad17 [2012-06-07 00:00:17.027559] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d69: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.066276] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d90: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.097118] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : ----0001 When the volume is mounted on the ESXi servers, we get messages like these in nfs.log: [2012-06-06 23:57:34.697460] W [socket.c:195:__socket_rwv] 0-socket.nfs-server: readv failed (Connection reset by peer) The same volumes mounted via NFS on a linux box work fine and we did a couple of benchmarks with bonnie++ with very promising results. Curiously, if we ssh into the ESXi boxes and go to the mount point of the volume, we can see it contents and write. Any clues of what might be going on? Thanks in advance. Cheers, Atha ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
I don't think there are many people using it with VMware specifically nor the people who develop it have probably tested it much. I also suspect is some problem with NFS settings and wonder if it is possible to change it if you use that environment only for running virtual machines. I know that different from a normal Linux mount ESXi has some special way to mount it, that if not configured on the server side things won't work. Fernando -Original Message- From: Atha Kouroussis [mailto:akourous...@gmail.com] Sent: 08 June 2012 05:46 To: Fernando Frediani (Qube) Cc: gluster-users@gluster.org Subject: Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Hi Fernando, thanks for the reply. I'm seeing exactly the same behavior. I'm wondering if it somehow has to do with locking. I read here (http://community.gluster.org/q/can-not-mount-nfs-share-without-nolock-option/) that locking on NFS was not implemented in 3.2.x and it is now in 3.3. I tested 3.2.x with ESXi a few months ago and it seemed to work fine but the lack of granular locking made it a no-go back then. Anybody care to chime in with any suggestions? Is there a way to revert NFS to 3.2.x behavior to test? Cheers, Atha On Thursday, June 7, 2012 at 11:52 AM, Fernando Frediani (Qube) wrote: > Hi Atha, > > I have a very similar setup and behaviour here. > I have two bricks with replication and I am able to mount the NFS, deploy a > machine there, but when I try to Power it On it simply doesn't work and gives > a different message saying that it couldn't find some files. > > I wonder if anyone actually got it working with VMware ESXi and can share > with us their scenario setup. Here I have two CentOS 6.2 and Gluster 3.3.0. > > Fernando > > -Original Message- > From: gluster-users-boun...@gluster.org > [mailto:gluster-users-boun...@gluster.org] On Behalf Of Atha Kouroussis > Sent: 07 June 2012 15:29 > To: gluster-users@gluster.org (mailto:gluster-users@gluster.org) > Subject: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > Hi everybody, > we are testing Gluster 3.3 as an alternative to our current Nexenta based > storage. With the introduction of granular based locking gluster seems like a > viable alternative for VM storage. > > Regrettably we cannot get it to work even for the most rudimentary tests. We > have a two brick setup with two ESXi 5 servers. We created both distributed > and replicated volumes. We can mount the volumes via NFS on the ESXi servers > without any issues but that is as far as we can go. > > When we try to migrate a VM to the gluster backed datastore there is no > activity on the bricks and eventually the operation times out on the ESXi > side. The nfs.log shows messages like these (distributed volume): > > [2012-06-07 00:00:16.992649] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: > Unable to resolve FH: (192.168.11.11:646) vmvol : > 7d25cb9a-b9c8-440d-bbd8-973694ccad17 > [2012-06-07 00:00:17.027559] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: > 3bb48d69: /TEST => -1 (Directory not empty) > [2012-06-07 00:00:17.066276] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: > 3bb48d90: /TEST => -1 (Directory not empty) > [2012-06-07 00:00:17.097118] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: > Unable to resolve FH: (192.168.11.11:646) vmvol : > ----0001 > > > When the volume is mounted on the ESXi servers, we get messages like these in > nfs.log: > > [2012-06-06 23:57:34.697460] W [socket.c:195:__socket_rwv] > 0-socket.nfs-server: readv failed (Connection reset by peer) > > > The same volumes mounted via NFS on a linux box work fine and we did a couple > of benchmarks with bonnie++ with very promising results. > Curiously, if we ssh into the ESXi boxes and go to the mount point of the > volume, we can see it contents and write. > > Any clues of what might be going on? Thanks in advance. > > Cheers, > Atha > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org (mailto:Gluster-users@gluster.org) > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi Fernando, thanks for the reply. I'm seeing exactly the same behavior. I'm wondering if it somehow has to do with locking. I read here (http://community.gluster.org/q/can-not-mount-nfs-share-without-nolock-option/) that locking on NFS was not implemented in 3.2.x and it is now in 3.3. I tested 3.2.x with ESXi a few months ago and it seemed to work fine but the lack of granular locking made it a no-go back then. Anybody care to chime in with any suggestions? Is there a way to revert NFS to 3.2.x behavior to test? Cheers, Atha On Thursday, June 7, 2012 at 11:52 AM, Fernando Frediani (Qube) wrote: > Hi Atha, > > I have a very similar setup and behaviour here. > I have two bricks with replication and I am able to mount the NFS, deploy a > machine there, but when I try to Power it On it simply doesn't work and gives > a different message saying that it couldn't find some files. > > I wonder if anyone actually got it working with VMware ESXi and can share > with us their scenario setup. Here I have two CentOS 6.2 and Gluster 3.3.0. > > Fernando > > -Original Message- > From: gluster-users-boun...@gluster.org > [mailto:gluster-users-boun...@gluster.org] On Behalf Of Atha Kouroussis > Sent: 07 June 2012 15:29 > To: gluster-users@gluster.org (mailto:gluster-users@gluster.org) > Subject: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 > > Hi everybody, > we are testing Gluster 3.3 as an alternative to our current Nexenta based > storage. With the introduction of granular based locking gluster seems like a > viable alternative for VM storage. > > Regrettably we cannot get it to work even for the most rudimentary tests. We > have a two brick setup with two ESXi 5 servers. We created both distributed > and replicated volumes. We can mount the volumes via NFS on the ESXi servers > without any issues but that is as far as we can go. > > When we try to migrate a VM to the gluster backed datastore there is no > activity on the bricks and eventually the operation times out on the ESXi > side. The nfs.log shows messages like these (distributed volume): > > [2012-06-07 00:00:16.992649] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: > Unable to resolve FH: (192.168.11.11:646) vmvol : > 7d25cb9a-b9c8-440d-bbd8-973694ccad17 > [2012-06-07 00:00:17.027559] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: > 3bb48d69: /TEST => -1 (Directory not empty) > [2012-06-07 00:00:17.066276] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: > 3bb48d90: /TEST => -1 (Directory not empty) > [2012-06-07 00:00:17.097118] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: > Unable to resolve FH: (192.168.11.11:646) vmvol : > ----0001 > > > When the volume is mounted on the ESXi servers, we get messages like these in > nfs.log: > > [2012-06-06 23:57:34.697460] W [socket.c:195:__socket_rwv] > 0-socket.nfs-server: readv failed (Connection reset by peer) > > > The same volumes mounted via NFS on a linux box work fine and we did a couple > of benchmarks with bonnie++ with very promising results. > Curiously, if we ssh into the ESXi boxes and go to the mount point of the > volume, we can see it contents and write. > > Any clues of what might be going on? Thanks in advance. > > Cheers, > Atha > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org (mailto:Gluster-users@gluster.org) > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi Atha, I have a very similar setup and behaviour here. I have two bricks with replication and I am able to mount the NFS, deploy a machine there, but when I try to Power it On it simply doesn't work and gives a different message saying that it couldn't find some files. I wonder if anyone actually got it working with VMware ESXi and can share with us their scenario setup. Here I have two CentOS 6.2 and Gluster 3.3.0. Fernando -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Atha Kouroussis Sent: 07 June 2012 15:29 To: gluster-users@gluster.org Subject: [Gluster-users] Gluster 3.3.0 and VMware ESXi 5 Hi everybody, we are testing Gluster 3.3 as an alternative to our current Nexenta based storage. With the introduction of granular based locking gluster seems like a viable alternative for VM storage. Regrettably we cannot get it to work even for the most rudimentary tests. We have a two brick setup with two ESXi 5 servers. We created both distributed and replicated volumes. We can mount the volumes via NFS on the ESXi servers without any issues but that is as far as we can go. When we try to migrate a VM to the gluster backed datastore there is no activity on the bricks and eventually the operation times out on the ESXi side. The nfs.log shows messages like these (distributed volume): [2012-06-07 00:00:16.992649] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : 7d25cb9a-b9c8-440d-bbd8-973694ccad17 [2012-06-07 00:00:17.027559] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d69: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.066276] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d90: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.097118] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : ----0001 When the volume is mounted on the ESXi servers, we get messages like these in nfs.log: [2012-06-06 23:57:34.697460] W [socket.c:195:__socket_rwv] 0-socket.nfs-server: readv failed (Connection reset by peer) The same volumes mounted via NFS on a linux box work fine and we did a couple of benchmarks with bonnie++ with very promising results. Curiously, if we ssh into the ESXi boxes and go to the mount point of the volume, we can see it contents and write. Any clues of what might be going on? Thanks in advance. Cheers, Atha ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Gluster 3.3.0 and VMware ESXi 5
Hi everybody, we are testing Gluster 3.3 as an alternative to our current Nexenta based storage. With the introduction of granular based locking gluster seems like a viable alternative for VM storage. Regrettably we cannot get it to work even for the most rudimentary tests. We have a two brick setup with two ESXi 5 servers. We created both distributed and replicated volumes. We can mount the volumes via NFS on the ESXi servers without any issues but that is as far as we can go. When we try to migrate a VM to the gluster backed datastore there is no activity on the bricks and eventually the operation times out on the ESXi side. The nfs.log shows messages like these (distributed volume): [2012-06-07 00:00:16.992649] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : 7d25cb9a-b9c8-440d-bbd8-973694ccad17 [2012-06-07 00:00:17.027559] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d69: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.066276] W [nfs3.c:3525:nfs3svc_rmdir_cbk] 0-nfs: 3bb48d90: /TEST => -1 (Directory not empty) [2012-06-07 00:00:17.097118] E [nfs3.c:3551:nfs3_rmdir_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.11.11:646) vmvol : ----0001 When the volume is mounted on the ESXi servers, we get messages like these in nfs.log: [2012-06-06 23:57:34.697460] W [socket.c:195:__socket_rwv] 0-socket.nfs-server: readv failed (Connection reset by peer) The same volumes mounted via NFS on a linux box work fine and we did a couple of benchmarks with bonnie++ with very promising results. Curiously, if we ssh into the ESXi boxes and go to the mount point of the volume, we can see it contents and write. Any clues of what might be going on? Thanks in advance. Cheers, Atha ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
