[Gluster-users] root squashing
I have a really simple glusterfs setup. Used glusterfs-volgen --name glusterStore --transport tcp host1:/mnt/raid10 host2:/mnt/raid10 to create the necessary files. And mounted the system with glusterfs --volfile=/etc/glusterfs/glusterfs.vol /mnt/glusterfs/ on the clients. However, with this setup there is no "root squashing", which is something I need. How can this be achieved? Regards, /jon___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Question About GlusterFS Backed Xen DomU's
Hi Everyone, Hope your day has been going well. I have a quick question for everyone. I am currently using GlusterFS as a distributed SAN backend for the Xen based cloud platform we are developing. We deploy Xen virtuals on pairs of servers using GlusterFs V3 in replicate mode on Debian Stable (Lenny) with Xen 3.2.1 as the hypervisor. I am currently experiencing a weird issue where within the virtual machines (DomU) running on a GlusterFS mount only receive around 10-18MB/s write speeds, but full speed reads. Our hardware for each node is Dual Core Xeon Processors, 8GB of RAM and 4 * High Speed SATA drives (RAID 10, around 160MB/s writes and reads). If I write a file to the Gluster mount in the Dom0 (host) we receive around 90-100MB/s writes (maxing out the GigE link). If I run the virtual machine on the disks without Gluster I get much higher speeds within the DomU of around 80-90MB/s. This slow down only appears to occur on writes. Does anyone with a better understanding of GlusterFS, Fuse and filesystems have an idea why this is slowing down. The underlying file system is Ext3. Many Thanks Sheng ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git
Hello, Do you have MMAP enabled in httpd.conf? If so, try turning it off (EnableMMAP off). The default setting is on. I'm using Centos 5.5 with GlusterFS 3.0.4 and haven't seen such problems with my setup. -samuli - Alkuperäinen viesti - > If I use --disable-direct-io-mode when mounting the glusterfs document > root, I don't seem to have this issue with apache. > > Is this normal? > > > > This message (including any attachments) contains confidential > information intended for a specific individual and purpose, and is > protected by law. If you are not the intended recipient, you should > delete this message. Any disclosure, copying, or distribution of this > message, or the taking of any action based on it, is strictly > prohibited. ___ > Gluster-users mailing list Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Release announcements?
Is there a channel I'm not seeing to get release announcements? I only found out 3.0.5 was out by seeing it recommended in mailing list threads. Just wondering... -- Joshua Baker-LePain QB3 Shared Cluster Sysadmin UCSF ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git
If I use --disable-direct-io-mode when mounting the glusterfs document root, I don't seem to have this issue with apache. Is this normal? This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] fuse-bridge.c:796:fuse_getattr] glusterfs-fuse (fuse_loc_fill() failed)
Hello, I'm running glusterfs v3.0.5 with 4 servers and 4 clients. Kernel version is 2.6.24-9. I'm getting an error like this when trying to list files on a directory: ls: cannot access /mnt/glusterfs/directory: No such file or directory At glusterfs client log i see multiple messages like this one: [2010-07-14 20:28:06] W [fuse-bridge.c:796:fuse_getattr] glusterfs-fuse: 13289745: GETATTR 3045123208 (fuse_loc_fill() failed) This directory is accessed by a web server, more specifically, Tomcat. I read other topic at this list that says that this is caused by race conditions, but no solution was provided. Is there something that i can do solve this issue? Thanks! --- Fernando Marcelo ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git
Hmm. Ok. I'm using Centos 5.5, tried both the fusemount and fusemount-glusterfs. -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Alex/AT Sent: Wednesday, July 14, 2010 1:31 PM To: Gluster General Discussion List Subject: Re: [Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git > I seem to be getting partial 206 erros on images when I serve web > content from a glusterfs fuse mount. It happens to all images it appears. > I don't see any errors in the logs for gluster. > I have taken the exact image, copied it to tmp, and added a symlink from > the glusterfs to the hard mount point and the apache server will serve > it. > Am I missing anything to get apache to serve content from a glusterfs? > Scratching my head on this one. Probably not a GlusterFS issue. I am serving three sites and a forum from GlusterFS, and everything is OK. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git
I seem to be getting partial 206 erros on images when I serve web content from a glusterfs fuse mount. It happens to all images it appears. I don't see any errors in the logs for gluster. I have taken the exact image, copied it to tmp, and added a symlink from the glusterfs to the hard mount point and the apache server will serve it. Am I missing anything to get apache to serve content from a glusterfs? Scratching my head on this one. Probably not a GlusterFS issue. I am serving three sites and a forum from GlusterFS, and everything is OK. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Slow Writes on GlusterFS Backed Xen DomU's
Hi Everyone, Hope your day has been going well. I have a quick question for everyone. I am currently using GlusterFS as a distributed SAN backend for the Xen based cloud platform we are developing. We deploy Xen virtuals on pairs of servers using GlusterFs V3 in replicate mode on Debian Stable (Lenny) with Xen 3.2.1 as the hypervisor. I am currently experiencing a weird issue where within the virtual machines (DomU) running on a GlusterFS mount only receive around 10-18MB/s write speeds, but full speed reads. Our hardware for each node is Dual Core Xeon Processors, 8GB of RAM and 4 * High Speed SATA drives (RAID 10, around 160MB/s writes and reads). If I write a file to the Gluster mount in the Dom0 (host) we receive around 90-100MB/s writes (maxing out the GigE link). If I run the virtual machine on the disks without Gluster I get much higher speeds within the DomU of around 80-90MB/s. This slow down only appears to occur on writes. Does anyone with a better understanding of GlusterFS, Fuse and filesystems have an idea why this is slowing down. The underlying file system is Ext3. Many Thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Odd issue with apache 2.2.15 and glusterfs git 3.1.0git
I seem to be getting partial 206 erros on images when I serve web content from a glusterfs fuse mount. It happens to all images it appears. I don't see any errors in the logs for gluster. I have taken the exact image, copied it to tmp, and added a symlink from the glusterfs to the hard mount point and the apache server will serve it. Am I missing anything to get apache to serve content from a glusterfs? Scratching my head on this one. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Issue with Samba over Gluster
Pavan, As i said before that i cannot switch to gluster 3.0.5 as most of my applications dont work properly in 3.0.5 so i am giving u all the info required using 3.0.4 version. I used to get the same issue with 3.0.5 also with samba. Here are the trace logs : http://dpaste.com/217973/ I have attached core dump file. Please have a look and reply. Irshad On Tue, Jul 13, 2010 at 1:10 PM, Irshad Ahmed wrote: > ThanxI will surely do what you have asked. > I was using 3.0.5...But had to remove it as my Jira Atlassian applications > were crashing and i dont know why. > So reverted back to 3.0.4 and they started working normally. > Whenever i mounted my gluster volume it used to show split brain afr error > in the logs. > But as far as samba is concerned it not working in 3.0.4 nor in 3.0.5. > I will send in logs asap. > > Irshad > > On Tue, Jul 13, 2010 at 1:07 PM, Pavan Sondur wrote: > >> Hi Irshad, >> Are you using GlusterFS 3.0.5? If not, give 3.0.5 a try and let us know >> how it goes. If you run into this problem, try capturing some data which can >> be useful for debugging - >> * Use 'option trace on' in the locks translator in the vol file >> The will print debug info in the log (in normal logging mode). >> * If users are experiencing hung like behavior, initiate a process state >> dump - >> kill -USR1 >> This will dump the state of glusterfs in /tmp/glusterdump. >> >> Send us the state dump and the logs if you run into the issues you >> mentioned. >> >> Pavan >> - Original Message - >> From: "Irshad Ahmed" >> To: "Gluster General Discussion List" >> Sent: Sunday, July 11, 2010 10:41:43 AM >> Subject: [Gluster-users] Issue with Samba over Gluster >> >> Hey guys, >> >> I am facing an issue with samba. >> When i expose a gluster volume using samba so my windows users can connect >> to it. >> My windows users normally use Microsoft Office and some use OpenOffice as >> they store all Excel/PPT/Word file in samba share. >> Users who use OpenOffice are able to work properly without any issues. >> But users who use Microsoft Office are not able to even open the files >> exposed by samba. >> When they try to open a file they get downloading file and then it >> timeouts >> and gives an error cannot read from source. >> >> This is my Gluster Server Config >> >> # Filestore Volume to be mounted >> volume gluster-filestore-data >> type storage/posix >> option directory /data/mpath2/filestore-data >> end-volume >> >> volume gluster-filestore-data-lock >> type features/locks >> subvolumes gluster-filestore-data >> end-volume >> >> volume gluster-filestore-data-brick >> type performance/io-threads >> option thread-count 16 >> subvolumes gluster-filestore-data-lock >> end-volume >> >> volume gluster-filestore-data-server >> type protocol/server >> option transport-type tcp >> option transport.socket.listen-port 7001 >> option auth.addr.gluster-filestore-data-brick.allow 10.15.* >> subvolumes gluster-filestore-data-brick >> end-volume >> >> Gluster Client Config >> >> volume gluster-filestore-data-1 >> type protocol/client >> option transport-type tcp >> option remote-host 10.15.6.10 >> option remote-port 7001 >> option remote-subvolume gluster-filestore-data-brick >> end-volume >> >> volume gluster-filestore-data-2 >> type protocol/client >> option transport-type tcp >> option remote-host 10.15.6.11 >> option remote-port 7001 >> option remote-subvolume gluster-filestore-data-brick >> end-volume >> >> volume replicate >> type cluster/replicate >> subvolumes gluster-filestore-data-1 gluster-filestore-data-2 >> end-volume >> >> volume writebehind >> type performance/write-behind >> option cache-size 1MB >> subvolumes replicate >> end-volume >> >> volume cache >> type performance/io-cache >> option cache-size 512MB >> subvolumes writebehind >> end-volume >> >> I tested samba without gluster and it works fine so this is not a samba >> issue. >> I did some more testing to find out what happening with samba over >> gluster. >> I check the logs of samba and when i access files using MS Office i get >> oplock errors. >> I tried disabling oplock and MS Office application was able to open the >> file >> but in read-only mode but OpenOffice stopepd working now. >> If i enable oplocks everything started working other than MS Office. >> I dont see any kind of crash logs or any other logging in gluster logs. >> >> Any help please >> >> -- >> Irshad Ahmed >> >> ___ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> > > > > -- > Irshad Ahmed > -- Irshad Ahmed ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] HA NFS
Hi Koen, We are infact looking at integrating this into the platform. Regards, Tejas. - Original Message - From: "Layer7 Consultancy" To: "Gluster General Discussion List" Sent: Wednesday, July 14, 2010 1:06:38 PM Subject: Re: [Gluster-users] HA NFS Thanks Vikas, I think I have enough information now to set up a proof of concept. Perhaps this (ucarp install by user) is something to be mentioned on the website? Even better would be integration into Gluster Platform ;-) Best regards, Koen 2010/7/14 Shehjar Tikoo : > Layer7 Consultancy wrote: >> >> Hello Vikas, >> >> How do the NFS clients react to such a failover? Will the I/O just >> temporarily stall and then proceed once connectivity is resumed or >> will running transactions be aborted? >> > > There may be some delay as the IP migrates but overall, the NFS client's > retransmission behaviour will be able continue the transactions with no > interruptions. > >> Best regards, >> Koen >> >> 2010/7/13 Vikas Gorur : >> Your understanding is correct. Whether using the native NFS >> translator or re-export through unfsd, NFS clients only connect to the >> management IP. If you want failover for the NFS server, you'd setup a >> virtual IP using ucarp (http://www.ucarp.org) and the clients would >> only use this virtual IP. >>> >>> -- >>> Vikas Gorur >>> Engineer - Gluster, Inc. >>> -- >> >> ___ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] HA NFS
Thanks Vikas, I think I have enough information now to set up a proof of concept. Perhaps this (ucarp install by user) is something to be mentioned on the website? Even better would be integration into Gluster Platform ;-) Best regards, Koen 2010/7/14 Shehjar Tikoo : > Layer7 Consultancy wrote: >> >> Hello Vikas, >> >> How do the NFS clients react to such a failover? Will the I/O just >> temporarily stall and then proceed once connectivity is resumed or >> will running transactions be aborted? >> > > There may be some delay as the IP migrates but overall, the NFS client's > retransmission behaviour will be able continue the transactions with no > interruptions. > >> Best regards, >> Koen >> >> 2010/7/13 Vikas Gorur : >> Your understanding is correct. Whether using the native NFS >> translator or re-export through unfsd, NFS clients only connect to the >> management IP. If you want failover for the NFS server, you'd setup a >> virtual IP using ucarp (http://www.ucarp.org) and the clients would >> only use this virtual IP. >>> >>> -- >>> Vikas Gorur >>> Engineer - Gluster, Inc. >>> -- >> >> ___ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] HA NFS
Layer7 Consultancy wrote: Hello Vikas, How do the NFS clients react to such a failover? Will the I/O just temporarily stall and then proceed once connectivity is resumed or will running transactions be aborted? There may be some delay as the IP migrates but overall, the NFS client's retransmission behaviour will be able continue the transactions with no interruptions. Best regards, Koen 2010/7/13 Vikas Gorur : Your understanding is correct. Whether using the native NFS translator or re-export through unfsd, NFS clients only connect to the management IP. If you want failover for the NFS server, you'd setup a virtual IP using ucarp (http://www.ucarp.org) and the clients would only use this virtual IP. -- Vikas Gorur Engineer - Gluster, Inc. -- ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
