Re: [Gluster-users] Fwd: files not syncing up with glusterfs 3.1.2
On Mon, 21 Feb 2011, Joe Landman wrote: On 02/21/2011 09:53 AM, David Lloyd wrote: Likewise, we're aware of the very poor performance of gluster with small files. We serve a lot of large files, and we're now moved most of the small files off to a normal nfs server. Again small files aren't known to break gluster are they? Small files are the bane of every cluster file system. We recommend using NFS client with GlusterFS for smaller files, simply due to the additional caching you can get out of the NFS system. Those who are looking for better metadata performance might want to see if MooseFS fits their needs. It uses a metadata server which caches the entire filesystem metadata in RAM, so it seems to be very responsive. I have my home directory on GlusterFS and I did a quick trial on MooseFS. du on GlusterFS (native FUSE, not NFS) was 1 minute 8 seconds, du on MooseFS was 2 seconds. Good metadata performance should equate to good small-file performance (but you'll want to run your own tests to be sure). Note that MooseFS stores data with a fixed ~64KB block size, so it will be somewhat wasteful of space on very small files. Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] autofs problem
3.0.6 also appears to have fixed the autofs problem. Thanks, Brent On Wed, 20 Oct 2010, Brent A Nelson wrote: On Wed, 20 Oct 2010, Amar Tumballi wrote: Brent, Can you please try with 3.1.0 ? (if its a new setup) I remember seeing this issue long back when I was fixing 'autofs' issues with 3.0.x release, and fixing it. Let me recheck it again. Regards, Amar It's not a new setup, but I went ahead and created an extremely simple Gluster share in 3.0.5, confirmed that it still had the autofs issue, and then did the same for a little 3.1.0 test. 3.1.0 did not have an autofs problem in this quick test, but 3.0.5 did. So, it does look like 3.1 includes a fix for this issue. This makes me wonder how hard it would be to migrate my existing 3.0 setup to 3.1. My volume specs predate volgen, although they are pretty much straightforward distributed replicate volumes; would my existing backend data be picked up okay if I created fresh 3.1 volumes but used my existing backend data areas? Or would there be some extended attributes in the data areas that might not match up with the new situation and cause problems? Thanks, Brent On Wed, Oct 20, 2010 at 1:06 AM, Brent A Nelson wrote: I'm working on replacing my Ubuntu 8.04 desktops with Ubuntu 10.04, but I've hit a snag. Automount hangs on glusterfs (tried 3.0.4 and 3.0.5) in the same manner as described on the RedHat Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=603378 So, it's apparently a problem in Fedora, too. It also looks like Phil Packer reported the same issue in February and perhaps the same issue was also reported by Christopher Nelson in May. mount -t glusterfs ... works just fine by hand, but when autofs calls it, the result is 5 lingering processes: the mount command, the mount.glusterfs command that it called, a glusterfs command, a zombie glusterfs, and then another glusterfs. It seems clear that autofs only called the mount command once, and mount.glusterfs seesmt o have only called glusterfs once, but glusterfs somehow failed and respawned a couple of times... Is there a fix or workaround (other than to not use autofs)? Ubuntu 8.04 doesn't seem to have this issue, although I have had some machines hang up eventually (with heavy computing/network use), and the symptoms seem to match a hung-up autofs, so it's possible a similar issue is present but much more subtle... Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Some client problems with TCP-only NFS in Gluster 3.1
Alas, that does not work on Solaris 2.6 or 7. Solaris 7 was apparently
the first to support the WebNFS URL syntax, but it otherwise has the same
behavior as 2.6. Both seem to be hardwired to look for the UDP mountd
port, even when told to use TCP. On each, you can specify the NFS port,
but apparently not the mountd port (there's no mountport option
documented).
Perhaps they got it right in Solaris 8 or greater, at least for WebNFS,
but I no longer have any newer Solaris with which to test...
Thanks,
Brent
On Fri, 22 Oct 2010, Craig Carl wrote:
{Resending due to incomplete response]
Brent,
Thanks for your feedback . To mount with a Solaris client use -
` mount -o proto=tcp,vers=3 nfs://:38467/ `
As to UDP access we want to force users to use TCP. Everything about Gluster is
designed to be fast , as NFS over UDP approaches line speed it becomes
increasingly inefficient, [1] we want to avoid that.
I have updated our documentation to reflect the required tcp option and Solaris
instructions.
[1] http://nfs.sourceforge.net/#faq_b10
Thanks again,
Craig
-->
Craig Carl
Senior Systems Engineer
Gluster
From: "Brent A Nelson"
To: gluster-users@gluster.org
Sent: Thursday, October 21, 2010 8:18:02 AM
Subject: [Gluster-users] Some client problems with TCP-only NFS in Gluster 3.1
I see that the built-in NFS support registers mountd in portmap only with
tcp and not udp. While this makes sense for a TCP-only NFS
implementation, it does cause problems for some clients:
Ubuntu 10.04 and 7.04 mount just fine.
Ubuntu 8.04 gives "requested NFS version or transport protocol is not
supported", unless you specify "-o mountproto=tcp" as a mount option, in
which case it works just fine.
Solaris 2.6 & 7 both give "RPC: Program not registered". Solaris
apparently doesn't support the mountproto=tcp option, so there doesn't
seem to be any way for Solaris clients to mount.
There may be other clients that assume mountd will be contactable via
udp, even though they (otherwise) happily support TCP NFS...
Thanks,
Brent
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] autofs problem
I'm working on replacing my Ubuntu 8.04 desktops with Ubuntu 10.04, but I've hit a snag. Automount hangs on glusterfs (tried 3.0.4 and 3.0.5) in the same manner as described on the RedHat Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=603378 So, it's apparently a problem in Fedora, too. It also looks like Phil Packer reported the same issue in February and perhaps the same issue was also reported by Christopher Nelson in May. mount -t glusterfs ... works just fine by hand, but when autofs calls it, the result is 5 lingering processes: the mount command, the mount.glusterfs command that it called, a glusterfs command, a zombie glusterfs, and then another glusterfs. It seems clear that autofs only called the mount command once, and mount.glusterfs seesmt o have only called glusterfs once, but glusterfs somehow failed and respawned a couple of times... Is there a fix or workaround (other than to not use autofs)? Ubuntu 8.04 doesn't seem to have this issue, although I have had some machines hang up eventually (with heavy computing/network use), and the symptoms seem to match a hung-up autofs, so it's possible a similar issue is present but much more subtle... Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Some client problems with TCP-only NFS in Gluster 3.1
I see that the built-in NFS support registers mountd in portmap only with tcp and not udp. While this makes sense for a TCP-only NFS implementation, it does cause problems for some clients: Ubuntu 10.04 and 7.04 mount just fine. Ubuntu 8.04 gives "requested NFS version or transport protocol is not supported", unless you specify "-o mountproto=tcp" as a mount option, in which case it works just fine. Solaris 2.6 & 7 both give "RPC: Program not registered". Solaris apparently doesn't support the mountproto=tcp option, so there doesn't seem to be any way for Solaris clients to mount. There may be other clients that assume mountd will be contactable via udp, even though they (otherwise) happily support TCP NFS... Thanks, Brent ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] autofs problem
On Wed, 20 Oct 2010, Amar Tumballi wrote: Brent, Can you please try with 3.1.0 ? (if its a new setup) I remember seeing this issue long back when I was fixing 'autofs' issues with 3.0.x release, and fixing it. Let me recheck it again. Regards, Amar It's not a new setup, but I went ahead and created an extremely simple Gluster share in 3.0.5, confirmed that it still had the autofs issue, and then did the same for a little 3.1.0 test. 3.1.0 did not have an autofs problem in this quick test, but 3.0.5 did. So, it does look like 3.1 includes a fix for this issue. This makes me wonder how hard it would be to migrate my existing 3.0 setup to 3.1. My volume specs predate volgen, although they are pretty much straightforward distributed replicate volumes; would my existing backend data be picked up okay if I created fresh 3.1 volumes but used my existing backend data areas? Or would there be some extended attributes in the data areas that might not match up with the new situation and cause problems? Thanks, Brent On Wed, Oct 20, 2010 at 1:06 AM, Brent A Nelson wrote: I'm working on replacing my Ubuntu 8.04 desktops with Ubuntu 10.04, but I've hit a snag. Automount hangs on glusterfs (tried 3.0.4 and 3.0.5) in the same manner as described on the RedHat Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=603378 So, it's apparently a problem in Fedora, too. It also looks like Phil Packer reported the same issue in February and perhaps the same issue was also reported by Christopher Nelson in May. mount -t glusterfs ... works just fine by hand, but when autofs calls it, the result is 5 lingering processes: the mount command, the mount.glusterfs command that it called, a glusterfs command, a zombie glusterfs, and then another glusterfs. It seems clear that autofs only called the mount command once, and mount.glusterfs seesmt o have only called glusterfs once, but glusterfs somehow failed and respawned a couple of times... Is there a fix or workaround (other than to not use autofs)? Ubuntu 8.04 doesn't seem to have this issue, although I have had some machines hang up eventually (with heavy computing/network use), and the symptoms seem to match a hung-up autofs, so it's possible a similar issue is present but much more subtle... Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] autofs problem
I'm working on replacing my Ubuntu 8.04 desktops with Ubuntu 10.04, but I've hit a snag. Automount hangs on glusterfs (tried 3.0.4 and 3.0.5) in the same manner as described on the RedHat Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=603378 So, it's apparently a problem in Fedora, too. It also looks like Phil Packer reported the same issue in February and perhaps the same issue was also reported by Christopher Nelson in May. mount -t glusterfs ... works just fine by hand, but when autofs calls it, the result is 5 lingering processes: the mount command, the mount.glusterfs command that it called, a glusterfs command, a zombie glusterfs, and then another glusterfs. It seems clear that autofs only called the mount command once, and mount.glusterfs seesmt o have only called glusterfs once, but glusterfs somehow failed and respawned a couple of times... Is there a fix or workaround (other than to not use autofs)? Ubuntu 8.04 doesn't seem to have this issue, although I have had some machines hang up eventually (with heavy computing/network use), and the symptoms seem to match a hung-up autofs, so it's possible a similar issue is present but much more subtle... Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
