Re: [Gluster-users] Replicate over WAN?
Count Zero, Just to clarify, the three patches were not "released", they will make into a future release after proper testing. They are not developed to fix WAN issues per se, just to help with small files and slow "ls" kind of problems which are seen with huge numbers of files or huge number of small files or else deep directories. As a side effect, they *may* fix some of your WAN issues :-). Let me know how things go. Please don't use in production. Regards, Tejas. - Original Message - From: "Count Zero" To: "Vikas Gorur" Cc: gluster-users@gluster.org Sent: Thursday, May 6, 2010 1:45:32 AM Subject: Re: [Gluster-users] Replicate over WAN? > > > Replicate is not really designed for a WAN environment. A couple of things > that are probably affecting you are: > > 1) "lookup" (first access to any file or directory) needs to be sent to all > subvolumes to gather information to determine if self-heal is needed. There > is no way to fix this without losing the ability to self-heal. > > 2) "readdir" (ls) is always sent to the first subvolume. This is necessary to > ensure consistent inode numbers. Perhaps you could ensure that the first > subvolume is local? (Make sure the order of subvolumes is the same on all > your clients.) > Wait a sec. I believe there's a possible conflict in point (2), or perhaps I misunderstood: - Ensure the first subvolume is the local one - I am assuming that in order to do this, I need make the local subvolume the first one in the list of volumes, in the 'subvolumes' option - If I do this on every client, it means the order of subvolumes can not be the same, since on every client the local subvolume will be the first in the list. Can you confirm the above is ok? Thanks! :-) Also, I noticed this morning 3 patches were released to address the WAN issue? I will try them on a parallel test system first, and run some measurements. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Replicate over WAN?
On May 5, 2010, at 1:15 PM, Count Zero wrote: > > Wait a sec. I believe there's a possible conflict in point (2), or perhaps I > misunderstood: > > - Ensure the first subvolume is the local one - I am assuming that in order > to do this, I need make the local subvolume the first one in the list of > volumes, in the 'subvolumes' option > - If I do this on every client, it means the order of subvolumes can not be > the same, since on every client the local subvolume will be the first in the > list. Your understanding is correct. You can set first = local on only one of your sites. -- Vikas Gorur Engineer - Gluster, Inc. -- ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Replicate over WAN?
> > > Replicate is not really designed for a WAN environment. A couple of things > that are probably affecting you are: > > 1) "lookup" (first access to any file or directory) needs to be sent to all > subvolumes to gather information to determine if self-heal is needed. There > is no way to fix this without losing the ability to self-heal. > > 2) "readdir" (ls) is always sent to the first subvolume. This is necessary to > ensure consistent inode numbers. Perhaps you could ensure that the first > subvolume is local? (Make sure the order of subvolumes is the same on all > your clients.) > Wait a sec. I believe there's a possible conflict in point (2), or perhaps I misunderstood: - Ensure the first subvolume is the local one - I am assuming that in order to do this, I need make the local subvolume the first one in the list of volumes, in the 'subvolumes' option - If I do this on every client, it means the order of subvolumes can not be the same, since on every client the local subvolume will be the first in the list. Can you confirm the above is ok? Thanks! :-) Also, I noticed this morning 3 patches were released to address the WAN issue? I will try them on a parallel test system first, and run some measurements. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Replicate over WAN?
On May 4, 2010, at 9:01 PM, Count Zero wrote: > I have some machines over WAN, in a Replicate cluster, with around 50ms ~ > 60ms between them. > However, I have read-subvolume specified so that it will always use the local > brick instead of the WAN. > I just need this in order to replicate files as easily and as quickly as > possible between various systems... > > My question is - Why is it still painfully slow when all I do is read > operations? > Even just listing a directory takes ages. > I can understand about Write operations, but read? and from the local > sub-volume? Replicate is not really designed for a WAN environment. A couple of things that are probably affecting you are: 1) "lookup" (first access to any file or directory) needs to be sent to all subvolumes to gather information to determine if self-heal is needed. There is no way to fix this without losing the ability to self-heal. 2) "readdir" (ls) is always sent to the first subvolume. This is necessary to ensure consistent inode numbers. Perhaps you could ensure that the first subvolume is local? (Make sure the order of subvolumes is the same on all your clients.) -- Vikas Gorur Engineer - Gluster, Inc. -- ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Compilation problem on Solaris 10
Joris Vuffray wrote: Hi List, I'm unable to compile glusterfs on Solaris 10. Getting this error: - Making all in src .deps/posix.Tpo -c posix.c -fPIC -DPIC -o .libs/posix.o posix.c: In function `posix_lstat_with_gen': posix.c:274: error: structure has no member named `st_mode' posix.c:274: error: structure has no member named `st_mode' posix.c: In function `posix_fstat_with_gen': posix.c:345: error: structure has no member named `st_mode' posix.c:345: error: structure has no member named `st_mode' Can you please confirm the version of glusterfs that you are trying to compile? Regards, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Compilation problem on Solaris 10
Hi List, I'm unable to compile glusterfs on Solaris 10. Getting this error: - Making all in src /bin/bash ../../../../libtool --tag=CC--mode=compile gcc -DHAVE_CONFIG_H -I. -I../../../.. -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DGF_SOLARIS_HOST_OS -Wall -I../../../../libglusterfs/src -shared -nostartfiles -I../../../../argp-standalone -D_REENTRANT -g -O2 -MT posix.lo -MD -MP -MF .deps/posix.Tpo -c -o posix.lo posix.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../../.. -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DGF_SOLARIS_HOST_OS -Wall -I../../../../libglusterfs/src -nostartfiles -I../../../../argp-standalone -D_REENTRANT -g -O2 -MT posix.lo -MD -MP -MF .deps/posix.Tpo -c posix.c -fPIC -DPIC -o .libs/posix.o posix.c: In function `posix_lstat_with_gen': posix.c:274: error: structure has no member named `st_mode' posix.c:274: error: structure has no member named `st_mode' posix.c: In function `posix_fstat_with_gen': posix.c:345: error: structure has no member named `st_mode' posix.c:345: error: structure has no member named `st_mode' *** Error code 1 make: Fatal error: Command failed for target `posix.lo' Current working directory /root/glusterfs/xlators/storage/posix/src *** Error code 1 - Any Idea? Regards, Jo ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Compilation problem on Solaris 10
Hi List, I'm unable to compile glusterfs on Solaris 10. Getting this error: - Making all in src /bin/bash ../../../../libtool --tag=CC--mode=compile gcc -DHAVE_CONFIG_H -I. -I../../../.. -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DGF_SOLARIS_HOST_OS -Wall -I../../../../libglusterfs/src -shared -nostartfiles -I../../../../argp-standalone -D_REENTRANT -g -O2 -MT posix.lo -MD -MP -MF .deps/posix.Tpo -c -o posix.lo posix.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../../.. -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DGF_SOLARIS_HOST_OS -Wall -I../../../../libglusterfs/src -nostartfiles -I../../../../argp-standalone -D_REENTRANT -g -O2 -MT posix.lo -MD -MP -MF .deps/posix.Tpo -c posix.c -fPIC -DPIC -o .libs/posix.o posix.c: In function `posix_lstat_with_gen': posix.c:274: error: structure has no member named `st_mode' posix.c:274: error: structure has no member named `st_mode' posix.c: In function `posix_fstat_with_gen': posix.c:345: error: structure has no member named `st_mode' posix.c:345: error: structure has no member named `st_mode' *** Error code 1 make: Fatal error: Command failed for target `posix.lo' Current working directory /root/glusterfs/xlators/storage/posix/src *** Error code 1 - Any Idea? Regards, Jo ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
