Re: [Gluster-users] Gluster 3.2.5 client crash report
Hello Krish I changed my log-level in my log files to DEBUG to see if things are clearer, will attach it here if it crash again ( i think it will happen soon :( ) Our old log had nothing special but lines i posted up here before, plus a ton of message about NFS stale file like this: /E [client3_1-fops.c:2228:client3_1_lookup_cbk] 0-cdt_store-client-1: remote operation failed: Stale NFS file handle/ even though i'm using native gluster client ( fuse mount ) and disable nfs on gluster server. I had this error in log file many time before and my node still worked well, so i don't think it's the cause of problem. Will update info for you guys soon. Many thanks for your help On 7/2/2012 1:28 PM, Krishnan Parthasarathi wrote: Hi Tungdam, Could you attach all the log files present under /var/log/glusterfs (in the case of an RPM installation) ? The log file corresponding to the crashing process would have a backtrace indicating potential reason for a crash. ~krish - Original Message - From: "tungdam" To: "Vijay Bellur" Cc: gluster-users@gluster.org Sent: Monday, July 2, 2012 7:29:39 AM Subject: Re: [Gluster-users] Gluster 3.2.5 client crash report Hello Vijay My gluster client keep being crashed. Reinstalled it from RPM package from gluster ( to exclude the problem related to my c compiler ) but didn't help. This time it crashed without any log and my server was rebooted. Any clue, guys ? :( Any help would be highly appreciated On 6/30/2012 10:16 AM, Vijay Bellur wrote:) On 06/29/2012 09:29 PM, tungdam wrote: Hello Vijay You mean the core dump file ? It is pretty big, i think, it's about > 1G after gzip-ed. Is that what you need ? ( I need to make sure that i'm not going to send you the wrong - and big - info ) No, I would need the next 15-20 lines after a reference to "package-string: glusterfs 3.2.5" in the log file of the client where the crash was seen. Regards, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hi Tungdam, Could you attach all the log files present under /var/log/glusterfs (in the case of an RPM installation) ? The log file corresponding to the crashing process would have a backtrace indicating potential reason for a crash. ~krish - Original Message - From: "tungdam" To: "Vijay Bellur" Cc: gluster-users@gluster.org Sent: Monday, July 2, 2012 7:29:39 AM Subject: Re: [Gluster-users] Gluster 3.2.5 client crash report Hello Vijay My gluster client keep being crashed. Reinstalled it from RPM package from gluster ( to exclude the problem related to my c compiler ) but didn't help. This time it crashed without any log and my server was rebooted. Any clue, guys ? :( Any help would be highly appreciated On 6/30/2012 10:16 AM, Vijay Bellur wrote:) > On 06/29/2012 09:29 PM, tungdam wrote: >> Hello Vijay >> >> You mean the core dump file ? It is pretty big, i think, it's about > 1G >> after gzip-ed. >> Is that what you need ? ( I need to make sure that i'm not going to send >> you the wrong - and big - info ) >> > > > No, I would need the next 15-20 lines after a reference to > "package-string: glusterfs 3.2.5" in the log file of the client where > the crash was seen. > > Regards, > Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hello Vijay My gluster client keep being crashed. Reinstalled it from RPM package from gluster ( to exclude the problem related to my c compiler ) but didn't help. This time it crashed without any log and my server was rebooted. Any clue, guys ? :( Any help would be highly appreciated On 6/30/2012 10:16 AM, Vijay Bellur wrote:) On 06/29/2012 09:29 PM, tungdam wrote: Hello Vijay You mean the core dump file ? It is pretty big, i think, it's about > 1G after gzip-ed. Is that what you need ? ( I need to make sure that i'm not going to send you the wrong - and big - info ) No, I would need the next 15-20 lines after a reference to "package-string: glusterfs 3.2.5" in the log file of the client where the crash was seen. Regards, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hello Vijay There's nothing after that string in my log file. After crashing, log file stopped printing output, and i had to remount it then. So the log file looks like this: /package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900] - [2012-06-27 23:02:43.446042] W [write-behind.c:3023:init] 0-cdt_store-write-behind: disabling write-behind for first 0 bytes Given volfile: +--+ 1: volume cdt_store-client-0 2: type protocol/client 3: option remote-host Gnode1 4: option remote-subvolume /mnt/store 5: option transport-type tcp 6: end-volume 7: 8: volume cdt_store-client-1 9: type protocol/client 10: option remote-host Gnode2 11: option remote-subvolume /mnt/store2 12: option transport-type tcp 13: end-volume 14: 15: volume cdt_store-dht 16: type cluster/distribute 17: option min-free-disk 5 18: subvolumes cdt_store-client-0 cdt_store-client-1 19: end-volume 20: 21: volume cdt_store-write-behind 22: type performance/write-behind 23: option cache-size 4MB 24: subvolumes cdt_store-dht 25: end-volume 26: 27: volume cdt_store-read-ahead 28: type performance/read-ahead 29: subvolumes cdt_store-write-behind 30: end-volume 31: 32: volume cdt_store-io-cache 33: type performance/io-cache 34: option cache-size 512MB 35: subvolumes cdt_store-read-ahead 36: end-volume/ Many thanks On 6/30/2012 10:16 AM, Vijay Bellur wrote: On 06/29/2012 09:29 PM, tungdam wrote: Hello Vijay You mean the core dump file ? It is pretty big, i think, it's about > 1G after gzip-ed. Is that what you need ? ( I need to make sure that i'm not going to send you the wrong - and big - info ) No, I would need the next 15-20 lines after a reference to "package-string: glusterfs 3.2.5" in the log file of the client where the crash was seen. Regards, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
On 06/29/2012 09:29 PM, tungdam wrote: Hello Vijay You mean the core dump file ? It is pretty big, i think, it's about > 1G after gzip-ed. Is that what you need ? ( I need to make sure that i'm not going to send you the wrong - and big - info ) No, I would need the next 15-20 lines after a reference to "package-string: glusterfs 3.2.5" in the log file of the client where the crash was seen. Regards, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hello Vijay You mean the core dump file ? It is pretty big, i think, it's about > 1G after gzip-ed. Is that what you need ? ( I need to make sure that i'm not going to send you the wrong - and big - info ) Best regards On 6/29/2012 8:39 PM, Vijay Bellur wrote: On 06/27/2012 09:20 PM, tungdam wrote: Hello there Our gluster client was crashed 2 times last night. The gluster partition was unavailable for a long time and i have to remount it manually. On the gluster log, i saw things like these: /pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-06-28 00:19:24 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900]/ Can you please post the backtrace that usually follows the above in the log file? Thanks, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hi George I tried installing gluster from rpm ( download from here: http://download.gluster.org/pub/gluster/glusterfs/3.2/3.2.5/CentOS/6/ ) yesterday and waiting for the result now. Will try your suggestion soon, if it crashes once again. Many thanks for your help On 6/29/2012 8:26 PM, George Machitidze wrote: Try EPEL repo RPM's http://fedoraproject.org/wiki/EPEL Best regards, George Machitidze On Fri, Jun 29, 2012 at 12:58 PM, tungdam wrote: Hi George On 6/29/2012 3:46 PM, George Machitidze wrote: Hi Any similarities in activities, circumstances, crash time? No, it's kind of random crashing. Run well after 10 or 12 hours and then crashed, without any performance degrading signature. This server is a web server, we have some changes 2 days ago, maybe it's making heavier I/O operation than before. Just let u know :) rpm -qa|grep -E "gluster|fuse|kernel";uname -a;cat /etc/*-release Here is info you asked: rpm -qa | grep -E "gluster|fuse|kernel" kernel-firmware-2.6.32-220.el6.noarch dracut-kernel-004-256.el6.noarch abrt-addon-kerneloops-2.0.4-14.el6.centos.x86_64 kernel-headers-2.6.32-220.4.2.el6.x86_64 kernel-2.6.32-220.el6.x86_64 libreport-plugin-kerneloops-2.0.5-20.el6.x86_64 uname -a Linux web244 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux cat /etc/*-release CentOS release 6.2 (Final) CentOS release 6.2 (Final) CentOS release 6.2 (Final) lsmod | grep fuse fuse 66285 3 I have fuse module already, and didn't install any package from yum before, do i need to install it ? Many thanks Best regards, George Machitidze On Fri, Jun 29, 2012 at 12:40 PM, tungdam wrote: it keep being crashed everyday. I tried reinstalling ( from source code ) but it didn't help, will try installing from RPM to see if it's because of my compiler or not. Any idea, guys ? Any help would be highly appreciated :) On 6/28/2012 8:20 AM, tungdam wrote: Hello there Our gluster client was crashed 2 times last night. The gluster partition was unavailable for a long time and i have to remount it manually. On the gluster log, i saw things like these: pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-06-28 00:19:24 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900] Here is my "gluster volume info" output: Volume Name: xx Type: Distribute Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: Gnode1:/mnt/store Brick2: Gnode2:/mnt/store2 Options Reconfigured: nfs.addr-namelookup: off nfs.rpc-auth-allow: 10.0.0.245,10.0.0.244,10.0.0.247,10.0.0.54,10.0.0.55 auth.allow: 10.* cluster.min-free-disk: 5 performance.io-thread-count: 64 performance.cache-size: 512MB nfs.disable: off performance.write-behind-window-size: 4MB cluster.data-self-heal: off performance.stat-prefetch: off I googled around and found some people have the same problem with us but haven't found solution or any clues of what happened yet. Is it a bug ? and if it's could you please tell me how can i overcome it ? Many thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
On 06/27/2012 09:20 PM, tungdam wrote: Hello there Our gluster client was crashed 2 times last night. The gluster partition was unavailable for a long time and i have to remount it manually. On the gluster log, i saw things like these: /pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-06-28 00:19:24 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900]/ Can you please post the backtrace that usually follows the above in the log file? Thanks, Vijay ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Try EPEL repo RPM's http://fedoraproject.org/wiki/EPEL Best regards, George Machitidze On Fri, Jun 29, 2012 at 12:58 PM, tungdam wrote: > Hi George > > On 6/29/2012 3:46 PM, George Machitidze wrote: > > Hi > > Any similarities in activities, circumstances, crash time? > > No, it's kind of random crashing. Run well after 10 or 12 hours and then > crashed, without any performance degrading signature. This server is a web > server, we have some changes 2 days ago, maybe it's making heavier I/O > operation than before. Just let u know :) > > > rpm -qa|grep -E "gluster|fuse|kernel";uname -a;cat /etc/*-release > > Here is info you asked: > > rpm -qa | grep -E "gluster|fuse|kernel" > > kernel-firmware-2.6.32-220.el6.noarch > dracut-kernel-004-256.el6.noarch > abrt-addon-kerneloops-2.0.4-14.el6.centos.x86_64 > kernel-headers-2.6.32-220.4.2.el6.x86_64 > kernel-2.6.32-220.el6.x86_64 > libreport-plugin-kerneloops-2.0.5-20.el6.x86_64 > > uname -a > Linux web244 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 > x86_64 x86_64 GNU/Linux > > cat /etc/*-release > CentOS release 6.2 (Final) > CentOS release 6.2 (Final) > CentOS release 6.2 (Final) > > lsmod | grep fuse > fuse 66285 3 > > I have fuse module already, and didn't install any package from yum before, > do i need to install it ? > > Many thanks > > > > Best regards, > George Machitidze > > > On Fri, Jun 29, 2012 at 12:40 PM, tungdam wrote: > > it keep being crashed everyday. I tried reinstalling ( from source code ) > but it didn't help, will try installing from RPM to see if it's because of > my compiler or not. Any idea, guys ? > > Any help would be highly appreciated :) > > On 6/28/2012 8:20 AM, tungdam wrote: > > Hello there > > Our gluster client was crashed 2 times last night. The gluster partition was > unavailable for a long time and i have to remount it manually. On the > gluster log, i saw things like these: > > pending frames: > frame : type(1) op(LOOKUP) > frame : type(1) op(LOOKUP) > > patchset: git://git.gluster.com/glusterfs.git > signal received: 11 > time of crash: 2012-06-28 00:19:24 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > fdatasync 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 3.2.5 > /lib64/libc.so.6[0x3cf0c32900] > > > Here is my "gluster volume info" output: > > Volume Name: xx > Type: Distribute > Status: Started > Number of Bricks: 2 > Transport-type: tcp > Bricks: > Brick1: Gnode1:/mnt/store > Brick2: Gnode2:/mnt/store2 > Options Reconfigured: > nfs.addr-namelookup: off > nfs.rpc-auth-allow: 10.0.0.245,10.0.0.244,10.0.0.247,10.0.0.54,10.0.0.55 > auth.allow: 10.* > cluster.min-free-disk: 5 > performance.io-thread-count: 64 > performance.cache-size: 512MB > nfs.disable: off > performance.write-behind-window-size: 4MB > cluster.data-self-heal: off > performance.stat-prefetch: off > > I googled around and found some people have the same problem with us but > haven't found solution or any clues of what happened yet. Is it a bug ? and > if it's could you please tell me how can i overcome it ? > > Many thanks > > > > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hi George On 6/29/2012 3:46 PM, George Machitidze wrote: Hi Any similarities in activities, circumstances, crash time? No, it's kind of random crashing. Run well after 10 or 12 hours and then crashed, without any performance degrading signature. This server is a web server, we have some changes 2 days ago, maybe it's making heavier I/O operation than before. Just let u know :) rpm -qa|grep -E "gluster|fuse|kernel";uname -a;cat /etc/*-release Here is info you asked: /rpm -qa | grep -E "gluster|fuse|kernel" kernel-firmware-2.6.32-220.el6.noarch dracut-kernel-004-256.el6.noarch abrt-addon-kerneloops-2.0.4-14.el6.centos.x86_64 kernel-headers-2.6.32-220.4.2.el6.x86_64 kernel-2.6.32-220.el6.x86_64 libreport-plugin-kerneloops-2.0.5-20.el6.x86_64/ uname -a Linux web244 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux cat /etc/*-release CentOS release 6.2 (Final) CentOS release 6.2 (Final) CentOS release 6.2 (Final) /lsmod | grep fuse fuse 66285 3/ I have fuse module already, and didn't install any package from yum before, do i need to install it ? Many thanks Best regards, George Machitidze On Fri, Jun 29, 2012 at 12:40 PM, tungdam wrote: it keep being crashed everyday. I tried reinstalling ( from source code ) but it didn't help, will try installing from RPM to see if it's because of my compiler or not. Any idea, guys ? Any help would be highly appreciated :) On 6/28/2012 8:20 AM, tungdam wrote: Hello there Our gluster client was crashed 2 times last night. The gluster partition was unavailable for a long time and i have to remount it manually. On the gluster log, i saw things like these: pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-06-28 00:19:24 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900] Here is my "gluster volume info" output: Volume Name: xx Type: Distribute Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: Gnode1:/mnt/store Brick2: Gnode2:/mnt/store2 Options Reconfigured: nfs.addr-namelookup: off nfs.rpc-auth-allow: 10.0.0.245,10.0.0.244,10.0.0.247,10.0.0.54,10.0.0.55 auth.allow: 10.* cluster.min-free-disk: 5 performance.io-thread-count: 64 performance.cache-size: 512MB nfs.disable: off performance.write-behind-window-size: 4MB cluster.data-self-heal: off performance.stat-prefetch: off I googled around and found some people have the same problem with us but haven't found solution or any clues of what happened yet. Is it a bug ? and if it's could you please tell me how can i overcome it ? Many thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
Hi Any similarities in activities, circumstances, crash time? rpm -qa|grep -E "gluster|fuse|kernel";uname -a;cat /etc/*-release Best regards, George Machitidze On Fri, Jun 29, 2012 at 12:40 PM, tungdam wrote: > it keep being crashed everyday. I tried reinstalling ( from source code ) > but it didn't help, will try installing from RPM to see if it's because of > my compiler or not. Any idea, guys ? > > Any help would be highly appreciated :) > > On 6/28/2012 8:20 AM, tungdam wrote: > > Hello there > > Our gluster client was crashed 2 times last night. The gluster partition was > unavailable for a long time and i have to remount it manually. On the > gluster log, i saw things like these: > > pending frames: > frame : type(1) op(LOOKUP) > frame : type(1) op(LOOKUP) > > patchset: git://git.gluster.com/glusterfs.git > signal received: 11 > time of crash: 2012-06-28 00:19:24 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > fdatasync 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 3.2.5 > /lib64/libc.so.6[0x3cf0c32900] > > > Here is my "gluster volume info" output: > > Volume Name: xx > Type: Distribute > Status: Started > Number of Bricks: 2 > Transport-type: tcp > Bricks: > Brick1: Gnode1:/mnt/store > Brick2: Gnode2:/mnt/store2 > Options Reconfigured: > nfs.addr-namelookup: off > nfs.rpc-auth-allow: 10.0.0.245,10.0.0.244,10.0.0.247,10.0.0.54,10.0.0.55 > auth.allow: 10.* > cluster.min-free-disk: 5 > performance.io-thread-count: 64 > performance.cache-size: 512MB > nfs.disable: off > performance.write-behind-window-size: 4MB > cluster.data-self-heal: off > performance.stat-prefetch: off > > I googled around and found some people have the same problem with us but > haven't found solution or any clues of what happened yet. Is it a bug ? and > if it's could you please tell me how can i overcome it ? > > Many thanks > > > > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster 3.2.5 client crash report
it keep being crashed everyday. I tried reinstalling ( from source code ) but it didn't help, will try installing from RPM to see if it's because of my compiler or not. Any idea, guys ? Any help would be highly appreciated :) On 6/28/2012 8:20 AM, tungdam wrote: Hello there Our gluster client was crashed 2 times last night. The gluster partition was unavailable for a long time and i have to remount it manually. On the gluster log, i saw things like these: /pending frames: frame : type(1) op(LOOKUP) frame : type(1) op(LOOKUP) patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-06-28 00:19:24 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.2.5 /lib64/libc.so.6[0x3cf0c32900]/ Here is my "gluster volume info" output: /Volume Name: xx Type: Distribute Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: Gnode1:/mnt/store Brick2: Gnode2:/mnt/store2 Options Reconfigured: nfs.addr-namelookup: off nfs.rpc-auth-allow: 10.0.0.245,10.0.0.244,10.0.0.247,10.0.0.54,10.0.0.55 auth.allow: 10.* cluster.min-free-disk: 5 performance.io-thread-count: 64 performance.cache-size: 512MB nfs.disable: off performance.write-behind-window-size: 4MB cluster.data-self-heal: off performance.stat-prefetch: off/ I googled around and found some people have the same problem with us but haven't found solution or any clues of what happened yet. Is it a bug ? and if it's could you please tell me how can i overcome it ? Many thanks ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users