Re: [Gluster-users] Problems with Gluster 3.0.4 on Debian Squeeze
On 10/06/10 19:09, Vijay Bellur wrote: On Thursday 10 June 2010 09:27 PM, stephen mulcahy wrote: [2010-06-10 16:55:27] D [client-protocol.c:6178:client_setvolume_cbk] node34-1: setvolume failed (Stale NFS file handle) [2010-06-10 16:55:27] D [fuse-bridge.c:3100:fuse_thread_proc] fuse: pthread_cond_timedout returned non zero value ret: 0 errno: 0 [2010-06-10 16:55:27] N [client-protocol.c:6246:client_setvolume_cbk] node33-1: Connected to 10.141.0.33:6996, attached to remote volume 'brick1'. [2010-06-10 16:55:27] N [fuse-bridge.c:2950:fuse_init] glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.13 [2010-06-10 16:55:27] D [client-protocol.c:6178:client_setvolume_cbk] node34-1: setvolume failed (Stale NFS file handle) So it seems to connect to the node33 server but fails to connect to the node34 server. Can you please provide the glusterfs log file on node34 server too? That might give a clue on why this connection attempt failed. Sure, see below. It didn't contain any obvious error messages [2010-06-10 18:05:07] N [glusterfsd.c:1408:main] glusterfs: Successfully started [2010-06-10 18:09:18] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1020 disconnected [2010-06-10 18:09:18] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1019 disconnected [2010-06-10 18:09:18] N [server-helpers.c:842:server_connection_destroy] server- tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 [2010-06-10 18:09:29] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1018 disconnected [2010-06-10 18:09:29] N [server-helpers.c:842:server_connection_destroy] server-tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 [2010-06-10 18:09:29] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1017 disconnected [2010-06-10 18:09:29] N [server-helpers.c:842:server_connection_destroy] server-tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 [2010-06-10 18:09:39] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1016 disconnected [2010-06-10 18:09:39] N [server-helpers.c:842:server_connection_destroy] server-tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 [2010-06-10 18:09:39] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1015 disconnected [2010-06-10 18:09:39] N [server-helpers.c:842:server_connection_destroy] server-tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 [2010-06-10 18:09:49] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1014 disconnected [2010-06-10 18:09:49] N [server-protocol.c:6788:notify] server-tcp: 10.141.0.35:1013 disconnected [2010-06-10 18:09:49] N [server-helpers.c:842:server_connection_destroy] server-tcp: destroyed connection of node35-1695-2010/06/10-18:09:18:493118-node34-1 -stephen -- Stephen Mulcahy, DI2, Digital Enterprise Research Institute, NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland http://di2.deri.iehttp://webstar.deri.iehttp://sindice.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Problems with Gluster 3.0.4 on Debian Squeeze
I reinstalled node33 and node34 with centos 5.5 x86_64 and gluster 3.0.4 and repeated the same scenario with node35 running debian squeeze x86_64 and gluster 3.0.4 and didn't get the stale nfs file handle error. so there may be an issue with gluster server 3.0.4 on debian squeeze (or at least my installation of it). -stephen -- Stephen Mulcahy, DI2, Digital Enterprise Research Institute, NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland http://di2.deri.iehttp://webstar.deri.iehttp://sindice.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] health monitoring of replicated volume
I am curious about this - anyone? -Jenn On Jun 9, 2010, at 8:00 AM, Deyan Chepishev wrote: Hello, Is there any reasonable way to monitor the health of replicated volume and sync it, if out of sync ? Regards, ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] health monitoring of replicated volume
We plan to add a method to check if a replicated volume is out of sync. We also plan to add a way to find out which copy is good and which out of sync - we intend to do this for both volume/subvolume and also at the file level. The file level is helpful to monitor vm image file copies and their currency. There is however no planned date as yet by when this would be implemented. Will post a note when it's done. Let me know if you have any questions. Regards, Tejas. - Original Message - From: Jenn Fountain jfoun...@comcast.net To: Gluster General Discussion List gluster-users@gluster.org Sent: Friday, June 11, 2010 6:50:37 PM Subject: Re: [Gluster-users] health monitoring of replicated volume I am curious about this - anyone? -Jenn On Jun 9, 2010, at 8:00 AM, Deyan Chepishev wrote: Hello, Is there any reasonable way to monitor the health of replicated volume and sync it, if out of sync ? Regards, ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Performance
On 06/10/2010 10:03 PM, Todd Daugherty wrote: I agree it is a matter of tuning. So what do we tune? I am not stuck on 2.0.9, there just was no performance benefit for my data set. (Millions of 8-50 megabyte files) My test system has 12 Gigabytes of RAM which is why I used a size of 16 Gigabytes so that cache is not a factor in the test. But away what did you learn from these results? Read performance is the most important to me. (That is because the write performance is pretty good already) Thanks again. Todd Right now we can increase read-ahead page count. Since the best performance of glusterfs is seen when run from multiple clients its better to run iozone in cluster mode over multiple different clients. How many servers do you have? and how many clients?. Also remember to add sysctl vm.swappiness = 0 on server side to make sure that dirty cache in not filled up since the increase writebehind will cause aggressiveness on server side RAM usage as infiniband will receive lot of data. There are other parameters for transport ib-verbs on server side option transport.ib-verbs.work-request-send-count 256 option transport.ib-verbs.work-request-recv-count 256 These values are 32 by default. Try increasing these and rerun the benchmarks. If you still don't have enough performance benefit. We will apply write-behind and read-ahead on server side to optimally utilize RAM, since its not a well tested configuration its not recommended. But i think you should be getting enough benefit from the above details. Regards -- Harshavardhana Gluster Inc - http://www.gluster.com +1(408)-770-1887, Ext-113 +1(408)-480-1730 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
