A hard reboot solved it for me too, but I am a little worried on the stability of it.
The entire wordpress web root is stored on the gluster volume, a day later I attempted to tar/gzip the web directory on the gluster volume and again the load on one particular server sky rocketed, server became unresponsive and kernel hang messages appeared claiming the gzip process was hanging. It seems that if I cause any serious IO on the volume, things seem to snowball. There are 3 web nodes with a replicated gluster volume between them all. Each web node also mounts the gluster volume locally using NFS (an attempt to improve performance for lots of small php files). I tried mounting native glusterfs instead of NFS for client mount, but I had the same issues. ii glusterfs-client 3.6.2-ubuntu1~trusty3 amd64 clustered file-system (client package) ii glusterfs-common 3.6.2-ubuntu1~trusty3 amd64 GlusterFS common libraries and translator modules ii glusterfs-server 3.6.2-ubuntu1~trusty3 amd64 clustered file-system (server package) Volume Name: my_filestore_vol Type: Replicate Volume ID: xyz Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: my-web01:/export/brick0 Brick2: my-web02:/export/brick0 Brick3: my-web03:/export/brick0 Options Reconfigured: nfs.drc: off diagnostics.brick-log-level: WARNING Status of volume: my_filestore_vol Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick my-web01:/export/brick0 49152 Y 2138 Brick my-web02:/export/brick0 49152 Y 21104 Brick my-web03:/export/brick0 49152 Y 1827 NFS Server on localhost 2049 Y 2145 Self-heal Daemon on localhost N/A Y 2152 NFS Server on my-web03 2049 Y 1834 Self-heal Daemon on my-web03 N/A Y 1841 NFS Server on my-web02 2049 Y 21118 Self-heal Daemon on my-web02 N/A Y 21123 Task Status of Volume my_filestore_vol ------------------------------------------------------------------------------ There are no active volume tasks Regards, A ----- Original Message ----- From: "Hoggins!" <fucks...@wheres5.com> To: gluster-users@gluster.org Sent: Friday, 8 May, 2015 1:25:26 PM Subject: Re: [Gluster-users] High load / hang Well, that's "funny", because the exact same thing happened to me this morning, except that I could hard reboot the machine, and it got up and running normally again. But the symptoms you describe are oddly similar, and strangely simultaneous. Le 08/05/2015 10:36, Alun James a écrit : Hi folks, I have a 3 node gluster/web/db cluster running a Wordpress site . This morning one of the nodes is under very high load and the mounted gluster partition is inaccessible. Attempts to reboot that node have failed with the server seemingly hung/blocked. Has anyone else experienced this or can give any pointers in how to diagnose the cause of gluster going awry? _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
