date:20171017

[Gluster-users] Gluster processes remaining after stopping glusterd

2017-10-17 Thread ismael mondiu

Hi,

I noticed that when i stop my gluster server via systemctl stop glusterd 
command , one glusterfs process is still up.

Which is the correct way to stop all gluster processes in my host?



That's we see after run the command:

***

[root@xx ~]# ps -ef |  grep -i glu
root  1825 1  0 Oct05 ?00:05:07 /usr/sbin/glusterfsd -s 
dvihcasc0s --volfile-id advdemo.dvihcasc0s.opt-glusterfs-advdemo -p 
/var/lib/glusterd/vols/advdemo/run/dvihcasc0s-opt-glusterfs-advdemo.pid -S 
/var/run/gluster/b7cbd8cac308062ef1ad823a3abf54f5.socket --brick-name 
/opt/glusterfs/advdemo -l /var/log/glusterfs/bricks/opt-glusterfs-advdemo.log 
--xlator-option *-posix.glusterd-uuid=30865b77-4da5-4f40-9945-0bd2cf55ac2a 
--brick-port 49152 --xlator-option advdemo-server.listen-port=49152
root  2058 1  0 Oct05 ?00:00:28 /usr/sbin/glusterfs -s 
localhost --volfile-id gluster/glustershd -p 
/var/lib/glusterd/glustershd/run/glustershd.pid -l 
/var/log/glusterfs/glustershd.log -S 
/var/run/gluster/fe4d4b13937be47e8fef6fd69be60899.socket --xlator-option 
*replicate*.node-uuid=30865b77-4da5-4f40-9945-0bd2cf55ac2a
root 40044 39906  0 11:55 pts/000:00:00 grep --color=auto -i glu

*


Thanks

Ismael

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster CLI Feedback

2017-10-17 Thread Mahdi Adnan

Hi,


  1.  How often do you use the Gluster CLI? Is it a preferred method to manage 
Gluster? It's the only way we manage our volumes.
  2.  What operations do you commonly perform using the CLI? Create, replace, 
set, and healing info.
  3.  How intuitive/easy to use do you find the CLI ? it's easy enough and not 
confusing.
  4.  Is the help/information displayed sufficient? Yes, the help is suffecint 
enough.
  5.  Are there operations that are difficult to perform? no.



--

Respectfully
Mahdi A. Mahdi


From: gluster-users-boun...@gluster.org  on 
behalf of Nithya Balachandran 
Sent: Monday, October 16, 2017 12:57 PM
To: gluster-users
Cc: Gluster Devel
Subject: Re: [Gluster-users] Gluster CLI Feedback

Gentle reminder.

Thanks to those who have already responded.
Nithya

On 11 October 2017 at 14:38, Nithya Balachandran 
> wrote:
Hi,

As part of our initiative to improve Gluster usability, we would like feedback 
on the current Gluster CLI. Gluster 4.0 upstream development is currently in 
progress and it is an ideal time to consider CLI changes. Answers to the 
following would be appreciated:

  1.  How often do you use the Gluster CLI? Is it a preferred method to manage 
Gluster?
  2.  What operations do you commonly perform using the CLI?
  3.  How intuitive/easy to use do you find the CLI ?
  4.  Is the help/information displayed sufficient?
  5.  Are there operations that are difficult to perform?

Regards,
Nithya

___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] gluster + synology

2017-10-17 Thread Ben Mabey

Hi,
Does anyone have any experience of using Synology NAS servers as bricks in a 
gluster setup?  The ops team at my work prefers Synology since that is what 
they are already using and some of the nice out-of-the-box admin features. From 
what I can tell Synology runs a custom linux flavor so it should be possible to 
compile gluster on it. Any first hand experience with it?

Thanks,
Ben
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid entries in volume heal info that do not heal

2017-10-17 Thread Matt Waymack

Attached is the heal log for the volume as well as the shd log. 

>> Run these commands on all the bricks of the replica pair to get the attrs 
>> set on the backend.

[root@tpc-cent-glus1-081017 ~]# getfattr -d -e hex -m . 
/exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
getfattr: Removing leading '/' from absolute path names
# file: exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.afr.gv0-client-2=0x0001
trusted.gfid=0x108694dbc0394b7cbd3dad6a15d811a2
trusted.gfid2path.9a2f5ada22eb9c45=0x38633262623330322d323466332d346463622d393630322d3839356136396461363131662f435f564f4c2d623030312d693637342d63642d63772e6d6435

[root@tpc-cent-glus2-081017 ~]# getfattr -d -e hex -m . 
/exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
getfattr: Removing leading '/' from absolute path names
# file: exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.afr.gv0-client-2=0x0001
trusted.gfid=0x108694dbc0394b7cbd3dad6a15d811a2
trusted.gfid2path.9a2f5ada22eb9c45=0x38633262623330322d323466332d346463622d393630322d3839356136396461363131662f435f564f4c2d623030312d693637342d63642d63772e6d6435

[root@tpc-arbiter1-100617 ~]# getfattr -d -e hex -m . 
/exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
getfattr: /exp/b1/gv0/.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2: No 
such file or directory


[root@tpc-cent-glus1-081017 ~]# getfattr -d -e hex -m . 
/exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3
getfattr: Removing leading '/' from absolute path names
# file: exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.afr.gv0-client-11=0x0001
trusted.gfid=0xe0c56bf78bfe46cabde1e46b92d33df3
trusted.gfid2path.be3ba24c3ef95ff2=0x63323366353834652d353566652d343033382d393131622d3866373063656334616136662f435f564f4c2d623030332d69313331342d63642d636d2d63722e6d6435

[root@tpc-cent-glus2-081017 ~]# getfattr -d -e hex -m . 
/exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3
getfattr: Removing leading '/' from absolute path names
# file: exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.afr.gv0-client-11=0x0001
trusted.gfid=0xe0c56bf78bfe46cabde1e46b92d33df3
trusted.gfid2path.be3ba24c3ef95ff2=0x63323366353834652d353566652d343033382d393131622d3866373063656334616136662f435f564f4c2d623030332d69313331342d63642d636d2d63722e6d6435

[root@tpc-arbiter1-100617 ~]# getfattr -d -e hex -m . 
/exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3
getfattr: /exp/b4/gv0/.glusterfs/e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3: No 
such file or directory

>> And the output of "gluster volume heal  info split-brain"

[root@tpc-cent-glus1-081017 ~]# gluster volume heal gv0 info split-brain
Brick tpc-cent-glus1-081017:/exp/b1/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus2-081017:/exp/b1/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-arbiter1-100617:/exp/b1/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus1-081017:/exp/b2/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus2-081017:/exp/b2/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-arbiter1-100617:/exp/b2/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus1-081017:/exp/b3/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus2-081017:/exp/b3/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-arbiter1-100617:/exp/b3/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus1-081017:/exp/b4/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-cent-glus2-081017:/exp/b4/gv0
Status: Connected
Number of entries in split-brain: 0

Brick tpc-arbiter1-100617:/exp/b4/gv0
Status: Connected
Number of entries in split-brain: 0

-Matt

From: Karthik Subrahmanya [mailto:ksubr...@redhat.com] 
Sent: Tuesday, October 17, 2017 1:26 AM
To: Matt Waymack 
Cc: gluster-users 
Subject: Re: [Gluster-users] gfid entries in volume heal info that do not heal

Hi Matt,

Run these commands on all the bricks of the replica pair to get the attrs set 
on the backend.

On the bricks of first replica set:
getfattr -d -e hex -m . /.glusterfs/10/86/108694db-c039-4b7c-bd3d-ad6a15d811a2
On the fourth

Re: [Gluster-users] Distribute rebalance issues

2017-10-17 Thread Nithya Balachandran

On 17 October 2017 at 16:39, Stephen Remde 
wrote:

> Nithya,
>
> Is there any way to increase the logging level of the brick? There is
> nothing obvious (to me) in the log (see below for the same time period as
> the latest rebalance failure). This is the only brick on that server that
> has disconnects like this.
>

You can use
gluster volume set  brick-log-level DEBUG
or
 gluster volume set  brick-log-level TRACE


@Milind and Raghavendra G , can you take a look at this to see why there
are so many disconnects?

Regards,
Nithya

>
> Steve
>
> [2017-10-17 02:22:13.453575] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-03-5825-2017/08/30-20:45:55:170091-video-client-4-2-318 
> (version: 3.8.15)
> [2017-10-17 02:22:31.353286] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-403
> [2017-10-17 02:22:31.353326] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-403
> [2017-10-17 02:22:42.288856] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404 
> (version: 3.8.13)
> [2017-10-17 02:29:04.889303] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404
> [2017-10-17 02:29:04.889347] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404
> [2017-10-17 02:29:15.327604] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-405 
> (version: 3.8.13)
> [2017-10-17 02:33:30.745314] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-594
> [2017-10-17 02:33:30.745360] I [MSGID: 115013] 
> [server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
> [2017-10-17 02:33:30.745396] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-594
> [2017-10-17 02:33:41.563748] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595 
> (version: 3.8.13)
> [2017-10-17 02:36:43.833304] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595
> [2017-10-17 02:36:43.833342] I [MSGID: 115013] 
> [server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
> [2017-10-17 02:36:43.833371] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595
> [2017-10-17 02:36:54.569836] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596 
> (version: 3.8.13)
> [2017-10-17 02:38:16.697306] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596
> [2017-10-17 02:38:16.697370] I [MSGID: 115013] 
> [server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
> [2017-10-17 02:38:16.697432] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596
> [2017-10-17 02:38:34.591506] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-597 
> (version: 3.8.13)
> [2017-10-17 02:55:56.473306] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-19
> [2017-10-17 02:55:56.473366] I [MSGID: 101055] 
> [client_t.c:415:gf_client_unref] 0-video-server: Shutting down connection 
> titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-19
> [2017-10-17 02:56:07.161790] I [MSGID: 115029] 
> [server-handshake.c:692:server_setvolume] 0-video-server: accepted client 
> from titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-20 (version: 
> 3.8.8)
> [2017-10-17 03:15:13.529281] I [MSGID: 115036] 
> [server.c:548:server_rpc_notify] 0-video-server: disconnecting connection 
> from

Re: [Gluster-users] Distribute rebalance issues

2017-10-17 Thread Stephen Remde

Nithya,

Is there any way to increase the logging level of the brick? There is
nothing obvious (to me) in the log (see below for the same time period as
the latest rebalance failure). This is the only brick on that server that
has disconnects like this.

Steve

[2017-10-17 02:22:13.453575] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-03-5825-2017/08/30-20:45:55:170091-video-client-4-2-318
(version: 3.8.15)
[2017-10-17 02:22:31.353286] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-403
[2017-10-17 02:22:31.353326] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-403
[2017-10-17 02:22:42.288856] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404
(version: 3.8.13)
[2017-10-17 02:29:04.889303] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404
[2017-10-17 02:29:04.889347] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-404
[2017-10-17 02:29:15.327604] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-02-29040-2017/08/04-09:31:22:842268-video-client-4-7-405
(version: 3.8.13)
[2017-10-17 02:33:30.745314] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-594
[2017-10-17 02:33:30.745360] I [MSGID: 115013]
[server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
[2017-10-17 02:33:30.745396] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-594
[2017-10-17 02:33:41.563748] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595
(version: 3.8.13)
[2017-10-17 02:36:43.833304] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595
[2017-10-17 02:36:43.833342] I [MSGID: 115013]
[server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
[2017-10-17 02:36:43.833371] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-595
[2017-10-17 02:36:54.569836] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596
(version: 3.8.13)
[2017-10-17 02:38:16.697306] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596
[2017-10-17 02:38:16.697370] I [MSGID: 115013]
[server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
[2017-10-17 02:38:16.697432] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-596
[2017-10-17 02:38:34.591506] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-597
(version: 3.8.13)
[2017-10-17 02:55:56.473306] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-19
[2017-10-17 02:55:56.473366] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-19
[2017-10-17 02:56:07.161790] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from titan-17527-2017/09/18-19:57:41:611709-video-client-4-0-20
(version: 3.8.8)
[2017-10-17 03:15:13.529281] I [MSGID: 115036]
[server.c:548:server_rpc_notify] 0-video-server: disconnecting
connection from
node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-597
[2017-10-17 03:15:13.529330] I [MSGID: 115013]
[server-helpers.c:293:do_fd_cleanup] 0-video-server: fd cleanup on /xx
[2017-10-17 03:15:13.529400] I [MSGID: 101055]
[client_t.c:415:gf_client_unref] 0-video-server: Shutting down
connection node-dc4-01-6174-2017/07/13-10:46:48:503667-video-client-4-7-597
[2017-10-17 03:15:41.764247] I [MSGID: 115029]
[server-handshake.c:692:server_setvolume] 0-video-server: accepted
client from

[Gluster-users] gluster usecase CDN

2017-10-17 Thread atris adam

hi everybody,

as "
http://www.itzgeek.com/how-tos/linux/centos-how-tos/install-and-configure-glusterfs-on-centos-7-rhel-7.html;
says:

*GlusterFS *is an open-source, scalable network filesystem *suitable *for
high data-intensive workloads such as media streaming, cloud storage, and*
CDN (Content Delivery Network).*


I have googled alot, but couldn't find any architecture for CDN as a use
case of glusterfs. Can anybody help me in this case?


Thx alot gluster-users;)
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Distribute rebalance issues

2017-10-17 Thread Nithya Balachandran

On 17 October 2017 at 14:48, Stephen Remde 
wrote:

> Hi,
>
>
> I have a rebalance that has failed on one peer twice now. Rebalance logs 
> below (directories anonomised and some irrelevant log lines cut). It looks 
> like it loses connection to the brick, but immediately stops the rebalance on 
> that peer instead of waiting for reconnection - which happens a second or so 
> later.
> Is this normal behaviour? So far it has been the same server and the same 
> (remote) brick.
>
>
> The brick shows a high number of disconnects compared to the other bricks on 
> the same server
>
>
> ./export-md0-brick.log.1  2
> ./export-md1-brick.log.1  2
> ./export-md2-brick.log.1181
> ./export-md3-brick.log.1  2
>
>
> Any clues? What could be causing this because there is nothing in the log to 
> indicate cause.
>
> The rebalance process requires that all DHT child subvols be up during the
operation as it needs to reapply the directory layouts (which requires all
child subvols to be up). As this is a pure distribute volume, even a single
brick getting disconnected is enough to cause the process to stop.

You would need to figure out why that brick is disconnecting so often. The
brick logs might help with that.

Regards,
Nithya


>
> Steve
>
>
> gluster volume info video
>
> Volume Name: video
> Type: Distribute
> Volume ID: ccdac37f-9b0e-415f-b62e-9071d8168199
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 9
> Transport-type: tcp
> Bricks:
> Brick1: 10.0.0.31:/export/md0/brick
> Brick2: 10.0.0.32:/export/md0/brick
> Brick3: 10.0.0.31:/export/md1/brick
> Brick4: 10.0.0.32:/export/md1/brick
> Brick5: 10.0.0.31:/export/md2/brick
> Brick6: 10.0.0.32:/export/md2/brick
> Brick7: 10.0.0.31:/export/md3/brick
> Brick8: 10.0.0.32:/export/md3/brick
> Brick9: 10.0.0.33:/export/md0/brick
> Options Reconfigured:
> network.ping-timeout: 10
> cluster.min-free-disk: 1%
> transport.address-family: inet
> performance.readdir-ahead: on
> nfs.disable: on
> cluster.rebal-throttle: lazy
>
> [2017-10-12 23:00:55.099153] W [socket.c:590:__socket_rwv] 0-video-client-4: 
> readv on 10.0.0.31:49164 failed (Connection reset by peer)
> [2017-10-12 23:00:55.099709] I [MSGID: 114018] 
> [client.c:2280:client_rpc_notify] 0-video-client-4: disconnected from 
> video-client-4. Client process will keep trying to connect to glusterd until 
> brick's port is available
> [2017-10-12 23:00:55.099741] W [MSGID: 109073] [dht-common.c:8839:dht_notify] 
> 0-video-dht: Received CHILD_DOWN. Exiting
> [2017-10-12 23:00:55.099752] I [MSGID: 109029] 
> [dht-rebalance.c:4195:gf_defrag_stop] 0-: Received stop command on rebalance
> [2017-10-12 23:01:05.478462] I [rpc-clnt.c:1947:rpc_clnt_reconfig] 
> 0-video-client-4: changing port to 49164 (from 0)
> [2017-10-12 23:01:05.481180] I [MSGID: 114057] 
> [client-handshake.c:1446:select_server_supported_programs] 0-video-client-4: 
> Using Program GlusterFS 3.3, Num (1298437), Version (330)
> [2017-10-12 23:01:05.482630] I [MSGID: 114046] 
> [client-handshake.c:1222:client_setvolume_cbk] 0-video-client-4: Connected to 
> video-client-4, attached to remote volume '/export/md2/brick'.
> [2017-10-12 23:01:05.482659] I [MSGID: 114047] 
> [client-handshake.c:1233:client_setvolume_cbk] 0-video-client-4: Server and 
> Client lk-version numbers are not same, reopening the fds
> [2017-10-12 23:01:05.483365] I [MSGID: 114035] 
> [client-handshake.c:201:client_set_lk_version_cbk] 0-video-client-4: Server 
> lk version = 1
> [2017-10-12 23:01:30.310089] I [dht-rebalance.c:2819:gf_defrag_process_dir] 
> 0-DHT: Found critical error from gf_defrag_get_entry
> [2017-10-12 23:01:30.310166] E [MSGID: 109111] 
> [dht-rebalance.c:3090:gf_defrag_fix_layout] 0-video-dht: 
> gf_defrag_process_dir failed for directory: /y/y/y/y/y
> [2017-10-12 23:01:30.380574] E [MSGID: 109016] 
> [dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout failed 
> for /y/y/y/y/y
> [2017-10-12 23:01:30.380756] E [MSGID: 109016] 
> [dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout failed 
> for /y/y/y/y
> [2017-10-12 23:01:30.380879] E [MSGID: 109016] 
> [dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout failed 
> for /y/y/y
> [2017-10-12 23:01:30.380965] E [MSGID: 109016] 
> [dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout failed 
> for /y/y
> [2017-10-12 23:03:09.285157] W [glusterfsd.c:1327:cleanup_and_exit] 
> (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f112b6d16ba] 
> -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x55b325019545] 
> -->/usr/sbin/glusterfs(cleanup_and_exit+0x54) [0x55b3250193b4] ) 0-: received 
> signum (15), shutting down
>
> [2017-10-17 03:20:28.921512] W [socket.c:590:__socket_rwv] 0-video-client-4: 
> readv on 10.0.0.31:49164 failed (Connection reset by peer)
> [2017-10-17 03:20:28.921554] I [MSGID: 114018] 
> [client.c:2280:client_rpc_notify] 0-video-client-4: disconnected from 
> video-client-4. Client process

[Gluster-users] Distribute rebalance issues

2017-10-17 Thread Stephen Remde

Hi,


I have a rebalance that has failed on one peer twice now. Rebalance
logs below (directories anonomised and some irrelevant log lines cut).
It looks like it loses connection to the brick, but immediately stops
the rebalance on that peer instead of waiting for reconnection - which
happens a second or so later.
Is this normal behaviour? So far it has been the same server and the
same (remote) brick.


The brick shows a high number of disconnects compared to the other
bricks on the same server


./export-md0-brick.log.1  2
./export-md1-brick.log.1  2
./export-md2-brick.log.1181
./export-md3-brick.log.1  2


Any clues? What could be causing this because there is nothing in the
log to indicate cause.


Steve


gluster volume info video

Volume Name: video
Type: Distribute
Volume ID: ccdac37f-9b0e-415f-b62e-9071d8168199
Status: Started
Snapshot Count: 0
Number of Bricks: 9
Transport-type: tcp
Bricks:
Brick1: 10.0.0.31:/export/md0/brick
Brick2: 10.0.0.32:/export/md0/brick
Brick3: 10.0.0.31:/export/md1/brick
Brick4: 10.0.0.32:/export/md1/brick
Brick5: 10.0.0.31:/export/md2/brick
Brick6: 10.0.0.32:/export/md2/brick
Brick7: 10.0.0.31:/export/md3/brick
Brick8: 10.0.0.32:/export/md3/brick
Brick9: 10.0.0.33:/export/md0/brick
Options Reconfigured:
network.ping-timeout: 10
cluster.min-free-disk: 1%
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
cluster.rebal-throttle: lazy

[2017-10-12 23:00:55.099153] W [socket.c:590:__socket_rwv]
0-video-client-4: readv on 10.0.0.31:49164 failed (Connection reset by
peer)
[2017-10-12 23:00:55.099709] I [MSGID: 114018]
[client.c:2280:client_rpc_notify] 0-video-client-4: disconnected from
video-client-4. Client process will keep trying to connect to glusterd
until brick's port is available
[2017-10-12 23:00:55.099741] W [MSGID: 109073]
[dht-common.c:8839:dht_notify] 0-video-dht: Received CHILD_DOWN.
Exiting
[2017-10-12 23:00:55.099752] I [MSGID: 109029]
[dht-rebalance.c:4195:gf_defrag_stop] 0-: Received stop command on
rebalance
[2017-10-12 23:01:05.478462] I [rpc-clnt.c:1947:rpc_clnt_reconfig]
0-video-client-4: changing port to 49164 (from 0)
[2017-10-12 23:01:05.481180] I [MSGID: 114057]
[client-handshake.c:1446:select_server_supported_programs]
0-video-client-4: Using Program GlusterFS 3.3, Num (1298437), Version
(330)
[2017-10-12 23:01:05.482630] I [MSGID: 114046]
[client-handshake.c:1222:client_setvolume_cbk] 0-video-client-4:
Connected to video-client-4, attached to remote volume
'/export/md2/brick'.
[2017-10-12 23:01:05.482659] I [MSGID: 114047]
[client-handshake.c:1233:client_setvolume_cbk] 0-video-client-4:
Server and Client lk-version numbers are not same, reopening the fds
[2017-10-12 23:01:05.483365] I [MSGID: 114035]
[client-handshake.c:201:client_set_lk_version_cbk] 0-video-client-4:
Server lk version = 1
[2017-10-12 23:01:30.310089] I
[dht-rebalance.c:2819:gf_defrag_process_dir] 0-DHT: Found critical
error from gf_defrag_get_entry
[2017-10-12 23:01:30.310166] E [MSGID: 109111]
[dht-rebalance.c:3090:gf_defrag_fix_layout] 0-video-dht:
gf_defrag_process_dir failed for directory: /y/y/y/y/y
[2017-10-12 23:01:30.380574] E [MSGID: 109016]
[dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout
failed for /y/y/y/y/y
[2017-10-12 23:01:30.380756] E [MSGID: 109016]
[dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout
failed for /y/y/y/y
[2017-10-12 23:01:30.380879] E [MSGID: 109016]
[dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout
failed for /y/y/y
[2017-10-12 23:01:30.380965] E [MSGID: 109016]
[dht-rebalance.c:3267:gf_defrag_fix_layout] 0-video-dht: Fix layout
failed for /y/y
[2017-10-12 23:03:09.285157] W [glusterfsd.c:1327:cleanup_and_exit]
(-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f112b6d16ba]
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x55b325019545]
-->/usr/sbin/glusterfs(cleanup_and_exit+0x54) [0x55b3250193b4] ) 0-:
received signum (15), shutting down

[2017-10-17 03:20:28.921512] W [socket.c:590:__socket_rwv]
0-video-client-4: readv on 10.0.0.31:49164 failed (Connection reset by
peer)
[2017-10-17 03:20:28.921554] I [MSGID: 114018]
[client.c:2280:client_rpc_notify] 0-video-client-4: disconnected from
video-client-4. Client process will keep trying to connect to glusterd
until brick's port is available
[2017-10-17 03:20:28.921570] W [MSGID: 109073]
[dht-common.c:8839:dht_notify] 0-video-dht: Received CHILD_DOWN.
Exiting
[2017-10-17 03:20:28.921578] I [MSGID: 109029]
[dht-rebalance.c:4195:gf_defrag_stop] 0-: Received stop command on
rebalance
[2017-10-17 03:20:39.344417] I [rpc-clnt.c:1947:rpc_clnt_reconfig]
0-video-client-4: changing port to 49164 (from 0)
[2017-10-17 03:20:39.347440] I [MSGID: 114057]
[client-handshake.c:1446:select_server_supported_programs]
0-video-client-4: Using Program GlusterFS 3.3, Num (1298437), Version
(330)
[2017-10-17 03:20:39.349244] I [MSGID: 114046]
[client-handshake.c:1222:client_setvolume_cbk] 0-video-client-4:
Connected to

Re: [Gluster-users] Gluster CLI reference

2017-10-17 Thread Marcin Dulak

Hi,

I think the cli-main should contain all, even most obscure options.
Additionally, I'm not sure whether this is feasible, but maybe the cli-main
could include links to all places where an example usage of a option is
given.
The goal would be: I go to the cli-main page and if there is something I
want to do I have links with the examples of the usage.

Marcin

On Tue, Oct 17, 2017 at 7:55 AM, Nithya Balachandran 
wrote:

> Hi,
>
> As part of our initiative to improve our docs, we have made a few changes
> over the past few weeks. One of this is a CLI reference [1]. This is still
> a WIP so not all commands have been documented.
>
> Is this something you would find useful?
> Would you like to see more information captured as part of this? If yes,
> what else should we include?
>
> Thanks,
> Nithya
>
> [1]: http://docs.gluster.org/en/latest/CLI-Reference/cli-main/
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid entries in volume heal info that do not heal

2017-10-17 Thread Karthik Subrahmanya

Hi Matt,

Run these commands on all the bricks of the replica pair to get the attrs
set on the backend.

On the bricks of first replica set:
getfattr -d -e hex -m . /.glusterfs/10/86/
108694db-c039-4b7c-bd3d-ad6a15d811a2

On the fourth replica set:
getfattr -d -e hex -m . /.glusterfs/
e0/c5/e0c56bf7-8bfe-46ca-bde1-e46b92d33df3

Also run the "gluster volume heal " once and send the shd log.
And the output of "gluster volume heal  info split-brain"

Regards,
Karthik

On Mon, Oct 16, 2017 at 9:51 PM, Matt Waymack  wrote:

> OK, so here’s my output of the volume info and the heal info. I have not
> yet tracked down physical location of these files, any tips to finding them
> would be appreciated, but I’m definitely just wanting them gone.  I forgot
> to mention earlier that the cluster is running 3.12 and was upgraded from
> 3.10; these files were likely stuck like this when it was on 3.10.
>
>
>
> [root@tpc-cent-glus1-081017 ~]# gluster volume info gv0
>
>
>
> Volume Name: gv0
>
> Type: Distributed-Replicate
>
> Volume ID: 8f07894d-e3ab-4a65-bda1-9d9dd46db007
>
> Status: Started
>
> Snapshot Count: 0
>
> Number of Bricks: 4 x (2 + 1) = 12
>
> Transport-type: tcp
>
> Bricks:
>
> Brick1: tpc-cent-glus1-081017:/exp/b1/gv0
>
> Brick2: tpc-cent-glus2-081017:/exp/b1/gv0
>
> Brick3: tpc-arbiter1-100617:/exp/b1/gv0 (arbiter)
>
> Brick4: tpc-cent-glus1-081017:/exp/b2/gv0
>
> Brick5: tpc-cent-glus2-081017:/exp/b2/gv0
>
> Brick6: tpc-arbiter1-100617:/exp/b2/gv0 (arbiter)
>
> Brick7: tpc-cent-glus1-081017:/exp/b3/gv0
>
> Brick8: tpc-cent-glus2-081017:/exp/b3/gv0
>
> Brick9: tpc-arbiter1-100617:/exp/b3/gv0 (arbiter)
>
> Brick10: tpc-cent-glus1-081017:/exp/b4/gv0
>
> Brick11: tpc-cent-glus2-081017:/exp/b4/gv0
>
> Brick12: tpc-arbiter1-100617:/exp/b4/gv0 (arbiter)
>
> Options Reconfigured:
>
> nfs.disable: on
>
> transport.address-family: inet
>
>
>
> [root@tpc-cent-glus1-081017 ~]# gluster volume heal gv0 info
>
> Brick tpc-cent-glus1-081017:/exp/b1/gv0
>
> 
>
> 
>
> 
>
> 
>
> 
>
>
>
> 
>
>
>
> Status: Connected
>
> Number of entries: 118
>
>
>
> Brick tpc-cent-glus2-081017:/exp/b1/gv0
>
> 
>
> 
>
> 
>
> 
>
> 
>
>
>
> 
>
>
>
> Status: Connected
>
> Number of entries: 118
>
>
>
> Brick tpc-arbiter1-100617:/exp/b1/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-cent-glus1-081017:/exp/b2/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-cent-glus2-081017:/exp/b2/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-arbiter1-100617:/exp/b2/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-cent-glus1-081017:/exp/b3/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-cent-glus2-081017:/exp/b3/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-arbiter1-100617:/exp/b3/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Brick tpc-cent-glus1-081017:/exp/b4/gv0
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> Status: Connected
>
> Number of entries: 24
>
>
>
> Brick tpc-cent-glus2-081017:/exp/b4/gv0
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> 
>
> Status: Connected
>
> Number of entries: 24
>
>
>
> Brick tpc-arbiter1-100617:/exp/b4/gv0
>
> Status: Connected
>
> Number of entries: 0
>
>
>
> Thank you for your help!
>
>
>
> *From:* Karthik Subrahmanya [mailto:ksubr...@redhat.com]
> *Sent:* Monday, October 16, 2017 10:27 AM
> *To:* Matt Waymack 
> *Cc:* gluster-users 
> *Subject:* Re: [Gluster-users] gfid entries in volume heal info that do
> not heal
>
>
>
> Hi Matt,
>
>
>
> The files might be in split brain. Could you please send the outputs of
> these?
>
> gluster volume info 
>
> gluster volume heal  info
>
> And also the getfattr output of the files which are in the heal info
> output from all the bricks of that replica pair.
>
> getfattr -d -e hex -m . 
>
>
>
> Thanks &  Regards
>
> Karthik
>
>
>
> On 16-Oct-2017 8:16 PM, "Matt Waymack"  wrote:
>
> Hi all,
>
>
>
> I have a volume where the output of volume heal info shows several gfid
> entries to be healed, but they’ve been there for weeks and have not
> healed.  Any normal file that shows up on the heal info does get healed as
> expected, but these gfid entries do not.  Is there any way to remove these
> orphaned entries from the volume so they are no longer stuck in the heal
> process?
>
>
>
> Thank you!
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Gluster processes remaining after stopping glusterd

Re: [Gluster-users] Gluster CLI Feedback

[Gluster-users] gluster + synology

Re: [Gluster-users] gfid entries in volume heal info that do not heal

Re: [Gluster-users] Distribute rebalance issues

Re: [Gluster-users] Distribute rebalance issues

[Gluster-users] gluster usecase CDN

Re: [Gluster-users] Distribute rebalance issues

[Gluster-users] Distribute rebalance issues

Re: [Gluster-users] Gluster CLI reference

Re: [Gluster-users] gfid entries in volume heal info that do not heal

11 matches

Site Navigation

Mail list logo

Footer information