Re: [Gluster-users] Fwd: Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and checkpatch warnings fixed

[Anand Subramanian]

This is not a must-have fix for pnfs functionality. pnfs mode can be 
enabled and it will work without the libgfapi fix.


The fix is only important from a performance optimization standpoint 
alone (esp for large files).


Also note that currently for GlusterFS, we only support the 
PNFS-FILE-LAYOUT with stripe-count as 1. And though  both configurations 
(i) single MDS with multiple DS'es as well as the (ii) all-symmetric 
architecture of every ganesha head behaving as both MDS|DS are supported 
and work right now, the latter is not functionally complete as we will 
need to add an upcall support for Layout recalls.


There is additional thought needed once EC support is completed, 
sharding work is in and once the FLEX-FILE-LAYOUT IETF draft is complete.


Anand


On 03/23/2015 03:11 PM, Niels de Vos wrote:

On Mon, Mar 23, 2015 at 12:49:56PM +0530, Anand Subramanian wrote:

FYI.

GlusterFS vols can now be accessed via NFSv4.1 pNFS protocol (mount -t nfs
-o minorversion=1 ...) from nfs-ganesha 2.2-rc5 onwards.

Note: one fix is to go into libgfapi to fix up using anonymous fd's in
ds_write/make_ds_handle() (Avati's sugeestion that really helps here).
Once Jiffin or myself get that fix in, a good large file performance can be
seen with pNFS vs V4.

I could not find the needed change for libgfapi. Could you post the link
to the review?

Thanks,
Niels


All thanks and credit to Jiffin for his terrific effort in coding things up
quickly and for fixing bugs.

Anand


 Forwarded Message 
Subject:Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and
checkpatch warnings fixed
Date:   Sat, 21 Mar 2015 01:04:30 +0100
From:   GerritHub 
Reply-To:   ffilz...@mindspring.com
To: Anand Subramanian 
CC: onnfrhvruutnzhnaq.-g...@noclue.notk.org



 From Frank Filz :

Frank Filz has submitted this change and it was merged.

Change subject: pNFS code drop enablement and checkpatch warnings fixed
..


pNFS code drop enablement and checkpatch warnings fixed

Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
Signed-off-by: Anand Subramanian 
---
A src/FSAL/FSAL_GLUSTER/ds.c
M src/FSAL/FSAL_GLUSTER/export.c
M src/FSAL/FSAL_GLUSTER/gluster_internal.h
M src/FSAL/FSAL_GLUSTER/handle.c
M src/FSAL/FSAL_GLUSTER/main.c
A src/FSAL/FSAL_GLUSTER/mds.c
6 files changed, 993 insertions(+), 0 deletions(-)



--
To view, visit https://review.gerrithub.io/221683
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: Anand Subramanian 
Gerrit-Reviewer: Frank Filz 
Gerrit-Reviewer: onnfrhvruutnzhnaq.-g...@noclue.notk.org



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[Atin Mukherjee]

If you are okay to do a fresh set up I would recommend you to clean up
/var/lib/glusterd/peers/* and then restart glusterd in both the nodes
and then try peer probing.

~Atin

On 03/23/2015 06:44 PM, Andreas wrote:
> Hi,
> 
> # gluster peer detach 10.32.1.144
> (No output here. Similar to the problem with 'gluster peer probe'.)
> # gluster peer detach 10.32.1.144 force
> peer detach: failed: Peer is already being detached from cluster.
> Check peer status by running gluster peer status
> # gluster peer status
> Number of Peers: 1
> 
> Hostname: 10.32.1.144
> Uuid: 82cdb873-28cc-4ed0-8cfe-2b6275770429
> State: Probe Sent to Peer (Connected)
> 
> # ping 10.32.1.144
> PING 10.32.1.144 (10.32.1.144): 56 data bytes
> 64 bytes from 10.32.1.144: seq=0 ttl=64 time=1.811 ms
> 64 bytes from 10.32.1.144: seq=1 ttl=64 time=1.834 ms
> ^C
> --- 10.32.1.144 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip min/avg/max = 1.811/1.822/1.834 ms
> 
> 
> As previously stated, this problem seems to be similar to what I experienced 
> with
> 'gluster peer probe'. I can reboot the server, but the situation will be the 
> same
> (I've tried this many times).
> Any ideas of which ports to investigate and how to do it to get the most 
> reliable result?
> Anything else that could cause this?
> 
> 
> 
> Regards
> Andreas
> 
> 
> On 03/23/15 11:10, Atin Mukherjee wrote:
>>
>> On 03/23/2015 03:28 PM, Andreas Hollaus wrote:
>>> 2Hi,
>>>
>>> This network problem is persistent. However, I can ping the server so guess 
>>> it
>>> depends on the port no, right?
>>> I tried to telnet to port 24007, but I was not sure how to interpret the 
>>> result as I
>>> got no respons and no timeout (it just seemed to be waiting for something).
>>> That's why I decided to install nmap, but according to that tool the port 
>>> was
>>> accessible. Are there any other ports that are vital to gluster peer probe?
>>>
>>> When you say 'deprobe', I guess you mean 'gluster peer detach'? That 
>>> command shows
>>> similar behaviour to gluster peer probe.
>> Yes I meant peer detach. How about gluster peer detach force?
> 
> 
>>>
>>> Regards
>>> Andreas
>>>
>>> On 03/23/15 05:34, Atin Mukherjee wrote:
 On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
> Hi,
>
> I hope that these are the logs that you requested.
>
> Logs from 10.32.0.48:
> --
> # more /var/log/glusterfs/.cmd_log_history
> [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
> returned
>  with unknown errno -1
>
> # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
> [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
> 0-/usr/s
> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
> /usr/sbin/
> glusterd -p /var/run/glusterd.pid)
> [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: 
> Maximum allo
> wed open file descriptors set to 65536
> [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: Using 
> /var/l
> ib/glusterd as working directory
> [2015-03-19 13:41:31.247826] I 
> [glusterd-store.c:2063:glusterd_restore_op_versio
> n] 0-management: Detected new install. Setting op-version to maximum : 
> 30600
> [2015-03-19 13:41:31.247902] I 
> [glusterd-store.c:3497:glusterd_store_retrieve_mi
> ssed_snaps_list] 0-management: No missed snaps list.
> Final graph:
> +--+
>   1: volume management
>   2: type mgmt/glusterd
>   3: option rpc-auth.auth-glusterfs on
>   4: option rpc-auth.auth-unix on
>   5: option rpc-auth.auth-null on
>   6: option transport.socket.listen-backlog 128
>   7: option ping-timeout 30
>   8: option transport.socket.read-fail-log off
>   9: option transport.socket.keepalive-interval 2
>  10: option transport.socket.keepalive-time 10
>  11: option transport-type socket
>  12: option working-directory /var/lib/glusterd
>  13: end-volume
>  14: 
> +--+
> [2015-03-19 13:42:02.258403] I 
> [glusterd-handler.c:1015:__glusterd_handle_cli_pr
> obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
> [2015-03-19 13:42:02.259456] I 
> [glusterd-handler.c:3165:glusterd_probe_begin] 0-
> glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
> [2015-03-19 13:42:02.259664] I [rpc-clnt.c:969:rpc_clnt_connection_init] 
> 0-manag
> ement: setting frame-timeout to 600
> [2015-03-19 13:42:02.260488] I 
> [glusterd-handler.c:3098:glusterd_friend_add] 0-m
> anagement: connect returned 0
> [2015-03-19 13:42:02.270316] I 
> [glusterd.c:176:glusterd_uuid_generate_save] 0-ma
> nagement: g

Re: [Gluster-users] GlusterFS for production use with Ovirt

[Punit Dambiwal]

Hi Paul,

Thanks for the updates...

A few things to clarify
1) are all 4 for Gluster or are they the whole cluster.keep in mind that
Ovirt expects that you are running Gluster on different nodes than your
VM's. while I do not completely agree with that idea that is how it is now.
Comment :- All 4 servers are the part of the one gluster volume...

2) RAID hardware is not necessarily a bad thing, in fact the cache may help
significantly. that said  it is not required.things to keep in mind if you
are using RAID hardware for performance..
2.a) RAID 0 will give you the best speed if you are using 3 nodes for quorum.
keep in mind this will require more work in the case of a drive failure to
remake the brick.
2.b) if you are only doing a mirror across 2 bricks RAID 0 may be acceptable
but consider using 1+0 or RAID 5 the performance on the two may vary
depending on the RAID controller but 1+0 is generally faster.
2.c) always optimize for the stripe which in theory should be automatic if
the RAID controller reports the required information to do it, but in
practice most do not so you should do it manually and Red Hat's support
site just added a nice tool to do it for you.
Comment :- What about if i will split my 24 SSD in to two virtual drive
with RAID 6...actually i want to leverage the performance of the
SSD..without wastage too much of the disk capacity and redundancy...

Thanks,
Punit

On Tue, Mar 24, 2015 at 11:01 AM, Paul Robert Marino 
wrote:

> A few things to clarify
> 1) are all 4 for Gluster or are they the whole cluster.
> keep in mind that Ovirt expects that you are running Gluster on
> different nodes than your VM's. while I do not completely agree with
> that idea that is how it is now.
>
> 2) RAID hardware is not necessarily a bad thing, in fact the cache may
> help significantly. that said  it is not required.
> things to keep in mind if you are using RAID hardware for performance
> 2.a) RAID 0 will give you the best speed if you are using 3 nodes for
> quorum. keep in mind this will require more work in the case of a
> drive failure to remake the brick.
> 2.b) if you are only doing a mirror across 2 bricks RAID 0 may be
> acceptable but consider using 1+0 or RAID 5 the performance on the two
> may vary depending on the RAID controller but 1+0 is generally faster.
> 2.c) always optimize for the stripe which in theory should be
> automatic if the RAID controller reports the required information to
> do it, but in practice most do not so you should do it manually and
> Red Hat's support site just added a nice tool to do it for you.
>
> 3) for performance more RAM is rarely a bad thing; however it can be
> with sustained writes.If you are constantly writing faster than the
> disks can handle you will eventually fill the buffers and block till
> they flush. so the answer here is if you are mostly dealing with reads
> more and bursts of writes more ram is better. if you are dealing with
> constant writes it may cause uneven performance and may in fact be
> detrimental.
>
> 4) On the 10Gb consider using consider using RDMA Iverbs over Ethernet
> it may help.
>
> 5) Look closely at the scheduler on the disks the default these days
> is CFQ (Completely Fair Queuing). I find DEADLINE works better for
> Gluster; however depending on if you are using a RAID and you have a
> properly aligned file system you may find NOOP may provide better
> performance you need to do some testing with your hardware to
> determine if this is the case.
>
> On Mon, Mar 23, 2015 at 9:43 PM, Punit Dambiwal  wrote:
> > Hi,
> >
> > I want to use Glusterfs with Ovirt 3.5...please help me to make the
> > architecture stable for the production use :-
> >
> > I have 4 servers...every server can host 24 SSD disk(As bricks)..i want
> to
> > deploy distributed replicated storage with replica =2i don't want to
> use
> > the Hardware RAID...as i think it will badly impact the performance...
> >
> > 1. Glusterfs 3.5 or 3.6 ?? (which one will be stable for the production
> > use).
> > 2. Do i use the Hardware RAID or Not ??
> > 3. IF HW RAID then which RAID level and does it impact the performance...
> > 4. I want to make it rock solid...so it can use for production purpose...
> > 5. How much RAM should be sufficient on each server...on the each server
> i
> > have two E5 CPU's...
> > 6. For Network Connectivity i have 2*10G NIC with bonding on each
> server...
> >
> > Thanks,
> > Punit
> >
> > ___
> > Gluster-users mailing list
> > Gluster-users@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] GlusterFS for production use with Ovirt

[Paul Robert Marino]

A few things to clarify
1) are all 4 for Gluster or are they the whole cluster.
keep in mind that Ovirt expects that you are running Gluster on
different nodes than your VM's. while I do not completely agree with
that idea that is how it is now.

2) RAID hardware is not necessarily a bad thing, in fact the cache may
help significantly. that said  it is not required.
things to keep in mind if you are using RAID hardware for performance
2.a) RAID 0 will give you the best speed if you are using 3 nodes for
quorum. keep in mind this will require more work in the case of a
drive failure to remake the brick.
2.b) if you are only doing a mirror across 2 bricks RAID 0 may be
acceptable but consider using 1+0 or RAID 5 the performance on the two
may vary depending on the RAID controller but 1+0 is generally faster.
2.c) always optimize for the stripe which in theory should be
automatic if the RAID controller reports the required information to
do it, but in practice most do not so you should do it manually and
Red Hat's support site just added a nice tool to do it for you.

3) for performance more RAM is rarely a bad thing; however it can be
with sustained writes.If you are constantly writing faster than the
disks can handle you will eventually fill the buffers and block till
they flush. so the answer here is if you are mostly dealing with reads
more and bursts of writes more ram is better. if you are dealing with
constant writes it may cause uneven performance and may in fact be
detrimental.

4) On the 10Gb consider using consider using RDMA Iverbs over Ethernet
it may help.

5) Look closely at the scheduler on the disks the default these days
is CFQ (Completely Fair Queuing). I find DEADLINE works better for
Gluster; however depending on if you are using a RAID and you have a
properly aligned file system you may find NOOP may provide better
performance you need to do some testing with your hardware to
determine if this is the case.

On Mon, Mar 23, 2015 at 9:43 PM, Punit Dambiwal  wrote:
> Hi,
>
> I want to use Glusterfs with Ovirt 3.5...please help me to make the
> architecture stable for the production use :-
>
> I have 4 servers...every server can host 24 SSD disk(As bricks)..i want to
> deploy distributed replicated storage with replica =2i don't want to use
> the Hardware RAID...as i think it will badly impact the performance...
>
> 1. Glusterfs 3.5 or 3.6 ?? (which one will be stable for the production
> use).
> 2. Do i use the Hardware RAID or Not ??
> 3. IF HW RAID then which RAID level and does it impact the performance...
> 4. I want to make it rock solid...so it can use for production purpose...
> 5. How much RAM should be sufficient on each server...on the each server i
> have two E5 CPU's...
> 6. For Network Connectivity i have 2*10G NIC with bonding on each server...
>
> Thanks,
> Punit
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] GlusterFS for production use with Ovirt

[Punit Dambiwal]

Hi,

I want to use Glusterfs with Ovirt 3.5...please help me to make the
architecture stable for the production use :-

I have 4 servers...every server can host 24 SSD disk(As bricks)..i want to
deploy distributed replicated storage with replica =2i don't want to
use the Hardware RAID...as i think it will badly impact the performance...

1. Glusterfs 3.5 or 3.6 ?? (which one will be stable for the production
use).
2. Do i use the Hardware RAID or Not ??
3. IF HW RAID then which RAID level and does it impact the performance...
4. I want to make it rock solid...so it can use for production purpose...
5. How much RAM should be sufficient on each server...on the each server i
have two E5 CPU's...
6. For Network Connectivity i have 2*10G NIC with bonding on each server...

Thanks,
Punit
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] Revamping the GlusterFS Documentation...

[Justin Clift]

On 23 Mar 2015, at 07:01, Shravan Chandrashekar  wrote:
> Hi All, 
> 
> "The Gluster Filesystem documentation is not user friendly and fragmented" 
> and this has been the feedback we have been receiving. 
> 
> We got back to our drawing board and blueprints and realized that the content 
> was scattered at various places. These include: 
> 
> [Static HTML] http://www.gluster.org/documentation/ 
> [Mediawiki] http://www.gluster.org/community/documentation/ 
> [In-source] https://github.com/gluster/glusterfs/tree/master/doc 
> [Markdown] https://github.com/GlusterFS/Notes 
> 
> and so on… 
> 
> Hence, we started by curating content from various sources including 
> gluster.org static HTML documentation, glusterfs github repository, 
> various blog posts and the Community wiki. We also felt the need to improve 
> the community member's experience with Gluster documentation. This led us to 
> put some thought into the user interface. As a result we came up with a page 
> which links all content into a single landing page: 
> 
> http://www.gluster.org/community/documentation/index.php/Staged_Docs 
>  
> This is just our first step to improve our community docs and enhance the 
> community contribution towards documentation. I would like to thank Humble 
> Chirammal and Anjana Sriram for the suggestions and directions during the 
> entire process. I am sure there is lot of scope for improvement. 
> Hence, request you all to review the content and provide your suggestions. 

Looks like a good effort.  Is the general concept for this to
become the front/landing page for the main wiki?

Also some initial thoughts:

 * Gluster Ant Logo image - The first letter REALLY looks like a C
   (to me), not a G.  Reads as "Cluster" for me...

   That aside, it looks really good. :)


 * "Getting Started" section ... move it up maybe, before the
   Terminology / Architecture / Additional Resources bit

   This is to make it more obvious for new people.


 * "Terminologies" should probably be "Terminology", as
   "Terminology" is kind of both singular and plural.


 * "All that Developers need to know" → "Everything Developers
   need to know"

They're my first thoughts anyway. :)

+ Justin

--
GlusterFS - http://www.gluster.org

An open source, distributed file system scaling to several
petabytes, and handling thousands of clients.

My personal twitter: twitter.com/realjustinclift

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume creation time?

[Niels de Vos]

On Mon, Mar 23, 2015 at 02:03:22PM +, Jason Hilton wrote:
> Thank you Niels!  That did the trick.  I have everything worked out
> now and I'm even able to access the NFS shares via windows clients.
> Your help was much appreciated.

Glad to hear it helped, thanks for the confirmation!

Niels

> 
> 
> -Original Message-
> From: Niels de Vos [mailto:nde...@redhat.com] 
> Sent: Monday, March 23, 2015 4:58 AM
> To: Jason Hilton
> Cc: 'gluster-users@gluster.org'
> Subject: Re: [Gluster-users] Volume creation time?
> 
> On Sun, Mar 22, 2015 at 06:21:52PM +, Jason Hilton wrote:
> > So I have been trying to get this going and the volume status command 
> > shows that the "NFS Server on localhost" is offline, port and pid are 
> > "N/A".  I have double checked everything and NFS is disabled on start 
> > up an are not currently running.  The showmount command returns
> > "clnt_create: RPC: Program not registered"
> 
> Please make sure that none of the RPC programs for NFS are registered at 
> rpcbind before starting Gluster/NFS (or rebooting).
> 
> You can check by executing 'rpcinfo' on the Gluster/NFS server. Any of the 
> mountd, nlockmgr, status and nfs can block the registration of the NFS-server 
> (and its helper protocols). Unregistering one of the protocols can be done 
> like this:
> 
> # rpcinfo
> ...
> 153tcp   0.0.0.0.150.65 mountd superuser
> ...
> 
> # rpcinfo -d 15 3
> 
> Do this for all the protocols (+versions) mentioned above and restart the 
> Gluster/NFS server process:
> 
> # gluster volume start $VOLUME force
> 
> The 'start force' will only start any missing processes, like the Gluster/NFS 
> (a 'glusterfs' process). Running processes (like the ones for the bricks) 
> should not be impacted.
> 
> Niels
> 
> > 
> > Any ideas?
> > Thanks again!
> >  
> > -Original Message-
> > From: Niels de Vos [mailto:nde...@redhat.com]
> > Sent: Sunday, March 22, 2015 11:45 AM
> > To: Jason Hilton
> > Cc: 'gluster-users@gluster.org'
> > Subject: Re: [Gluster-users] Volume creation time?
> > 
> > On Sun, Mar 22, 2015 at 02:59:53PM +, Jason Hilton wrote:
> > > Thank you for the quick reply!  I didn't expect to see any response 
> > > on a Sunday.  I did as you suggested and found  some messages 
> > > stating that the address and port were failing to bind because it 
> > > was already in use.  It turned out that the NFS service was running 
> > > and interfered with glusterd.  I was intending to share my gluster 
> > > volumes via NFS and I thought I had read that as of V3, gluster 
> > > exported NFS shares by default, so I had started the service.  Does 
> > > gluster provide its own NFS services?
> > 
> > Yes, Gluster comes indeed with its own NFS-server. You should not start any 
> > NFS-services, Gluster takes care of starting them. The only service that 
> > you need to have running (or activated for systemd environments), is 
> > rpcbind.
> > 
> > Once your volume has been created and started, you should be able to see 
> > that there is a NFS-server running with this command:
> > 
> > # gluster volume status
> > 
> > And, with 'showmount -e' the volme should be listed as an export.
> > 
> > Cheers,
> > Niels
> > 
> > > 
> > > ***
> > > Jason Hilton
> > > Director of Technology Development
> > > 601 Madison Street, Suite 400
> > > Alexandria, VA 22314
> > > jason.hil...@aaae.org
> > > Desk: 703.824.0500x167
> > > FAX: 703.578.4952
> > > 
> > > AAAE Tech support:
> > > i...@aaae.org
> > > 703.797.2555, opt. 2
> > > ***
> > > 
> > > 
> > > -Original Message-
> > > From: Niels de Vos [mailto:nde...@redhat.com]
> > > Sent: Sunday, March 22, 2015 10:13 AM
> > > To: Jason Hilton
> > > Cc: 'gluster-users@gluster.org'
> > > Subject: Re: [Gluster-users] Volume creation time?
> > > 
> > > On Sun, Mar 22, 2015 at 01:34:24PM +, Jason Hilton wrote:
> > > > Hi-
> > > > I'm new to GlusterFS and I have been trying to set up a gluster 
> > > > volume.  The volume is 150 TB.  I started the create volume 
> > > > command on Friday morning and it has not yet completed.  Since I 
> > > > have no prior experience with GlusterFS, is this an expected 
> > > > duration?  The server is no power house, a pair of older Xeon Quad 
> > > > core processors at 2 GHz and only 4 GB of RAM.  TOP shows very 
> > > > little processor usage, but IOTOP shows some disk I/O.  I don't 
> > > > mind waiting it out, I just want to be sure that the process is 
> > > > still proceeding.  Is there a way to monitor Gluster volume creation 
> > > > progress?
> > > 
> > > Volume creation should be very fast, there is not a lot to do for Gluster 
> > > to create a volume. A couple of seconds should be sufficient.
> > > 
> > > Check the /var/log/glusterfs/etc-*.log to see if there are any errors 
> > > listed there.
> > > 
> > > HTH,
> >

Re: [Gluster-users] What should I do to improve performance ?

[Ben Turner]

- Original Message -
> From: "marianna cattani" 
> To: gluster-users@gluster.org
> Sent: Monday, March 23, 2015 6:09:41 AM
> Subject: [Gluster-users] What should I do to improve performance ?
> 
> Dear all,
> I followed the tutorial I read at this link :
> http://www.gluster.org/documentation/use_cases/Virt-store-usecase/
> 
> I have 4 nodes configured as a linked list , each node also performs virtual
> machines with KVM and mounts on its ip address, like this:
> 
> 172.16.155.12:/nova /var/lib/nova/instances glusterfs defaults,_netdev 0 0
> 
> Each node has two nic (ten giga) bonded in mode 4.
> 
> What can I do to further improve the speed ?

What kind of disks are back ending your 10G NICs?  Are you using FUSE or 
libgfapi to connect to gluster from your hypervisor?  What kind of speeds are 
you expecting vs seeing in your environment?  We need to understand what your 
HW can do first then gather some data running on gluster and compare the two.  
As a rule of thumb with replica 2 you should see about:

throughput = ( NIC line speed / 2 ) - 20% overhead

As long as your disks can service it.  If you are seeing about that on the 
gluster mounts then go inside one of the VMs and run the same test, the VM 
should get something similar.  If you aren't seeing at least 400 MB / sec on 
sequential writes and 500-700 MB /sec on reads then there may be something off 
in your storage stack.

-b

> BR.
> 
> M.
> 
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume creation time?

[Jason Hilton]

Thank you Niels!  That did the trick.  I have everything worked out now and I'm 
even able to access the NFS shares via windows clients.  Your help was much 
appreciated.


-Original Message-
From: Niels de Vos [mailto:nde...@redhat.com] 
Sent: Monday, March 23, 2015 4:58 AM
To: Jason Hilton
Cc: 'gluster-users@gluster.org'
Subject: Re: [Gluster-users] Volume creation time?

On Sun, Mar 22, 2015 at 06:21:52PM +, Jason Hilton wrote:
> So I have been trying to get this going and the volume status command 
> shows that the "NFS Server on localhost" is offline, port and pid are 
> "N/A".  I have double checked everything and NFS is disabled on start 
> up an are not currently running.  The showmount command returns
> "clnt_create: RPC: Program not registered"

Please make sure that none of the RPC programs for NFS are registered at 
rpcbind before starting Gluster/NFS (or rebooting).

You can check by executing 'rpcinfo' on the Gluster/NFS server. Any of the 
mountd, nlockmgr, status and nfs can block the registration of the NFS-server 
(and its helper protocols). Unregistering one of the protocols can be done like 
this:

# rpcinfo
...
153tcp   0.0.0.0.150.65 mountd superuser
...

# rpcinfo -d 15 3

Do this for all the protocols (+versions) mentioned above and restart the 
Gluster/NFS server process:

# gluster volume start $VOLUME force

The 'start force' will only start any missing processes, like the Gluster/NFS 
(a 'glusterfs' process). Running processes (like the ones for the bricks) 
should not be impacted.

Niels

> 
> Any ideas?
> Thanks again!
>  
> -Original Message-
> From: Niels de Vos [mailto:nde...@redhat.com]
> Sent: Sunday, March 22, 2015 11:45 AM
> To: Jason Hilton
> Cc: 'gluster-users@gluster.org'
> Subject: Re: [Gluster-users] Volume creation time?
> 
> On Sun, Mar 22, 2015 at 02:59:53PM +, Jason Hilton wrote:
> > Thank you for the quick reply!  I didn't expect to see any response 
> > on a Sunday.  I did as you suggested and found  some messages 
> > stating that the address and port were failing to bind because it 
> > was already in use.  It turned out that the NFS service was running 
> > and interfered with glusterd.  I was intending to share my gluster 
> > volumes via NFS and I thought I had read that as of V3, gluster 
> > exported NFS shares by default, so I had started the service.  Does 
> > gluster provide its own NFS services?
> 
> Yes, Gluster comes indeed with its own NFS-server. You should not start any 
> NFS-services, Gluster takes care of starting them. The only service that you 
> need to have running (or activated for systemd environments), is rpcbind.
> 
> Once your volume has been created and started, you should be able to see that 
> there is a NFS-server running with this command:
> 
> # gluster volume status
> 
> And, with 'showmount -e' the volme should be listed as an export.
> 
> Cheers,
> Niels
> 
> > 
> > ***
> > Jason Hilton
> > Director of Technology Development
> > 601 Madison Street, Suite 400
> > Alexandria, VA 22314
> > jason.hil...@aaae.org
> > Desk: 703.824.0500x167
> > FAX: 703.578.4952
> > 
> > AAAE Tech support:
> > i...@aaae.org
> > 703.797.2555, opt. 2
> > ***
> > 
> > 
> > -Original Message-
> > From: Niels de Vos [mailto:nde...@redhat.com]
> > Sent: Sunday, March 22, 2015 10:13 AM
> > To: Jason Hilton
> > Cc: 'gluster-users@gluster.org'
> > Subject: Re: [Gluster-users] Volume creation time?
> > 
> > On Sun, Mar 22, 2015 at 01:34:24PM +, Jason Hilton wrote:
> > > Hi-
> > > I'm new to GlusterFS and I have been trying to set up a gluster 
> > > volume.  The volume is 150 TB.  I started the create volume 
> > > command on Friday morning and it has not yet completed.  Since I 
> > > have no prior experience with GlusterFS, is this an expected 
> > > duration?  The server is no power house, a pair of older Xeon Quad 
> > > core processors at 2 GHz and only 4 GB of RAM.  TOP shows very 
> > > little processor usage, but IOTOP shows some disk I/O.  I don't 
> > > mind waiting it out, I just want to be sure that the process is 
> > > still proceeding.  Is there a way to monitor Gluster volume creation 
> > > progress?
> > 
> > Volume creation should be very fast, there is not a lot to do for Gluster 
> > to create a volume. A couple of seconds should be sufficient.
> > 
> > Check the /var/log/glusterfs/etc-*.log to see if there are any errors 
> > listed there.
> > 
> > HTH,
> > Niels
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[Andreas]

Hi,

Thanks, but no firewall involved in my distro.

FYI:
# netstat -tan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address   Foreign Address State  
tcp0  0 0.0.0.0:24007   0.0.0.0:*   LISTEN


Regards
Andreas

On 03/23/15 14:21, JF Le Fillâtre wrote:
> Hello,
>
> If you're running RHEL 7 or CentOS 7, what is the state of firewalld on
> your systems?
>
> firewall-cmd --list-all-zones
>
> Thanks,
> JF
>
>
> On 23/03/15 14:14, Andreas wrote:
>> Hi,
>>
>> # gluster peer detach 10.32.1.144
>> (No output here. Similar to the problem with 'gluster peer probe'.)
>> # gluster peer detach 10.32.1.144 force
>> peer detach: failed: Peer is already being detached from cluster.
>> Check peer status by running gluster peer status
>> # gluster peer status
>> Number of Peers: 1
>>
>> Hostname: 10.32.1.144
>> Uuid: 82cdb873-28cc-4ed0-8cfe-2b6275770429
>> State: Probe Sent to Peer (Connected)
>>
>> # ping 10.32.1.144
>> PING 10.32.1.144 (10.32.1.144): 56 data bytes
>> 64 bytes from 10.32.1.144: seq=0 ttl=64 time=1.811 ms
>> 64 bytes from 10.32.1.144: seq=1 ttl=64 time=1.834 ms
>> ^C
>> --- 10.32.1.144 ping statistics ---
>> 2 packets transmitted, 2 packets received, 0% packet loss
>> round-trip min/avg/max = 1.811/1.822/1.834 ms
>>
>>
>> As previously stated, this problem seems to be similar to what I experienced 
>> with
>> 'gluster peer probe'. I can reboot the server, but the situation will be the 
>> same
>> (I've tried this many times).
>> Any ideas of which ports to investigate and how to do it to get the most 
>> reliable result?
>> Anything else that could cause this?
>>
>>
>>
>> Regards
>> Andreas
>>
>>
>> On 03/23/15 11:10, Atin Mukherjee wrote:
>>> On 03/23/2015 03:28 PM, Andreas Hollaus wrote:
 2Hi,

 This network problem is persistent. However, I can ping the server so 
 guess it
 depends on the port no, right?
 I tried to telnet to port 24007, but I was not sure how to interpret the 
 result as I
 got no respons and no timeout (it just seemed to be waiting for something).
 That's why I decided to install nmap, but according to that tool the port 
 was
 accessible. Are there any other ports that are vital to gluster peer probe?

 When you say 'deprobe', I guess you mean 'gluster peer detach'? That 
 command shows
 similar behaviour to gluster peer probe.
>>> Yes I meant peer detach. How about gluster peer detach force?
>>
 Regards
 Andreas

 On 03/23/15 05:34, Atin Mukherjee wrote:
> On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
>> Hi,
>>
>> I hope that these are the logs that you requested.
>>
>> Logs from 10.32.0.48:
>> --
>> # more /var/log/glusterfs/.cmd_log_history
>> [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
>> returned
>>  with unknown errno -1
>>
>> # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
>> [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
>> 0-/usr/s
>> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
>> /usr/sbin/
>> glusterd -p /var/run/glusterd.pid)
>> [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: 
>> Maximum allo
>> wed open file descriptors set to 65536
>> [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: 
>> Using /var/l
>> ib/glusterd as working directory
>> [2015-03-19 13:41:31.247826] I 
>> [glusterd-store.c:2063:glusterd_restore_op_versio
>> n] 0-management: Detected new install. Setting op-version to maximum : 
>> 30600
>> [2015-03-19 13:41:31.247902] I 
>> [glusterd-store.c:3497:glusterd_store_retrieve_mi
>> ssed_snaps_list] 0-management: No missed snaps list.
>> Final graph:
>> +--+
>>   1: volume management
>>   2: type mgmt/glusterd
>>   3: option rpc-auth.auth-glusterfs on
>>   4: option rpc-auth.auth-unix on
>>   5: option rpc-auth.auth-null on
>>   6: option transport.socket.listen-backlog 128
>>   7: option ping-timeout 30
>>   8: option transport.socket.read-fail-log off
>>   9: option transport.socket.keepalive-interval 2
>>  10: option transport.socket.keepalive-time 10
>>  11: option transport-type socket
>>  12: option working-directory /var/lib/glusterd
>>  13: end-volume
>>  14: 
>> +--+
>> [2015-03-19 13:42:02.258403] I 
>> [glusterd-handler.c:1015:__glusterd_handle_cli_pr
>> obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
>> [2015-03-19 13:42:02.259456] I 
>> [glusterd-handler.c:3165:glusterd_probe_begin] 0-
>> glusterd: Unable to find pe

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[JF Le Fillâtre]

Hello,

If you're running RHEL 7 or CentOS 7, what is the state of firewalld on
your systems?

firewall-cmd --list-all-zones

Thanks,
JF


On 23/03/15 14:14, Andreas wrote:
> Hi,
> 
> # gluster peer detach 10.32.1.144
> (No output here. Similar to the problem with 'gluster peer probe'.)
> # gluster peer detach 10.32.1.144 force
> peer detach: failed: Peer is already being detached from cluster.
> Check peer status by running gluster peer status
> # gluster peer status
> Number of Peers: 1
> 
> Hostname: 10.32.1.144
> Uuid: 82cdb873-28cc-4ed0-8cfe-2b6275770429
> State: Probe Sent to Peer (Connected)
> 
> # ping 10.32.1.144
> PING 10.32.1.144 (10.32.1.144): 56 data bytes
> 64 bytes from 10.32.1.144: seq=0 ttl=64 time=1.811 ms
> 64 bytes from 10.32.1.144: seq=1 ttl=64 time=1.834 ms
> ^C
> --- 10.32.1.144 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip min/avg/max = 1.811/1.822/1.834 ms
> 
> 
> As previously stated, this problem seems to be similar to what I experienced 
> with
> 'gluster peer probe'. I can reboot the server, but the situation will be the 
> same
> (I've tried this many times).
> Any ideas of which ports to investigate and how to do it to get the most 
> reliable result?
> Anything else that could cause this?
> 
> 
> 
> Regards
> Andreas
> 
> 
> On 03/23/15 11:10, Atin Mukherjee wrote:
>>
>> On 03/23/2015 03:28 PM, Andreas Hollaus wrote:
>>> 2Hi,
>>>
>>> This network problem is persistent. However, I can ping the server so guess 
>>> it
>>> depends on the port no, right?
>>> I tried to telnet to port 24007, but I was not sure how to interpret the 
>>> result as I
>>> got no respons and no timeout (it just seemed to be waiting for something).
>>> That's why I decided to install nmap, but according to that tool the port 
>>> was
>>> accessible. Are there any other ports that are vital to gluster peer probe?
>>>
>>> When you say 'deprobe', I guess you mean 'gluster peer detach'? That 
>>> command shows
>>> similar behaviour to gluster peer probe.
>> Yes I meant peer detach. How about gluster peer detach force?
> 
> 
>>>
>>> Regards
>>> Andreas
>>>
>>> On 03/23/15 05:34, Atin Mukherjee wrote:
 On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
> Hi,
>
> I hope that these are the logs that you requested.
>
> Logs from 10.32.0.48:
> --
> # more /var/log/glusterfs/.cmd_log_history
> [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
> returned
>  with unknown errno -1
>
> # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
> [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
> 0-/usr/s
> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
> /usr/sbin/
> glusterd -p /var/run/glusterd.pid)
> [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: 
> Maximum allo
> wed open file descriptors set to 65536
> [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: Using 
> /var/l
> ib/glusterd as working directory
> [2015-03-19 13:41:31.247826] I 
> [glusterd-store.c:2063:glusterd_restore_op_versio
> n] 0-management: Detected new install. Setting op-version to maximum : 
> 30600
> [2015-03-19 13:41:31.247902] I 
> [glusterd-store.c:3497:glusterd_store_retrieve_mi
> ssed_snaps_list] 0-management: No missed snaps list.
> Final graph:
> +--+
>   1: volume management
>   2: type mgmt/glusterd
>   3: option rpc-auth.auth-glusterfs on
>   4: option rpc-auth.auth-unix on
>   5: option rpc-auth.auth-null on
>   6: option transport.socket.listen-backlog 128
>   7: option ping-timeout 30
>   8: option transport.socket.read-fail-log off
>   9: option transport.socket.keepalive-interval 2
>  10: option transport.socket.keepalive-time 10
>  11: option transport-type socket
>  12: option working-directory /var/lib/glusterd
>  13: end-volume
>  14: 
> +--+
> [2015-03-19 13:42:02.258403] I 
> [glusterd-handler.c:1015:__glusterd_handle_cli_pr
> obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
> [2015-03-19 13:42:02.259456] I 
> [glusterd-handler.c:3165:glusterd_probe_begin] 0-
> glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
> [2015-03-19 13:42:02.259664] I [rpc-clnt.c:969:rpc_clnt_connection_init] 
> 0-manag
> ement: setting frame-timeout to 600
> [2015-03-19 13:42:02.260488] I 
> [glusterd-handler.c:3098:glusterd_friend_add] 0-m
> anagement: connect returned 0
> [2015-03-19 13:42:02.270316] I 
> [glusterd.c:176:glusterd_uuid_generate_save] 0-ma
> nagement: generated UUID: 4441e237-89d6-4cdf-a212-f1

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[Andreas]

Hi,

# gluster peer detach 10.32.1.144
(No output here. Similar to the problem with 'gluster peer probe'.)
# gluster peer detach 10.32.1.144 force
peer detach: failed: Peer is already being detached from cluster.
Check peer status by running gluster peer status
# gluster peer status
Number of Peers: 1

Hostname: 10.32.1.144
Uuid: 82cdb873-28cc-4ed0-8cfe-2b6275770429
State: Probe Sent to Peer (Connected)

# ping 10.32.1.144
PING 10.32.1.144 (10.32.1.144): 56 data bytes
64 bytes from 10.32.1.144: seq=0 ttl=64 time=1.811 ms
64 bytes from 10.32.1.144: seq=1 ttl=64 time=1.834 ms
^C
--- 10.32.1.144 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.811/1.822/1.834 ms


As previously stated, this problem seems to be similar to what I experienced 
with
'gluster peer probe'. I can reboot the server, but the situation will be the 
same
(I've tried this many times).
Any ideas of which ports to investigate and how to do it to get the most 
reliable result?
Anything else that could cause this?



Regards
Andreas


On 03/23/15 11:10, Atin Mukherjee wrote:
>
> On 03/23/2015 03:28 PM, Andreas Hollaus wrote:
>> 2Hi,
>>
>> This network problem is persistent. However, I can ping the server so guess 
>> it
>> depends on the port no, right?
>> I tried to telnet to port 24007, but I was not sure how to interpret the 
>> result as I
>> got no respons and no timeout (it just seemed to be waiting for something).
>> That's why I decided to install nmap, but according to that tool the port was
>> accessible. Are there any other ports that are vital to gluster peer probe?
>>
>> When you say 'deprobe', I guess you mean 'gluster peer detach'? That command 
>> shows
>> similar behaviour to gluster peer probe.
> Yes I meant peer detach. How about gluster peer detach force?


>>
>> Regards
>> Andreas
>>
>> On 03/23/15 05:34, Atin Mukherjee wrote:
>>> On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
 Hi,

 I hope that these are the logs that you requested.

 Logs from 10.32.0.48:
 --
 # more /var/log/glusterfs/.cmd_log_history
 [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
 returned
  with unknown errno -1

 # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
 [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
 0-/usr/s
 bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
 /usr/sbin/
 glusterd -p /var/run/glusterd.pid)
 [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: 
 Maximum allo
 wed open file descriptors set to 65536
 [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: Using 
 /var/l
 ib/glusterd as working directory
 [2015-03-19 13:41:31.247826] I 
 [glusterd-store.c:2063:glusterd_restore_op_versio
 n] 0-management: Detected new install. Setting op-version to maximum : 
 30600
 [2015-03-19 13:41:31.247902] I 
 [glusterd-store.c:3497:glusterd_store_retrieve_mi
 ssed_snaps_list] 0-management: No missed snaps list.
 Final graph:
 +--+
   1: volume management
   2: type mgmt/glusterd
   3: option rpc-auth.auth-glusterfs on
   4: option rpc-auth.auth-unix on
   5: option rpc-auth.auth-null on
   6: option transport.socket.listen-backlog 128
   7: option ping-timeout 30
   8: option transport.socket.read-fail-log off
   9: option transport.socket.keepalive-interval 2
  10: option transport.socket.keepalive-time 10
  11: option transport-type socket
  12: option working-directory /var/lib/glusterd
  13: end-volume
  14: 
 +--+
 [2015-03-19 13:42:02.258403] I 
 [glusterd-handler.c:1015:__glusterd_handle_cli_pr
 obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
 [2015-03-19 13:42:02.259456] I 
 [glusterd-handler.c:3165:glusterd_probe_begin] 0-
 glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
 [2015-03-19 13:42:02.259664] I [rpc-clnt.c:969:rpc_clnt_connection_init] 
 0-manag
 ement: setting frame-timeout to 600
 [2015-03-19 13:42:02.260488] I 
 [glusterd-handler.c:3098:glusterd_friend_add] 0-m
 anagement: connect returned 0
 [2015-03-19 13:42:02.270316] I 
 [glusterd.c:176:glusterd_uuid_generate_save] 0-ma
 nagement: generated UUID: 4441e237-89d6-4cdf-a212-f17ecb953b58
 [2015-03-19 13:42:02.273427] I 
 [glusterd-rpc-ops.c:244:__glusterd_probe_cbk] 0-m
 anagement: Received probe resp from uuid: 
 82cdb873-28cc-4ed0-8cfe-2b6275770429,
 host: 10.32.1.144
 [2015-03-19 13:42:02.273681] I 
 [glusterd-rpc-ops.c:386:__glusterd_probe_cbk] 0-g
 lusterd: Received resp to probe re

[Gluster-users] Gluster mount failing

[Mark Lewis]

Hello
Vanilla client install on 14.04 ubuntu, servers running centos.

# mount -t glusterfs 10.0.1.63:/s2cVol1  /var/www/data

[2015-03-23 10:51:22.568090] I [glusterfsd.c:1910:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.4.2
(/usr/sbin/glusterfs --volfile-id=/s2cVol1 --volfile-server=10.0.1.63
/var/www/data)
[2015-03-23 10:51:22.609516] I [socket.c:3480:socket_init] 0-glusterfs: SSL
support is NOT enabled
[2015-03-23 10:51:22.609630] I [socket.c:3495:socket_init] 0-glusterfs:
using system polling thread
[2015-03-23 10:51:22.624985] I [socket.c:3480:socket_init]
0-s2cVol1-client-2: SSL support is NOT enabled
[2015-03-23 10:51:22.625212] I [socket.c:3495:socket_init]
0-s2cVol1-client-2: using system polling thread
[2015-03-23 10:51:22.627370] I [socket.c:3480:socket_init]
0-s2cVol1-client-1: SSL support is NOT enabled
[2015-03-23 10:51:22.627565] I [socket.c:3495:socket_init]
0-s2cVol1-client-1: using system polling thread
[2015-03-23 10:51:22.629733] I [socket.c:3480:socket_init]
0-s2cVol1-client-0: SSL support is NOT enabled
[2015-03-23 10:51:22.629910] I [socket.c:3495:socket_init]
0-s2cVol1-client-0: using system polling thread
[2015-03-23 10:51:22.630116] I [client.c:2154:notify] 0-s2cVol1-client-0:
parent translators are ready, attempting connect on transport
[2015-03-23 10:51:22.632624] E [common-utils.c:211:gf_resolve_ip6]
0-resolver: getaddrinfo failed (Name or service not known)
[2015-03-23 10:51:22.632837] E
[name.c:249:af_inet_client_get_remote_sockaddr] 0-s2cVol1-client-0: DNS
resolution failed on host ealcent0008
[2015-03-23 10:51:22.633020] I [client.c:2154:notify] 0-s2cVol1-client-1:
parent translators are ready, attempting connect on transport
[2015-03-23 10:51:22.634215] E [common-utils.c:211:gf_resolve_ip6]
0-resolver: getaddrinfo failed (Name or service not known)
[2015-03-23 10:51:22.634405] E
[name.c:249:af_inet_client_get_remote_sockaddr] 0-s2cVol1-client-1: DNS
resolution failed on host ealcent0009
[2015-03-23 10:51:22.634555] I [client.c:2154:notify] 0-s2cVol1-client-2:
parent translators are ready, attempting connect on transport
[2015-03-23 10:51:22.635703] E [common-utils.c:211:gf_resolve_ip6]
0-resolver: getaddrinfo failed (Name or service not known)
[2015-03-23 10:51:22.635892] E
[name.c:249:af_inet_client_get_remote_sockaddr] 0-s2cVol1-client-2: DNS
resolution failed on host ealcent0010
[2015-03-23 10:51:22.636042] E [afr-common.c:3735:afr_notify]
0-s2cVol1-replicate-0: All subvolumes are down. Going offline until atleast
one of them comes back up.
[2015-03-23 10:51:22.654066] I [fuse-bridge.c:4769:fuse_graph_setup] 0-fuse:
switched to graph 0
Given volfile:


+---
---+
[2015-03-23 10:51:22.659826] I [fuse-bridge.c:3724:fuse_init]
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel
7.22
[2015-03-23 10:51:22.660618] W [fuse-bridge.c:705:fuse_attr_cbk]
0-glusterfs-fuse: 2: LOOKUP() / => -1 (No such file or directory)
Mount failed. Please check the log file for more details.
[2015-03-23 10:51:22.666530] I [fuse-bridge.c:4628:fuse_thread_proc] 0-fuse:
unmounting /var/www/data
[2015-03-23 10:51:22.667179] W [glusterfsd.c:1002:cleanup_and_exit]
(-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f524f7ad47d]
(-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f524fa80182]
(-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xd5) [0x7f5250569ef5]))) 0-:
received signum (15), shutting down
[2015-03-23 10:51:22.667212] I [fuse-bridge.c:5260:fini] 0-fuse: Unmounting
'/var/www/data'.

Can anyone suggest what to do next?  I can telnet to one of the replicas on
111 OK, no firewall ports in the way I don¹t think, and the volume was
mounted OK before I installed the new client.



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster mount failing

[Mark Lewis]

Ignore that ­ I¹ve just seen the DNS error.



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Fwd: Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and checkpatch warnings fixed

[Jiffin Tony Thottan]

Yup

On 23/03/15 17:01, Humble Devassy Chirammal wrote:

Isnt this one http://review.gluster.org/#/c/9971 ?

--Humble


On Mon, Mar 23, 2015 at 3:11 PM, Niels de Vos > wrote:


On Mon, Mar 23, 2015 at 12:49:56PM +0530, Anand Subramanian wrote:
> FYI.
>
> GlusterFS vols can now be accessed via NFSv4.1 pNFS protocol
(mount -t nfs
> -o minorversion=1 ...) from nfs-ganesha 2.2-rc5 onwards.
>
> Note: one fix is to go into libgfapi to fix up using anonymous
fd's in
> ds_write/make_ds_handle() (Avati's sugeestion that really helps
here).
> Once Jiffin or myself get that fix in, a good large file
performance can be
> seen with pNFS vs V4.

I could not find the needed change for libgfapi. Could you post
the link
to the review?

Thanks,
Niels

>
> All thanks and credit to Jiffin for his terrific effort in
coding things up
> quickly and for fixing bugs.
>
> Anand
>
>
>  Forwarded Message 
> Subject:  Change in ffilz/nfs-ganesha[next]: pNFS code drop
enablement and
> checkpatch warnings fixed
> Date: Sat, 21 Mar 2015 01:04:30 +0100
> From: GerritHub mailto:supp...@gerritforge.com>>
> Reply-To: ffilz...@mindspring.com 
> To:   Anand Subramanian mailto:ana...@redhat.com>>
> CC: onnfrhvruutnzhnaq.-g...@noclue.notk.org

>
>
>
> From Frank Filz mailto:ffilz...@mindspring.com>>:
>
> Frank Filz has submitted this change and it was merged.
>
> Change subject: pNFS code drop enablement and checkpatch
warnings fixed
>
..
>
>
> pNFS code drop enablement and checkpatch warnings fixed
>
> Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> Signed-off-by: Anand Subramanian mailto:ana...@redhat.com>>
> ---
> A src/FSAL/FSAL_GLUSTER/ds.c
> M src/FSAL/FSAL_GLUSTER/export.c
> M src/FSAL/FSAL_GLUSTER/gluster_internal.h
> M src/FSAL/FSAL_GLUSTER/handle.c
> M src/FSAL/FSAL_GLUSTER/main.c
> A src/FSAL/FSAL_GLUSTER/mds.c
> 6 files changed, 993 insertions(+), 0 deletions(-)
>
>
>
> --
> To view, visit https://review.gerrithub.io/221683
> To unsubscribe, visit https://review.gerrithub.io/settings
>
> Gerrit-MessageType: merged
> Gerrit-Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> Gerrit-PatchSet: 1
> Gerrit-Project: ffilz/nfs-ganesha
> Gerrit-Branch: next
> Gerrit-Owner: Anand Subramanian mailto:ana...@redhat.com>>
> Gerrit-Reviewer: Frank Filz mailto:ffilz...@mindspring.com>>
> Gerrit-Reviewer: onnfrhvruutnzhnaq.-g...@noclue.notk.org

>
>
>

> ___
> Gluster-users mailing list
> Gluster-users@gluster.org 
> http://www.gluster.org/mailman/listinfo/gluster-users

___
Gluster-users mailing list
Gluster-users@gluster.org 
http://www.gluster.org/mailman/listinfo/gluster-users




___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Fwd: Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and checkpatch warnings fixed

[Humble Devassy Chirammal]

Isnt this one http://review.gluster.org/#/c/9971 ?

--Humble


On Mon, Mar 23, 2015 at 3:11 PM, Niels de Vos  wrote:

> On Mon, Mar 23, 2015 at 12:49:56PM +0530, Anand Subramanian wrote:
> > FYI.
> >
> > GlusterFS vols can now be accessed via NFSv4.1 pNFS protocol (mount -t
> nfs
> > -o minorversion=1 ...) from nfs-ganesha 2.2-rc5 onwards.
> >
> > Note: one fix is to go into libgfapi to fix up using anonymous fd's in
> > ds_write/make_ds_handle() (Avati's sugeestion that really helps here).
> > Once Jiffin or myself get that fix in, a good large file performance can
> be
> > seen with pNFS vs V4.
>
> I could not find the needed change for libgfapi. Could you post the link
> to the review?
>
> Thanks,
> Niels
>
> >
> > All thanks and credit to Jiffin for his terrific effort in coding things
> up
> > quickly and for fixing bugs.
> >
> > Anand
> >
> >
> >  Forwarded Message 
> > Subject:  Change in ffilz/nfs-ganesha[next]: pNFS code drop
> enablement and
> > checkpatch warnings fixed
> > Date: Sat, 21 Mar 2015 01:04:30 +0100
> > From: GerritHub 
> > Reply-To: ffilz...@mindspring.com
> > To:   Anand Subramanian 
> > CC:   onnfrhvruutnzhnaq.-g...@noclue.notk.org
> >
> >
> >
> > From Frank Filz :
> >
> > Frank Filz has submitted this change and it was merged.
> >
> > Change subject: pNFS code drop enablement and checkpatch warnings fixed
> > ..
> >
> >
> > pNFS code drop enablement and checkpatch warnings fixed
> >
> > Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> > Signed-off-by: Anand Subramanian 
> > ---
> > A src/FSAL/FSAL_GLUSTER/ds.c
> > M src/FSAL/FSAL_GLUSTER/export.c
> > M src/FSAL/FSAL_GLUSTER/gluster_internal.h
> > M src/FSAL/FSAL_GLUSTER/handle.c
> > M src/FSAL/FSAL_GLUSTER/main.c
> > A src/FSAL/FSAL_GLUSTER/mds.c
> > 6 files changed, 993 insertions(+), 0 deletions(-)
> >
> >
> >
> > --
> > To view, visit https://review.gerrithub.io/221683
> > To unsubscribe, visit https://review.gerrithub.io/settings
> >
> > Gerrit-MessageType: merged
> > Gerrit-Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> > Gerrit-PatchSet: 1
> > Gerrit-Project: ffilz/nfs-ganesha
> > Gerrit-Branch: next
> > Gerrit-Owner: Anand Subramanian 
> > Gerrit-Reviewer: Frank Filz 
> > Gerrit-Reviewer: onnfrhvruutnzhnaq.-g...@noclue.notk.org
> >
> >
> >
>
> > ___
> > Gluster-users mailing list
> > Gluster-users@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Is rebalance completely broken on 3.5.3 ?

[Alessandro Ipe]

Hi Olav,


Thanks for the info. I read the whole thread that you sent me... and I am more 
scared 
than ever... The fact that the developers do not have a clue of what is causing 
this 
issue is just frightening.

Concerning my issue, apparently after two days (a full heal is ongoing on the 
volume), 
I did not get any error messages from the client when trying to list the 
incriminate 
files, but I got twice the same file .forward with the same content, size, 
permissions 
and date... which is consistent to what you got previously... I simply remove 
TWICE 
the file with rm on the client and copy back a sane version. The one million 
dollar 
question is : are there more files in a similar state on my 90 TB volume ? I am 
delaying a find on the whole volume to find out...

What also concerns me is the absence of aknowledgement or reply from the 
developers concerning this severe issue... The fact that only end-users on 
production 
setup hit this issue while it cannot be reproduced in labs should be a clear 
signal that 
this should addressed in priority, from my point of view. And lab testing 
should also try 
to mimic real life use, with bricks servers under heavy load (> 10), with 
several tens 
of client accessing the gluster volume to track down all possible issues 
resulting from 
either network, i/o, ... timeouts.


Thanks for your help,


Alessandro.


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[Atin Mukherjee]

On 03/23/2015 03:28 PM, Andreas Hollaus wrote:
> 2Hi,
> 
> This network problem is persistent. However, I can ping the server so guess it
> depends on the port no, right?
> I tried to telnet to port 24007, but I was not sure how to interpret the 
> result as I
> got no respons and no timeout (it just seemed to be waiting for something).
> That's why I decided to install nmap, but according to that tool the port was
> accessible. Are there any other ports that are vital to gluster peer probe?
> 
> When you say 'deprobe', I guess you mean 'gluster peer detach'? That command 
> shows
> similar behaviour to gluster peer probe.
Yes I meant peer detach. How about gluster peer detach force?
> 
> 
> Regards
> Andreas
> 
> On 03/23/15 05:34, Atin Mukherjee wrote:
>>
>> On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
>>> Hi,
>>>
>>> I hope that these are the logs that you requested.
>>>
>>> Logs from 10.32.0.48:
>>> --
>>> # more /var/log/glusterfs/.cmd_log_history
>>> [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
>>> returned
>>>  with unknown errno -1
>>>
>>> # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
>>> [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
>>> 0-/usr/s
>>> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
>>> /usr/sbin/
>>> glusterd -p /var/run/glusterd.pid)
>>> [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: Maximum 
>>> allo
>>> wed open file descriptors set to 65536
>>> [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: Using 
>>> /var/l
>>> ib/glusterd as working directory
>>> [2015-03-19 13:41:31.247826] I 
>>> [glusterd-store.c:2063:glusterd_restore_op_versio
>>> n] 0-management: Detected new install. Setting op-version to maximum : 30600
>>> [2015-03-19 13:41:31.247902] I 
>>> [glusterd-store.c:3497:glusterd_store_retrieve_mi
>>> ssed_snaps_list] 0-management: No missed snaps list.
>>> Final graph:
>>> +--+
>>>   1: volume management
>>>   2: type mgmt/glusterd
>>>   3: option rpc-auth.auth-glusterfs on
>>>   4: option rpc-auth.auth-unix on
>>>   5: option rpc-auth.auth-null on
>>>   6: option transport.socket.listen-backlog 128
>>>   7: option ping-timeout 30
>>>   8: option transport.socket.read-fail-log off
>>>   9: option transport.socket.keepalive-interval 2
>>>  10: option transport.socket.keepalive-time 10
>>>  11: option transport-type socket
>>>  12: option working-directory /var/lib/glusterd
>>>  13: end-volume
>>>  14: 
>>> +--+
>>> [2015-03-19 13:42:02.258403] I 
>>> [glusterd-handler.c:1015:__glusterd_handle_cli_pr
>>> obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
>>> [2015-03-19 13:42:02.259456] I 
>>> [glusterd-handler.c:3165:glusterd_probe_begin] 0-
>>> glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
>>> [2015-03-19 13:42:02.259664] I [rpc-clnt.c:969:rpc_clnt_connection_init] 
>>> 0-manag
>>> ement: setting frame-timeout to 600
>>> [2015-03-19 13:42:02.260488] I 
>>> [glusterd-handler.c:3098:glusterd_friend_add] 0-m
>>> anagement: connect returned 0
>>> [2015-03-19 13:42:02.270316] I [glusterd.c:176:glusterd_uuid_generate_save] 
>>> 0-ma
>>> nagement: generated UUID: 4441e237-89d6-4cdf-a212-f17ecb953b58
>>> [2015-03-19 13:42:02.273427] I 
>>> [glusterd-rpc-ops.c:244:__glusterd_probe_cbk] 0-m
>>> anagement: Received probe resp from uuid: 
>>> 82cdb873-28cc-4ed0-8cfe-2b6275770429,
>>> host: 10.32.1.144
>>> [2015-03-19 13:42:02.273681] I 
>>> [glusterd-rpc-ops.c:386:__glusterd_probe_cbk] 0-g
>>> lusterd: Received resp to probe req
>>> [2015-03-19 13:42:02.278863] I 
>>> [glusterd-handshake.c:1119:__glusterd_mgmt_hndsk_
>>> versions_ack] 0-management: using the op-version 30600
>>> [2015-03-19 13:52:03.277422] E [rpc-clnt.c:201:call_bail] 0-management: 
>>> bailing
>>> out frame type(Peer mgmt) op(--(2)) xid = 0x6 sent = 2015-03-19 
>>> 13:42:02.273482.
>>>  timeout = 600 for 10.32.1.144:24007
>> Here is the issue, there was some problem in the network at the time
>> when peer probe was issued. This is why the call bail is seen. Could you
>> try to deprobe and then probe it back again?
>>> [2015-03-19 13:52:03.277453] I [socket.c:3366:socket_submit_reply] 
>>> 0-socket.mana
>>> gement: not connected (priv->connected = 255)
>>> [2015-03-19 13:52:03.277468] E [rpcsvc.c:1247:rpcsvc_submit_generic] 
>>> 0-rpc-servi
>>> ce: failed to submit message (XID: 0x1, Program: GlusterD svc cli, 
>>> ProgVers: 2,
>>> Proc: 1) to rpc-transport (socket.management)
>>> [2015-03-19 13:52:03.277483] E [glusterd-utils.c:387:glusterd_submit_reply] 
>>> 0-:
>>> Reply submission failed
>>>
>>>
>>>
>>> Logs from 10.32.1.144:
>>> -
>>> # more ./.cmd_log_history
>>>
>>> # more ./etc-glusterfs-glusterd.vol.lo

[Gluster-users] What should I do to improve performance ?

[marianna cattani]

Dear all,
I followed the tutorial I read at this link :
http://www.gluster.org/documentation/use_cases/Virt-store-usecase/

I have 4 nodes configured as a linked list , each node also performs
virtual machines with KVM and mounts on its ip address, like this:

172.16.155.12:/nova /var/lib/nova/instances glusterfs defaults,_netdev 0 0

Each node has two nic (ten giga) bonded in mode 4.

What can I do to further improve the speed ?
BR.

M.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster peer probe error (v3.6.2)

[Andreas Hollaus]

2Hi,

This network problem is persistent. However, I can ping the server so guess it
depends on the port no, right?
I tried to telnet to port 24007, but I was not sure how to interpret the result 
as I
got no respons and no timeout (it just seemed to be waiting for something).
That's why I decided to install nmap, but according to that tool the port was
accessible. Are there any other ports that are vital to gluster peer probe?

When you say 'deprobe', I guess you mean 'gluster peer detach'? That command 
shows
similar behaviour to gluster peer probe.


Regards
Andreas

On 03/23/15 05:34, Atin Mukherjee wrote:
>
> On 03/22/2015 07:11 PM, Andreas Hollaus wrote:
>> Hi,
>>
>> I hope that these are the logs that you requested.
>>
>> Logs from 10.32.0.48:
>> --
>> # more /var/log/glusterfs/.cmd_log_history
>> [2015-03-19 13:52:03.277438]  : peer probe 10.32.1.144 : FAILED : Probe 
>> returned
>>  with unknown errno -1
>>
>> # more /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
>> [2015-03-19 13:41:31.241768] I [MSGID: 100030] [glusterfsd.c:2018:main] 
>> 0-/usr/s
>> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
>> /usr/sbin/
>> glusterd -p /var/run/glusterd.pid)
>> [2015-03-19 13:41:31.245352] I [glusterd.c:1214:init] 0-management: Maximum 
>> allo
>> wed open file descriptors set to 65536
>> [2015-03-19 13:41:31.245432] I [glusterd.c:1259:init] 0-management: Using 
>> /var/l
>> ib/glusterd as working directory
>> [2015-03-19 13:41:31.247826] I 
>> [glusterd-store.c:2063:glusterd_restore_op_versio
>> n] 0-management: Detected new install. Setting op-version to maximum : 30600
>> [2015-03-19 13:41:31.247902] I 
>> [glusterd-store.c:3497:glusterd_store_retrieve_mi
>> ssed_snaps_list] 0-management: No missed snaps list.
>> Final graph:
>> +--+
>>   1: volume management
>>   2: type mgmt/glusterd
>>   3: option rpc-auth.auth-glusterfs on
>>   4: option rpc-auth.auth-unix on
>>   5: option rpc-auth.auth-null on
>>   6: option transport.socket.listen-backlog 128
>>   7: option ping-timeout 30
>>   8: option transport.socket.read-fail-log off
>>   9: option transport.socket.keepalive-interval 2
>>  10: option transport.socket.keepalive-time 10
>>  11: option transport-type socket
>>  12: option working-directory /var/lib/glusterd
>>  13: end-volume
>>  14: 
>> +--+
>> [2015-03-19 13:42:02.258403] I 
>> [glusterd-handler.c:1015:__glusterd_handle_cli_pr
>> obe] 0-glusterd: Received CLI probe req 10.32.1.144 24007
>> [2015-03-19 13:42:02.259456] I 
>> [glusterd-handler.c:3165:glusterd_probe_begin] 0-
>> glusterd: Unable to find peerinfo for host: 10.32.1.144 (24007)
>> [2015-03-19 13:42:02.259664] I [rpc-clnt.c:969:rpc_clnt_connection_init] 
>> 0-manag
>> ement: setting frame-timeout to 600
>> [2015-03-19 13:42:02.260488] I [glusterd-handler.c:3098:glusterd_friend_add] 
>> 0-m
>> anagement: connect returned 0
>> [2015-03-19 13:42:02.270316] I [glusterd.c:176:glusterd_uuid_generate_save] 
>> 0-ma
>> nagement: generated UUID: 4441e237-89d6-4cdf-a212-f17ecb953b58
>> [2015-03-19 13:42:02.273427] I [glusterd-rpc-ops.c:244:__glusterd_probe_cbk] 
>> 0-m
>> anagement: Received probe resp from uuid: 
>> 82cdb873-28cc-4ed0-8cfe-2b6275770429,
>> host: 10.32.1.144
>> [2015-03-19 13:42:02.273681] I [glusterd-rpc-ops.c:386:__glusterd_probe_cbk] 
>> 0-g
>> lusterd: Received resp to probe req
>> [2015-03-19 13:42:02.278863] I 
>> [glusterd-handshake.c:1119:__glusterd_mgmt_hndsk_
>> versions_ack] 0-management: using the op-version 30600
>> [2015-03-19 13:52:03.277422] E [rpc-clnt.c:201:call_bail] 0-management: 
>> bailing
>> out frame type(Peer mgmt) op(--(2)) xid = 0x6 sent = 2015-03-19 
>> 13:42:02.273482.
>>  timeout = 600 for 10.32.1.144:24007
> Here is the issue, there was some problem in the network at the time
> when peer probe was issued. This is why the call bail is seen. Could you
> try to deprobe and then probe it back again?
>> [2015-03-19 13:52:03.277453] I [socket.c:3366:socket_submit_reply] 
>> 0-socket.mana
>> gement: not connected (priv->connected = 255)
>> [2015-03-19 13:52:03.277468] E [rpcsvc.c:1247:rpcsvc_submit_generic] 
>> 0-rpc-servi
>> ce: failed to submit message (XID: 0x1, Program: GlusterD svc cli, ProgVers: 
>> 2,
>> Proc: 1) to rpc-transport (socket.management)
>> [2015-03-19 13:52:03.277483] E [glusterd-utils.c:387:glusterd_submit_reply] 
>> 0-:
>> Reply submission failed
>>
>>
>>
>> Logs from 10.32.1.144:
>> -
>> # more ./.cmd_log_history
>>
>> # more ./etc-glusterfs-glusterd.vol.log
>> [1970-01-01 00:00:53.225739] I [MSGID: 100030] [glusterfsd.c:2018:main] 
>> 0-/usr/s
>> bin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: 
>> /usr/sbin/
>> glusterd -p /var/run/glusterd.pid)
>> [1970-01-01 00:00:53.229222] I 

Re: [Gluster-users] Fwd: Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and checkpatch warnings fixed

[Niels de Vos]

On Mon, Mar 23, 2015 at 12:49:56PM +0530, Anand Subramanian wrote:
> FYI.
> 
> GlusterFS vols can now be accessed via NFSv4.1 pNFS protocol (mount -t nfs
> -o minorversion=1 ...) from nfs-ganesha 2.2-rc5 onwards.
> 
> Note: one fix is to go into libgfapi to fix up using anonymous fd's in
> ds_write/make_ds_handle() (Avati's sugeestion that really helps here).
> Once Jiffin or myself get that fix in, a good large file performance can be
> seen with pNFS vs V4.

I could not find the needed change for libgfapi. Could you post the link
to the review?

Thanks,
Niels

> 
> All thanks and credit to Jiffin for his terrific effort in coding things up
> quickly and for fixing bugs.
> 
> Anand
> 
> 
>  Forwarded Message 
> Subject:  Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and
> checkpatch warnings fixed
> Date: Sat, 21 Mar 2015 01:04:30 +0100
> From: GerritHub 
> Reply-To: ffilz...@mindspring.com
> To:   Anand Subramanian 
> CC:   onnfrhvruutnzhnaq.-g...@noclue.notk.org
> 
> 
> 
> From Frank Filz :
> 
> Frank Filz has submitted this change and it was merged.
> 
> Change subject: pNFS code drop enablement and checkpatch warnings fixed
> ..
> 
> 
> pNFS code drop enablement and checkpatch warnings fixed
> 
> Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> Signed-off-by: Anand Subramanian 
> ---
> A src/FSAL/FSAL_GLUSTER/ds.c
> M src/FSAL/FSAL_GLUSTER/export.c
> M src/FSAL/FSAL_GLUSTER/gluster_internal.h
> M src/FSAL/FSAL_GLUSTER/handle.c
> M src/FSAL/FSAL_GLUSTER/main.c
> A src/FSAL/FSAL_GLUSTER/mds.c
> 6 files changed, 993 insertions(+), 0 deletions(-)
> 
> 
> 
> -- 
> To view, visit https://review.gerrithub.io/221683
> To unsubscribe, visit https://review.gerrithub.io/settings
> 
> Gerrit-MessageType: merged
> Gerrit-Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
> Gerrit-PatchSet: 1
> Gerrit-Project: ffilz/nfs-ganesha
> Gerrit-Branch: next
> Gerrit-Owner: Anand Subramanian 
> Gerrit-Reviewer: Frank Filz 
> Gerrit-Reviewer: onnfrhvruutnzhnaq.-g...@noclue.notk.org
> 
> 
> 

> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume creation time?

[Niels de Vos]

On Sun, Mar 22, 2015 at 06:21:52PM +, Jason Hilton wrote:
> So I have been trying to get this going and the volume status command
> shows that the "NFS Server on localhost" is offline, port and pid are
> "N/A".  I have double checked everything and NFS is disabled on start
> up an are not currently running.  The showmount command returns
> "clnt_create: RPC: Program not registered"

Please make sure that none of the RPC programs for NFS are registered at
rpcbind before starting Gluster/NFS (or rebooting).

You can check by executing 'rpcinfo' on the Gluster/NFS server. Any of
the mountd, nlockmgr, status and nfs can block the registration of the
NFS-server (and its helper protocols). Unregistering one of the
protocols can be done like this:

# rpcinfo
...
153tcp   0.0.0.0.150.65 mountd superuser
...

# rpcinfo -d 15 3

Do this for all the protocols (+versions) mentioned above and restart
the Gluster/NFS server process:

# gluster volume start $VOLUME force

The 'start force' will only start any missing processes, like the
Gluster/NFS (a 'glusterfs' process). Running processes (like the ones
for the bricks) should not be impacted.

Niels

> 
> Any ideas?
> Thanks again!
>  
> -Original Message-
> From: Niels de Vos [mailto:nde...@redhat.com] 
> Sent: Sunday, March 22, 2015 11:45 AM
> To: Jason Hilton
> Cc: 'gluster-users@gluster.org'
> Subject: Re: [Gluster-users] Volume creation time?
> 
> On Sun, Mar 22, 2015 at 02:59:53PM +, Jason Hilton wrote:
> > Thank you for the quick reply!  I didn't expect to see any response on 
> > a Sunday.  I did as you suggested and found  some messages stating 
> > that the address and port were failing to bind because it was already 
> > in use.  It turned out that the NFS service was running and interfered 
> > with glusterd.  I was intending to share my gluster volumes via NFS 
> > and I thought I had read that as of V3, gluster exported NFS shares by 
> > default, so I had started the service.  Does gluster provide its own 
> > NFS services?
> 
> Yes, Gluster comes indeed with its own NFS-server. You should not start any 
> NFS-services, Gluster takes care of starting them. The only service that you 
> need to have running (or activated for systemd environments), is rpcbind.
> 
> Once your volume has been created and started, you should be able to see that 
> there is a NFS-server running with this command:
> 
> # gluster volume status
> 
> And, with 'showmount -e' the volme should be listed as an export.
> 
> Cheers,
> Niels
> 
> > 
> > ***
> > Jason Hilton
> > Director of Technology Development
> > 601 Madison Street, Suite 400
> > Alexandria, VA 22314
> > jason.hil...@aaae.org
> > Desk: 703.824.0500x167
> > FAX: 703.578.4952
> > 
> > AAAE Tech support:
> > i...@aaae.org
> > 703.797.2555, opt. 2
> > ***
> > 
> > 
> > -Original Message-
> > From: Niels de Vos [mailto:nde...@redhat.com]
> > Sent: Sunday, March 22, 2015 10:13 AM
> > To: Jason Hilton
> > Cc: 'gluster-users@gluster.org'
> > Subject: Re: [Gluster-users] Volume creation time?
> > 
> > On Sun, Mar 22, 2015 at 01:34:24PM +, Jason Hilton wrote:
> > > Hi-
> > > I'm new to GlusterFS and I have been trying to set up a gluster 
> > > volume.  The volume is 150 TB.  I started the create volume command 
> > > on Friday morning and it has not yet completed.  Since I have no 
> > > prior experience with GlusterFS, is this an expected duration?  The 
> > > server is no power house, a pair of older Xeon Quad core processors 
> > > at 2 GHz and only 4 GB of RAM.  TOP shows very little processor 
> > > usage, but IOTOP shows some disk I/O.  I don't mind waiting it out, 
> > > I just want to be sure that the process is still proceeding.  Is 
> > > there a way to monitor Gluster volume creation progress?
> > 
> > Volume creation should be very fast, there is not a lot to do for Gluster 
> > to create a volume. A couple of seconds should be sufficient.
> > 
> > Check the /var/log/glusterfs/etc-*.log to see if there are any errors 
> > listed there.
> > 
> > HTH,
> > Niels
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] I/O error on replicated volume

[Mohammed Rafi K C]

On 03/23/2015 11:28 AM, Jonathan Heese wrote:
> On Mar 23, 2015, at 1:20 AM, "Mohammed Rafi K C"  > wrote:
>
>>
>> On 03/21/2015 07:49 PM, Jonathan Heese wrote:
>>>
>>> Mohamed,
>>>
>>>
>>> I have completed the steps you suggested (unmount all, stop the
>>> volume, set the config.transport to tcp, start the volume, mount,
>>> etc.), and the behavior has indeed changed.
>>>
>>>
>>> [root@duke ~]# gluster volume info
>>>
>>> Volume Name: gluster_disk
>>> Type: Replicate
>>> Volume ID: 2307a5a8-641e-44f4-8eaf-7cc2b704aafd
>>> Status: Started
>>> Number of Bricks: 1 x 2 = 2
>>> Transport-type: tcp
>>> Bricks:
>>> Brick1: duke-ib:/bricks/brick1
>>> Brick2: duchess-ib:/bricks/brick1
>>> Options Reconfigured:
>>> config.transport: tcp
>>>
>>>
>>> [root@duke ~]# gluster volume status
>>> Status of volume: gluster_disk
>>> Gluster process Port   
>>> Online  Pid
>>> --
>>> Brick duke-ib:/bricks/brick149152  
>>> Y   16362
>>> Brick duchess-ib:/bricks/brick1 49152  
>>> Y   14155
>>> NFS Server on localhost 2049   
>>> Y   16374
>>> Self-heal Daemon on localhost   N/A
>>> Y   16381
>>> NFS Server on duchess-ib2049   
>>> Y   14167
>>> Self-heal Daemon on duchess-ib  N/A
>>> Y   14174
>>>
>>> Task Status of Volume gluster_disk
>>> --
>>> There are no active volume tasks
>>>
>>> I am no longer seeing the I/O errors during prolonged periods of
>>> write I/O that I was seeing when the transport was set to rdma.
>>> However, I am seeing this message on both nodes every 3 seconds
>>> (almost exactly):
>>>
>>>
>>> ==> /var/log/glusterfs/nfs.log <==
>>> [2015-03-21 14:17:40.379719] W
>>> [rdma.c:1076:gf_rdma_cm_event_handler] 0-gluster_disk-client-1: cma
>>> event RDMA_CM_EVENT_REJECTED, error 8 (me:10.10.10.1:1023
>>> peer:10.10.10.2:49152)
>>>
>>>
>>> Is this something to worry about?
>>>
>> If you are not using nfs to export the volumes, there is nothing to
>> worry.
>
> I'm using the native glusterfs FUSE component to mount the volume
> locally on both servers -- I assume that you're referring to the
> standard NFS protocol stuff, which I'm not using here.
>
> Incidentally, I would like to keep my logs from filling up with junk
> if possible.  Is there something I can do to get rid of these
> (useless?) error messages?

If i understand correctly, you are getting this enormous log message
from nfs log only, all other logs and everything are fine now, right ?
If that is the case, and you are not at all using nfs for exporting the
volume, as  a workaround you can disable nfs for your volume or cluster.
(gluster v set nfs.disable on). This will turnoff your gluster nfs
server, and you will no longer get those log messages.


>>> Any idea why there are rdma pieces in play when I've set my
>>> transport to tcp?
>>>
>>
>> there should not be any piece of rdma,if possible, can you paste the
>> volfile for nfs server. You can find the volfile in
>> /var/lib/glusterd/nfs/nfs-server.vol or
>> /usr/local/var/lib/glusterd/nfs/nfs-server.vol
>
> I will get this for you when I can.  Thanks.

If you can make it, that will be great help to understand the problem.


Rafi KC

>
> Regards,
> Jon Heese
>
>> Rafi KC
>>>
>>> The actual I/O appears to be handled properly and I've seen no
>>> further errors in the testing I've done so far.
>>>
>>>
>>> Thanks.
>>>
>>>
>>> Regards,
>>>
>>> Jon Heese
>>>
>>>
>>> 
>>> *From:* gluster-users-boun...@gluster.org
>>>  on behalf of Jonathan Heese
>>> 
>>> *Sent:* Friday, March 20, 2015 7:04 AM
>>> *To:* Mohammed Rafi K C
>>> *Cc:* gluster-users
>>> *Subject:* Re: [Gluster-users] I/O error on replicated volume
>>>  
>>> Mohammed,
>>>
>>> Thanks very much for the reply.  I will try that and report back.
>>>
>>> Regards,
>>> Jon Heese
>>>
>>> On Mar 20, 2015, at 3:26 AM, "Mohammed Rafi K C"
>>> mailto:rkavu...@redhat.com>> wrote:
>>>

 On 03/19/2015 10:16 PM, Jonathan Heese wrote:
>
> Hello all,
>
>  
>
> Does anyone else have any further suggestions for troubleshooting
> this?
>
>  
>
> To sum up: I have a 2 node 2 brick replicated volume, which holds
> a handful of iSCSI image files which are mounted and served up by
> tgtd (CentOS 6) to a handful of devices on a dedicated iSCSI
> network.  The most important iSCSI clients (initiators) are four
> VMware ESXi 5.5 hosts that use the iSCSI volumes as backing for
> their datastores for virtual machine storage.
>
>  
>
> After a few minutes of sustained writing to the volume, I am
> seeing

[Gluster-users] Fwd: Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement and checkpatch warnings fixed

[Anand Subramanian]

FYI.

GlusterFS vols can now be accessed via NFSv4.1 pNFS protocol (mount -t 
nfs -o minorversion=1 ...) from nfs-ganesha 2.2-rc5 onwards.


Note: one fix is to go into libgfapi to fix up using anonymous fd's in 
ds_write/make_ds_handle() (Avati's sugeestion that really helps here).
Once Jiffin or myself get that fix in, a good large file performance can 
be seen with pNFS vs V4.


All thanks and credit to Jiffin for his terrific effort in coding things 
up quickly and for fixing bugs.


Anand


 Forwarded Message 
Subject: 	Change in ffilz/nfs-ganesha[next]: pNFS code drop enablement 
and checkpatch warnings fixed

Date:   Sat, 21 Mar 2015 01:04:30 +0100
From:   GerritHub 
Reply-To:   ffilz...@mindspring.com
To: Anand Subramanian 
CC: onnfrhvruutnzhnaq.-g...@noclue.notk.org



From Frank Filz :

Frank Filz has submitted this change and it was merged.

Change subject: pNFS code drop enablement and checkpatch warnings fixed
..


pNFS code drop enablement and checkpatch warnings fixed

Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
Signed-off-by: Anand Subramanian 
---
A src/FSAL/FSAL_GLUSTER/ds.c
M src/FSAL/FSAL_GLUSTER/export.c
M src/FSAL/FSAL_GLUSTER/gluster_internal.h
M src/FSAL/FSAL_GLUSTER/handle.c
M src/FSAL/FSAL_GLUSTER/main.c
A src/FSAL/FSAL_GLUSTER/mds.c
6 files changed, 993 insertions(+), 0 deletions(-)



--
To view, visit https://review.gerrithub.io/221683
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8c58dd6d6326f692681f76b96f29c630db21a92
Gerrit-PatchSet: 1
Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-Owner: Anand Subramanian 
Gerrit-Reviewer: Frank Filz 
Gerrit-Reviewer: onnfrhvruutnzhnaq.-g...@noclue.notk.org



___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Revamping the GlusterFS Documentation...

[Shravan Chandrashekar]

Hi All, 

"The Gluster Filesystem documentation is not user friendly and fragmented" and 
this has been the feedback we have been receiving. 

We got back to our drawing board and blueprints and realized that the content 
was scattered at various places. These include: 

[Static HTML] http://www.gluster.org/documentation/ 
[Mediawiki] http://www.gluster.org/community/documentation/ 
[In-source] https://github.com/gluster/glusterfs/tree/master/doc 
[Markdown] https://github.com/GlusterFS/Notes 

and so on… 

Hence, we started by curating content from various sources including 
gluster.org static HTML documentation, glusterfs github repository, 
various blog posts and the Community wiki. We also felt the need to improve the 
community member's experience with Gluster documentation. This led us to put 
some thought into the user interface. As a result we came up with a page which 
links all content into a single landing page: 

http://www.gluster.org/community/documentation/index.php/Staged_Docs 

This is just our first step to improve our community docs and enhance the 
community contribution towards documentation. I would like to thank Humble 
Chirammal and Anjana Sriram for the suggestions and directions during the 
entire process. I am sure there is lot of scope for improvement. 
Hence, request you all to review the content and provide your suggestions. 

Regards, 
Shravan Chandra 

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users