Udo - We are tracking these sort of client side configuration issues as bug #2014 , however in this particular case the hosts file/DNS workaround appears to be working well at several sites, is there any reason that process won't work for you?
Thanks, Craig --> Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "Udo Waechter" <udo.waech...@uni-osnabrueck.de> To: gluster-users@gluster.org Cc: "Craig Carl" <cr...@gluster.com> Sent: Tuesday, November 9, 2010 11:33:56 PM Subject: Re: [Gluster-users] 3.1: Multiple networks and Client access Hi, any news on this? Thanks for the effort, udo. On 01.11.2010, at 15:02, Udo Waechter wrote: > Hi and thanks for the answer. > I think there is a bug/problem with gluster. > First some pre-knowledge > > - our internal network does not resolve vie DNS. > - the internal hosts are only resolvable via /etc/hosts. > > Now, my idea would be this: > > 1. on the cluster nodes, use the internal (bonding) interfaces for > communication > 2. the rest of the network should use the external interfaces to communicate > with the storage cloud. > > To achieve this, the idea was now to: > > a) create the volume with the internal names > $ gluster volume create ... hostname1.internal:/exp hostname2.internal:/exp > hostname3.internal:/exp > > b) On all the (external) nodes, that should reach the gluster-cluster, add > hostnam1...X.internal to /etc/hosts resolving to their external ip-address. > > > Now to all the problems: > > Regarding point 1 above: > - when I add a peer with its internal name it works, except for the peer that > I add the other peers from. > ---hostname2.internal: $ gluster peer probe hostname1.internal > ---hostname1.internal $ gluster peer status > Number of Peers: 1 > > Hostname: 10.10.33.142 > Uuid: a19fc9d3-d00f-4440-b096-c974db1cd8c7 > State: Peer in Cluster (Connected) > > This should be hostname2.internal > > When I do gluster peer probe hostname1.internal (on the host > hostname1.internal) I get: > > "hostname1.internal is already part of another cluster" > so here, ip/name resolution works... > > this works in all permutations. The peer from which I do "gluster peer probe > ..." always does not resolve to its internal name, but its ip-adress > > As a result from all this, point a) can not succeed, since: > > gluster volume create .... hostname... hostname... hostname... results in: > > "Host hostnameX is not a friend", where hostnameX ist the host where the > volume creation was attempted. > > > I have tried and installed pdnsd for the internal network, but this does not > solve the problems either. > > As a last resort, I edited /etc/glusterd/peers/* and replaced the ip-adresses > by hand. Now, "gluster peer status" gives me the names instead of the > ip-adress. > but "volume create" still tells me about the host (where I create the volume > from) not being a friend. > > Any help, solution is highly apreciated. > Thanks, > udo. > > On 18.10.2010, at 07:10, Craig Carl wrote: > >> Udo - >> With 3.1 when you mount/create/change a volume those changes are propagated >> via RPC to all of the other Gluster servers and clients. When you created >> the volume using 10.10.x.x IP addresses those IPs where what got sent to the >> client. In previous versions you could have just edited the client side >> configuration file and change or added the 192. addresses but not in this >> version, due to DVM. There should be a way to make multiple networks work so >> I will file a bug. >> In the meantime I think I have a workaround. If you use names instead of IP >> addresses and then make sure DNS or host files are setup properly is should >> work as Gluster does export via all interfaces. For example if the servers >> have these IPs - >> >> server1 - 10.10.1.1, 192.168.1.1 >> server2 - 10.10.1.2, 192.168.1.2 >> server3 - 10.10.1.3, 192.168.1.3 >> >> #gluster volume create test-ext stripe 3 server1:/ext server2:/ext >> server3:/ext >> >> You would just need to make sure that hosts on the 10.10.x.x resolve the >> servername to its 10. IP, and clients on the 192.x resolve to the 192 IP. >> Should be a simple change to the /etc/host files. >> >> Please let me know if this works so I can include that information in my bug >> report. >> >> Thanks, >> >> Craig >> >> -- >> Craig Carl >> Senior Systems Engineer; Gluster, Inc. >> Cell - (408) 829-9953 (California, USA) >> Office - (408) 770-1884 >> Gtalk - craig.c...@gmail.com >> Twitter - @gluster >> Installing Gluster Storage Platform, the movie! >> http://rackerhacker.com/2010/08/11/one-month-with-glusterfs-in-production/ >> >> >> From: "Udo Waechter" <udo.waech...@uni-osnabrueck.de> >> To: gluster-users@gluster.org >> Sent: Sunday, October 17, 2010 12:57:18 AM >> Subject: Re: [Gluster-users] 3.1: Multiple networks and Client access >> >> Hi, >> although I totally forgot about the firewall, this problem is not related to >> it. >> >> The ports you mentioned are open. >> >> I have created another volume using the ip-adresses from the external >> interfaces >> >> $ gluster volume create test-ext stripe 3 192.168.x.x1:/ext >> 192.168.x.x2:/ext 192.168.x.x3:/ext >> >> and this can not only be mounted, it also works perfectly. >> >> How could this work >> $ gluster volume create test-ext stripe 3 10.10.x.x1:/ext 10.10.x.x2:/ext >> 10.10.x.x3:/ext >> if the ip-addresses of the 10.10.x.x network are not reachable from the >> 192.168.x.x network? >> >> I have read somewhere in the docs that by default, glusterd listens on all >> interfaces, but does it also by default export everything to all interfaces? >> >> Thanks again, >> udo. >> >> >> On 16.10.2010, at 16:37, Jacob Shucart wrote: >> >>> Udo, >>> >>> It sounds to me like a firewall is blocking access to the Gluster system >>> preventing some of the traffic from happening. Please see: >>> >>> http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Installing_GlusterFS_on_Red_Hat_Package_Manager_%28RPM%29_Distributions >>> >>> >>> This lists the ports that need to be open both between Gluster storage >>> nodes and between the client. This also needs to be open on all storage >>> nodes to all clients. >>> >>> Please try this and let us know if this helps. >>> >>> -Jacob >>> >>> ----- Original Message ----- >>> From: "Udo Waechter" <udo.waech...@uni-osnabrueck.de> >>> To: gluster-users@gluster.org >>> Sent: Saturday, October 16, 2010 5:07:15 AM >>> Subject: [Gluster-users] 3.1: Multiple networks and Client access >>> >>> Hello all, >>> I just ran over Gluster 3.1 and got it up and running in notime. Thanks for >>> the great FS. >>> >>> I have a question regarding multiple network. >>> >>> We have all our servers connected via an non-routed internal switch (a >>> private subnet through which only the servers shall communicate with each >>> other), additionally the official subnet to enable their reachability from >>> the outside world. >>> >>> The internal IPnetwork is 10.10.x.x and the external shall be 192.168.x.x >>> (for the sake of the question). >>> >>> I have now set up 3 servers as a GlusterFS. With a Striped Volume over the >>> 10.10.x.x network. >>> All clients that are within this network can mount and use the volume. >>> >>> The problem arises with the clients from the other 192.168.x network. They >>> can mount the volume via: >>> >>> mount -t glusterfs 192.168.x.x:volname /mnt >>> >>> but they fail to use the mountpoint (ls and so) >>> >>> I have read another post (http://bit.ly/a3Tm6n) but the solution mentioned >>> there does not seem to work with version 3.1 any more. >>> The server(s) complain about "transport.socket..." not being allowed in the >>> volume file. >>> >>> The client from the 192.168.x.x network tries to contact the bricks via the >>> 10.10.x network and not via 192.168.x.x (which would work) >>> >>> Is there a solution to this problem? >>> >>> As I understand it, I would need to define the bricks with their >>> 192.168.x.x ip-adresses but this would mean that the internal network won't >>> be used. >>> >>> Can I define one brick with both ip-adresses (or 0.0.0.0). >>> Setting the "auth.allow" option to both subnets did not solve the problem >>> (except for the mount being possible after setting it) >>> >>> Any help is highly appreciated. >>> Thanks, >>> udo. >>> -- >>> ---[ Institute of Cognitive Science @ University of Osnabrueck >>> ---[ Albrechtstrasse 28, D-49076 Osnabrueck, 969-3362 >>> ---[ Documentation: https://doc.ikw.uni-osnabrueck.de >>> >>> >>> >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users@gluster.org >>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >>> >> >> -- >> :: udo waechter - r...@zoide.net :: N 52º16'30.5" E 8º3'10.1" >> :: genuine input for your ears: http://auriculabovinari.de >> :: your eyes: http://ezag.zoide.net >> :: your brain: http://zoide.net >> >> >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users@gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > -- > ---[ Institute of Cognitive Science @ University of Osnabrueck > ---[ Albrechtstrasse 28, D-49076 Osnabrueck, 969-3362 > ---[ Documentation: https://doc.ikw.uni-osnabrueck.de > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > -- :: udo waechter - r...@zoide.net :: N 52º16'30.5" E 8º3'10.1" :: genuine input for your ears: http://auriculabovinari.de :: your eyes: http://ezag.zoide.net :: your brain: http://zoide.net
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
