Re: [Gluster-users] Configure separate network for inter-node communication
Thanks Jeff for this blog post, looking forward to NSR and its chain replication! On Monday, March 9, 2015 1:00 PM, Jeff Darcy wrote: > I would be very interested to read your blog post as soon as its out and I > guess many others too. Please do post the link to this list as soon as its > online. Sorry, forgot to do this earlier. It's here: http://pl.atyp.us/2015-03-life-on-the-server-side.html ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Configure separate network for inter-node communication
On 9 Mar 2015, at 12:00, Jeff Darcy wrote: >> I would be very interested to read your blog post as soon as its out and I >> guess many others too. Please do post the link to this list as soon as its >> online. > > Sorry, forgot to do this earlier. It's here: > > http://pl.atyp.us/2015-03-life-on-the-server-side.html Awesome. Might be worth taking advantage of the newly offered "unlimited" Rackspace-account-for-GlusterFS-tuning-guide to try this out with some large node numbers? (eg 100-200 maybe? :>) + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Configure separate network for inter-node communication
> I would be very interested to read your blog post as soon as its out and I > guess many others too. Please do post the link to this list as soon as its > online. Sorry, forgot to do this earlier. It's here: http://pl.atyp.us/2015-03-life-on-the-server-side.html ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Configure separate network for inter-node communication
Thank you for the detailed explanation. Due to the fact that right now it does not make much difference to split the traffic I will refrain from doing that and simply wait for the new style replication. This looks like a very promising feature and I am looking forward to it. My other concern here would be that I need to implement a GlusterFS cluster right now with 3.5 so I hope it will possible to upgrade without much pain from AFR to NSR. For now, and if I understand everything correclty, it means that if I want better performance I will need to have all my gluster nodes as well as clients on a 10 Gigabit network and not just the gluster servers themselves, right? My main issue here is that I am serving a lot of small web files from my gluster servers using fuse on the clients so it's not really the amount of data but the latency which matters. I would be very interested to read your blog post as soon as its out and I guess many others too. Please do post the link to this list as soon as its online. On Thursday, March 5, 2015 2:23 PM, Jeff Darcy wrote: > I have two gluster nodes in a replicated setup and have connected the two > nodes together directly through a 10 Gbit/s crossover cable. Now I would > like to tell gluster to use this seperate private network for any > communications between the two nodes. Does that make sense? Will this bring > me any performance gain? and if yes how do I configure that? It is possible, but it's not likely to improve performance much (yet). The easiest way to do this is to use a custom /etc/hosts on the servers, so that *on a server* every other server's name resolves to its private back-end address. Meanwhile, clients resolve that same name to the server's front-end address. You can get a similar effect with explicit host routes or iptables rules on the servers. The reason this won't have much effect on performance is that the servers do not (currently) replicate to one another. Instead, clients send data directly to every replica themselves. The only time time a private network would see much traffic would be when the "clients" are actually the servers performing administrative operations - self heal, rebalance, and so on. In 4.0, both parts of this answer would be different. First, we expect to have better handling of multiple networks and multi-homed hosts, including user specification of which networks to use for which traffic[1]. Second, 4.0 will have a new form of replication which *does* replicate directly between servers[2]. Parts of this second feature are in fact likely to appear well before the rest of 4.0, using the server-to-server data flow but retaining our current methods of tracking changes and re-syncing servers after a failure. In fact I'm writing a blog post right now about this, including some performance measurements. I'll respond again here when it's done. [1] http://www.gluster.org/community/documentation/index.php/Features/SplitNetwork [2] http://www.gluster.org/community/documentation/index.php/Features/new-style-replication ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Configure separate network for inter-node communication
> I have two gluster nodes in a replicated setup and have connected the two > nodes together directly through a 10 Gbit/s crossover cable. Now I would > like to tell gluster to use this seperate private network for any > communications between the two nodes. Does that make sense? Will this bring > me any performance gain? and if yes how do I configure that? It is possible, but it's not likely to improve performance much (yet). The easiest way to do this is to use a custom /etc/hosts on the servers, so that *on a server* every other server's name resolves to its private back-end address. Meanwhile, clients resolve that same name to the server's front-end address. You can get a similar effect with explicit host routes or iptables rules on the servers. The reason this won't have much effect on performance is that the servers do not (currently) replicate to one another. Instead, clients send data directly to every replica themselves. The only time time a private network would see much traffic would be when the "clients" are actually the servers performing administrative operations - self heal, rebalance, and so on. In 4.0, both parts of this answer would be different. First, we expect to have better handling of multiple networks and multi-homed hosts, including user specification of which networks to use for which traffic[1]. Second, 4.0 will have a new form of replication which *does* replicate directly between servers[2]. Parts of this second feature are in fact likely to appear well before the rest of 4.0, using the server-to-server data flow but retaining our current methods of tracking changes and re-syncing servers after a failure. In fact I'm writing a blog post right now about this, including some performance measurements. I'll respond again here when it's done. [1] http://www.gluster.org/community/documentation/index.php/Features/SplitNetwork [2] http://www.gluster.org/community/documentation/index.php/Features/new-style-replication ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Configure separate network for inter-node communication
On 03/04/2015 11:24 PM, ML mail wrote: > Hello, > > I have two gluster nodes in a replicated setup and have connected the two > nodes together directly through a 10 Gbit/s crossover cable. Now I would like > to tell gluster to use this seperate private network for any communications > between the two nodes. Does that make sense? Will this bring me any > performance gain? and if yes how do I configure that? > > My clients are intended to access my gluster volumes using FUSE through the > main 1 Gbit/s interface. > > Regards > ML > Makes a lot of sense, just give the 10gbe nic's a different ip subnet and put these in the /etc/hosts file or dns. Use these hostname.storage names to setup the volume. There should be some way to convert an existing volume but I don't have the details at the moment. Read this for some info: http://www.ovirt.org/Change_network_interface_for_Gluster Met vriendelijke groet, With kind regards, Jorick Astrego Netbulae Virtualization Experts Tel: 053 20 30 270 i...@netbulae.euStaalsteden 4-3A KvK 08198180 Fax: 053 20 30 271 www.netbulae.eu 7547 TA Enschede BTW NL821234584B01 ___ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users
