Hi, I'm no expert on this - more like a somewhat informed amateur - but your configuration sounds perhaps more suitable for a grid architecture using each location as a node in the grid. Grids often communicate via WANs like you have, and are tolerant of the slow bandwidth and possible loss of connections, which is not a strength of any parallel file systems that I know of. The disadvantage of the grid architecture is that it is not real-time. I guess your solution will depend on what latency (measure in seconds to minutes) you can accommodate in syncing the data.
More info and leads can be found here: http://www.isgtw.org/?pid=1002049 James -----Original Message----- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Count Zero Sent: Friday, April 09, 2010 12:56 PM To: Tejas N. Bhise Subject: [Gluster-users] WAN Challenge Hi Guys, I've sent this once but I did not even get it myself from the mailing list so i'm not sure it was even received correctly so I am re-posting. My apologies if this is a re-post. I have an interesting situation, and I'm wondering if there's a solution for it in the glusterfs realm or if I will have to resort to other solutions that complement glusterfs (such as rsync or unison). I have 9 servers in 3 locations on the internet (3 servers per location). Unfortunately, the network distance between them is such that setting up a Distribute or NUFA cluster between them all is difficult (I'm not saying impossible, because it may be possible and I just don't know how to pull it off). There are 3 servers in each data center, and they are all clustered via NUFA: DC-A -+ NUFA-Cluster ---+ SRV-A1 ---+ SRV-A2 ---+ SRV-A3 DC-B ( >> rsync from A) -+ NUFA-Cluster ---+ SRV-B1 ---+ SRV-B2 ---+ SRV-B3 DC-C ( >> rsync from B) -+ NUFA-Cluster ---+ SRV-C1 ---+ SRV-C2 ---+ SRV-C3 The reason I did it like this, so far: 1) I needed file reads to be fast on each local node, so I have the "option local-volume-name `hostname`" trick in my glusterfs.vol file (like in the cookbook). 2) Bandwidth between DC-A and DC-B and DC-C is kinda low... and since glusterfs waits for the last server to finish, this severely slows down the entire cluster for any operation, including just listing the files in a directory. Is there a better way to implement this? All the examples I find are about 4 node replication, etc. What about inter-continent replication of data between NUFA Clusters? Any advice would be greatly appreciated :-) At the moment, out of lack of options, I plan to sync between the 3 NUFA clusters with "INOSYNC". Thanks, Count Zero P.S. Below is my configuration file, from /etc/glusterfs/glusterfs.vol: ---------------------8<--------------------8<------------------ volume posix type storage/posix option directory /data/export end-volume volume locks type features/locks subvolumes posix end-volume volume brick type performance/io-threads subvolumes locks end-volume volume server type protocol/server option transport-type tcp option auth.addr.brick.allow * subvolumes brick end-volume volume srv-a1 type protocol/client option transport-type tcp option remote-host srv-a1 option remote-subvolume brick end-volume volume srv-a2 type protocol/client option transport-type tcp option remote-host srv-a2 option remote-subvolume brick end-volume volume srv-a3 type protocol/client option transport-type tcp option remote-host srv-a3 option remote-subvolume brick end-volume volume nufa type cluster/nufa option local-volume-name `hostname` subvolumes srv-a1 srv-a2 srv-a3 end-volume volume writebehind type performance/write-behind option cache-size 1MB subvolumes nufa end-volume volume cache type performance/io-cache option cache-size 512MB subvolumes writebehind end-volume --------------------->8-------------------->8------------------ _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
