Re: Running Cassandra across different Amazon EC2 regions
vpn on ubuntu should be easy if you ask your good friend google... you should not have to pay for it (but paying might get you a fancy GUI, or perhaps very optimized performance that could squeeze a few more %) - Stephen --- Sent from my Android phone, so random spelling mistakes, random nonsense words and other nonsense are a direct result of using swype to type on the screen On 6 May 2011 23:25, Sameer Farooqui cassandral...@gmail.com wrote:
Re: Running Cassandra across different Amazon EC2 regions
Thanks, Shasha. It looks like there are two ways to make Cassandra work across regions, changing the source code or using software like Vyatta and VIPs. ++ Changing the source code ++ Milind documents how to change the source code in 0.7.4 but warns that there are significant limitations to this, which are not really discussed in his paper. I think some people ran into a problem running nodetool repair after this change, but it worked fine for others. Also, people worried about inter-node communication not being encrypted last month, but with the release of 0.8 this isn't a problem. Are there any other problems we can run into by changing the source code this way? How to change source code in 0.7.4: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/attachment/6191521/0/CassandraInstancesindifferentregions--EC2%5B1%5D.pdf?by-user=t +++ Configuring VIPs +++ Shasha documented a method of using Vyatta (vyatta.com) and VIPs to configure Cassandra across regions. This technique configures a VIP interface (tunXXX) for each node and sets the listen address as the VIP, so all inter-node communication occurs over the VPN tunnel. The RPC address is set to 0.0.0.0 so it listens on all interfaces and does not use the VPN tunnel. - This approach seems to be better, but does anyone have instructions for doing this in EC2 without Vyatta, which is paid software and costs around $700? We're using Ubuntu 11.04 on EC2 and would prefer to use an open source, free software vendor. - Sameer On Thu, May 5, 2011 at 10:48 PM, Sasha Dolgy sdo...@gmail.com wrote: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-IP-address-resolution-in-MultiDC-setup-EC2-VIP-td6306635.html On May 6, 2011 3:07 AM, Sameer Farooqui cassandral...@gmail.com wrote: We're trying to set up a Cassandra 0.8.0beta1 cluster across Amazon East West regions. It does not work out of the box with the binaries and the nodes in different regions end up setting this own clusters. The problem is with Cassandra's Listening Address as described by Rui: Using external IP causes Binding exception, because Cassandra, and java socket, cannot listen on the external IP, which does not belong to this node. Using private IP prevents nodes in other region from connecting back to this node with the private IP, which has been sent to other nodes for cluster establishing. In Feb 2011, Peter Fales posted: ... the trick is that you have must use the external addresses in your storage-conf.xml, but since you don't have a NIC that can actually bind to those addresses, you need to listen on the internal addresses (or, more simply, all interfaces). At the time, I was not able to get the cross-region cluster to work without making changes to the Cassandra code. Perhaps things have evolved so that there are other ways to do it now. Does anyone know if Cassandra 0.8 can be configured to run in multiple Amazon regions without modifying the source code? And if not, has anybody successfully done this with the 0.8 source code and shared the technique? After some research we found that other people have also ran into this issue with versions 0.6.x and 0.7.x of Cassandra: Fix for 0.6.5 by Peter Fales in Sept 2010: http://www.mail-archive.com/dev@cassandra.apache.org/msg01832.html http://www.mail-archive.com/user@cassandra.apache.org/msg05692.html Fix for 0.7 by Rui: http://rliu09.wordpress.com/2011/04/23/cassandra-across-ec2-regions/ - Sameer Accenture Technology Labs
Running Cassandra across different Amazon EC2 regions
We're trying to set up a Cassandra 0.8.0beta1 cluster across Amazon East West regions. It does not work out of the box with the binaries and the nodes in different regions end up setting this own clusters. The problem is with Cassandra's Listening Address as described by Rui: Using external IP causes Binding exception, because Cassandra, and java socket, cannot listen on the external IP, which does not belong to this node. Using private IP prevents nodes in other region from connecting back to this node with the private IP, which has been sent to other nodes for cluster establishing. In Feb 2011, Peter Fales posted: ... the trick is that you have must use the external addresses in your storage-conf.xml, but since you don't have a NIC that can actually bind to those addresses, you need to listen on the internal addresses (or, more simply, all interfaces). At the time, I was not able to get the cross-region cluster to work without making changes to the Cassandra code. Perhaps things have evolved so that there are other ways to do it now. Does anyone know if Cassandra 0.8 can be configured to run in multiple Amazon regions without modifying the source code? And if not, has anybody successfully done this with the 0.8 source code and shared the technique? After some research we found that other people have also ran into this issue with versions 0.6.x and 0.7.x of Cassandra: Fix for 0.6.5 by Peter Fales in Sept 2010: http://www.mail-archive.com/dev@cassandra.apache.org/msg01832.html http://www.mail-archive.com/user@cassandra.apache.org/msg05692.html Fix for 0.7 by Rui: http://rliu09.wordpress.com/2011/04/23/cassandra-across-ec2-regions/ - Sameer Accenture Technology Labs
Re: Running Cassandra across different Amazon EC2 regions
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-IP-address-resolution-in-MultiDC-setup-EC2-VIP-td6306635.html On May 6, 2011 3:07 AM, Sameer Farooqui cassandral...@gmail.com wrote: We're trying to set up a Cassandra 0.8.0beta1 cluster across Amazon East West regions. It does not work out of the box with the binaries and the nodes in different regions end up setting this own clusters. The problem is with Cassandra's Listening Address as described by Rui: Using external IP causes Binding exception, because Cassandra, and java socket, cannot listen on the external IP, which does not belong to this node. Using private IP prevents nodes in other region from connecting back to this node with the private IP, which has been sent to other nodes for cluster establishing. In Feb 2011, Peter Fales posted: ... the trick is that you have must use the external addresses in your storage-conf.xml, but since you don't have a NIC that can actually bind to those addresses, you need to listen on the internal addresses (or, more simply, all interfaces). At the time, I was not able to get the cross-region cluster to work without making changes to the Cassandra code. Perhaps things have evolved so that there are other ways to do it now. Does anyone know if Cassandra 0.8 can be configured to run in multiple Amazon regions without modifying the source code? And if not, has anybody successfully done this with the 0.8 source code and shared the technique? After some research we found that other people have also ran into this issue with versions 0.6.x and 0.7.x of Cassandra: Fix for 0.6.5 by Peter Fales in Sept 2010: http://www.mail-archive.com/dev@cassandra.apache.org/msg01832.html http://www.mail-archive.com/user@cassandra.apache.org/msg05692.html Fix for 0.7 by Rui: http://rliu09.wordpress.com/2011/04/23/cassandra-across-ec2-regions/ - Sameer Accenture Technology Labs
