Another possibility is this: why not setup 2 nodes in 1 region in 1 az, and get that to work. Then, open a third node in the same region, but different AZ, and get that to work. Then, once you have that working, open a fourth node in a different region and get that to work.
Seems like taking a piece-meal approach would be beneficial here. Dave Viner On Thu, Feb 24, 2011 at 6:11 AM, Daniel van Ham Colchete < daniel.colch...@gmail.com> wrote: > Himanshi, > > my bad, try this for iptables: > > # SNAT outgoing connections > iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j > SNAT --to-source INTERNALIP > > As for tcpdump the argument for the -i option is the interface name (eth0, > cassth0, etc...), and not the IP. So, it should be > tcpdump -i cassth0 -n port 7000 > or > tcpdump -i eth0 -n port 7000 > > I`m assuming your main network card is eth0, but that should be the case. > > Does it work? > > Best, > Daniel > > > On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma > <himanshi.sha...@tcs.com>wrote: > >> >> Thanks Daniel. >> >> But SNAT command is not working and when i try tcpdump it gives >> >> [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 >> tcpdump: Invalid adapter index >> >> Not able to figure out wats this ?? >> >> Thanks, >> Himanshi >> >> >> >> From: Daniel van Ham Colchete <daniel.colch...@gmail.com> To: >> user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> ------------------------------ >> >> >> >> Himanshi, >> >> you could try adding your public IP address to an internal interface and >> DNAT the packets to it. This shouldn't give you any problems with your >> normal traffic. Tell Cassandra on listen on the public IPs and it should >> work. >> >> Linux commands would be: >> >> # Create an internal interface using bridge-utils >> brctl addbr cassth0 >> >> # add the ip >> ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32> >> >> # DNAT incoming connections >> iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT >> --to-destination 50.18.60.117 >> >> # SNAT outgoing connections >> iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT >> --to-source INTERNALIP >> >> This should work since Amazon you re-SNAT your outgoing packets to your >> public IP again, so the other cassandra instance will see your public IP as >> your source address. >> >> I didn't test this setup here but it should work unless I forgot some >> small detail. If you need to troubleshoot use the command "tcpdump -i >> INTERFACE -n port 7000" where INTERFACE should be your public interface or >> your cassth0. >> >> Please let me know if it worked. >> >> Best regards, >> Daniel Colchete >> >> On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <* >> himanshi.sha...@tcs.com* <himanshi.sha...@tcs.com>> wrote: >> giving private ip to rpc address gives the same exception >> and the keeping it blank and providing public to listen also fails. I >> tried keeping both blank and did telnet on 7000 so i get following o/p >> >> [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 >> Trying 122.248.193.37... >> Connected to 122.248.193.37. >> Escape character is '^]'. >> >> Similarly from another achine >> >> [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 >> Trying 184.72.22.87... >> Connected to 184.72.22.87. >> Escape character is '^]'. >> >> >> >> -----Dave Viner wrote: ----- >> To: *user@cassandra.apache.org* <user@cassandra.apache.org> >> From: Dave Viner <*davevi...@gmail.com* <davevi...@gmail.com>> >> Date: 02/24/2011 11:59AM >> cc: Himanshi Sharma <*himanshi.sha...@tcs.com* <himanshi.sha...@tcs.com>> >> >> Subject: Re: Cassandra nodes on EC2 in two different regions not >> communicating >> >> Try using the private ipv4 address in the rpc_address field, and the >> public ipv4 (NOT the elastic ip) in the listen_address. >> >> If that fails, go back to rpc_address empty, and start up cassandra. >> >> Then from the other node, please telnet to port 7000 on the first node. >> And show the output of that session in your reply. >> >> I haven't actually constructed a cross-region cluster nor have I used >> v0.7, but this really sounds like it should be easy. >> >> On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> wrote: >> Hi Dave, >> >> I tried with the public ips. If i mention the public ip in rpc address >> field, Cassandra gives the same exception but if leave it blank then >> Cassandra runs but again in the nodetool command with ring option it does'nt >> show the node in another region. >> >> Thanks, >> Himanshi >> >> >> -----Dave Viner wrote: ----- >> To: *user@cassandra.apache.org * <user@cassandra.apache.org> >> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> >> Date: 02/24/2011 10:43AM >> >> Subject: Re: Cassandra nodes on EC2 in two different regions not >> communicating >> >> That looks like it's not an issue of communicating between nodes. It >> appears that the node can not bind to the address on the localhost that >> you're asking for. >> >> " java.net.BindException: Cannot assign requested address " >> >> I think the issue is that the Elastic IP address is not actually an IP >> address that's on the localhost. So the daemon can not bind to that IP. >> Instead of using the EIP, use the local IP address for the rpc_address (i >> think that's what you need since that is what Thrift will bind to). Then >> for the listen_address should be the ip address that is routable from the >> other node. I would first try with the actual public IP address (not the >> Elastic IP). Once you get that to work, then shutdown the cluster, change >> the listen_address to the EIP, boot up and try again. >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> wrote: >> >> Hey Dave, >> >> Sorry i forgot to mention the Non-seed configuration. >> >> for first node in us-west its as below....i.e its own elastic ip >> >> listen_address: 50.18.60.117 >> rpc_address: 50.18.60.117 >> >> and for second node in ap-southeast-1 its as below....i.e again its own >> elastic ip >> >> listen_address: 175.41.143.192 >> rpc_address: 175.41.143.192 >> >> Thanks, >> Himanshi >> >> >> >> >> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> To: >> *user@cassandra.apache.org >> * <user@cassandra.apache.org> Date: 02/23/2011 11:01 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> >> ------------------------------ >> >> >> >> internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., >> from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east >> to us-west). To do regions, you must use the public ip address assigned by >> amazon. >> >> Himanshi, when you log into 1 node, and telnet to port 7000 on the other >> node, which IP address did you use - the 10.x address or the public ip >> address? >> And what is the seed/non-seed configuration in both cassandra.yaml files? >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < *fr...@isidorey.com >> *<fr...@isidorey.com>> >> wrote: >> The internal Amazon IP address is what you will want to use so you don't >> have to go through DNS anyways; not sure if this works from US-East to >> US-West, but it does make things quicker in between zones, e.g. us-east-1a >> to us-east-1b. >> >> >> On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < *davevi...@gmail.com >> *<davevi...@gmail.com>> >> wrote: >> Try using the IP address, not the dns name in the cassandra.yaml. >> >> If you can telnet from one to the other on port 7000, and both nodes have >> the other node in their config, it should work. >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> wrote: >> >> Ya they do. Have specified Public DNS in seed field of each node in >> Cassandra.yaml...nt able to figure out what the problem is ??? >> >> >> From: Sasha Dolgy < *sdo...@gmail.com * <sdo...@gmail.com>> To: >> *user@cassandra.apache.org >> * <user@cassandra.apache.org> Date: 02/23/2011 02:56 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> >> >> ------------------------------ >> >> >> >> did you define the other host in the cassandra.yaml ? on both servers >> .... they need to know about each other >> >> On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> wrote: >> >> Thanks Dave but I am able to telnet to other instances on port 7000 >> and when i run ./nodetool --host >> *ec2-50-18-60-117.us-west-1.compute.amazonaws.com >> * <http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> ring... I >> can see only one node. >> >> Do we need to configure anything else in Cassandra.yaml or >> Cassandra-env.sh ??? >> >> >> >> >> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> To: >> *user@cassandra.apache.org >> * <user@cassandra.apache.org> Cc: Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> Date: 02/23/2011 11:36 AM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> >> >> >> ------------------------------ >> >> >> >> If you login to one of the nodes, can you telnet to port 7000 on the other >> node? >> >> If not, then almost certainly it's a firewall/Security Group issue. >> >> You can find out the security groups for any node by logging in, and then >> running: >> >> % curl " *http://169.254.169.254/latest/meta-data/security-groups >> *<http://169.254.169.254/latest/meta-data/security-groups>" >> >> >> Assuming that both nodes are in the same security group, ensure that the >> SG is configured to allow other members of the SG to communicate on port >> 7000 to each other. >> >> HTH, >> Dave Viner >> >> >> On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * <himanshi.sha...@tcs.com>> wrote: >> >> Hi, >> >> I am new to Cassandra. I m running Cassandra on EC2. I configured >> Cassandra cluster on two instances in different regions. >> But when I am trying the nodetool command with ring option, I am getting >> only single node. >> >> How to make these two nodes communicate with each other. I have already >> opened required ports. i.e 7000, 8080, 9160 in respective >> security groups. Plz help me with this. >> >> Regards, >> Himanshi Sharma >> >> >> =====-----=====-----===== >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> >> >> =====-----=====-----===== >> >> >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> >> >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> >> >> and any attachments. Thank you >> >> >> >> >> >> -- >> Sasha Dolgy * >> **sasha.do...@gmail.com * <sasha.do...@gmail.com> >> >> =====-----=====-----===== >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> >> >> >> >> -- >> Frank LoVecchio >> Senior Software Engineer | Isidorey, LLC >> Google Voice +1.720.295.9179 * >> **isidorey.com * <http://isidorey.com/>| *facebook.com/franklovecchio >> *<http://facebook.com/franklovecchio>| >> *franklovecchio.com * <http://franklovecchio.com/> >> >> >> >> =====-----=====-----===== >> >> Notice: The information contained in this e-mail >> >> message and/or attachments to it may contain >> >> confidential or privileged information. If you are >> >> >> not the intended recipient, any dissemination, use, >> >> review, distribution, printing or copying of the >> >> information contained in this e-mail message >> >> and/or attachments to it are strictly prohibited. If >> >> you have received this communication in error, >> >> >> please notify us by reply e-mail or telephone and >> >> immediately and permanently delete the message >> >> and any attachments. Thank you >> >> >> >> >> =====-----=====-----===== >> >> Notice: The information contained in this e-mail >> >> message and/or attachments to it may contain >> >> confidential or privileged information. If you are >> >> >> not the intended recipient, any dissemination, use, >> >> review, distribution, printing or copying of the >> >> information contained in this e-mail message >> >> and/or attachments to it are strictly prohibited. If >> >> you have received this communication in error, >> >> >> please notify us by reply e-mail or telephone and >> >> immediately and permanently delete the message >> >> and any attachments. Thank you >> >> >> >> >> =====-----=====-----===== >> >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> >> >> =====-----=====-----===== >> >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> >
