Re: Cassandra nodes on EC2 in two different regions not communicating
Not sure if there is a particular reason for you using different regions, but Amazon states that each zone is a different physical location completely separate from others, e.g. us-east-1a and us-east-1b. Using the Amazon internal IPs (10.x. etc) reduces latency greatly by not going outbound through DNS (though us-east-1c is twice as latent), you have an RF 4 possible (in different physical locations!), and most importantly sh*t just works. Maybe this won't help you, but it may be useful for others :) On Thu, Feb 24, 2011 at 10:53 AM, Dave Viner wrote: > Another possibility is this: > > why not setup 2 nodes in 1 region in 1 az, and get that to work. > Then, open a third node in the same region, but different AZ, and get that > to work. > Then, once you have that working, open a fourth node in a different region > and get that to work. > > Seems like taking a piece-meal approach would be beneficial here. > > Dave Viner > > > On Thu, Feb 24, 2011 at 6:11 AM, Daniel van Ham Colchete < > daniel.colch...@gmail.com> wrote: > >> Himanshi, >> >> my bad, try this for iptables: >> >> # SNAT outgoing connections >> iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j >> SNAT --to-source INTERNALIP >> >> As for tcpdump the argument for the -i option is the interface name (eth0, >> cassth0, etc...), and not the IP. So, it should be >> tcpdump -i cassth0 -n port 7000 >> or >> tcpdump -i eth0 -n port 7000 >> >> I`m assuming your main network card is eth0, but that should be the case. >> >> Does it work? >> >> Best, >> Daniel >> >> >> On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma > > wrote: >> >>> >>> Thanks Daniel. >>> >>> But SNAT command is not working and when i try tcpdump it gives >>> >>> [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 >>> tcpdump: Invalid adapter index >>> >>> Not able to figure out wats this ?? >>> >>> Thanks, >>> Himanshi >>> >>> >>> >>> From: Daniel van Ham Colchete To: >>> user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: >>> Cassandra nodes on EC2 in two different regions not communicating >>> -- >>> >>> >>> >>> Himanshi, >>> >>> you could try adding your public IP address to an internal interface and >>> DNAT the packets to it. This shouldn't give you any problems with your >>> normal traffic. Tell Cassandra on listen on the public IPs and it should >>> work. >>> >>> Linux commands would be: >>> >>> # Create an internal interface using bridge-utils >>> brctl addbr cassth0 >>> >>> # add the ip >>> ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32> >>> >>> # DNAT incoming connections >>> iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT >>> --to-destination 50.18.60.117 >>> >>> # SNAT outgoing connections >>> iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT >>> --to-source INTERNALIP >>> >>> This should work since Amazon you re-SNAT your outgoing packets to your >>> public IP again, so the other cassandra instance will see your public IP as >>> your source address. >>> >>> I didn't test this setup here but it should work unless I forgot some >>> small detail. If you need to troubleshoot use the command "tcpdump -i >>> INTERFACE -n port 7000" where INTERFACE should be your public interface or >>> your cassth0. >>> >>> Please let me know if it worked. >>> >>> Best regards, >>> Daniel Colchete >>> >>> On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <* >>> himanshi.sha...@tcs.com* > wrote: >>> giving private ip to rpc address gives the same exception >>> and the keeping it blank and providing public to listen also fails. I >>> tried keeping both blank and did telnet on 7000 so i get following o/p >>> >>> [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 >>> Trying 122.248.193.37... >>> Connected to 122.248.193.37. >>> Escape character is '^]'. >>> >>> Similarly from another achine >>> >>> [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 >>> Trying 184.72.22.87... >>> Connected to 184.72.22.87. >>> Escape character is
Re: Cassandra nodes on EC2 in two different regions not communicating
Another possibility is this: why not setup 2 nodes in 1 region in 1 az, and get that to work. Then, open a third node in the same region, but different AZ, and get that to work. Then, once you have that working, open a fourth node in a different region and get that to work. Seems like taking a piece-meal approach would be beneficial here. Dave Viner On Thu, Feb 24, 2011 at 6:11 AM, Daniel van Ham Colchete < daniel.colch...@gmail.com> wrote: > Himanshi, > > my bad, try this for iptables: > > # SNAT outgoing connections > iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j > SNAT --to-source INTERNALIP > > As for tcpdump the argument for the -i option is the interface name (eth0, > cassth0, etc...), and not the IP. So, it should be > tcpdump -i cassth0 -n port 7000 > or > tcpdump -i eth0 -n port 7000 > > I`m assuming your main network card is eth0, but that should be the case. > > Does it work? > > Best, > Daniel > > > On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma > wrote: > >> >> Thanks Daniel. >> >> But SNAT command is not working and when i try tcpdump it gives >> >> [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 >> tcpdump: Invalid adapter index >> >> Not able to figure out wats this ?? >> >> Thanks, >> Himanshi >> >> >> >> From: Daniel van Ham Colchete To: >> user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> -- >> >> >> >> Himanshi, >> >> you could try adding your public IP address to an internal interface and >> DNAT the packets to it. This shouldn't give you any problems with your >> normal traffic. Tell Cassandra on listen on the public IPs and it should >> work. >> >> Linux commands would be: >> >> # Create an internal interface using bridge-utils >> brctl addbr cassth0 >> >> # add the ip >> ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32> >> >> # DNAT incoming connections >> iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT >> --to-destination 50.18.60.117 >> >> # SNAT outgoing connections >> iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT >> --to-source INTERNALIP >> >> This should work since Amazon you re-SNAT your outgoing packets to your >> public IP again, so the other cassandra instance will see your public IP as >> your source address. >> >> I didn't test this setup here but it should work unless I forgot some >> small detail. If you need to troubleshoot use the command "tcpdump -i >> INTERFACE -n port 7000" where INTERFACE should be your public interface or >> your cassth0. >> >> Please let me know if it worked. >> >> Best regards, >> Daniel Colchete >> >> On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <* >> himanshi.sha...@tcs.com* > wrote: >> giving private ip to rpc address gives the same exception >> and the keeping it blank and providing public to listen also fails. I >> tried keeping both blank and did telnet on 7000 so i get following o/p >> >> [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 >> Trying 122.248.193.37... >> Connected to 122.248.193.37. >> Escape character is '^]'. >> >> Similarly from another achine >> >> [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 >> Trying 184.72.22.87... >> Connected to 184.72.22.87. >> Escape character is '^]'. >> >> >> >> -Dave Viner wrote: - >> To: *user@cassandra.apache.org* >> From: Dave Viner <*davevi...@gmail.com* > >> Date: 02/24/2011 11:59AM >> cc: Himanshi Sharma <*himanshi.sha...@tcs.com* > >> >> Subject: Re: Cassandra nodes on EC2 in two different regions not >> communicating >> >> Try using the private ipv4 address in the rpc_address field, and the >> public ipv4 (NOT the elastic ip) in the listen_address. >> >> If that fails, go back to rpc_address empty, and start up cassandra. >> >> Then from the other node, please telnet to port 7000 on the first node. >> And show the output of that session in your reply. >> >> I haven't actually constructed a cross-region cluster nor have I used >> v0.7, but this really sounds like it should be easy. >> >> On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * > wrot
Re: Cassandra nodes on EC2 in two different regions not communicating
Himanshi, my bad, try this for iptables: # SNAT outgoing connections iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j SNAT --to-source INTERNALIP As for tcpdump the argument for the -i option is the interface name (eth0, cassth0, etc...), and not the IP. So, it should be tcpdump -i cassth0 -n port 7000 or tcpdump -i eth0 -n port 7000 I`m assuming your main network card is eth0, but that should be the case. Does it work? Best, Daniel On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma wrote: > > Thanks Daniel. > > But SNAT command is not working and when i try tcpdump it gives > > [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 > tcpdump: Invalid adapter index > > Not able to figure out wats this ?? > > Thanks, > Himanshi > > > > From: Daniel van Ham Colchete To: > user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: Cassandra > nodes on EC2 in two different regions not communicating > -- > > > > Himanshi, > > you could try adding your public IP address to an internal interface and > DNAT the packets to it. This shouldn't give you any problems with your > normal traffic. Tell Cassandra on listen on the public IPs and it should > work. > > Linux commands would be: > > # Create an internal interface using bridge-utils > brctl addbr cassth0 > > # add the ip > ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32> > > # DNAT incoming connections > iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT > --to-destination 50.18.60.117 > > # SNAT outgoing connections > iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT > --to-source INTERNALIP > > This should work since Amazon you re-SNAT your outgoing packets to your > public IP again, so the other cassandra instance will see your public IP as > your source address. > > I didn't test this setup here but it should work unless I forgot some small > detail. If you need to troubleshoot use the command "tcpdump -i INTERFACE -n > port 7000" where INTERFACE should be your public interface or your cassth0. > > Please let me know if it worked. > > Best regards, > Daniel Colchete > > On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <*himanshi.sha...@tcs.com > * > wrote: > giving private ip to rpc address gives the same exception > and the keeping it blank and providing public to listen also fails. I tried > keeping both blank and did telnet on 7000 so i get following o/p > > [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 > Trying 122.248.193.37... > Connected to 122.248.193.37. > Escape character is '^]'. > > Similarly from another achine > > [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 > Trying 184.72.22.87... > Connected to 184.72.22.87. > Escape character is '^]'. > > > > -Dave Viner wrote: - > To: *user@cassandra.apache.org* > From: Dave Viner <*davevi...@gmail.com* > > Date: 02/24/2011 11:59AM > cc: Himanshi Sharma <*himanshi.sha...@tcs.com* > > > Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > Try using the private ipv4 address in the rpc_address field, and the public > ipv4 (NOT the elastic ip) in the listen_address. > > If that fails, go back to rpc_address empty, and start up cassandra. > > Then from the other node, please telnet to port 7000 on the first node. > And show the output of that session in your reply. > > I haven't actually constructed a cross-region cluster nor have I used v0.7, > but this really sounds like it should be easy. > > On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < *himanshi.sha...@tcs.com > * > wrote: > Hi Dave, > > I tried with the public ips. If i mention the public ip in rpc address > field, Cassandra gives the same exception but if leave it blank then > Cassandra runs but again in the nodetool command with ring option it does'nt > show the node in another region. > > Thanks, > Himanshi > > > -Dave Viner wrote: - > To: *user@cassandra.apache.org * > From: Dave Viner < *davevi...@gmail.com * > > Date: 02/24/2011 10:43AM > > Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > That looks like it's not an issue of communicating between nodes. It > appears that the node can not bind to the address on the localhost that > you're asking for. > > " java.net.BindException: Cannot assign requested address " > > I think the issue is that the Elastic IP address is not actually an IP > address that's on the localhost.
Re: Cassandra nodes on EC2 in two different regions not communicating
Thanks Daniel. But SNAT command is not working and when i try tcpdump it gives [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 tcpdump: Invalid adapter index Not able to figure out wats this ?? Thanks, Himanshi From: Daniel van Ham Colchete To: user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating Himanshi, you could try adding your public IP address to an internal interface and DNAT the packets to it. This shouldn't give you any problems with your normal traffic. Tell Cassandra on listen on the public IPs and it should work. Linux commands would be: # Create an internal interface using bridge-utils brctl addbr cassth0 # add the ip ip addr add dev cassth0 50.18.60.117/32 # DNAT incoming connections iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT --to-destination 50.18.60.117 # SNAT outgoing connections iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT --to-source INTERNALIP This should work since Amazon you re-SNAT your outgoing packets to your public IP again, so the other cassandra instance will see your public IP as your source address. I didn't test this setup here but it should work unless I forgot some small detail. If you need to troubleshoot use the command "tcpdump -i INTERFACE -n port 7000" where INTERFACE should be your public interface or your cassth0. Please let me know if it worked. Best regards, Daniel Colchete On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma wrote: giving private ip to rpc address gives the same exception and the keeping it blank and providing public to listen also fails. I tried keeping both blank and did telnet on 7000 so i get following o/p [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 Trying 122.248.193.37... Connected to 122.248.193.37. Escape character is '^]'. Similarly from another achine [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 Trying 184.72.22.87... Connected to 184.72.22.87. Escape character is '^]'. -Dave Viner wrote: - To: user@cassandra.apache.org From: Dave Viner Date: 02/24/2011 11:59AM cc: Himanshi Sharma Subject: Re: Cassandra nodes on EC2 in two different regions not communicating Try using the private ipv4 address in the rpc_address field, and the public ipv4 (NOT the elastic ip) in the listen_address. If that fails, go back to rpc_address empty, and start up cassandra. Then from the other node, please telnet to port 7000 on the first node. And show the output of that session in your reply. I haven't actually constructed a cross-region cluster nor have I used v0.7, but this really sounds like it should be easy. On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hi Dave, I tried with the public ips. If i mention the public ip in rpc address field, Cassandra gives the same exception but if leave it blank then Cassandra runs but again in the nodetool command with ring option it does'nt show the node in another region. Thanks, Himanshi -Dave Viner wrote: - To: user@cassandra.apache.org From: Dave Viner < davevi...@gmail.com > Date: 02/24/2011 10:43AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating That looks like it's not an issue of communicating between nodes. It appears that the node can not bind to the address on the localhost that you're asking for. " java.net.BindException: Cannot assign requested address " I think the issue is that the Elastic IP address is not actually an IP address that's on the localhost. So the daemon can not bind to that IP. Instead of using the EIP, use the local IP address for the rpc_address (i think that's what you need since that is what Thrift will bind to). Then for the listen_address should be the ip address that is routable from the other node. I would first try with the actual public IP address (not the Elastic IP). Once you get that to work, then shutdown the cluster, change the listen_address to the EIP, boot up and try again. Dave Viner On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hey Dave, Sorry i forgot to mention the Non-seed configuration. for first node in us-west its as belowi.e its own elastic ip listen_address: 50.18.60.117 rpc_address: 50.18.60.117 and for second node in ap-southeast-1 its as belowi.e again its own elastic ip listen_address: 175.41.143.192 rpc_address: 175.41.143.192 Thanks, Himanshi From: Dave Viner < davevi...@gmail.com > To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east
Re: Cassandra nodes on EC2 in two different regions not communicating
Himanshi, you could try adding your public IP address to an internal interface and DNAT the packets to it. This shouldn't give you any problems with your normal traffic. Tell Cassandra on listen on the public IPs and it should work. Linux commands would be: # Create an internal interface using bridge-utils brctl addbr cassth0 # add the ip ip addr add dev cassth0 50.18.60.117/32 # DNAT incoming connections iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT --to-destination 50.18.60.117 # SNAT outgoing connections iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT --to-source INTERNALIP This should work since Amazon you re-SNAT your outgoing packets to your public IP again, so the other cassandra instance will see your public IP as your source address. I didn't test this setup here but it should work unless I forgot some small detail. If you need to troubleshoot use the command "tcpdump -i INTERFACE -n port 7000" where INTERFACE should be your public interface or your cassth0. Please let me know if it worked. Best regards, Daniel Colchete On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma wrote: > giving private ip to rpc address gives the same exception > and the keeping it blank and providing public to listen also fails. I tried > keeping both blank and did telnet on 7000 so i get following o/p > > [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 > Trying 122.248.193.37... > Connected to 122.248.193.37. > Escape character is '^]'. > > Similarly from another achine > > [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 > Trying 184.72.22.87... > Connected to 184.72.22.87. > Escape character is '^]'. > > > > -Dave Viner wrote: - > > To: user@cassandra.apache.org > From: Dave Viner > Date: 02/24/2011 11:59AM > cc: Himanshi Sharma > > Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > Try using the private ipv4 address in the rpc_address field, and the public > ipv4 (NOT the elastic ip) in the listen_address. > > If that fails, go back to rpc_address empty, and start up cassandra. > > Then from the other node, please telnet to port 7000 on the first node. > And show the output of that session in your reply. > > I haven't actually constructed a cross-region cluster nor have I used v0.7, > but this really sounds like it should be easy. > > On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < himanshi.sha...@tcs.com > > wrote: > >> Hi Dave, >> >> I tried with the public ips. If i mention the public ip in rpc address >> field, Cassandra gives the same exception but if leave it blank then >> Cassandra runs but again in the nodetool command with ring option it does'nt >> show the node in another region. >> >> Thanks, >> Himanshi >> >> >> -Dave Viner wrote: - >> >> To: user@cassandra.apache.org >> From: Dave Viner < davevi...@gmail.com > >> Date: 02/24/2011 10:43AM >> >> Subject: Re: Cassandra nodes on EC2 in two different regions not >> communicating >> >> That looks like it's not an issue of communicating between nodes. It >> appears that the node can not bind to the address on the localhost that >> you're asking for. >> >> " java.net.BindException: Cannot assign requested address " >> >> I think the issue is that the Elastic IP address is not actually an IP >> address that's on the localhost. So the daemon can not bind to that IP. >> Instead of using the EIP, use the local IP address for the rpc_address (i >> think that's what you need since that is what Thrift will bind to). Then >> for the listen_address should be the ip address that is routable from the >> other node. I would first try with the actual public IP address (not the >> Elastic IP). Once you get that to work, then shutdown the cluster, change >> the listen_address to the EIP, boot up and try again. >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sha...@tcs.com >> > wrote: >> >>> >>> Hey Dave, >>> >>> Sorry i forgot to mention the Non-seed configuration. >>> >>> for first node in us-west its as belowi.e its own elastic ip >>> >>> listen_address: 50.18.60.117 >>> rpc_address: 50.18.60.117 >>> >>> and for second node in ap-southeast-1 its as belowi.e again its own >>> elastic ip >>> >>> listen_address: 175.41.143.192 >>> rpc_address: 175.41.143.192 >>> >>> Thanks, >>&g
Re: Cassandra nodes on EC2 in two different regions not communicating
giving private ip to rpc address gives the same exception and the keeping it blank and providing public to listen also fails. I tried keeping both blank and did telnet on 7000 so i get following o/p [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000Trying 122.248.193.37...Connected to 122.248.193.37.Escape character is '^]'. Similarly from another achine [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000Trying 184.72.22.87...Connected to 184.72.22.87.Escape character is '^]'. -Dave Viner wrote: - To: user@cassandra.apache.orgFrom: Dave Viner Date: 02/24/2011 11:59AMcc: Himanshi Sharma Subject: Re: Cassandra nodes on EC2 in two different regions not communicatingTry using the private ipv4 address in the rpc_address field, and the public ipv4 (NOT the elastic ip) in the listen_address. If that fails, go back to rpc_address empty, and start up cassandra. Then from the other node, please telnet to port 7000 on the first node. And show the output of that session in your reply. I haven't actually constructed a cross-region cluster nor have I used v0.7, but this really sounds like it should be easy. On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hi Dave, I tried with the public ips. If i mention the public ip in rpc address field, Cassandra gives the same exception but if leave it blank then Cassandra runs but again in the nodetool command with ring option it does'nt show the node in another region. Thanks, Himanshi -Dave Viner wrote: - To: user@cassandra.apache.org From: Dave Viner < davevi...@gmail.com > Date: 02/24/2011 10:43AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating That looks like it's not an issue of communicating between nodes. It appears that the node can not bind to the address on the localhost that you're asking for. " java.net.BindException: Cannot assign requested address " I think the issue is that the Elastic IP address is not actually an IP address that's on the localhost. So the daemon can not bind to that IP. Instead of using the EIP, use the local IP address for the rpc_address (i think that's what you need since that is what Thrift will bind to). Then for the listen_address should be the ip address that is routable from the other node. I would first try with the actual public IP address (not the Elastic IP). Once you get that to work, then shutdown the cluster, change the listen_address to the EIP, boot up and try again. Dave Viner On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hey Dave, Sorry i forgot to mention the Non-seed configuration. for first node in us-west its as belowi.e its own elastic ip listen_address: 50.18.60.117 rpc_address: 50.18.60.117 and for second node in ap-southeast-1 its as belowi.e again its own elastic ip listen_address: 175.41.143.192 rpc_address: 175.41.143.192 Thanks, Himanshi From: Dave Viner < davevi...@gmail.com > To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east to us-west). To do regions, you must use the public ip address assigned by amazon. Himanshi, when you log into 1 node, and telnet to port 7000 on the other node, which IP address did you use - the 10.x address or the public ip address? And what is the seed/non-seed configuration in both cassandra.yaml files? Dave Viner On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < fr...@isidorey.com > wrote: The internal Amazon IP address is what you will want to use so you don't have to go through DNS anyways; not sure if this works from US-East to US-West, but it does make things quicker in between zones, e.g. us-east-1a to us-east-1b. On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < davevi...@gmail.com > wrote: Try using the IP address, not the dns name in the cassandra.yaml. If you can telnet from one to the other on port 7000, and both nodes have the other node in their config, it should work. Dave Viner On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Ya they do. Have specified Public DNS in seed field of each node in Cassandra.yaml...nt able to figure out what the problem is ??? From: Sasha Dolgy < sdo...@gmail.com > To: user@cassandra.apache.org Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Thanks Dave but I am able to telnet to other instances on p
Re: Cassandra nodes on EC2 in two different regions not communicating
Try using the private ipv4 address in the rpc_address field, and the public ipv4 (NOT the elastic ip) in the listen_address. If that fails, go back to rpc_address empty, and start up cassandra. Then from the other node, please telnet to port 7000 on the first node. And show the output of that session in your reply. I haven't actually constructed a cross-region cluster nor have I used v0.7, but this really sounds like it should be easy. On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma wrote: > Hi Dave, > > I tried with the public ips. If i mention the public ip in rpc address > field, Cassandra gives the same exception but if leave it blank then > Cassandra runs but again in the nodetool command with ring option it does'nt > show the node in another region. > > Thanks, > Himanshi > > > -Dave Viner wrote: - > > To: user@cassandra.apache.org > From: Dave Viner > Date: 02/24/2011 10:43AM > > Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > That looks like it's not an issue of communicating between nodes. It > appears that the node can not bind to the address on the localhost that > you're asking for. > > " java.net.BindException: Cannot assign requested address " > > I think the issue is that the Elastic IP address is not actually an IP > address that's on the localhost. So the daemon can not bind to that IP. > Instead of using the EIP, use the local IP address for the rpc_address (i > think that's what you need since that is what Thrift will bind to). Then > for the listen_address should be the ip address that is routable from the > other node. I would first try with the actual public IP address (not the > Elastic IP). Once you get that to work, then shutdown the cluster, change > the listen_address to the EIP, boot up and try again. > > Dave Viner > > > On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sha...@tcs.com > > wrote: > >> >> Hey Dave, >> >> Sorry i forgot to mention the Non-seed configuration. >> >> for first node in us-west its as belowi.e its own elastic ip >> >> listen_address: 50.18.60.117 >> rpc_address: 50.18.60.117 >> >> and for second node in ap-southeast-1 its as belowi.e again its own >> elastic ip >> >> listen_address: 175.41.143.192 >> rpc_address: 175.41.143.192 >> >> Thanks, >> Himanshi >> >> >> >> >> >> From: >> Dave Viner < davevi...@gmail.com > >> To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> -- >> >> >> >> internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., >> from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east >> to us-west). To do regions, you must use the public ip address assigned by >> amazon. >> >> Himanshi, when you log into 1 node, and telnet to port 7000 on the other >> node, which IP address did you use - the 10.x address or the public ip >> address? >> And what is the seed/non-seed configuration in both cassandra.yaml files? >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < *fr...@isidorey.com >> *> >> wrote: >> The internal Amazon IP address is what you will want to use so you don't >> have to go through DNS anyways; not sure if this works from US-East to >> US-West, but it does make things quicker in between zones, e.g. us-east-1a >> to us-east-1b. >> >> >> On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < *davevi...@gmail.com >> *> >> wrote: >> Try using the IP address, not the dns name in the cassandra.yaml. >> >> If you can telnet from one to the other on port 7000, and both nodes have >> the other node in their config, it should work. >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < *himanshi.sha...@tcs.com >> * > wrote: >> >> Ya they do. Have specified Public DNS in seed field of each node in >> Cassandra.yaml...nt able to figure out what the problem is ??? >> >> >> >> From: Sasha Dolgy < *sdo...@gmail.com * > To: >> *user@cassandra.apache.org >> * Date: 02/23/2011 02:56 PM Subject: Re: >> Cassandra nodes on EC2 in two different regions not communicating >> >> -- >> >> >> >> did you define the other host in the cassandra.yaml ? on both serve
Re: Cassandra nodes on EC2 in two different regions not communicating
Hi Dave, I tried with the public ips. If i mention the public ip in rpc address field, Cassandra gives the same exception but if leave it blank then Cassandra runs but again in the nodetool command with ring option it does'nt show the node in another region. Thanks, Himanshi-Dave Viner wrote: - To: user@cassandra.apache.orgFrom: Dave Viner Date: 02/24/2011 10:43AMSubject: Re: Cassandra nodes on EC2 in two different regions not communicatingThat looks like it's not an issue of communicating between nodes. It appears that the node can not bind to the address on the localhost that you're asking for. " java.net.BindException: Cannot assign requested address " I think the issue is that the Elastic IP address is not actually an IP address that's on the localhost. So the daemon can not bind to that IP. Instead of using the EIP, use the local IP address for the rpc_address (i think that's what you need since that is what Thrift will bind to). Then for the listen_address should be the ip address that is routable from the other node. I would first try with the actual public IP address (not the Elastic IP). Once you get that to work, then shutdown the cluster, change the listen_address to the EIP, boot up and try again. Dave Viner On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hey Dave, Sorry i forgot to mention the Non-seed configuration. for first node in us-west its as belowi.e its own elastic ip listen_address: 50.18.60.117 rpc_address: 50.18.60.117 and for second node in ap-southeast-1 its as belowi.e again its own elastic ip listen_address: 175.41.143.192 rpc_address: 175.41.143.192 Thanks, Himanshi From: Dave Viner < davevi...@gmail.com > To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east to us-west). To do regions, you must use the public ip address assigned by amazon. Himanshi, when you log into 1 node, and telnet to port 7000 on the other node, which IP address did you use - the 10.x address or the public ip address? And what is the seed/non-seed configuration in both cassandra.yaml files? Dave Viner On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < fr...@isidorey.com > wrote: The internal Amazon IP address is what you will want to use so you don't have to go through DNS anyways; not sure if this works from US-East to US-West, but it does make things quicker in between zones, e.g. us-east-1a to us-east-1b. On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < davevi...@gmail.com > wrote: Try using the IP address, not the dns name in the cassandra.yaml. If you can telnet from one to the other on port 7000, and both nodes have the other node in their config, it should work. Dave Viner On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Ya they do. Have specified Public DNS in seed field of each node in Cassandra.yaml...nt able to figure out what the problem is ??? From: Sasha Dolgy < sdo...@gmail.com > To: user@cassandra.apache.org Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Thanks Dave but I am able to telnet to other instances on port 7000 and when i run ./nodetool --host ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only one node. Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh ??? From: Dave Viner < davevi...@gmail.com > To: user@cassandra.apache.org Cc: Himanshi Sharma < himanshi.sha...@tcs.com > Date: 02/23/2011 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl " http://169.254.169.254/latest/meta-data/security-groups " Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma < himanshi.sha...@tcs.com > wrote: Hi, I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra cluster on two instances in different regions. But when I am trying the nodetool command with ring option, I am getting only single node. How to make these two nodes communicate
Re: Cassandra nodes on EC2 in two different regions not communicating
That looks like it's not an issue of communicating between nodes. It appears that the node can not bind to the address on the localhost that you're asking for. "java.net.BindException: Cannot assign requested address " I think the issue is that the Elastic IP address is not actually an IP address that's on the localhost. So the daemon can not bind to that IP. Instead of using the EIP, use the local IP address for the rpc_address (i think that's what you need since that is what Thrift will bind to). Then for the listen_address should be the ip address that is routable from the other node. I would first try with the actual public IP address (not the Elastic IP). Once you get that to work, then shutdown the cluster, change the listen_address to the EIP, boot up and try again. Dave Viner On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma wrote: > > Hey Dave, > > Sorry i forgot to mention the Non-seed configuration. > > for first node in us-west its as belowi.e its own elastic ip > > listen_address: 50.18.60.117 > rpc_address: 50.18.60.117 > > and for second node in ap-southeast-1 its as belowi.e again its own > elastic ip > > listen_address: 175.41.143.192 > rpc_address: 175.41.143.192 > > Thanks, > Himanshi > > > > > > From: > Dave Viner > To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: > Cassandra nodes on EC2 in two different regions not communicating > -- > > > > internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., > from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east > to us-west). To do regions, you must use the public ip address assigned by > amazon. > > Himanshi, when you log into 1 node, and telnet to port 7000 on the other > node, which IP address did you use - the 10.x address or the public ip > address? > And what is the seed/non-seed configuration in both cassandra.yaml files? > > Dave Viner > > > On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio > <*fr...@isidorey.com*> > wrote: > The internal Amazon IP address is what you will want to use so you don't > have to go through DNS anyways; not sure if this works from US-East to > US-West, but it does make things quicker in between zones, e.g. us-east-1a > to us-east-1b. > > > On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner > <*davevi...@gmail.com*> > wrote: > Try using the IP address, not the dns name in the cassandra.yaml. > > If you can telnet from one to the other on port 7000, and both nodes have > the other node in their config, it should work. > > Dave Viner > > > On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma <*himanshi.sha...@tcs.com > * > wrote: > > Ya they do. Have specified Public DNS in seed field of each node in > Cassandra.yaml...nt able to figure out what the problem is ??? > > > > From: Sasha Dolgy <*sdo...@gmail.com* > To: * > user@cassandra.apache.org* Date: 02/23/2011 > 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > -- > > > > did you define the other host in the cassandra.yaml ? on both servers > they need to know about each other > > On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma <* > himanshi.sha...@tcs.com* > wrote: > > Thanks Dave but I am able to telnet to other instances on port 7000 > and when i run ./nodetool --host * > ec2-50-18-60-117.us-west-1.compute.amazonaws.com*<http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> > ring... I can see only one node. > > Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh > ??? > > > > > > From: Dave Viner <*davevi...@gmail.com* > To: * > user@cassandra.apache.org* Cc: Himanshi > Sharma <*himanshi.sha...@tcs.com* > Date: 02/23/2011 > 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > > -- > > > > If you login to one of the nodes, can you telnet to port 7000 on the other > node? > > If not, then almost certainly it's a firewall/Security Group issue. > > You can find out the security groups for any node by logging in, and then > running: > > % curl > "*http://169.254.169.254/latest/meta-data/security-groups*<http://169.254.169.254/latest/meta-data/security-groups>" > > > Assuming that both nodes are in the same security group, ensure that the SG > is configured to allow other members of the SG to communicate on port 7000 > to each other. > > HTH, > Dave Viner > > > On Tue, Feb 22, 2011 at
Re: Cassandra nodes on EC2 in two different regions not communicating
Hey Dave, Sorry i forgot to mention the Non-seed configuration. for first node in us-west its as belowi.e its own elastic ip listen_address: 50.18.60.117 rpc_address: 50.18.60.117 and for second node in ap-southeast-1 its as belowi.e again its own elastic ip listen_address: 175.41.143.192 rpc_address: 175.41.143.192 Thanks, Himanshi From: Dave Viner To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east to us-west). To do regions, you must use the public ip address assigned by amazon. Himanshi, when you log into 1 node, and telnet to port 7000 on the other node, which IP address did you use - the 10.x address or the public ip address? And what is the seed/non-seed configuration in both cassandra.yaml files? Dave Viner On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio wrote: The internal Amazon IP address is what you will want to use so you don't have to go through DNS anyways; not sure if this works from US-East to US-West, but it does make things quicker in between zones, e.g. us-east-1a to us-east-1b. On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner wrote: Try using the IP address, not the dns name in the cassandra.yaml. If you can telnet from one to the other on port 7000, and both nodes have the other node in their config, it should work. Dave Viner On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma wrote: Ya they do. Have specified Public DNS in seed field of each node in Cassandra.yaml...nt able to figure out what the problem is ??? From: Sasha Dolgy To: user@cassandra.apache.org Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma wrote: Thanks Dave but I am able to telnet to other instances on port 7000 and when i run ./nodetool --host ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only one node. Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh ??? From: Dave Viner To: user@cassandra.apache.org Cc: Himanshi Sharma Date: 02/23/2011 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl "http://169.254.169.254/latest/meta-data/security-groups"; Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma wrote: Hi, I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra cluster on two instances in different regions. But when I am trying the nodetool command with ring option, I am getting only single node. How to make these two nodes communicate with each other. I have already opened required ports. i.e 7000, 8080, 9160 in respective security groups. Plz help me with this. Regards, Himanshi Sharma =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you -- Sasha Dolgy sasha.do...@gmail.com =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the i
Re: Cassandra nodes on EC2 in two different regions not communicating
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:161) at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:217) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:134) I am able to ping both the nodes through public ips. From: Dave Viner To: user@cassandra.apache.org Date: 02/23/2011 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east to us-west). To do regions, you must use the public ip address assigned by amazon. Himanshi, when you log into 1 node, and telnet to port 7000 on the other node, which IP address did you use - the 10.x address or the public ip address? And what is the seed/non-seed configuration in both cassandra.yaml files? Dave Viner On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio wrote: The internal Amazon IP address is what you will want to use so you don't have to go through DNS anyways; not sure if this works from US-East to US-West, but it does make things quicker in between zones, e.g. us-east-1a to us-east-1b. On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner wrote: Try using the IP address, not the dns name in the cassandra.yaml. If you can telnet from one to the other on port 7000, and both nodes have the other node in their config, it should work. Dave Viner On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma wrote: Ya they do. Have specified Public DNS in seed field of each node in Cassandra.yaml...nt able to figure out what the problem is ??? From: Sasha Dolgy To: user@cassandra.apache.org Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma wrote: Thanks Dave but I am able to telnet to other instances on port 7000 and when i run ./nodetool --host ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only one node. Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh ??? From: Dave Viner To: user@cassandra.apache.org Cc: Himanshi Sharma Date: 02/23/2011 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl "http://169.254.169.254/latest/meta-data/security-groups"; Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma wrote: Hi, I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra cluster on two instances in different regions. But when I am trying the nodetool command with ring option, I am getting only single node. How to make these two nodes communicate with each other. I have already opened required ports. i.e 7000, 8080, 9160 in respective security groups. Plz help me with this. Regards, Himanshi Sharma =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you -- Sasha Dolgy sasha.do...@gmail.com =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended reci
Re: Cassandra nodes on EC2 in two different regions not communicating
I posted on this topic last September. (See http://www.mail-archive.com/user@cassandra.apache.org/msg05692.html) I was able to use Cassandra across EC2regions. However, the trick is that you have must use the "external" addresses in your storage-conf.xml, but since you don't have a NIC that can actually bind to those addresses, you need to listen on the "internal" addresses (or, more simply, all interfaces) At the time, I was not able to get the cross-region cluster to work without making changes to the Cassandra code. Perhaps things have evolved so that there are other ways to do it now. On Wed, Feb 23, 2011 at 11:31:05AM -0600, Dave Viner wrote: > >internal EC2 ips (10.xxx.xxx.xxx) work across availability zones >(e.g., from us-east-1a to us-east-1b) but do not work across regions >(e.g., us-east to us-west). To do regions, you must use the public ip >address assigned by amazon. > >Himanshi, when you log into 1 node, and telnet to port 7000 on the >other node, which IP address did you use - the 10.x address or the >public ip address? > >And what is the seed/non-seed configuration in both cassandra.yaml >files? > >Dave Viner > >On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio ><[1]fr...@isidorey.com> wrote: > > The internal Amazon IP address is what you will want to use so you > don't have to go through DNS anyways; not sure if this works from > US-East to US-West, but it does make things quicker in between > zones, e.g. us-east-1a to us-east-1b. > >On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner <[2]davevi...@gmail.com> >wrote: > > Try using the IP address, not the dns name in the cassandra.yaml. > >If you can telnet from one to the other on port 7000, and both nodes >have the other node in their config, it should work. > >Dave Viner > >On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma ><[3]himanshi.sha...@tcs.com> wrote: > > Ya they do. Have specified Public DNS in seed field of each node in > Cassandra.yaml...nt able to figure out what the problem is ??? > > From: Sasha Dolgy <[4]sdo...@gmail.com> >To: [5]user@cassandra.apache.org >Date: 02/23/2011 02:56 PM >Subject: Re: Cassandra nodes on EC2 in two different regions not >communicating >__ > > did you define the other host in the cassandra.yaml ? on both > servers they need to know about each other > On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma > <[6]himanshi.sha...@tcs.com> wrote: > Thanks Dave but I am able to telnet to other instances on port 7000 > and when i run ./nodetool --host > [7]ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can > see only one node. > Do we need to configure anything else in Cassandra.yaml or > Cassandra-env.sh ??? > > From: Dave Viner <[8]davevi...@gmail.com> >To: [9]user@cassandra.apache.org >Cc: Himanshi Sharma <[10]himanshi.sha...@tcs.com> >Date: 02/23/2011 11:36 AM >Subject: Re: Cassandra nodes on EC2 in two different regions not >communicating >__ > > If you login to one of the nodes, can you telnet to port 7000 on > the other node? > If not, then almost certainly it's a firewall/Security Group issue. > You can find out the security groups for any node by logging in, > and then running: > % curl > "[11]http://169.254.169.254/latest/meta-data/security-groups"; > Assuming that both nodes are in the same security group, ensure > that the SG is configured to allow other members of the SG to > communicate on port 7000 to each other. > HTH, > Dave Viner > On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma > <[12]himanshi.sha...@tcs.com> wrote: > Hi, > I am new to Cassandra. I m running Cassandra on EC2. I configured > Cassandra cluster on two instances in different regions. > But when I am trying the nodetool command with ring option, I am > getting only single node. > How to make these two nodes communicate with each other. I have > already opened required ports. i.e 7000, 8080, 9160 in respective > security groups. Plz help me with this. > Regards, > Himanshi Sharma > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are
Re: Cassandra nodes on EC2 in two different regions not communicating
internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east to us-west). To do regions, you must use the public ip address assigned by amazon. Himanshi, when you log into 1 node, and telnet to port 7000 on the other node, which IP address did you use - the 10.x address or the public ip address? And what is the seed/non-seed configuration in both cassandra.yaml files? Dave Viner On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio wrote: > The internal Amazon IP address is what you will want to use so you don't > have to go through DNS anyways; not sure if this works from US-East to > US-West, but it does make things quicker in between zones, e.g. us-east-1a > to us-east-1b. > > > On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner wrote: > >> Try using the IP address, not the dns name in the cassandra.yaml. >> >> If you can telnet from one to the other on port 7000, and both nodes have >> the other node in their config, it should work. >> >> Dave Viner >> >> >> On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma > > wrote: >> >>> >>> Ya they do. Have specified Public DNS in seed field of each node in >>> Cassandra.yaml...nt able to figure out what the problem is ??? >>> >>> >>> >>> >>> From: Sasha Dolgy To: user@cassandra.apache.org >>> Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two >>> different regions not communicating >>> -- >>> >>> >>> >>> did you define the other host in the cassandra.yaml ? on both servers >>> they need to know about each other >>> >>> On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma <* >>> himanshi.sha...@tcs.com* > wrote: >>> >>> Thanks Dave but I am able to telnet to other instances on port 7000 >>> and when i run ./nodetool --host * >>> ec2-50-18-60-117.us-west-1.compute.amazonaws.com*<http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> >>> ring... I can see only one node. >>> >>> Do we need to configure anything else in Cassandra.yaml or >>> Cassandra-env.sh ??? >>> >>> >>> >>> >>> >>> >>> From: Dave Viner <*davevi...@gmail.com* > To: * >>> user@cassandra.apache.org* Cc: Himanshi >>> Sharma <*himanshi.sha...@tcs.com* > Date: >>> 02/23/2011 >>> 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions >>> not communicating >>> >>> -- >>> >>> >>> >>> If you login to one of the nodes, can you telnet to port 7000 on the >>> other node? >>> >>> If not, then almost certainly it's a firewall/Security Group issue. >>> >>> You can find out the security groups for any node by logging in, and then >>> running: >>> >>> % curl >>> "*http://169.254.169.254/latest/meta-data/security-groups*<http://169.254.169.254/latest/meta-data/security-groups>" >>> >>> >>> Assuming that both nodes are in the same security group, ensure that the >>> SG is configured to allow other members of the SG to communicate on port >>> 7000 to each other. >>> >>> HTH, >>> Dave Viner >>> >>> >>> On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma <* >>> himanshi.sha...@tcs.com* > wrote: >>> >>> Hi, >>> >>> I am new to Cassandra. I m running Cassandra on EC2. I configured >>> Cassandra cluster on two instances in different regions. >>> But when I am trying the nodetool command with ring option, I am getting >>> only single node. >>> >>> How to make these two nodes communicate with each other. I have already >>> opened required ports. i.e 7000, 8080, 9160 in respective >>> security groups. Plz help me with this. >>> >>> Regards, >>> Himanshi Sharma >>> >>> >>> =-=-= >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have r
Re: Cassandra nodes on EC2 in two different regions not communicating
The internal Amazon IP address is what you will want to use so you don't have to go through DNS anyways; not sure if this works from US-East to US-West, but it does make things quicker in between zones, e.g. us-east-1a to us-east-1b. On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner wrote: > Try using the IP address, not the dns name in the cassandra.yaml. > > If you can telnet from one to the other on port 7000, and both nodes have > the other node in their config, it should work. > > Dave Viner > > > On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma > wrote: > >> >> Ya they do. Have specified Public DNS in seed field of each node in >> Cassandra.yaml...nt able to figure out what the problem is ??? >> >> >> >> >> From: Sasha Dolgy To: user@cassandra.apache.org Date: >> 02/23/2011 >> 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not >> communicating >> -- >> >> >> >> did you define the other host in the cassandra.yaml ? on both servers >> they need to know about each other >> >> On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma <* >> himanshi.sha...@tcs.com* > wrote: >> >> Thanks Dave but I am able to telnet to other instances on port 7000 >> and when i run ./nodetool --host * >> ec2-50-18-60-117.us-west-1.compute.amazonaws.com*<http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> >> ring... I can see only one node. >> >> Do we need to configure anything else in Cassandra.yaml or >> Cassandra-env.sh ??? >> >> >> >> >> >> >> From: Dave Viner <*davevi...@gmail.com* > To: * >> user@cassandra.apache.org* Cc: Himanshi >> Sharma <*himanshi.sha...@tcs.com* > Date: >> 02/23/2011 >> 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions >> not communicating >> >> -- >> >> >> >> If you login to one of the nodes, can you telnet to port 7000 on the other >> node? >> >> If not, then almost certainly it's a firewall/Security Group issue. >> >> You can find out the security groups for any node by logging in, and then >> running: >> >> % curl >> "*http://169.254.169.254/latest/meta-data/security-groups*<http://169.254.169.254/latest/meta-data/security-groups>" >> >> >> Assuming that both nodes are in the same security group, ensure that the >> SG is configured to allow other members of the SG to communicate on port >> 7000 to each other. >> >> HTH, >> Dave Viner >> >> >> On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma <* >> himanshi.sha...@tcs.com* > wrote: >> >> Hi, >> >> I am new to Cassandra. I m running Cassandra on EC2. I configured >> Cassandra cluster on two instances in different regions. >> But when I am trying the nodetool command with ring option, I am getting >> only single node. >> >> How to make these two nodes communicate with each other. I have already >> opened required ports. i.e 7000, 8080, 9160 in respective >> security groups. Plz help me with this. >> >> Regards, >> Himanshi Sharma >> >> >> =-=-= >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> >> >> =-=-= >> >> >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> >> >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> >> >> and any attachments. Thank you >> >> >> >> >> >> -- >> Sasha Dolgy* >> **sasha.do...@gmail.com* >> >> =-=-= >> Notice: The information contained in this e-mail >> message and/or attachments to it may contain >> confidential or privileged information. If you are >> >> not the intended recipient, any dissemination, use, >> review, distribution, printing or copying of the >> information contained in this e-mail message >> and/or attachments to it are strictly prohibited. If >> you have received this communication in error, >> >> please notify us by reply e-mail or telephone and >> immediately and permanently delete the message >> and any attachments. Thank you >> >> >> > -- Frank LoVecchio Senior Software Engineer | Isidorey, LLC Google Voice +1.720.295.9179 isidorey.com | facebook.com/franklovecchio | franklovecchio.com
Re: Cassandra nodes on EC2 in two different regions not communicating
Try using the IP address, not the dns name in the cassandra.yaml. If you can telnet from one to the other on port 7000, and both nodes have the other node in their config, it should work. Dave Viner On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma wrote: > > Ya they do. Have specified Public DNS in seed field of each node in > Cassandra.yaml...nt able to figure out what the problem is ??? > > > > > From: Sasha Dolgy To: user@cassandra.apache.org Date: > 02/23/2011 > 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > -- > > > > did you define the other host in the cassandra.yaml ? on both servers > they need to know about each other > > On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma <* > himanshi.sha...@tcs.com* > wrote: > > Thanks Dave but I am able to telnet to other instances on port 7000 > and when i run ./nodetool --host * > ec2-50-18-60-117.us-west-1.compute.amazonaws.com*<http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> > ring... I can see only one node. > > Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh > ??? > > > > > > > From: Dave Viner <*davevi...@gmail.com* > To: * > user@cassandra.apache.org* Cc: Himanshi > Sharma <*himanshi.sha...@tcs.com* > Date: 02/23/2011 > 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not > communicating > > -- > > > > If you login to one of the nodes, can you telnet to port 7000 on the other > node? > > If not, then almost certainly it's a firewall/Security Group issue. > > You can find out the security groups for any node by logging in, and then > running: > > % curl > "*http://169.254.169.254/latest/meta-data/security-groups*<http://169.254.169.254/latest/meta-data/security-groups>" > > > Assuming that both nodes are in the same security group, ensure that the SG > is configured to allow other members of the SG to communicate on port 7000 > to each other. > > HTH, > Dave Viner > > > On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma <*himanshi.sha...@tcs.com > * > wrote: > > Hi, > > I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra > cluster on two instances in different regions. > But when I am trying the nodetool command with ring option, I am getting > only single node. > > How to make these two nodes communicate with each other. I have already > opened required ports. i.e 7000, 8080, 9160 in respective > security groups. Plz help me with this. > > Regards, > Himanshi Sharma > > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > > > > =-=-= > > > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > > > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > > > and any attachments. Thank you > > > > > > -- > Sasha Dolgy* > **sasha.do...@gmail.com* > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > >
Re: Cassandra nodes on EC2 in two different regions not communicating
Ya they do. Have specified Public DNS in seed field of each node in Cassandra.yaml...nt able to figure out what the problem is ??? From: Sasha Dolgy To: user@cassandra.apache.org Date: 02/23/2011 02:56 PM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma wrote: Thanks Dave but I am able to telnet to other instances on port 7000 and when i run ./nodetool --host ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only one node. Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh ??? From: Dave Viner To: user@cassandra.apache.org Cc: Himanshi Sharma Date: 02/23/2011 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl "http://169.254.169.254/latest/meta-data/security-groups"; Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma wrote: Hi, I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra cluster on two instances in different regions. But when I am trying the nodetool command with ring option, I am getting only single node. How to make these two nodes communicate with each other. I have already opened required ports. i.e 7000, 8080, 9160 in respective security groups. Plz help me with this. Regards, Himanshi Sharma =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you -- Sasha Dolgy sasha.do...@gmail.com =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you
Re: Cassandra nodes on EC2 in two different regions not communicating
did you define the other host in the cassandra.yaml ? on both servers they need to know about each other On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma wrote: > > Thanks Dave but I am able to telnet to other instances on port 7000 > and when i run ./nodetool --host > ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only > one node. > > Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh > ??? > > > > > > > > From: Dave Viner To: user@cassandra.apache.org Cc: > Himanshi > Sharma Date: 02/23/2011 11:36 AM Subject: Re: > Cassandra nodes on EC2 in two different regions not communicating > -- > > > > If you login to one of the nodes, can you telnet to port 7000 on the other > node? > > If not, then almost certainly it's a firewall/Security Group issue. > > You can find out the security groups for any node by logging in, and then > running: > > % curl > "*http://169.254.169.254/latest/meta-data/security-groups*<http://169.254.169.254/latest/meta-data/security-groups> > " > > Assuming that both nodes are in the same security group, ensure that the SG > is configured to allow other members of the SG to communicate on port 7000 > to each other. > > HTH, > Dave Viner > > > On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma <*himanshi.sha...@tcs.com > * > wrote: > > Hi, > > I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra > cluster on two instances in different regions. > But when I am trying the nodetool command with ring option, I am getting > only single node. > > How to make these two nodes communicate with each other. I have already > opened required ports. i.e 7000, 8080, 9160 in respective > security groups. Plz help me with this. > > Regards, > Himanshi Sharma > > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > > > > =-=-= > > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > > and any attachments. Thank you > > > -- Sasha Dolgy sasha.do...@gmail.com
Re: Cassandra nodes on EC2 in two different regions not communicating
Thanks Dave but I am able to telnet to other instances on port 7000 and when i run ./nodetool --host ec2-50-18-60-117.us-west-1.compute.amazonaws.com ring... I can see only one node. Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh ??? From: Dave Viner To: user@cassandra.apache.org Cc: Himanshi Sharma Date: 02/23/2011 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions not communicating If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl "http://169.254.169.254/latest/meta-data/security-groups"; Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma wrote: Hi, I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra cluster on two instances in different regions. But when I am trying the nodetool command with ring option, I am getting only single node. How to make these two nodes communicate with each other. I have already opened required ports. i.e 7000, 8080, 9160 in respective security groups. Plz help me with this. Regards, Himanshi Sharma =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you
Re: Cassandra nodes on EC2 in two different regions not communicating
If you login to one of the nodes, can you telnet to port 7000 on the other node? If not, then almost certainly it's a firewall/Security Group issue. You can find out the security groups for any node by logging in, and then running: % curl "http://169.254.169.254/latest/meta-data/security-groups"; Assuming that both nodes are in the same security group, ensure that the SG is configured to allow other members of the SG to communicate on port 7000 to each other. HTH, Dave Viner On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma wrote: > > Hi, > > I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra > cluster on two instances in different regions. > But when I am trying the nodetool command with ring option, I am getting > only single node. > > How to make these two nodes communicate with each other. I have already > opened required ports. i.e 7000, 8080, 9160 in respective > security groups. Plz help me with this. > > Regards, > Himanshi Sharma > > > =-=-= > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > >
