Re: Ec2 snitch with network topology strategy
Hmm... Just tested the config. It works, got confused with the options, my bad. On Wed, Jun 29, 2011 at 2:26 PM, pankajsoni0126 wrote: > I was thinking of leveraging ec2 snitch. But my question is then how do I > give replica placement options? > > Or can I give snitch as ec2snitch and write the nodes > cassandra-topology.prop and in give locator strategy at time of creating > keyspace as network topology strategy. But will it work? > > And those who are struggling to deploy cassandra with across ec2 regions. > > 1. approach is to use milind's patch, it works but has some limitation. > https://issues.apache.org/jira/browse/CASSANDRA-2362 > 2. openvpn is a good option but neverthless is futile with encryption > available in 0.8.0 cassandra > 3. Vijay has come up with a patch and so far tested I have not seen any > jerks. > https://issues.apache.org/jira/browse/CASSANDRA-2452 - its marked to be > there in 0.8.2 release. > > > -pankaj > > > > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Ec2-snitch-with-network-topology-strategy-tp6528188p6528188.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com. >
Re: advice for EC2 deployment
hey, I have got my ec2 multi-dc across AZ's but in same region us-east. Now I am trying to deploy cassandra over multiple regions that is ec2 us west, singapore and us-east. I have edited the config file as sasha's reply below. though when I run nodetool in each DC, I only see the nodes from that region. That is EC2 US west is showing only 2 nodes which are up in that region but not the other 2 which are there in US-east. Kindly suggest a solution. -thanks On Wed, Apr 27, 2011 at 5:45 PM, Sasha Dolgy wrote: > Hi, > > If I understand you correctly, you are trying to get a private ip in > us-east speaking to the private ip in us-west. to make your life > easier, configure your nodes to use hostname of the server. if it's > in a different region, it will use the public ip (ec2 dns will handle > this for you) and if it's in the same region, it will use the private > ip. this way you can stop worrying about if you are using the public > or private ip to communicate with another node. let the aws dns do > the work for you. > > just make sure you are using v0.8 with SSL turned on and have the > appropriate security group definitions ... > > -sasha > > > > On Wed, Apr 27, 2011 at 1:55 PM, pankajsoni0126 > wrote: > > I have been trying to deploy Cassandra cluster across regions and for > that I > > posted this "IP address resolution in MultiDC setup". > > > > But when it is to get nodes talking to each other on different regions > say, > > us-east and us-west over private IP's of EC2 nodes I am facing problems. > > > > I am assuming if Cassandra is built for multi-DC setup it should be > easily > > deployed with node1's DC1's public IP listed as seed in all nodes in DC2 > and > > to gain idea about network topology? I have hit a dud for deployment in > such > > scenario. > > > > Or is it there any way possible to use Private IP's for such a scenario > in > > EC2, as Public Ip are less secure and costly? >
Re: IP address resolution in MultiDC setup
Hi, I have a question regarding Vyatta or any providing VIP in general. While routing through gateway do we bind it to ec2 nodes private IP or public IP? Also, in general could you explain how VIP might help for I am new towards this side of field. thanks On Mon, Apr 25, 2011 at 9:47 PM, Sasha Dolgy wrote: > honest opinion? smoke and mirrors. i really have no idea. i was > surprised to see the latency drop when we started using the VIP's we > assigned routing through our ec2 vyatta gateways. it makes it nice > because it unties you from being 100% stuck on amazon. you can design > your environment for cassandra with local nodes in an office if you > wanted ... it also solved the security problems i was coming across in > that before cassandra 0.8, intra-node communication IS NOT encrypted > or secured > > anyway .. the biggest thing for me was to ensure we are not tied to > one provider. this was the best for my business casealso allowed > us to not be harmed by the > https://twitter.com/#!/search/amazonpocalypse ... > > -sd > > > On Mon, Apr 25, 2011 at 6:11 PM, Milind Parikh > wrote: > > @Sasha > > Very interesting that you find a big difference in latency between nodes. > > Any hypothesis on what is going on in internal aws routing that makes it > > inefficient? > > Milind >
Re: IP address resolution in MultiDC setup
one last doubt is pending after reading your document: 1. when deploying cassandra across multiple dcs using your patch, is it possible to have internal network of nodes in each data center talking over private ip? then I assume the node with public ip will act as coordinator. But if it goes down the link between data centers will be down? could you clear this one. thnks pankaj On Mon, Apr 25, 2011 at 7:00 PM, pankaj soni wrote: > scrap the last mail, just finished reading Amazon ec2 resource policy. > > @milind when deploying cassandra across multiple dcs using your patch, is > it possible to have internal network of nodes in each data center talking > over private ip? > then I assume the node with public ip will act as co-ordinator. If it goes > down the link between data centers will be down? > > Thanks > pankaj > > > On Mon, Apr 25, 2011 at 6:09 PM, pankaj soni wrote: > >> Just read your paper on this. Must say helped a great deal. >> >> 1 more query does amazon by default award both external and internal IP >> address for each node? or we have to explicitly buy the external IP's? >> >> I am looking into overlay n/w's. >> >> >> On Mon, Apr 25, 2011 at 5:20 PM, Milind Parikh wrote: >> >>> I stand correctedI show how cassandra can be deployed in multiple dcs >>> through a simple patch; using public ips. In your scenario with an overlay >>> n/w, you will not require this patch. >>> >>> /*** >>> sent from my android...please pardon occasional typos as I respond @ the >>> speed of thought >>> / >>> >>> On Apr 25, 2011 7:43 AM, "Milind Parikh" wrote: >>> >>> I have authored exactly this paperplease search this ml. Please be >>> aware about ec2's internal network as you design your deployment. Ec2 also >>> does not support multicast; which is a pain,but not unsurmountable. >>> >>> >>> >>> /*** >>> sent from my android...please pardon occasional typos as I respond @ the >>> ... >>> >>> >>> > >>> > On Apr 25, 2011 7:31 AM, "pankaj soni" >>> wrote: >>> > >>> > We are expecting t... >>> >>> pankaj >>> >>> >>> > >>> > >>> > >>> > On Mon, Apr 25, 2011 at 4:55 PM, Milind Parikh >>> wrote: >>> > > >>> > It will be thro... >>> >>> >> >
Re: IP address resolution in MultiDC setup
scrap the last mail, just finished reading Amazon ec2 resource policy. @milind when deploying cassandra across multiple dcs using your patch, is it possible to have internal network of nodes in each data center talking over private ip? then I assume the node with public ip will act as co-ordinator. If it goes down the link between data centers will be down? Thanks pankaj On Mon, Apr 25, 2011 at 6:09 PM, pankaj soni wrote: > Just read your paper on this. Must say helped a great deal. > > 1 more query does amazon by default award both external and internal IP > address for each node? or we have to explicitly buy the external IP's? > > I am looking into overlay n/w's. > > > On Mon, Apr 25, 2011 at 5:20 PM, Milind Parikh wrote: > >> I stand correctedI show how cassandra can be deployed in multiple dcs >> through a simple patch; using public ips. In your scenario with an overlay >> n/w, you will not require this patch. >> >> /*** >> sent from my android...please pardon occasional typos as I respond @ the >> speed of thought >> / >> >> On Apr 25, 2011 7:43 AM, "Milind Parikh" wrote: >> >> I have authored exactly this paperplease search this ml. Please be >> aware about ec2's internal network as you design your deployment. Ec2 also >> does not support multicast; which is a pain,but not unsurmountable. >> >> >> >> /*** >> sent from my android...please pardon occasional typos as I respond @ the >> ... >> >> >> > >> > On Apr 25, 2011 7:31 AM, "pankaj soni" >> wrote: >> > >> > We are expecting t... >> >> pankaj >> >> >> > >> > >> > >> > On Mon, Apr 25, 2011 at 4:55 PM, Milind Parikh >> wrote: >> > > >> > It will be thro... >> >> >
Re: IP address resolution in MultiDC setup
Just read your paper on this. Must say helped a great deal. 1 more query does amazon by default award both external and internal IP address for each node? or we have to explicitly buy the external IP's? I am looking into overlay n/w's. On Mon, Apr 25, 2011 at 5:20 PM, Milind Parikh wrote: > I stand correctedI show how cassandra can be deployed in multiple dcs > through a simple patch; using public ips. In your scenario with an overlay > n/w, you will not require this patch. > > /*** > sent from my android...please pardon occasional typos as I respond @ the > speed of thought > / > > On Apr 25, 2011 7:43 AM, "Milind Parikh" wrote: > > I have authored exactly this paperplease search this ml. Please be > aware about ec2's internal network as you design your deployment. Ec2 also > does not support multicast; which is a pain,but not unsurmountable. > > > > /*** > sent from my android...please pardon occasional typos as I respond @ the > ... > > > > > > On Apr 25, 2011 7:31 AM, "pankaj soni" wrote: > > > > We are expecting t... > > pankaj > > > > > > > > > > On Mon, Apr 25, 2011 at 4:55 PM, Milind Parikh > wrote: > > > > > It will be thro... > >
Re: IP address resolution in MultiDC setup
Could you give the exact name of your paper. It will be easier to search. thanks On Mon, Apr 25, 2011 at 5:13 PM, Milind Parikh wrote: > I have authored exactly this paperplease search this ml. Please be > aware about ec2's internal network as you design your deployment. Ec2 also > does not support multicast; which is a pain,but not unsurmountable. > > /*** > sent from my android...please pardon occasional typos as I respond @ the > speed of thought > / > > On Apr 25, 2011 7:31 AM, "pankaj soni" wrote: > > We are expecting to deploy it on amazon cloud ec2, if it may help. I am > sure people would have deployed Cassandra data centers in different regions > on cloud before. But I am unable to find documentation of any such > deployment online. > > Because of this multi-regions the public-private IP address issue is > important. > > pankaj > > > > On Mon, Apr 25, 2011 at 4:55 PM, Milind Parikh > wrote: > > > > It will be thro... > >
Re: IP address resolution in MultiDC setup
We are expecting to deploy it on amazon cloud ec2, if it may help. I am sure people would have deployed Cassandra data centers in different regions on cloud before. But I am unable to find documentation of any such deployment online. Because of this multi-regions the public-private IP address issue is important. pankaj On Mon, Apr 25, 2011 at 4:55 PM, Milind Parikh wrote: > It will be through an overlay n/w. unfortunately setting up such n/w is > complex. Look @ something like openvpn. > > If multicast is supported, it will be easier. With complex software such as > Cassandra, it is much better to go with the expected flow; rather than > devicing your own flows.my2c. > > /*** > sent from my android...please pardon occasional typos as I respond @ the > speed of thought > ****/ > > On Apr 25, 2011 5:27 AM, "pankaj soni" wrote: > > Hi, > > We have a scenario for which we are considering using apache Cassandra for > deployment for our data storage needs.The setup is to be spread across > multiple data centers in different regions(physical locations). With each > data center having multiple nodes. However we can afford at most 1 public IP > address for each data center. With nodes inside the data center > communicating over private IP. We plan to use RF=3 and > OldNetworkTopologyStrategy > for replica placement. > > > 1 . This leads us to question that how will node discovery take place and > how will Cassandra ring be formed between multiple data centers? > > 2. How is data partitioning to be carried on in this scenario? > > 3. If say data resides in Data center 1 node 2 and read query is sent to > Data center 2 node 1, assuming it DC2 has no local replica than how is read > query to be serviced? This is our biggest concern as articles relating to > public/private IPs for cassandra could not be found. > > > As in Cassandra any node can be queried for data and same goes for write > requests, cassandra is our first choice in environments we have to deploy. > > Any suggestion is welcome. > > pankaj > >
IP address resolution in MultiDC setup
Hi, We have a scenario for which we are considering using apache Cassandra for deployment for our data storage needs.The setup is to be spread across multiple data centers in different regions(physical locations). With each data center having multiple nodes. However we can afford at most 1 public IP address for each data center. With nodes inside the data center communicating over private IP. We plan to use RF=3 and OldNetworkTopologyStrategy for replica placement. 1 . This leads us to question that how will node discovery take place and how will Cassandra ring be formed between multiple data centers? 2. How is data partitioning to be carried on in this scenario? 3. If say data resides in Data center 1 node 2 and read query is sent to Data center 2 node 1, assuming it DC2 has no local replica than how is read query to be serviced? This is our biggest concern as articles relating to public/private IPs for cassandra could not be found. As in Cassandra any node can be queried for data and same goes for write requests, cassandra is our first choice in environments we have to deploy. Any suggestion is welcome. pankaj
