I took a few minutes to update the ipv6 draft spec. https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+VPC+Router
On Fri, Jan 17, 2014 at 2:20 PM, Marcus Sorensen <shadow...@gmail.com> wrote: > On Fri, Jan 17, 2014 at 11:59 AM, Chiradeep Vittal > <chiradeep.vit...@citrix.com> wrote: >> Yes, let's not shut out the ULA option. >> >> For the LB "Public" interface, will this be just another tier? That is, >> not an explicit public VLAN. > > Well, we do have the internal LB function, but I was thinking more > along the lines of what we have now with IPv4, that is, there's a > public network that all IPv4 VPCs are already attaching to, we just > need to drop an IPv6 range on that so that VPCs can grab addresses for > themselves and any routing/LB/NAT66 they may want to do. > >> >> Since IPV6 support is still sparse, it is probably best that we support >> dual stack for now. >> For instance some repositories have flaky IPv6. Also, within corporations >> is still mostly ipv4 and they will want to VPN in and provide services >> (e.g., Active Directory) from within their data center. >> >> Another aspect we should keep in mind is security. For example: >> Rogue IPv6 RA [http://tools.ietf.org/html/rfc6104] >> ND exhaustion [ >> http://blog.ipspace.net/2011/05/ipv6-neighbor-discovery-exhaustion.html] >> Unicast RPF [ >> http://www.cisco.com/web/about/security/intelligence/unicast-rpf.html] >> >> >> On 1/16/14 11:45 PM, "Marcus Sorensen" <shadow...@gmail.com> wrote: >> >>>Good point. These are the kinds of things we should be aware of right >>>now. Even though we don't necessarily want to bite off too much right >>>now in initial implementation, we want to plan so that certain things >>>we may want to do aren't impossible. >>> >>>I was thinking either the cloud provider would have a PI, or the end >>>user may bring their own to the cloud provider. Either way, >>>arrangements need to be made to get it routed up to the point where >>>the VPC can service it, and entered as the 'global cidr' for that VPC >>>(or at least a small portion of the PI). If we don't, then we have to >>>NAT, which as you mentioned isn't really widely supported. I suppose >>>nothing would preclude us from allowing a ULA space to be entered as >>>the VPC global space later on if we wanted to support NAT for IPv6 on >>>the router at some point. You could do either and we'd just route >>>public space and NAT/port forward private space. >>> >>>So, to recap a bit: >>> >>>VPC is assigned a prefix (say /60). Admin assigns sub-prefix from >>>that, per network tier, down to a /64. Route information is published >>>both to a plugin framework and the event bus to enable automation of >>>upstream route programming (point the /60 to the 'public traffic' ip >>>of the VPC router), or admin can manually set up the upstream route >>>per VPC. >>> >>>DNS can be provided by the (currently required) IPv4 recursor on the >>>VPC router, but ultimately dnsmasq on IPv6 address can also provide >>>it, looking forward to IPv6-only networks. >>> >>>'public traffic' addressing for VPC routers (the internet-facing >>>interface) will be handled by static assignment from a public range, >>>much how the IPv4 public traffic is now. This allows for assigning >>>extra LB addresses or static NAT addresses in the future. >>> >>>guest networks, my initial preference would be for SLAAC, but I think >>>ultimately we'd want to be able to assign multiple ips to a guest. >>>With the 64 bits of the SLAAC space dedicated to all of the unique MAC >>>address possibilities, we can't really do that. We may want to >>>consider DHCP with static assignments from the beginning. >>> >>>On Thu, Jan 16, 2014 at 11:32 PM, Chiradeep Vittal >>><chiradeep.vit...@citrix.com> wrote: >>>> Are we assuming that the Cloud Provider has a Provider-Independent (PI) >>>> space? >>>> Using a Unique Local Address (ULA) space for the VPC *might* to >>>>desirable. >>>> If VPC tiers (subnets) can span zones (not possible today), then NAT66 >>>>can >>>> help keep addresses stable as workloads move to a different zone for >>>> availability reasons. >>>> But I see that only Ubuntu has support for NAT66. >>>> >>>> >>>> On 1/16/14 11:33 AM, "Sheng Yang" <sh...@yasker.org> wrote: >>>> >>>>>On Thu, Jan 9, 2014 at 7:04 AM, Daan Hoogland >>>>><dhoogl...@schubergphilis.com>wrote: >>>>> >>>>>> H Marcus, We had a small session on how we plan to go about ipv6 >>>>>> configuration with your bullet list present. Comments are still >>>>>> brainstorming phase quality but hopefully they will lead to something. >>>>>> >>>>>> > * VPC has no global IPv6 prefix (super CIDR as current private >>>>>>space), >>>>>> it's simply IPv6 enabled or not. Admins can choose to route a /60 or a >>>>>>/48 >>>>>> to a vpc and carve it up among the networks there, but it's not >>>>>>required or >>>>>> enforced by cloudstack. >>>>>> The idea at Schuberg Philis is that VPCs should have a standard size >>>>>> network assigned to be taken from a pool. We should be able to >>>>>>configure >>>>>> the width of the network. The idea is that per level (Physical >>>>>> network/vpc/tier) only one width will be used but the size of it is >>>>>> configurable. Default block for a VR / VPC would be a /56 and each >>>>>> individual network would use a /64. Note that in a typical isolated >>>>>>network >>>>>> we only use the first. >>>>>> >>>>>> > * VPC networks get assigned one or multiple IPv6 prefixes, each >>>>>>prefix >>>>>> can be marked 'SLAAC', 'DHCP', or 'Manual'. >>>>>> For networks we are in favor of using SLAAC for configuring routing >>>>>>and >>>>>> addressing, but would advise DHCPv6 to configure DNS and potentially >>>>>>other >>>>>> things (NTP?). No need to register the addresses in CloudStack. We >>>>>>already >>>>>> store the MAC of the NIC so we know which address will be configured >>>>>>on >>>>>>the >>>>>> interface with the configured prefix. Only the router will receive a >>>>>> specific address (prefix::1/64) >>>>>> >>>>>> As mentioned earlier first implementation will do SLAAC. How will >>>>>>SLAAC >>>>>>be >>>>>> configured with ACL's? >>>>>> >>>>>> > * Mgmt server allows calling external plugins for pushing routes to >>>>>> routers upstream of VPCs (SDN or router API), but could be manually >>>>>>done by >>>>>> admins as well >>>>>> We discussed the routing issues in front of the VPC / VR. The goal is >>>>>>to >>>>>> provide cloudstack with a way to deal with routable addresses inside >>>>>>an >>>>>> isolated network, as actually this problem is relevant for both IPv4 >>>>>>and >>>>>> IPv6. However for IPv4 we have a perfectly workable solution with NAT >>>>>>and >>>>>> in the IPv6 world we don't/. So we need to address this issue on how >>>>>>to >>>>>> route a network from the provider edge into an isolated network. >>>>>> >>>>>> For IPv6 we have several options: >>>>>> * Dynamic routing protocols, here we can think of iBGP, OSPF or >>>>>>any >>>>>> other protocol. Basically cloudstack assigns a block to a VR / VPC and >>>>>>the >>>>>> VPC / VR tells the outside world to route that block to the outside IP >>>>>>of >>>>>> the router. >>>>>> * Block allocator, cloudstack knows about the supernet (/48 for >>>>>> example) and assigns a block (/56) to a VR or VPC. Cloudstack tells >>>>>>the >>>>>> block allocator to allocate (route) this block to the VR/VPC. Block >>>>>> allocater needs to know about the device being managed (upstream >>>>>>router >>>>>> like cisco/juniper) and configured the route using telnet or an API >>>>>> * Broker, We introduce a new systemvm, the block broker. The >>>>>>admin >>>>>> routes the /48 to the block broker and CloudStack tells the blok >>>>>>broker >>>>>>to >>>>>> reroute a specific subblock to this VR / VPC >>>>>> * Static, Admin configures a list of block/ip pairs in the >>>>>>upstream >>>>>> router. CloudStack knows about these pairs and assigns the ip to the >>>>>>VR >>>>>>/ >>>>>> VPC and the block will be routed. >>>>>> >>>>>> >>>>>> > * Work could be done in stages, e.g. SLAAC/manual network ranges >>>>>>would >>>>>> be fairly straightforward, whereas DHCP ranges would require >>>>>>programming >>>>>> scripts and ip allocation code. >>>>>> >>>>>> > * An issue was raised about privacy concerns with SLAAC using MAC, >>>>>>but >>>>>> we think this revolves more around clients than servers; that is, a >>>>>>client >>>>>> moving around the country would be traceable because of the MAC, but a >>>>>> server always has the same address anyway. >>>>>> >>>>>> > * Still need to figure out what to do about ACLs, that's a whole >>>>>> separate issue, but a plan needs to be in place. >>>>>> Are we going to put ACL's on networks or hosts or both? What is >>>>>>easiest >>>>>>to >>>>>> implement and enforce? >>>>>> >>>>> >>>>>The routing is controlled by upstream router, so it's straightforward >>>>>that >>>>>ACL would be done by upstream router. >>>>> >>>>>But after rethink, modifying the upstream router automatically make me >>>>>feel >>>>>unsafe... Probably host side is a better choice, though it would be more >>>>>work to do. >>>>> >>>>> >>>>>> Do we care about portforwarding, static NAT, Load Balancing etc as >>>>>>well? >>>>>> Or at least think >>>>>> about what impact these decisions have. >>>>>> This is not a version 1 consideration?!? >>>>>> >>>>>> > * We assume there will be an ipv6 public range assignable, allocated >>>>>>for >>>>>> VPC routers/static nat/loadbalancers to pull from. How will this range >>>>>>be >>>>>> made available? >>>>>> Is a cidr configured? >>>>>> Will we preconfigure all ranges or have a dynamic availability check? >>>>>> >>>>>> Radvd package is added to the systemvm template. (Done by Hugo this >>>>>> morning) >>>>>> >>>>>> Some standard UI components are needed. >>>>>> >>>>>> (test-)attention needs to go to: >>>>>> >>>>>> HaProxy version/feature matrix has to be reviewed to check for ipv6 >>>>>> compatability. >>>>> >>>>> >>>>>> Does dnsmasq work with ipv6? >>>>>> >>>>> >>>>>Yes. Already use it for IPv6 shared network implementation. >>>>> >>>>> >>>>>> >>>>>> Does keepalived work seemslessly? >>>>>> >>>>> >>>>>According to http://www.keepalived.org/, it should work. But seems works >>>>>are still going on. >>>>> >>>>>--Sheng >>>>> >>>>> >>>>>> Ipv6 tables configuration needs to be pushed to the VRs >>>>>> >>>>>> -----Original Message----- >>>>>> From: Marcus Sorensen [mailto:shadow...@gmail.com] >>>>>> Sent: maandag 6 januari 2014 9:11 >>>>>> To: cloudstack-...@incubator.apache.org >>>>>> Subject: IPv6 in VPC (was Re: IPv6 plan - questions) >>>>>> >>>>>> I've discussed this a bit with various subject matter experts at our >>>>>> datacenters/business, and so far we're leaning toward a rollout like >>>>>> this: >>>>>> >>>>>> * VPC has no global IPv6 prefix (super CIDR as current private space), >>>>>> it's simply IPv6 enabled or not. Admins can choose to route a /60 or a >>>>>> /48 to a vpc and carve it up among the networks there, but it's not >>>>>> required or enforced by cloudstack. >>>>>> >>>>>> * VPC networks get assigned one or multiple IPv6 prefixes, each prefix >>>>>>can >>>>>> be marked 'SLAAC', 'DHCP', or 'Manual'. >>>>>> >>>>>> * Mgmt server allows calling external plugins for pushing routes to >>>>>> routers upstream of VPCs (SDN or router API), but could be manually >>>>>>done by >>>>>> admins as well >>>>>> >>>>>> * Work could be done in stages, e.g. SLAAC/manual network ranges would >>>>>>be >>>>>> fairly straightforward, whereas DHCP ranges would require programming >>>>>> scripts and ip allocation code. >>>>>> >>>>>> * An issue was raised about privacy concerns with SLAAC using MAC, but >>>>>>we >>>>>> think this revolves more around clients than servers; that is, a >>>>>>client >>>>>> moving around the country would be traceable because of the MAC, but a >>>>>> server always has the same address anyway. >>>>>> >>>>>> * Still need to figure out what to do about ACLs, that's a whole >>>>>>separate >>>>>> issue, but a plan needs to be in place. Do we care about port >>>>>>forwarding, >>>>>> static NAT, Load Balancing etc as well? Or at least think about what >>>>>>impact >>>>>> these decisions have. >>>>>> >>>>>> * We assume there will be an ipv6 public range assignable, allocated >>>>>>for >>>>>> VPC routers/static nat/loadbalancers to pull from. >>>>>> >>>>>> On Sat, Jan 4, 2014 at 6:11 AM, Marcus Sorensen <shadow...@gmail.com> >>>>>> wrote: >>>>>> > I've put together a rough draft spec: >>>>>> > >>>>>> > >>>>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+VPC+Rou >>>>>> > ter >>>>>> > >>>>>> > I basically just laid out some rough ideas. I know there has been a >>>>>> > lot of discussion in the past about DHCPv6, etc. My hope is that we >>>>>> > can at least decide on a spec, for future reference. >>>>>> > >>>>>> > >>>>>> > On Fri, Jan 3, 2014 at 9:53 PM, Marcus Sorensen >>>>>><shadow...@gmail.com> >>>>>> wrote: >>>>>> >> It's been a long time since I've heard anything in regards to IPv6, >>>>>> >> let alone VPC support. Does anyone have plans for this at all? >>>>>>We'd >>>>>> >> like to support IPv6, and we have enough CS knowledge and external >>>>>> >> tools to hack something together, but I'd much prefer to build with >>>>>> >> the community and/or be forward compatible with what it deploys. >>>>>> >> >>>>>> >> I'd like to start with something simple, like perhaps optionally >>>>>> >> providing a /64 or larger as a parameter when creating a VPC (or a >>>>>> >> separate call to add an IPV6 block), and network on the vpc. Then >>>>>>it >>>>>> >> sounds like there's already a mechanism in place for tracking ipv6 >>>>>> >> assignments to nics, that could be leveraged to pass dhcp >>>>>>assignments >>>>>> >> to routers. >>>>>> >> >>>>>> >> Then there's the whole acl thing, that seems like at least as big >>>>>>of >>>>>> >> a project as mentioned previously. >>>>>> >> >>>>>> >> On Mon, Aug 12, 2013 at 3:47 PM, Marcus Sorensen >>>>>><shadow...@gmail.com> >>>>>> wrote: >>>>>> >>> has there been any further discussion that I might have missed >>>>>> >>> around >>>>>> >>> ipv6 in VPC? >>>>>> >>> >>>>>> >>> On Thu, Mar 7, 2013 at 12:09 PM, Sheng Yang <sh...@yasker.org> >>>>>>wrote: >>>>>> >>>> Hi Dave, >>>>>> >>>> >>>>>> >>>> I am glad it fits your need. That's our target. :) >>>>>> >>>> >>>>>> >>>> --Sheng >>>>>> >>>> >>>>>> >>>> On Thu, Mar 7, 2013 at 2:14 AM, Dave Cahill >>>>>><dcah...@midokura.com> >>>>>> wrote: >>>>>> >>>>> Hi Sheng, >>>>>> >>>>> >>>>>> >>>>> Thanks for the quick reply, that helps a lot. >>>>>> >>>>> >>>>>> >>>>> My main purpose was to figure out how these changes affect >>>>>>virtual >>>>>> >>>>> networking and pluggability. Having read through the IPv6 code >>>>>> >>>>> today, it looks like it will work very nicely with virtual >>>>>>networks. >>>>>> >>>>> >>>>>> >>>>> For example, when VMs are assigned an IPv6 address, the IPv6 >>>>>> >>>>> address is stored in the NicProfile object. So, taking DHCP as >>>>>>an >>>>>> >>>>> example, if the MidoNet plugin implements the >>>>>>DHCPServiceProvider >>>>>> >>>>> interface, it will receive the NicProfile as one of the >>>>>>parameters >>>>>> of addDhcpEntry. >>>>>> >>>>> If we want to implement IPv6, we can then take the IPv6 address >>>>>> >>>>> from the NicProfile, and just use it as needed. >>>>>> >>>>> >>>>>> >>>>> Thanks again for taking the time to respond, and for the >>>>>>detailed >>>>>>FS. >>>>>> >>>>> >>>>>> >>>>> Dave. >>>>>> >>>>> >>>>>> >>>>> On Thu, Mar 7, 2013 at 4:57 AM, Sheng Yang <sh...@yasker.org> >>>>>>wrote: >>>>>> >>>>> >>>>>> >>>>>> On Wed, Mar 6, 2013 at 1:36 AM, Dave Cahill >>>>>><dcah...@midokura.com> >>>>>> wrote: >>>>>> >>>>>> > Hi, >>>>>> >>>>>> >>>>>> >>>>>> Hi Dave, >>>>>> >>>>>> > >>>>>> >>>>>> > I've been catching up on IPv6 plans by reading the functional >>>>>> >>>>>> > specs and Jira tickets - it's great to have so much material >>>>>>to >>>>>> refer to. >>>>>> >>>>>> > >>>>>> >>>>>> > I still have a few questions though, and I'm hoping someone >>>>>> >>>>>> > involved with the feature can enlighten me. >>>>>> >>>>>> > >>>>>> >>>>>> > *[Support for Providers other than Virtual Router]* In [3], >>>>>>the >>>>>> >>>>>> > spec says "No external device support in plan." >>>>>> >>>>>> > What does this mean exactly? >>>>>> >>>>>> >>>>>> >>>>>> Because CloudStack also supports using external devices as >>>>>> >>>>>> network controller e.g. Juniper SRX as firewall and NetScaler >>>>>>as >>>>>> >>>>>> load balancer. The words here said is just we don't support >>>>>>these >>>>>> >>>>>> devices when using IPv6. >>>>>> >>>>>> > >>>>>> >>>>>> > For example, if using Providers other than the Virtual >>>>>>Router, >>>>>> >>>>>> > does the UI still allow setting IPv6 addresses? >>>>>> >>>>>> > >>>>>> >>>>>> > If so, do we attempt to pass IPv6 addresses to the Providers >>>>>>no >>>>>> >>>>>> > matter what, or do we check whether the Provider has IPv6 >>>>>>support? >>>>>> >>>>>> >>>>>> >>>>>> Yes, we checked it when you try to create a IPv6 >>>>>> >>>>>> network(currently only support advance shared network). >>>>>> >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>> > *[Networking Modes]* >>>>>> >>>>>> > Advanced Shared mode and Basic mode are mentioned in the Jira >>>>>> >>>>>> > ticket [1] - "Isolated Network" is mentioned briefly in [2], >>>>>> >>>>>> > but I wanted to check if the Advanced Isolated and VPC modes >>>>>> >>>>>> > are on the roadmap? >>>>>> >>>>>> >>>>>> >>>>>> There is no "basic isolated" network, so "Isolated" network is >>>>>> >>>>>> what we're talking about. We haven't got plan for VPC yet. >>>>>> >>>>>> >>>>>> >>>>>> And one correction: we didn't support "basic" mode for phase 1. >>>>>> >>>>>> We support only "advance shared network" in phase 1. The >>>>>> >>>>>> supported cases are described in FS. Jira ticket only provided >>>>>>a >>>>>> >>>>>> rough idea at the time. >>>>>> >>>>>> > >>>>>> >>>>>> > *[IP Address Management / IPAM]* From [1], re: handing out >>>>>>IPv6 >>>>>> >>>>>> > addresses: "One way could be that the network admin creates a >>>>>> >>>>>> > static route for a /48 towards a Virtual Router and then the >>>>>>VR >>>>>> >>>>>> > can hand out /64s to Instances." >>>>>> >>>>>> > >>>>>> >>>>>> > With IPv4, IPAM is handled by the CloudStack management >>>>>>server, >>>>>> >>>>>> > and the VR is told which IP address to give to the VM over >>>>>> >>>>>> > DHCP. Would this change with IPv6? "The VR can hand out /64s >>>>>>to >>>>>> >>>>>> > instances" sounds like the VR is handling IPAM to some >>>>>>extent. >>>>>> >>>>>> >>>>>> >>>>>> Well, it's not how it works now. Please refer to the FS. The >>>>>> >>>>>> current implementation works like before. VR get a /64 then >>>>>> >>>>>> handle out IPv6 addresses to VM. >>>>>> >>>>>> > >>>>>> >>>>>> > From [3], "Router advertisement should be sent by public >>>>>> >>>>>> > gateway in the network." - to double-check, does this mean >>>>>>the >>>>>> >>>>>> > router outside the CloudStack network should send RAs, but >>>>>>the >>>>>>VR >>>>>> won't send RAs? >>>>>> >>>>>> >>>>>> >>>>>> Yes. Because in phase 1, we support only "advance shared >>>>>> >>>>>> network", in which case, VR is NOT the gateway. So we assume >>>>>>the >>>>>> >>>>>> gateway router outside CloudStack should send out RA to the >>>>>>VMs. >>>>>> >>>>>> >>>>>> >>>>>> But in the phase 2, VR would acting as gateway, then it would >>>>>>send >>>>>> out RAs. >>>>>> >>>>>> >>>>>> >>>>>> --Sheng >>>>>> >>>>>> > >>>>>> >>>>>> > Thanks, >>>>>> >>>>>> > Dave. >>>>>> >>>>>> > >>>>>> >>>>>> > [1] IPv6 Support main Jira ticket >>>>>> >>>>>> > https://issues.apache.org/jira/browse/CLOUDSTACK-452 >>>>>> >>>>>> > >>>>>> >>>>>> > [2] IPv6 Support in CloudStack FS >>>>>> >>>>>> > >>>>>>https://cwiki.apache.org/CLOUDSTACK/ipv6-support-in-cloudstack. >>>>>> >>>>>> > html >>>>>> >>>>>> > >>>>>> >>>>>> > [3] IPv6 Support FS >>>>>> >>>>>> > https://cwiki.apache.org/CLOUDSTACK/ipv6-support.html >>>>>> >>>>>> >>>>>> >>>> >>
