On Thu, Jan 16, 2014 at 11:19 PM, Marcus Sorensen <shadow...@gmail.com>wrote:
> On Thu, Jan 16, 2014 at 12:33 PM, Sheng Yang <sh...@yasker.org> wrote: > > On Thu, Jan 9, 2014 at 7:04 AM, Daan Hoogland < > dhoogl...@schubergphilis.com> > > wrote: > >> > >> H Marcus, We had a small session on how we plan to go about ipv6 > >> configuration with your bullet list present. Comments are still > >> brainstorming phase quality but hopefully they will lead to something. > >> > >> > * VPC has no global IPv6 prefix (super CIDR as current private space), > >> > it's simply IPv6 enabled or not. Admins can choose to route a /60 or > a /48 > >> > to a vpc and carve it up among the networks there, but it's not > required or > >> > enforced by cloudstack. > >> The idea at Schuberg Philis is that VPCs should have a standard size > >> network assigned to be taken from a pool. We should be able to > configure the > >> width of the network. The idea is that per level (Physical > >> network/vpc/tier) only one width will be used but the size of it is > >> configurable. Default block for a VR / VPC would be a /56 and each > >> individual network would use a /64. Note that in a typical isolated > network > >> we only use the first. > >> > >> > * VPC networks get assigned one or multiple IPv6 prefixes, each prefix > >> > can be marked 'SLAAC', 'DHCP', or 'Manual'. > >> For networks we are in favor of using SLAAC for configuring routing and > >> addressing, but would advise DHCPv6 to configure DNS and potentially > other > >> things (NTP?). No need to register the addresses in CloudStack. We > already > >> store the MAC of the NIC so we know which address will be configured on > the > >> interface with the configured prefix. Only the router will receive a > >> specific address (prefix::1/64) > >> > >> As mentioned earlier first implementation will do SLAAC. How will SLAAC > be > >> configured with ACL's? > >> > >> > * Mgmt server allows calling external plugins for pushing routes to > >> > routers upstream of VPCs (SDN or router API), but could be manually > done by > >> > admins as well > >> We discussed the routing issues in front of the VPC / VR. The goal is to > >> provide cloudstack with a way to deal with routable addresses inside an > >> isolated network, as actually this problem is relevant for both IPv4 and > >> IPv6. However for IPv4 we have a perfectly workable solution with NAT > and in > >> the IPv6 world we don't/. So we need to address this issue on how to > route a > >> network from the provider edge into an isolated network. > >> > >> For IPv6 we have several options: > >> * Dynamic routing protocols, here we can think of iBGP, OSPF or > any > >> other protocol. Basically cloudstack assigns a block to a VR / VPC and > the > >> VPC / VR tells the outside world to route that block to the outside IP > of > >> the router. > >> * Block allocator, cloudstack knows about the supernet (/48 for > >> example) and assigns a block (/56) to a VR or VPC. Cloudstack tells the > >> block allocator to allocate (route) this block to the VR/VPC. Block > >> allocater needs to know about the device being managed (upstream router > like > >> cisco/juniper) and configured the route using telnet or an API > >> * Broker, We introduce a new systemvm, the block broker. The admin > >> routes the /48 to the block broker and CloudStack tells the blok broker > to > >> reroute a specific subblock to this VR / VPC > >> * Static, Admin configures a list of block/ip pairs in the > upstream > >> router. CloudStack knows about these pairs and assigns the ip to the VR > / > >> VPC and the block will be routed. > >> > >> > >> > * Work could be done in stages, e.g. SLAAC/manual network ranges would > >> > be fairly straightforward, whereas DHCP ranges would require > programming > >> > scripts and ip allocation code. > >> > >> > * An issue was raised about privacy concerns with SLAAC using MAC, but > >> > we think this revolves more around clients than servers; that is, a > client > >> > moving around the country would be traceable because of the MAC, but a > >> > server always has the same address anyway. > >> > >> > * Still need to figure out what to do about ACLs, that's a whole > >> > separate issue, but a plan needs to be in place. > >> Are we going to put ACL's on networks or hosts or both? What is easiest > to > >> implement and enforce? > > > > > > The routing is controlled by upstream router, so it's straightforward > that > > ACL would be done by upstream router. > > I think we're talking about ACLs as in the way ACLs are currently > implemented in VPC. The VPC router would block, for example, traffic > from it's public interface into a specific tier, via a rule that > Cloudstack manages. Also, we're easily in control here, whereas it's > difficult to do upstream. > Um, yes, I forgot the VPC router would still control the traffic for network, through router advertisement. Then it would be easily done as today. --Sheng > > > > > But after rethink, modifying the upstream router automatically make me > feel > > unsafe... Probably host side is a better choice, though it would be more > > work to do. > > > >> > >> Do we care about portforwarding, static NAT, Load Balancing etc as well? > >> Or at least think > >> about what impact these decisions have. > >> This is not a version 1 consideration?!? > >> > >> > * We assume there will be an ipv6 public range assignable, allocated > for > >> > VPC routers/static nat/loadbalancers to pull from. How will this > range be > >> > made available? > >> Is a cidr configured? > >> Will we preconfigure all ranges or have a dynamic availability check? > >> > >> Radvd package is added to the systemvm template. (Done by Hugo this > >> morning) > >> > >> Some standard UI components are needed. > >> > >> (test-)attention needs to go to: > >> > >> HaProxy version/feature matrix has to be reviewed to check for ipv6 > >> compatability. > >> > >> > >> Does dnsmasq work with ipv6? > > > > > > Yes. Already use it for IPv6 shared network implementation. > > > >> > >> > >> Does keepalived work seemslessly? > > > > > > According to http://www.keepalived.org/, it should work. But seems > works are > > still going on. > > > > --Sheng > > > >> > >> Ipv6 tables configuration needs to be pushed to the VRs > >> > >> -----Original Message----- > >> From: Marcus Sorensen [mailto:shadow...@gmail.com] > >> Sent: maandag 6 januari 2014 9:11 > >> To: cloudstack-...@incubator.apache.org > >> Subject: IPv6 in VPC (was Re: IPv6 plan - questions) > >> > >> I've discussed this a bit with various subject matter experts at our > >> datacenters/business, and so far we're leaning toward a rollout like > >> this: > >> > >> * VPC has no global IPv6 prefix (super CIDR as current private space), > >> it's simply IPv6 enabled or not. Admins can choose to route a /60 or a > >> /48 to a vpc and carve it up among the networks there, but it's not > >> required or enforced by cloudstack. > >> > >> * VPC networks get assigned one or multiple IPv6 prefixes, each prefix > can > >> be marked 'SLAAC', 'DHCP', or 'Manual'. > >> > >> * Mgmt server allows calling external plugins for pushing routes to > >> routers upstream of VPCs (SDN or router API), but could be manually > done by > >> admins as well > >> > >> * Work could be done in stages, e.g. SLAAC/manual network ranges would > be > >> fairly straightforward, whereas DHCP ranges would require programming > >> scripts and ip allocation code. > >> > >> * An issue was raised about privacy concerns with SLAAC using MAC, but > we > >> think this revolves more around clients than servers; that is, a client > >> moving around the country would be traceable because of the MAC, but a > >> server always has the same address anyway. > >> > >> * Still need to figure out what to do about ACLs, that's a whole > separate > >> issue, but a plan needs to be in place. Do we care about port > forwarding, > >> static NAT, Load Balancing etc as well? Or at least think about what > impact > >> these decisions have. > >> > >> * We assume there will be an ipv6 public range assignable, allocated for > >> VPC routers/static nat/loadbalancers to pull from. > >> > >> On Sat, Jan 4, 2014 at 6:11 AM, Marcus Sorensen <shadow...@gmail.com> > >> wrote: > >> > I've put together a rough draft spec: > >> > > >> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/IPv6+in+VPC+Rou > >> > ter > >> > > >> > I basically just laid out some rough ideas. I know there has been a > >> > lot of discussion in the past about DHCPv6, etc. My hope is that we > >> > can at least decide on a spec, for future reference. > >> > > >> > > >> > On Fri, Jan 3, 2014 at 9:53 PM, Marcus Sorensen <shadow...@gmail.com> > >> > wrote: > >> >> It's been a long time since I've heard anything in regards to IPv6, > >> >> let alone VPC support. Does anyone have plans for this at all? We'd > >> >> like to support IPv6, and we have enough CS knowledge and external > >> >> tools to hack something together, but I'd much prefer to build with > >> >> the community and/or be forward compatible with what it deploys. > >> >> > >> >> I'd like to start with something simple, like perhaps optionally > >> >> providing a /64 or larger as a parameter when creating a VPC (or a > >> >> separate call to add an IPV6 block), and network on the vpc. Then it > >> >> sounds like there's already a mechanism in place for tracking ipv6 > >> >> assignments to nics, that could be leveraged to pass dhcp assignments > >> >> to routers. > >> >> > >> >> Then there's the whole acl thing, that seems like at least as big of > >> >> a project as mentioned previously. > >> >> > >> >> On Mon, Aug 12, 2013 at 3:47 PM, Marcus Sorensen < > shadow...@gmail.com> > >> >> wrote: > >> >>> has there been any further discussion that I might have missed > >> >>> around > >> >>> ipv6 in VPC? > >> >>> > >> >>> On Thu, Mar 7, 2013 at 12:09 PM, Sheng Yang <sh...@yasker.org> > wrote: > >> >>>> Hi Dave, > >> >>>> > >> >>>> I am glad it fits your need. That's our target. :) > >> >>>> > >> >>>> --Sheng > >> >>>> > >> >>>> On Thu, Mar 7, 2013 at 2:14 AM, Dave Cahill <dcah...@midokura.com> > >> >>>> wrote: > >> >>>>> Hi Sheng, > >> >>>>> > >> >>>>> Thanks for the quick reply, that helps a lot. > >> >>>>> > >> >>>>> My main purpose was to figure out how these changes affect virtual > >> >>>>> networking and pluggability. Having read through the IPv6 code > >> >>>>> today, it looks like it will work very nicely with virtual > networks. > >> >>>>> > >> >>>>> For example, when VMs are assigned an IPv6 address, the IPv6 > >> >>>>> address is stored in the NicProfile object. So, taking DHCP as an > >> >>>>> example, if the MidoNet plugin implements the DHCPServiceProvider > >> >>>>> interface, it will receive the NicProfile as one of the parameters > >> >>>>> of addDhcpEntry. > >> >>>>> If we want to implement IPv6, we can then take the IPv6 address > >> >>>>> from the NicProfile, and just use it as needed. > >> >>>>> > >> >>>>> Thanks again for taking the time to respond, and for the detailed > >> >>>>> FS. > >> >>>>> > >> >>>>> Dave. > >> >>>>> > >> >>>>> On Thu, Mar 7, 2013 at 4:57 AM, Sheng Yang <sh...@yasker.org> > wrote: > >> >>>>> > >> >>>>>> On Wed, Mar 6, 2013 at 1:36 AM, Dave Cahill < > dcah...@midokura.com> > >> >>>>>> wrote: > >> >>>>>> > Hi, > >> >>>>>> > >> >>>>>> Hi Dave, > >> >>>>>> > > >> >>>>>> > I've been catching up on IPv6 plans by reading the functional > >> >>>>>> > specs and Jira tickets - it's great to have so much material to > >> >>>>>> > refer to. > >> >>>>>> > > >> >>>>>> > I still have a few questions though, and I'm hoping someone > >> >>>>>> > involved with the feature can enlighten me. > >> >>>>>> > > >> >>>>>> > *[Support for Providers other than Virtual Router]* In [3], the > >> >>>>>> > spec says "No external device support in plan." > >> >>>>>> > What does this mean exactly? > >> >>>>>> > >> >>>>>> Because CloudStack also supports using external devices as > >> >>>>>> network controller e.g. Juniper SRX as firewall and NetScaler as > >> >>>>>> load balancer. The words here said is just we don't support these > >> >>>>>> devices when using IPv6. > >> >>>>>> > > >> >>>>>> > For example, if using Providers other than the Virtual Router, > >> >>>>>> > does the UI still allow setting IPv6 addresses? > >> >>>>>> > > >> >>>>>> > If so, do we attempt to pass IPv6 addresses to the Providers no > >> >>>>>> > matter what, or do we check whether the Provider has IPv6 > >> >>>>>> > support? > >> >>>>>> > >> >>>>>> Yes, we checked it when you try to create a IPv6 > >> >>>>>> network(currently only support advance shared network). > >> >>>>>> > >> >>>>>> > > >> >>>>>> > *[Networking Modes]* > >> >>>>>> > Advanced Shared mode and Basic mode are mentioned in the Jira > >> >>>>>> > ticket [1] - "Isolated Network" is mentioned briefly in [2], > >> >>>>>> > but I wanted to check if the Advanced Isolated and VPC modes > >> >>>>>> > are on the roadmap? > >> >>>>>> > >> >>>>>> There is no "basic isolated" network, so "Isolated" network is > >> >>>>>> what we're talking about. We haven't got plan for VPC yet. > >> >>>>>> > >> >>>>>> And one correction: we didn't support "basic" mode for phase 1. > >> >>>>>> We support only "advance shared network" in phase 1. The > >> >>>>>> supported cases are described in FS. Jira ticket only provided a > >> >>>>>> rough idea at the time. > >> >>>>>> > > >> >>>>>> > *[IP Address Management / IPAM]* From [1], re: handing out IPv6 > >> >>>>>> > addresses: "One way could be that the network admin creates a > >> >>>>>> > static route for a /48 towards a Virtual Router and then the VR > >> >>>>>> > can hand out /64s to Instances." > >> >>>>>> > > >> >>>>>> > With IPv4, IPAM is handled by the CloudStack management server, > >> >>>>>> > and the VR is told which IP address to give to the VM over > >> >>>>>> > DHCP. Would this change with IPv6? "The VR can hand out /64s to > >> >>>>>> > instances" sounds like the VR is handling IPAM to some extent. > >> >>>>>> > >> >>>>>> Well, it's not how it works now. Please refer to the FS. The > >> >>>>>> current implementation works like before. VR get a /64 then > >> >>>>>> handle out IPv6 addresses to VM. > >> >>>>>> > > >> >>>>>> > From [3], "Router advertisement should be sent by public > >> >>>>>> > gateway in the network." - to double-check, does this mean the > >> >>>>>> > router outside the CloudStack network should send RAs, but the > VR > >> >>>>>> > won't send RAs? > >> >>>>>> > >> >>>>>> Yes. Because in phase 1, we support only "advance shared > >> >>>>>> network", in which case, VR is NOT the gateway. So we assume the > >> >>>>>> gateway router outside CloudStack should send out RA to the VMs. > >> >>>>>> > >> >>>>>> But in the phase 2, VR would acting as gateway, then it would > send > >> >>>>>> out RAs. > >> >>>>>> > >> >>>>>> --Sheng > >> >>>>>> > > >> >>>>>> > Thanks, > >> >>>>>> > Dave. > >> >>>>>> > > >> >>>>>> > [1] IPv6 Support main Jira ticket > >> >>>>>> > https://issues.apache.org/jira/browse/CLOUDSTACK-452 > >> >>>>>> > > >> >>>>>> > [2] IPv6 Support in CloudStack FS > >> >>>>>> > https://cwiki.apache.org/CLOUDSTACK/ipv6-support-in-cloudstack > . > >> >>>>>> > html > >> >>>>>> > > >> >>>>>> > [3] IPv6 Support FS > >> >>>>>> > https://cwiki.apache.org/CLOUDSTACK/ipv6-support.html > >> >>>>>> > > > > >
