RE: IPV6 in Isolated/VPC networks
Hi Kristaps, The operator will need to add two ranges to ACS. 1) /64 for the physical network (just as we do with public IPV4 at the moment), this IPs will be used for the outside interface of the VRs 2) 1 or more /64 or larger (ideally /56s I'd say), which ACS will subnet into /64s and assign them to the inside interfaces of the VRs. Cheers Alex -Original Message- From: Kristaps Cudars Sent: 12 August 2021 10:37 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Hi Rohit, I think its excellent and satisfies all involved sides. We will be in front seats bringing this to production. Suggestion: This * At the zone level root-admin specifies a /64 public range that will be used for VRs, then they can add a /48, or /56 IPv6 range for guest networks (to be used by isolated networks and VPC tiers) To * At the zone level root-admin specifies a /64 public range that will be used for VRs, then they can add one or more ranges equal or larger than /64 IPv6 and ACS will subnet it in /64s for Isolated networks or VPC tiers. Way: RIPE NCC in case of PI (Provider Independent) range is granting /48 and its important to have possibility of assigning several subnets, for example /58 to be used for internal /64 allocations. If you want to extend your /48 you will need to have good explanation why you want it. RIPE NCC: Organisations requesting an IPv6 PI assignment larger than a /48 need to provide additional documentation to justify the larger assignment size. The RIPE NCC will evaluate the PI assignment request and assign an independent IPv6 prefix to the End User organization directly. On Wed, 11 Aug 2021 at 15:26, Rohit Yadav wrote: > Hi all, > > Thanks for your feedback and ideas, I've gone ahead with discussing > them with Alex and came up with a PoC/design which can be implemented > in the following phases: > > * Phase1: implement ipv6 support in isolated networks and VPC with > static routing > * Phase2: discuss and implement support for dynamic routing (TBD) > > For Phase1 here's the high-level proposal: > > * IPv6 address management: > * At the zone level root-admin specifies a /64 public range that > will be used for VRs, then they can add a /48, or /56 IPv6 range for > guest networks (to be used by isolated networks and VPC tiers) > * On creation of any IPv6 enabled isolated network or VPC tier, > from the /48 or /56 block a /64 network is allocated/used > * We assume SLAAC and autoconfiguration, no DHCPv6 in the zone > (discuss: is privacy a concern, can privacy extensions rfc4941 of > slaac be > explored?) > * Network offerings: root-admin can create new network offerings (with > VPC too) that specifies a network stack option: > * ipv4 only (default, for backward compatibility all > networks/offerings post-upgrade migrate to this option) > * ipv4-and-ipv6 > * ipv6-only (this can be phase 1.b) > * A new routing option: static (phase1), dynamic (phase2, with > multiple sub-options such as ospf/bgp etc...) > * VR changes: > * VR gets its guest and public nics set to inet6 auto > * For each /64 allocated to guest network and VPC tiers, radvd is > configured to do RA > * Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6 > firewall rules (ACLs, ingress, egress) are implemented; ACLs between > VPC tiers are managed/implemented by ipv6 firewall on VR > * It is assumed that static routes are created on the core/main > router by the admin or automated using some scripts/tools; for this > CloudStack will announce events with details of /64 networks and VR's > public IPv6 address that can be consumed by a rabbitmq/message bus > client (for example), or a custom cron job or script as part of orchestration. > (this wouldn't be necessary for dynamic routing bgp with phase2) > * Guest Networking: With SLAAC, it's easy for CloudStack to calculate > allocate and use a /64 and determine the IPv6 address of VR nics and > guest VM nics > * A user create an isolated network/VPC with an offering that is > ipv6 enabled > * A user can manage firewall for the IPv6 address/guest nics; > there'll be no port forward and LB feature though for IPv6 > * A users can run workloads in the guest VMs that listen on > publically routable ipv6 addresses > * Usage/billing etc continue to work, no change needed > > Network layout: > > [core/ISP router] -> [VR] -> [guest netwokr or VPC tier on a VLAN] -> > [guest VMs/nics] *core/ISP router needs static routes to be added > (manually or automated), assumes a /48 or /56 configured for the zone > > Thoughts, feedback? > > Proof-of-concept commentary: h
Re: IPV6 in Isolated/VPC networks
>
> I create a LAN ipv6 (public network for CloudStack VR): at subnet/prefix 0:
> LAN IPv6 address: 2001:470:ed36:0::1/64
> Address mode: SLAAC+stateless DHCP (no dhcpv6)
> *
> *
> In the isolated VR, I enabled ipv6 as:
> net.ipv6.conf.all.disable_ipv6 = 0
> net.ipv6.conf.all.forwarding = 1
> net.ipv6.conf.all.accept_ra = 1
> net.ipv6.conf.all.accept_redirects = 1
> net.ipv6.conf.all.autoconf = 1
>
> Set up a IPv6 nameserver/dns in /etc/resolve.conf
> And configured the nics:
> echo iface eth0 inet6 auto >> /etc/network/interfaces
> echo iface eth2 inet6 auto >> /etc/network/interfaces
> /etc/init.d/networking restart
> Next, restart ACS isolated network without cleanup to have it reconfigure
> IPv4 nics, firewall, NAT etc
>
> *
> Next, I created a /64 network for the isolated guest network on eth0 of VR
> using radvd:
>
> # cat /etc/radvd.conf
> interface eth0
> {
> AdvSendAdvert on;
> MinRtrAdvInterval 5;
> MaxRtrAdvInterval 15;
> prefix 2001:470:ed36:1::/64
> {
> AdvOnLink on;
> AdvAutonomous on;
> };
> };
> systemctl restart radvd
> All guest VMs nics and VR's eth0 gets IPv6 address (SLAAC) in this
> ...:1::/64 network
> * Finally I added a static route in toy core-router for the new /64
> IPv6 range in the isolated network
> 2001:470:ed36:1::/64 via dev
>
> *
> ... and I enabled firewall rules to allow any traffic to pass for the new
> /64 network
>
> And voila all done! I create a domain record that points to my guest
> VM IPv6 address a test webserver on
> http://ipv6-isolated-ntwk-demo.yadav.cloud/
>
> (Note: I'll get rid of the tunnel and request a new /48 block after a few
> days, sharing this solely for testing purposes)
>
>
> Regards.
>
>
> From: Wido den Hollander
> Sent: Tuesday, July 20, 2021 12:46
> To: d...@cloudstack.apache.org
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> Op 19-07-2021 om 20:38 schreef Kristaps Cudars:
> > Hi Wido,
> >
> > I assume that flouting ip will not work grate with ingress/egress acl on
> VR.
> >
> > From regular ACS user perspective:
> > I have Instance with dualstack its running web app on 443.
> > I want to swap instances for whatever reason.
> > In case of IPv4 change d-nat rule.
> > In case of IPv6 if flouting IP was not created upfront he will need to
> change dns entry that usually has 24h ttl. Inconvenience degradation in
> experience.
> >
>
> Yes, but, keep in mind that the IP you are using can also be terminated
> on the VR where HAProxy proxies request to the backend VM (could even be
> v4!)
>
> I'm not against DHCPv6, but I have seen many issues with implementing
> it. Therefor I always stick to SLAAC.
>
> > From ACS admin perspective:
> > I don’t want to have these tickets in helpdesk.
> > You needed to create another flouting IP that it would be seamless- will
> not work as answer.
> >
>
> I understand that as well.
>
> Wido
>
> >
> > On 2021/07/19 09:05:54, Wido den Hollander wrote:
> >>
> >>
> >> Op 16-07-2021 om 21:46 schreef Kristaps Cudars:
> >>> Hi Wido,
> >>>
> >>> Your proposal is to sacrifice ability to reassign IPv6 to instance,
> have internal domain prefix, and list/db in ACS what IPv6 has been assigned
> to what instance and go with RA and SLAAC. For route signaling to switch
> use BGP/OSPFv3 or manual pre-creation.
> >>>
> >>
> >> You can still list the IPs which have been assigned. You'll know exactly
> >> what IPv6 address a VM has because of the prefix + MAC. Privacy
> >> Extensions need to be disabled in the VM.
> >>
> >> This already works in CloudStack in Shared Networks in this way.
> >>
> >> Using secondary IPs you can always have 'floating' IPv6 addressess.
> >>
> >> Wido
> >>
> >>> Option with RA and managed flag that DHCPv6 is in use to support
> preset information and ability to create route information from ACS is not
> an option as DHCPv6 its failing?
> >>>
> >>>
> >>> On 2021/07/16 15:17:42, Wido den Hollander wrote:
> >>>>
> >>>>
> >>>> Op 16-07-2021 om 16:42 schreef Hean Seng:
> >>>>> Hi Wido,
> >>>>>
> >>>>> In current setup, each Cloudstack have own VR, so in this new IPv6
> subnet
> >>>>> allocation , each VR (which have Frr) will need to have peering with
> I
Re: IPV6 in Isolated/VPC networks
p to have it reconfigure
> IPv4 nics, firewall, NAT etc
>
> *
> Next, I created a /64 network for the isolated guest network on eth0 of VR
> using radvd:
>
> # cat /etc/radvd.conf
> interface eth0
> {
> AdvSendAdvert on;
> MinRtrAdvInterval 5;
> MaxRtrAdvInterval 15;
> prefix 2001:470:ed36:1::/64
> {
> AdvOnLink on;
> AdvAutonomous on;
> };
> };
> systemctl restart radvd
> All guest VMs nics and VR's eth0 gets IPv6 address (SLAAC) in this
> ...:1::/64 network
> * Finally I added a static route in toy core-router for the new /64
> IPv6 range in the isolated network
> 2001:470:ed36:1::/64 via dev
>
> *
> ... and I enabled firewall rules to allow any traffic to pass for the new
> /64 network
>
> And voila all done! I create a domain record that points to my guest
> VM IPv6 address a test webserver on
> http://ipv6-isolated-ntwk-demo.yadav.cloud/
>
> (Note: I'll get rid of the tunnel and request a new /48 block after a few
> days, sharing this solely for testing purposes)
>
>
> Regards.
>
>
> From: Wido den Hollander
> Sent: Tuesday, July 20, 2021 12:46
> To: d...@cloudstack.apache.org
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> Op 19-07-2021 om 20:38 schreef Kristaps Cudars:
> > Hi Wido,
> >
> > I assume that flouting ip will not work grate with ingress/egress acl on
> VR.
> >
> > From regular ACS user perspective:
> > I have Instance with dualstack its running web app on 443.
> > I want to swap instances for whatever reason.
> > In case of IPv4 change d-nat rule.
> > In case of IPv6 if flouting IP was not created upfront he will need to
> change dns entry that usually has 24h ttl. Inconvenience degradation in
> experience.
> >
>
> Yes, but, keep in mind that the IP you are using can also be terminated
> on the VR where HAProxy proxies request to the backend VM (could even be
> v4!)
>
> I'm not against DHCPv6, but I have seen many issues with implementing
> it. Therefor I always stick to SLAAC.
>
> > From ACS admin perspective:
> > I don’t want to have these tickets in helpdesk.
> > You needed to create another flouting IP that it would be seamless- will
> not work as answer.
> >
>
> I understand that as well.
>
> Wido
>
> >
> > On 2021/07/19 09:05:54, Wido den Hollander wrote:
> >>
> >>
> >> Op 16-07-2021 om 21:46 schreef Kristaps Cudars:
> >>> Hi Wido,
> >>>
> >>> Your proposal is to sacrifice ability to reassign IPv6 to instance,
> have internal domain prefix, and list/db in ACS what IPv6 has been assigned
> to what instance and go with RA and SLAAC. For route signaling to switch
> use BGP/OSPFv3 or manual pre-creation.
> >>>
> >>
> >> You can still list the IPs which have been assigned. You'll know exactly
> >> what IPv6 address a VM has because of the prefix + MAC. Privacy
> >> Extensions need to be disabled in the VM.
> >>
> >> This already works in CloudStack in Shared Networks in this way.
> >>
> >> Using secondary IPs you can always have 'floating' IPv6 addressess.
> >>
> >> Wido
> >>
> >>> Option with RA and managed flag that DHCPv6 is in use to support
> preset information and ability to create route information from ACS is not
> an option as DHCPv6 its failing?
> >>>
> >>>
> >>> On 2021/07/16 15:17:42, Wido den Hollander wrote:
> >>>>
> >>>>
> >>>> Op 16-07-2021 om 16:42 schreef Hean Seng:
> >>>>> Hi Wido,
> >>>>>
> >>>>> In current setup, each Cloudstack have own VR, so in this new IPv6
> subnet
> >>>>> allocation , each VR (which have Frr) will need to have peering with
> ISP
> >>>>> router (and either BGP or Static Route) , and there is 1000
> Acocunts, it
> >>>>> will 1000 BGP session with ISP router , Am I right for this ? or I
> >>>>> understand wrong .
> >>>>>
> >>>>
> >>>> Yes, that is correct. A /56 would also be sufficient or a /60 which is
> >>>> enough to allocate a few /64 subnets.
> >>>>
> >>>> 1000 BGP connections isn't really a problem for a proper router at the
> >>>> ISP. OSPF(v3) would be better, but as I said that's poorly supported.
> >>>>
> >>>> The ISP could also install 1000 static routes, but that means that the
> >>>&g
Re: IPV6 in Isolated/VPC networks
and request a new /48 block after a few days, sharing this solely for testing purposes) Regards. From: Wido den Hollander Sent: Tuesday, July 20, 2021 12:46 To: d...@cloudstack.apache.org Subject: Re: IPV6 in Isolated/VPC networks Op 19-07-2021 om 20:38 schreef Kristaps Cudars: > Hi Wido, > > I assume that flouting ip will not work grate with ingress/egress acl on VR. > > From regular ACS user perspective: > I have Instance with dualstack its running web app on 443. > I want to swap instances for whatever reason. > In case of IPv4 change d-nat rule. > In case of IPv6 if flouting IP was not created upfront he will need to change > dns entry that usually has 24h ttl. Inconvenience degradation in experience. > Yes, but, keep in mind that the IP you are using can also be terminated on the VR where HAProxy proxies request to the backend VM (could even be v4!) I'm not against DHCPv6, but I have seen many issues with implementing it. Therefor I always stick to SLAAC. > From ACS admin perspective: > I don’t want to have these tickets in helpdesk. > You needed to create another flouting IP that it would be seamless- will not > work as answer. > I understand that as well. Wido > > On 2021/07/19 09:05:54, Wido den Hollander wrote: >> >> >> Op 16-07-2021 om 21:46 schreef Kristaps Cudars: >>> Hi Wido, >>> >>> Your proposal is to sacrifice ability to reassign IPv6 to instance, have >>> internal domain prefix, and list/db in ACS what IPv6 has been assigned to >>> what instance and go with RA and SLAAC. For route signaling to switch use >>> BGP/OSPFv3 or manual pre-creation. >>> >> >> You can still list the IPs which have been assigned. You'll know exactly >> what IPv6 address a VM has because of the prefix + MAC. Privacy >> Extensions need to be disabled in the VM. >> >> This already works in CloudStack in Shared Networks in this way. >> >> Using secondary IPs you can always have 'floating' IPv6 addressess. >> >> Wido >> >>> Option with RA and managed flag that DHCPv6 is in use to support preset >>> information and ability to create route information from ACS is not an >>> option as DHCPv6 its failing? >>> >>> >>> On 2021/07/16 15:17:42, Wido den Hollander wrote: >>>> >>>> >>>> Op 16-07-2021 om 16:42 schreef Hean Seng: >>>>> Hi Wido, >>>>> >>>>> In current setup, each Cloudstack have own VR, so in this new IPv6 >>>>> subnet >>>>> allocation , each VR (which have Frr) will need to have peering with ISP >>>>> router (and either BGP or Static Route) , and there is 1000 Acocunts, it >>>>> will 1000 BGP session with ISP router , Am I right for this ? or I >>>>> understand wrong . >>>>> >>>> >>>> Yes, that is correct. A /56 would also be sufficient or a /60 which is >>>> enough to allocate a few /64 subnets. >>>> >>>> 1000 BGP connections isn't really a problem for a proper router at the >>>> ISP. OSPF(v3) would be better, but as I said that's poorly supported. >>>> >>>> The ISP could also install 1000 static routes, but that means that the >>>> ISP's router needs to have those configured. >>>> >>>> http://docs.frrouting.org/en/latest/ospf6d.html >>>> (While looking up this URL I see that Frr recently put in a lot of work >>>> in OSPFv3, seems better now) >>>> >>>>> I understand IPv6 is different then IPv4, and in IPv6 it suppose each >>>>> devices have own IP. It just how to realize in easy way. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> > On Fri, Jul 16, 2021 at 8:17 PM Wido den Hollander wrote: >>>>> >>>>>> >>>>>> >>>>>> Op 16-07-2021 om 05:54 schreef Hean Seng: >>>>>>> Hi Wido, >>>>>>> >>>>>>> My initial thought is not like this, it is the /48 at ISP router, and >>>>>> /64 >>>>>>> subnet assign to AdvanceZoneVR, AdvanceZoneVR responsible is >>>>>>> distribule IPv6 ip (from the assigned /64 sunet) to VM, and not routing >>>>>>> the traffic, in the VM that get the IPv6 IP will default route to ISP >>>>>>> router as gw. It can may be a bridge over
Re: IPV6 in Isolated/VPC networks
;. I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account
basis
> 3) End-user driven AS number on a per network basis
(for
bring your
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
<mailto:w...@widodh.nl>>>
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>
<mailto:d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>>;
> Alex Mattioli mailto:alex.matti...@shapeblue.com>
> <mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>>
> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> mailto:rohit.ya...@shapeblue.com>
<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>>;
> Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
> <mailto:gabr...@pcextreme.nl
gabr...@pcextreme.nl
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > Hi all,
> > @Wei Zhou<mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>> @Rohit
> Yadav<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>
> <mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>> and myself are
investigating how
> to enable IPV6 support on Isolated and VPC networks
and
would like
> your input on it.
> > At the moment we are looking at implementing FRR
with
BGP (and
> possibly OSPF) on the ACS VR.
> >
> > We are looking for requirements, recommendations,
ideas, rants,
> etc...etc...
> >
>
> Ok! Here we go.
>
> I think that you mean that the VR will actually
route
the
IPv6
> traffic and for that you need to have a way of
getting
a
subnet
> routed to the VR.
>
> BGP is probably you best bet here. Although OSPFv3
technically
> supports this it is very badly implemented in Frr
for
example.
>
> Now FRR is a very good router and one of the fancy
features it
> supports is BGP Unnumered. This allows for auto
configuration of BGP
> over a L2 network when both sides are sending Router
Advertisements.
> This is very easy for flexible BGP configurations
where
both sides
> have dynamic IPs.
>
> What you want to do is that you get a /56, /48 or
something which is
> >/64 bits routed to the VR.
>
> Now you can sub-segment this into separate /64
subnets.
You don't
> want to go smaller then a /64 is that prevents you
from
using SLAAC
> for IPv6 address configuration. This is how it works
for
Shared
> Networks now in Basic and Advanced Zones.
>
> FRR can now also send out the Router Advertisements
on
the downlinks
> sending out:
>
> - DNS servers
> - DNS domain
> - Prefix (/64) to be used
>
> There is no need for DHCPv6. You can calculate the
IPv6
address the
> VM will obtain by using the MAC and the prefix.
>
> So in short:
>
> - Using BGP you routed a /48 to the VR
> - Now you split this into /64 subnets towards the
isolated networks
>
> Wido
>
> > Alex Mattioli
> >
> >
> >
> >
>
>
>
> --
> Regards,
> Hean Seng
--
Regards,
Hean Seng
--
Regards,
Hean Seng
Re: IPV6 in Isolated/VPC networks
IPv6 2000::200:201::1 > >>>>>> > >>>>>> So in cloudstack need to allow user to enter , IPv6 > >>>>> gwateway , and > >>>>>> the /48 Ipv6 prefix , then it will self allocate the > /64 > >> ip > >>>>> to the VR , > >>>>>> and maintain make sure not ovelap allocation > >>>>>> > >>>>>> > >>>>> > >>>>> But NAT is truly not the solution with IPv6. IPv6 is > supposed > >> to > >>>> be > >>>>> routable. In addition you should avoid DHCPv6 as much as > >>>>> possible as > >>>>> that's not really the intended use-case for address > allocation > >>>>> with IPv6. > >>>>> > >>>>> In order to route an /48 IPv6 subnet to the VR you have a > few > >>>>> possibilities: > >>>>> > >>>>> - Static route from the upperlying routers which are > outside > >> of > >>>>> CloudStack > >>>>> - BGP > >>>>> - OSPFv3 (broken in most cases!) > >>>>> - DHCPv6 Prefix Delegation > >>>>> > >>>>> BGP and/or Static routes are still the best bet here. > >>>>> > >>>>> So what you do is that you tell CloudStack that you will > route > >>>>> 2001:db8::/48 to the VR, the VR can then use that to split > it > >> up > >>>>> into > >>>>> multiple /64 subnets going towards the instances: > >>>>> > >>>>> - 2001:db8::/64 > >>>>> - 2001:db8:1::/64 > >>>>> - 2001:db8:2::/64 > >>>>> ... > >>>>> - 2001:db8:f::/64 > >>>>> > >>>>> And go on. > >>>>> > >>>>> In case of BGP you indeed have to tell the VR a few things: > >>>>> > >>>>> - It's own AS number > >>>>> - The peer's address(es) > >>>>> > >>>>> With FRR you can simply say: > >>>>> > >>>>> neighbor 2001:db8:4fa::179 remote-as external > >>>>> > >>>>> The /48 you need to have at the VR anyway in case of > either a > >>>>> static > >>>>> route or BGP. > >>>>> > >>>>> We just need to add a NullRoute on the VR for that /48 so > that > >>>>> traffic > >>>>> will not be routed to the upper gateway in case of the VR > >> can't > >>>>> find a > >>>>> route. > >>>>> > >>>>> Wido > >>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli > >>>>>> >>>>> <mailto:alex.matti...@shapeblue.com> > >>>>> <mailto:alex.matti...@shapeblue.com > >>>>> <mailto:alex.matti...@shapeblue.com>>> wrote: > >>>>>> > >>>>>> Hi Wido, > >>>>>> That's pretty much in line with our thoughts, thanks > >> for > >>>>> the input. > >>>>>> I believe we agree on the following points then: > >>>>>> > >>>>>> - FRR with BGP (no OSPF) > >>>>>> - Route /48 (or/56) down to the VR > >>>>>> - /64 per network > >>>>>> - SLACC for IP addressing > >>>>>> > >>>>>> I believe the next big question is then "on which > level > >
Re: IPV6 in Isolated/VPC networks
> that would be complicated .
>
> We only need assign IPv6 's /64 prefix to Virtual Router
(VR) in NAT
> zone, and the VR responsible to deliver single IPv6 to VM
via
DHCP6.
>
> In VR, you need to have Default IPv6 route to Physical
Router's /48. IP
> as IPv6 Gateway. Thens should be done .
>
> Example :
> Physical Router Interface
> IPv6 IP : 2000:::1/48
>
> Cloudstack virtual router : 2000::200:201::1/64 with
default ipv6
> route to router ip 2000:::1
> and Clodustack Virtual router dhcp allocate IP to VM , and
VM will have
> default route to VR. IPv6 2000::200:201::1
>
> So in cloudstack need to allow user to enter , IPv6
gwateway , and
> the /48 Ipv6 prefix , then it will self allocate the /64
ip
to the VR ,
> and maintain make sure not ovelap allocation
>
>
But NAT is truly not the solution with IPv6. IPv6 is supposed
to
be
routable. In addition you should avoid DHCPv6 as much as
possible as
that's not really the intended use-case for address allocation
with IPv6.
In order to route an /48 IPv6 subnet to the VR you have a few
possibilities:
- Static route from the upperlying routers which are outside
of
CloudStack
- BGP
- OSPFv3 (broken in most cases!)
- DHCPv6 Prefix Delegation
BGP and/or Static routes are still the best bet here.
So what you do is that you tell CloudStack that you will route
2001:db8::/48 to the VR, the VR can then use that to split it
up
into
multiple /64 subnets going towards the instances:
- 2001:db8::/64
- 2001:db8:1::/64
- 2001:db8:2::/64
...
- 2001:db8:f::/64
And go on.
In case of BGP you indeed have to tell the VR a few things:
- It's own AS number
- The peer's address(es)
With FRR you can simply say:
neighbor 2001:db8:4fa::179 remote-as external
The /48 you need to have at the VR anyway in case of either a
static
route or BGP.
We just need to add a NullRoute on the VR for that /48 so that
traffic
will not be routed to the upper gateway in case of the VR
can't
find a
route.
Wido
>
>
>
>
>
> On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> mailto:alex.matti...@shapeblue.com>
<mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>> wrote:
>
> Hi Wido,
> That's pretty much in line with our thoughts, thanks
for
the input.
> I believe we agree on the following points then:
>
> - FRR with BGP (no OSPF)
> - Route /48 (or/56) down to the VR
> - /64 per network
> - SLACC for IP addressing
>
> I believe the next big question is then "on which level
of ACS do we
> manage AS numbers?". I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account
basis
> 3) End-user driven AS number on a per network basis
(for
bring your
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
<mailto:w...@widodh.nl>>>
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>
<mailto:d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>>;
> Alex Mattioli mailto:alex.matti...@shapeblue.com>
> <mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>>
> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
>
Re: IPV6 in Isolated/VPC networks
gt;>> > >>> Wido > >>> > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli > >>> > >>> <mailto:alex.matti...@shapeblue.com> > >>> <mailto:alex.matti...@shapeblue.com > >>> <mailto:alex.matti...@shapeblue.com>>> wrote: > >>> > > >>> > Hi Wido, > >>> > That's pretty much in line with our thoughts, thanks > for > >>> the input. > >>> > I believe we agree on the following points then: > >>> > > >>> > - FRR with BGP (no OSPF) > >>> > - Route /48 (or/56) down to the VR > >>> > - /64 per network > >>> > - SLACC for IP addressing > >>> > > >>> > I believe the next big question is then "on which level > >>> of ACS do we > >>> > manage AS numbers?". I see two options: > >>> > 1) Private AS number on a per-zone basis > >>> > 2) Root Admin assigned AS number on a domain/account > basis > >>> > 3) End-user driven AS number on a per network basis > (for > >>> bring your > >>> > own AS and IP scenario) > >>> > > >>> > Thoughts? > >>> > > >>> > Cheers > >>> > Alex > >>> > > >>> > > >>> > > >>> > > >>> > -Original Message- > >>> > From: Wido den Hollander >>> <mailto:w...@widodh.nl> <mailto:w...@widodh.nl > >>> <mailto:w...@widodh.nl>>> > >>> > Sent: 13 July 2021 15:08 > >>> > To: d...@cloudstack.apache.org > >>> <mailto:d...@cloudstack.apache.org> > >>> <mailto:d...@cloudstack.apache.org > >>> <mailto:d...@cloudstack.apache.org>>; > >>> > Alex Mattioli >>> <mailto:alex.matti...@shapeblue.com> > >>> > <mailto:alex.matti...@shapeblue.com > >>> <mailto:alex.matti...@shapeblue.com>>> > >>> > Cc: Wei Zhou >>> <mailto:wei.z...@shapeblue.com> > >>> > <mailto:wei.z...@shapeblue.com > >>> <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav > >>> > >>> <mailto:rohit.ya...@shapeblue.com> > >>> <mailto:rohit.ya...@shapeblue.com > >>> <mailto:rohit.ya...@shapeblue.com>>>; > >>> > Gabriel Beims Bräscher >>> <mailto:gabr...@pcextreme.nl> > >>> > <mailto:gabr...@pcextreme.nl gabr...@pcextreme.nl > >>>>> > >>> > Subject: Re: IPV6 in Isolated/VPC networks > >>> > > >>> > > >>> > > >>> > On 7/7/21 1:16 PM, Alex Mattioli wrote: > >>> > > Hi all, > >>> > > @Wei Zhou<mailto:wei.z...@shapeblue.com > >>> <mailto:wei.z...@shapeblue.com> > >>> > <mailto:wei.z...@shapeblue.com > >>> <mailto:wei.z...@shapeblue.com>>> @Rohit > >>> > Yadav<mailto:rohit.ya...@shapeblue.com > >>> <mailto:rohit.ya...@shapeblue.com> > >>> > <mailto:rohit.ya...@shapeblue.com > >>> <mailto:rohit.ya...@shapeblue.com>>> and myself are > >>> investigating how > >>> > to enable IPV6 support on Isolated and VPC networks and > >>> would like > >>> > your input on it. > >>> > > At the moment we are looking at implementing FRR > with > >>> BGP (and > >>> > possibly OSPF) on the ACS VR. > >>> > > > >>> > > We are looking for requirements, recommendations, > >>> ideas, rants, > >>> > etc...etc... > >>> > > > >>> > > >>> > Ok! Here we go. > >>> > > >>> > I think that you mean that the VR will actually route > the > >>> IPv6 > >>> > traffic and for that you need to have a way of getting > a > >>> subnet > >>> > routed to the VR. > >>> > > >>> > BGP is probably you best bet here. Although OSPFv3 > >>> technically > >>> > supports this it is very badly implemented in Frr for > >>> example. > >>> > > >>> > Now FRR is a very good router and one of the fancy > >>> features it > >>> > supports is BGP Unnumered. This allows for auto > >>> configuration of BGP > >>> > over a L2 network when both sides are sending Router > >>> Advertisements. > >>> > This is very easy for flexible BGP configurations where > >>> both sides > >>> > have dynamic IPs. > >>> > > >>> > What you want to do is that you get a /56, /48 or > >>> something which is > >>> > >/64 bits routed to the VR. > >>> > > >>> > Now you can sub-segment this into separate /64 subnets. > >>> You don't > >>> > want to go smaller then a /64 is that prevents you from > >>> using SLAAC > >>> > for IPv6 address configuration. This is how it works > for > >>> Shared > >>> > Networks now in Basic and Advanced Zones. > >>> > > >>> > FRR can now also send out the Router Advertisements on > >>> the downlinks > >>> > sending out: > >>> > > >>> > - DNS servers > >>> > - DNS domain > >>> > - Prefix (/64) to be used > >>> > > >>> > There is no need for DHCPv6. You can calculate the IPv6 > >>> address the > >>> > VM will obtain by using the MAC and the prefix. > >>> > > >>> > So in short: > >>> > > >>> > - Using BGP you routed a /48 to the VR > >>> > - Now you split this into /64 subnets towards the > >>> isolated networks > >>> > > >>> > Wido > >>> > > >>> > > Alex Mattioli > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > >>> > > >>> > > >>> > -- > >>> > Regards, > >>> > Hean Seng > >>> > >>> > >>> > >>> -- > >>> Regards, > >>> Hean Seng > >>> > >>> > >>> > >>> -- > >>> Regards, > >>> Hean Seng > > > > > > > -- Regards, Hean Seng
Re: IPV6 in Isolated/VPC networks
AT is truly not the solution with IPv6. IPv6 is supposed to
be
routable. In addition you should avoid DHCPv6 as much as
possible as
that's not really the intended use-case for address allocation
with IPv6.
In order to route an /48 IPv6 subnet to the VR you have a few
possibilities:
- Static route from the upperlying routers which are outside of
CloudStack
- BGP
- OSPFv3 (broken in most cases!)
- DHCPv6 Prefix Delegation
BGP and/or Static routes are still the best bet here.
So what you do is that you tell CloudStack that you will route
2001:db8::/48 to the VR, the VR can then use that to split it up
into
multiple /64 subnets going towards the instances:
- 2001:db8::/64
- 2001:db8:1::/64
- 2001:db8:2::/64
...
- 2001:db8:f::/64
And go on.
In case of BGP you indeed have to tell the VR a few things:
- It's own AS number
- The peer's address(es)
With FRR you can simply say:
neighbor 2001:db8:4fa::179 remote-as external
The /48 you need to have at the VR anyway in case of either a
static
route or BGP.
We just need to add a NullRoute on the VR for that /48 so that
traffic
will not be routed to the upper gateway in case of the VR can't
find a
route.
Wido
>
>
>
>
>
> On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli
> mailto:alex.matti...@shapeblue.com>
<mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>> wrote:
>
> Hi Wido,
> That's pretty much in line with our thoughts, thanks for
the input.
> I believe we agree on the following points then:
>
> - FRR with BGP (no OSPF)
> - Route /48 (or/56) down to the VR
> - /64 per network
> - SLACC for IP addressing
>
> I believe the next big question is then "on which level
of ACS do we
> manage AS numbers?". I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account basis
> 3) End-user driven AS number on a per network basis (for
bring your
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
<mailto:w...@widodh.nl>>>
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>
<mailto:d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>>;
> Alex Mattioli mailto:alex.matti...@shapeblue.com>
> <mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>>
> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> mailto:rohit.ya...@shapeblue.com>
<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>>;
> Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
> <mailto:gabr...@pcextreme.nl <mailto:gabr...@pcextreme.nl
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > Hi all,
> > @Wei Zhou<mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>> @Rohit
> Yadav<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>
> <mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>> and myself are
investigating how
> to enable IPV6 support on Isolated and VPC networks and
would like
> your input on it.
> > At the moment we are looking at implementing FRR with
BGP (and
> possibly OSPF) on the ACS VR.
> >
> >
Re: IPV6 in Isolated/VPC networks
> and maintain make sure not ovelap allocation > > > > > > > > > > But NAT is truly not the solution with IPv6. IPv6 is supposed to > be > > routable. In addition you should avoid DHCPv6 as much as > > possible as > > that's not really the intended use-case for address allocation > > with IPv6. > > > > In order to route an /48 IPv6 subnet to the VR you have a few > > possibilities: > > > > - Static route from the upperlying routers which are outside of > > CloudStack > > - BGP > > - OSPFv3 (broken in most cases!) > > - DHCPv6 Prefix Delegation > > > > BGP and/or Static routes are still the best bet here. > > > > So what you do is that you tell CloudStack that you will route > > 2001:db8::/48 to the VR, the VR can then use that to split it up > > into > > multiple /64 subnets going towards the instances: > > > > - 2001:db8::/64 > > - 2001:db8:1::/64 > > - 2001:db8:2::/64 > > ... > > - 2001:db8:f::/64 > > > > And go on. > > > > In case of BGP you indeed have to tell the VR a few things: > > > > - It's own AS number > > - The peer's address(es) > > > > With FRR you can simply say: > > > > neighbor 2001:db8:4fa::179 remote-as external > > > > The /48 you need to have at the VR anyway in case of either a > > static > > route or BGP. > > > > We just need to add a NullRoute on the VR for that /48 so that > > traffic > > will not be routed to the upper gateway in case of the VR can't > > find a > > route. > > > > Wido > > > > > > > > > > > > > > > > > > > > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli > > > > <mailto:alex.matti...@shapeblue.com> > > <mailto:alex.matti...@shapeblue.com > > <mailto:alex.matti...@shapeblue.com>>> wrote: > > > > > > Hi Wido, > > > That's pretty much in line with our thoughts, thanks for > > the input. > > > I believe we agree on the following points then: > > > > > > - FRR with BGP (no OSPF) > > > - Route /48 (or/56) down to the VR > > > - /64 per network > > > - SLACC for IP addressing > > > > > > I believe the next big question is then "on which level > > of ACS do we > > > manage AS numbers?". I see two options: > > > 1) Private AS number on a per-zone basis > > > 2) Root Admin assigned AS number on a domain/account basis > > > 3) End-user driven AS number on a per network basis (for > > bring your > > > own AS and IP scenario) > > > > > > Thoughts? > > > > > > Cheers > > > Alex > > > > > > > > > > > > > > > -Original Message- > > > From: Wido den Hollander > <mailto:w...@widodh.nl> <mailto:w...@widodh.nl > > <mailto:w...@widodh.nl>>> > > > Sent: 13 July 2021 15:08 > > > To: d...@cloudstack.apache.org > > <mailto:d...@cloudstack.apache.org> > > <mailto:d...@cloudstack.apache.org > > <mailto:d...@cloudstack.apache.org>>; > > > Alex Mattioli > <mailto:alex.matti...@shapeblue.com> > > > <mailto:alex.matti...@shapeblue.com > > <mailto:alex.matti...@shapeblue.com>>> > > > Cc: Wei Zhou > <mailto:wei.z...@shapeblue.com> > > > <mailto:wei.z...@shapeblue.com > > <mailto:wei.z...@shapeblue.com>>>; Rohit Yadav > > > > <mailto:rohit.ya...@shapeblue.com> > > <mailto:rohit.ya...@shapeblue.com > > <mailto:rohit.ya...@shapeblue.com>>>; > > > Gabriel
Re: IPV6 in Isolated/VPC networks
level
of ACS do we
> manage AS numbers?". I see two options:
> 1) Private AS number on a per-zone basis
> 2) Root Admin assigned AS number on a domain/account basis
> 3) End-user driven AS number on a per network basis (for
bring your
> own AS and IP scenario)
>
> Thoughts?
>
> Cheers
> Alex
>
>
>
>
> -Original Message-
> From: Wido den Hollander mailto:w...@widodh.nl> <mailto:w...@widodh.nl
<mailto:w...@widodh.nl>>>
> Sent: 13 July 2021 15:08
> To: d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>
<mailto:d...@cloudstack.apache.org
<mailto:d...@cloudstack.apache.org>>;
> Alex Mattioli mailto:alex.matti...@shapeblue.com>
> <mailto:alex.matti...@shapeblue.com
<mailto:alex.matti...@shapeblue.com>>>
> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>>; Rohit Yadav
> mailto:rohit.ya...@shapeblue.com>
<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>>;
> Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>
> <mailto:gabr...@pcextreme.nl <mailto:gabr...@pcextreme.nl>>>
> Subject: Re: IPV6 in Isolated/VPC networks
>
>
>
> On 7/7/21 1:16 PM, Alex Mattioli wrote:
> > Hi all,
> > @Wei Zhou<mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>
> <mailto:wei.z...@shapeblue.com
<mailto:wei.z...@shapeblue.com>>> @Rohit
> Yadav<mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>
> <mailto:rohit.ya...@shapeblue.com
<mailto:rohit.ya...@shapeblue.com>>> and myself are
investigating how
> to enable IPV6 support on Isolated and VPC networks and
would like
> your input on it.
> > At the moment we are looking at implementing FRR with
BGP (and
> possibly OSPF) on the ACS VR.
> >
> > We are looking for requirements, recommendations,
ideas, rants,
> etc...etc...
> >
>
> Ok! Here we go.
>
> I think that you mean that the VR will actually route the
IPv6
> traffic and for that you need to have a way of getting a
subnet
> routed to the VR.
>
> BGP is probably you best bet here. Although OSPFv3
technically
> supports this it is very badly implemented in Frr for
example.
>
> Now FRR is a very good router and one of the fancy
features it
> supports is BGP Unnumered. This allows for auto
configuration of BGP
> over a L2 network when both sides are sending Router
Advertisements.
> This is very easy for flexible BGP configurations where
both sides
> have dynamic IPs.
>
> What you want to do is that you get a /56, /48 or
something which is
> >/64 bits routed to the VR.
>
> Now you can sub-segment this into separate /64 subnets.
You don't
> want to go smaller then a /64 is that prevents you from
using SLAAC
> for IPv6 address configuration. This is how it works for
Shared
> Networks now in Basic and Advanced Zones.
>
> FRR can now also send out the Router Advertisements on
the downlinks
> sending out:
>
> - DNS servers
> - DNS domain
> - Prefix (/64) to be used
>
> There is no need for DHCPv6. You can calculate the IPv6
address the
> VM will obtain by using the MAC and the prefix.
>
> So in short:
>
> - Using BGP you routed a /48 to the VR
> - Now you split this into /64 subnets towards the
isolated networks
>
> Wido
>
> > Alex Mattioli
> >
> >
> >
>
Re: IPV6 in Isolated/VPC networks
Or explain like this : 1) Cloudstack generate list of /64 subnet from /48 that Network admin assigned to Cloudstack 2) Cloudsack allocated the subnet (that generated from step1) to Virtual Router, one Virtual Router have one subniet /64 3) Virtual Router allocate single IPv6 (within the range of /64 allocated to VR) to VM On Thu, Jul 15, 2021 at 6:25 PM Hean Seng wrote: > Hi Wido, > > I think the /48 is at physical router as gateway , and subnet of /64 at VR > of Cloudstack. Cloudstack only keep which /48 prefix and vlan information > of this /48 to be later split the /64. to VR. > > And the instances is getting singe IPv6 of /64 IP. The VR is getting > /64. The default gateway shall goes to /48 of physical router ip . In > this case ,does not need any BGP router . > > > Similar concept as IPv4 : > > /48 subnet of IPv6 is equivalent to current /24 subnet of IPv4 that > created in Network. > and /64 of IPv6 is equivalent to single IP of IPv4 assign to VM. > > > > > On Thu, Jul 15, 2021 at 5:31 PM Wido den Hollander wrote: > >> >> >> Op 14-07-2021 om 16:44 schreef Hean Seng: >> > Hi >> > >> > I replied in another thread, i think do not need implement BGP or OSPF, >> > that would be complicated . >> > >> > We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT >> > zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. >> > >> > In VR, you need to have Default IPv6 route to Physical Router's /48. >> IP >> > as IPv6 Gateway. Thens should be done . >> > >> > Example : >> > Physical Router Interface >> > IPv6 IP : 2000:::1/48 >> > >> > Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 >> > route to router ip 2000:::1 >> > and Clodustack Virtual router dhcp allocate IP to VM , and VM will >> have >> > default route to VR. IPv6 2000::200:201::1 >> > >> > So in cloudstack need to allow user to enter , IPv6 gwateway , and >> > the /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR >> , >> > and maintain make sure not ovelap allocation >> > >> > >> >> But NAT is truly not the solution with IPv6. IPv6 is supposed to be >> routable. In addition you should avoid DHCPv6 as much as possible as >> that's not really the intended use-case for address allocation with IPv6. >> >> In order to route an /48 IPv6 subnet to the VR you have a few >> possibilities: >> >> - Static route from the upperlying routers which are outside of CloudStack >> - BGP >> - OSPFv3 (broken in most cases!) >> - DHCPv6 Prefix Delegation >> >> BGP and/or Static routes are still the best bet here. >> >> So what you do is that you tell CloudStack that you will route >> 2001:db8::/48 to the VR, the VR can then use that to split it up into >> multiple /64 subnets going towards the instances: >> >> - 2001:db8::/64 >> - 2001:db8:1::/64 >> - 2001:db8:2::/64 >> ... >> - 2001:db8:f::/64 >> >> And go on. >> >> In case of BGP you indeed have to tell the VR a few things: >> >> - It's own AS number >> - The peer's address(es) >> >> With FRR you can simply say: >> >> neighbor 2001:db8:4fa::179 remote-as external >> >> The /48 you need to have at the VR anyway in case of either a static >> route or BGP. >> >> We just need to add a NullRoute on the VR for that /48 so that traffic >> will not be routed to the upper gateway in case of the VR can't find a >> route. >> >> Wido >> >> > >> > >> > >> > >> > >> > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli >> > mailto:alex.matti...@shapeblue.com>> >> wrote: >> > >> > Hi Wido, >> > That's pretty much in line with our thoughts, thanks for the input. >> > I believe we agree on the following points then: >> > >> > - FRR with BGP (no OSPF) >> > - Route /48 (or/56) down to the VR >> > - /64 per network >> > - SLACC for IP addressing >> > >> > I believe the next big question is then "on which level of ACS do we >> > manage AS numbers?". I see two options: >> > 1) Private AS number on a per-zone basis >> > 2) Root Admin assigned AS number on a domain/account basis >> > 3) End-user driven AS number on a per network basis (for bring your >> > own AS and
Re: IPV6 in Isolated/VPC networks
Hi Wido, I think the /48 is at physical router as gateway , and subnet of /64 at VR of Cloudstack. Cloudstack only keep which /48 prefix and vlan information of this /48 to be later split the /64. to VR. And the instances is getting singe IPv6 of /64 IP. The VR is getting /64. The default gateway shall goes to /48 of physical router ip . In this case ,does not need any BGP router . Similar concept as IPv4 : /48 subnet of IPv6 is equivalent to current /24 subnet of IPv4 that created in Network. and /64 of IPv6 is equivalent to single IP of IPv4 assign to VM. On Thu, Jul 15, 2021 at 5:31 PM Wido den Hollander wrote: > > > Op 14-07-2021 om 16:44 schreef Hean Seng: > > Hi > > > > I replied in another thread, i think do not need implement BGP or OSPF, > > that would be complicated . > > > > We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT > > zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. > > > > In VR, you need to have Default IPv6 route to Physical Router's /48. IP > > as IPv6 Gateway. Thens should be done . > > > > Example : > > Physical Router Interface > > IPv6 IP : 2000:::1/48 > > > > Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 > > route to router ip 2000:::1 > > and Clodustack Virtual router dhcp allocate IP to VM , and VM will have > > default route to VR. IPv6 2000::200:201::1 > > > > So in cloudstack need to allow user to enter , IPv6 gwateway , and > > the /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , > > and maintain make sure not ovelap allocation > > > > > > But NAT is truly not the solution with IPv6. IPv6 is supposed to be > routable. In addition you should avoid DHCPv6 as much as possible as > that's not really the intended use-case for address allocation with IPv6. > > In order to route an /48 IPv6 subnet to the VR you have a few > possibilities: > > - Static route from the upperlying routers which are outside of CloudStack > - BGP > - OSPFv3 (broken in most cases!) > - DHCPv6 Prefix Delegation > > BGP and/or Static routes are still the best bet here. > > So what you do is that you tell CloudStack that you will route > 2001:db8::/48 to the VR, the VR can then use that to split it up into > multiple /64 subnets going towards the instances: > > - 2001:db8::/64 > - 2001:db8:1::/64 > - 2001:db8:2::/64 > ... > - 2001:db8:f::/64 > > And go on. > > In case of BGP you indeed have to tell the VR a few things: > > - It's own AS number > - The peer's address(es) > > With FRR you can simply say: > > neighbor 2001:db8:4fa::179 remote-as external > > The /48 you need to have at the VR anyway in case of either a static > route or BGP. > > We just need to add a NullRoute on the VR for that /48 so that traffic > will not be routed to the upper gateway in case of the VR can't find a > route. > > Wido > > > > > > > > > > > > > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli > > mailto:alex.matti...@shapeblue.com>> > wrote: > > > > Hi Wido, > > That's pretty much in line with our thoughts, thanks for the input. > > I believe we agree on the following points then: > > > > - FRR with BGP (no OSPF) > > - Route /48 (or/56) down to the VR > > - /64 per network > > - SLACC for IP addressing > > > > I believe the next big question is then "on which level of ACS do we > > manage AS numbers?". I see two options: > > 1) Private AS number on a per-zone basis > > 2) Root Admin assigned AS number on a domain/account basis > > 3) End-user driven AS number on a per network basis (for bring your > > own AS and IP scenario) > > > > Thoughts? > > > > Cheers > > Alex > > > > > > > > > > -Original Message- > > From: Wido den Hollander mailto:w...@widodh.nl>> > > Sent: 13 July 2021 15:08 > > To: d...@cloudstack.apache.org <mailto:d...@cloudstack.apache.org>; > > Alex Mattioli > <mailto:alex.matti...@shapeblue.com>> > > Cc: Wei Zhou > <mailto:wei.z...@shapeblue.com>>; Rohit Yadav > > mailto:rohit.ya...@shapeblue.com>>; > > Gabriel Beims Bräscher > <mailto:gabr...@pcextreme.nl>> > > Subject: Re: IPV6 in Isolated/VPC networks > > > > > > > > On 7/7/21 1:16 PM, Alex Mattioli wrote: > > > Hi all, > > > @Wei Zhou<mailto:wei.z...
Re: IPV6 in Isolated/VPC networks
Op 14-07-2021 om 16:44 schreef Hean Seng: Hi I replied in another thread, i think do not need implement BGP or OSPF, that would be complicated . We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. In VR, you need to have Default IPv6 route to Physical Router's /48. IP as IPv6 Gateway. Thens should be done . Example : Physical Router Interface IPv6 IP : 2000:::1/48 Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 route to router ip 2000:::1 and Clodustack Virtual router dhcp allocate IP to VM , and VM will have default route to VR. IPv6 2000::200:201::1 So in cloudstack need to allow user to enter , IPv6 gwateway , and the /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and maintain make sure not ovelap allocation But NAT is truly not the solution with IPv6. IPv6 is supposed to be routable. In addition you should avoid DHCPv6 as much as possible as that's not really the intended use-case for address allocation with IPv6. In order to route an /48 IPv6 subnet to the VR you have a few possibilities: - Static route from the upperlying routers which are outside of CloudStack - BGP - OSPFv3 (broken in most cases!) - DHCPv6 Prefix Delegation BGP and/or Static routes are still the best bet here. So what you do is that you tell CloudStack that you will route 2001:db8::/48 to the VR, the VR can then use that to split it up into multiple /64 subnets going towards the instances: - 2001:db8::/64 - 2001:db8:1::/64 - 2001:db8:2::/64 ... - 2001:db8:f::/64 And go on. In case of BGP you indeed have to tell the VR a few things: - It's own AS number - The peer's address(es) With FRR you can simply say: neighbor 2001:db8:4fa::179 remote-as external The /48 you need to have at the VR anyway in case of either a static route or BGP. We just need to add a NullRoute on the VR for that /48 so that traffic will not be routed to the upper gateway in case of the VR can't find a route. Wido On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli mailto:alex.matti...@shapeblue.com>> wrote: Hi Wido, That's pretty much in line with our thoughts, thanks for the input. I believe we agree on the following points then: - FRR with BGP (no OSPF) - Route /48 (or/56) down to the VR - /64 per network - SLACC for IP addressing I believe the next big question is then "on which level of ACS do we manage AS numbers?". I see two options: 1) Private AS number on a per-zone basis 2) Root Admin assigned AS number on a domain/account basis 3) End-user driven AS number on a per network basis (for bring your own AS and IP scenario) Thoughts? Cheers Alex -Original Message- From: Wido den Hollander mailto:w...@widodh.nl>> Sent: 13 July 2021 15:08 To: d...@cloudstack.apache.org <mailto:d...@cloudstack.apache.org>; Alex Mattioli mailto:alex.matti...@shapeblue.com>> Cc: Wei Zhou mailto:wei.z...@shapeblue.com>>; Rohit Yadav mailto:rohit.ya...@shapeblue.com>>; Gabriel Beims Bräscher mailto:gabr...@pcextreme.nl>> Subject: Re: IPV6 in Isolated/VPC networks On 7/7/21 1:16 PM, Alex Mattioli wrote: > Hi all, > @Wei Zhou<mailto:wei.z...@shapeblue.com <mailto:wei.z...@shapeblue.com>> @Rohit Yadav<mailto:rohit.ya...@shapeblue.com <mailto:rohit.ya...@shapeblue.com>> and myself are investigating how to enable IPV6 support on Isolated and VPC networks and would like your input on it. > At the moment we are looking at implementing FRR with BGP (and possibly OSPF) on the ACS VR. > > We are looking for requirements, recommendations, ideas, rants, etc...etc... > Ok! Here we go. I think that you mean that the VR will actually route the IPv6 traffic and for that you need to have a way of getting a subnet routed to the VR. BGP is probably you best bet here. Although OSPFv3 technically supports this it is very badly implemented in Frr for example. Now FRR is a very good router and one of the fancy features it supports is BGP Unnumered. This allows for auto configuration of BGP over a L2 network when both sides are sending Router Advertisements. This is very easy for flexible BGP configurations where both sides have dynamic IPs. What you want to do is that you get a /56, /48 or something which is >/64 bits routed to the VR. Now you can sub-segment this into separate /64 subnets. You don't want to go smaller then a /64 is that prevents you from using SLAAC for IPv6 address configuration. This is how it works for Shared Networks now in Basic and Advanced Zones. FRR can now also send out the Router Advertisements on the downlinks sen
Re: IPV6 in Isolated/VPC networks
Yes, sorry for that, can use NAT 6 also .I mentiioned DHCP6 , and you can point the gateway to /48 gw, and this does not need any BGP. Maintain BGP or OSPF is good, but is a lot more complicated , On Wed, Jul 14, 2021 at 10:57 PM Alex Mattioli wrote: > Hi Hean, > Do you mean using NAT66? Or did I miss something? > > Regards, > Alex > > > > > -Original Message- > From: Hean Seng > Sent: 14 July 2021 16:44 > To: users@cloudstack.apache.org > Cc: Wido den Hollander ; d...@cloudstack.apache.org; Wei > Zhou ; Rohit Yadav ; > Gabriel Beims Bräscher > Subject: Re: IPV6 in Isolated/VPC networks > > Hi > > I replied in another thread, i think do not need implement BGP or OSPF, > that would be complicated . > > We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT > zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. > > In VR, you need to have Default IPv6 route to Physical Router's /48. IP as > IPv6 Gateway. Thens should be done . > > Example : > Physical Router Interface > IPv6 IP : 2000:::1/48 > > Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 > route to router ip 2000:::1 and Clodustack Virtual router dhcp allocate > IP to VM , and VM will have default route to VR. IPv6 2000::200:201::1 > > So in cloudstack need to allow user to enter , IPv6 gwateway , and the > /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and > maintain make sure not ovelap allocation > > > > > > > > On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli > > wrote: > > > Hi Wido, > > That's pretty much in line with our thoughts, thanks for the input. I > > believe we agree on the following points then: > > > > - FRR with BGP (no OSPF) > > - Route /48 (or/56) down to the VR > > - /64 per network > > - SLACC for IP addressing > > > > I believe the next big question is then "on which level of ACS do we > > manage AS numbers?". I see two options: > > 1) Private AS number on a per-zone basis > > 2) Root Admin assigned AS number on a domain/account basis > > 3) End-user driven AS number on a per network basis (for bring your > > own AS and IP scenario) > > > > Thoughts? > > > > Cheers > > Alex > > > > > > > > > > -Original Message- > > From: Wido den Hollander > > Sent: 13 July 2021 15:08 > > To: d...@cloudstack.apache.org; Alex Mattioli > > > > Cc: Wei Zhou ; Rohit Yadav < > > rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher > > > > Subject: Re: IPV6 in Isolated/VPC networks > > > > > > > > On 7/7/21 1:16 PM, Alex Mattioli wrote: > > > Hi all, > > > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav > rohit.ya...@shapeblue.com> and myself are investigating how to enable > > IPV6 support on Isolated and VPC networks and would like your input on > it. > > > At the moment we are looking at implementing FRR with BGP (and > > > possibly > > OSPF) on the ACS VR. > > > > > > We are looking for requirements, recommendations, ideas, rants, > > etc...etc... > > > > > > > Ok! Here we go. > > > > I think that you mean that the VR will actually route the IPv6 traffic > > and for that you need to have a way of getting a subnet routed to the VR. > > > > BGP is probably you best bet here. Although OSPFv3 technically > > supports this it is very badly implemented in Frr for example. > > > > Now FRR is a very good router and one of the fancy features it > > supports is BGP Unnumered. This allows for auto configuration of BGP > > over a L2 network when both sides are sending Router Advertisements. > > This is very easy for flexible BGP configurations where both sides have > dynamic IPs. > > > > What you want to do is that you get a /56, /48 or something which is > > >/64 bits routed to the VR. > > > > Now you can sub-segment this into separate /64 subnets. You don't want > > to go smaller then a /64 is that prevents you from using SLAAC for > > IPv6 address configuration. This is how it works for Shared Networks > > now in Basic and Advanced Zones. > > > > FRR can now also send out the Router Advertisements on the downlinks > > sending out: > > > > - DNS servers > > - DNS domain > > - Prefix (/64) to be used > > > > There is no need for DHCPv6. You can calculate the IPv6 address the VM > > will obtain by using the MAC and the prefix. > > > > So in short: > > > > - Using BGP you routed a /48 to the VR > > - Now you split this into /64 subnets towards the isolated networks > > > > Wido > > > > > Alex Mattioli > > > > > > > > > > > > > > > > > > -- > Regards, > Hean Seng > -- Regards, Hean Seng
RE: IPV6 in Isolated/VPC networks
Hi Hean, Do you mean using NAT66? Or did I miss something? Regards, Alex -Original Message- From: Hean Seng Sent: 14 July 2021 16:44 To: users@cloudstack.apache.org Cc: Wido den Hollander ; d...@cloudstack.apache.org; Wei Zhou ; Rohit Yadav ; Gabriel Beims Bräscher Subject: Re: IPV6 in Isolated/VPC networks Hi I replied in another thread, i think do not need implement BGP or OSPF, that would be complicated . We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. In VR, you need to have Default IPv6 route to Physical Router's /48. IP as IPv6 Gateway. Thens should be done . Example : Physical Router Interface IPv6 IP : 2000:::1/48 Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 route to router ip 2000:::1 and Clodustack Virtual router dhcp allocate IP to VM , and VM will have default route to VR. IPv6 2000::200:201::1 So in cloudstack need to allow user to enter , IPv6 gwateway , and the /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and maintain make sure not ovelap allocation On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli wrote: > Hi Wido, > That's pretty much in line with our thoughts, thanks for the input. I > believe we agree on the following points then: > > - FRR with BGP (no OSPF) > - Route /48 (or/56) down to the VR > - /64 per network > - SLACC for IP addressing > > I believe the next big question is then "on which level of ACS do we > manage AS numbers?". I see two options: > 1) Private AS number on a per-zone basis > 2) Root Admin assigned AS number on a domain/account basis > 3) End-user driven AS number on a per network basis (for bring your > own AS and IP scenario) > > Thoughts? > > Cheers > Alex > > > > > -Original Message- > From: Wido den Hollander > Sent: 13 July 2021 15:08 > To: d...@cloudstack.apache.org; Alex Mattioli > > Cc: Wei Zhou ; Rohit Yadav < > rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher > > Subject: Re: IPV6 in Isolated/VPC networks > > > > On 7/7/21 1:16 PM, Alex Mattioli wrote: > > Hi all, > > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav rohit.ya...@shapeblue.com> and myself are investigating how to enable > IPV6 support on Isolated and VPC networks and would like your input on it. > > At the moment we are looking at implementing FRR with BGP (and > > possibly > OSPF) on the ACS VR. > > > > We are looking for requirements, recommendations, ideas, rants, > etc...etc... > > > > Ok! Here we go. > > I think that you mean that the VR will actually route the IPv6 traffic > and for that you need to have a way of getting a subnet routed to the VR. > > BGP is probably you best bet here. Although OSPFv3 technically > supports this it is very badly implemented in Frr for example. > > Now FRR is a very good router and one of the fancy features it > supports is BGP Unnumered. This allows for auto configuration of BGP > over a L2 network when both sides are sending Router Advertisements. > This is very easy for flexible BGP configurations where both sides have > dynamic IPs. > > What you want to do is that you get a /56, /48 or something which is > >/64 bits routed to the VR. > > Now you can sub-segment this into separate /64 subnets. You don't want > to go smaller then a /64 is that prevents you from using SLAAC for > IPv6 address configuration. This is how it works for Shared Networks > now in Basic and Advanced Zones. > > FRR can now also send out the Router Advertisements on the downlinks > sending out: > > - DNS servers > - DNS domain > - Prefix (/64) to be used > > There is no need for DHCPv6. You can calculate the IPv6 address the VM > will obtain by using the MAC and the prefix. > > So in short: > > - Using BGP you routed a /48 to the VR > - Now you split this into /64 subnets towards the isolated networks > > Wido > > > Alex Mattioli > > > > > > > > > > -- Regards, Hean Seng
Re: IPV6 in Isolated/VPC networks
Hi I replied in another thread, i think do not need implement BGP or OSPF, that would be complicated . We only need assign IPv6 's /64 prefix to Virtual Router (VR) in NAT zone, and the VR responsible to deliver single IPv6 to VM via DHCP6. In VR, you need to have Default IPv6 route to Physical Router's /48. IP as IPv6 Gateway. Thens should be done . Example : Physical Router Interface IPv6 IP : 2000:::1/48 Cloudstack virtual router : 2000::200:201::1/64 with default ipv6 route to router ip 2000:::1 and Clodustack Virtual router dhcp allocate IP to VM , and VM will have default route to VR. IPv6 2000::200:201::1 So in cloudstack need to allow user to enter , IPv6 gwateway , and the /48 Ipv6 prefix , then it will self allocate the /64 ip to the VR , and maintain make sure not ovelap allocation On Wed, Jul 14, 2021 at 8:55 PM Alex Mattioli wrote: > Hi Wido, > That's pretty much in line with our thoughts, thanks for the input. I > believe we agree on the following points then: > > - FRR with BGP (no OSPF) > - Route /48 (or/56) down to the VR > - /64 per network > - SLACC for IP addressing > > I believe the next big question is then "on which level of ACS do we > manage AS numbers?". I see two options: > 1) Private AS number on a per-zone basis > 2) Root Admin assigned AS number on a domain/account basis > 3) End-user driven AS number on a per network basis (for bring your own AS > and IP scenario) > > Thoughts? > > Cheers > Alex > > > > > -Original Message- > From: Wido den Hollander > Sent: 13 July 2021 15:08 > To: d...@cloudstack.apache.org; Alex Mattioli > Cc: Wei Zhou ; Rohit Yadav < > rohit.ya...@shapeblue.com>; Gabriel Beims Bräscher > Subject: Re: IPV6 in Isolated/VPC networks > > > > On 7/7/21 1:16 PM, Alex Mattioli wrote: > > Hi all, > > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit Yadav rohit.ya...@shapeblue.com> and myself are investigating how to enable > IPV6 support on Isolated and VPC networks and would like your input on it. > > At the moment we are looking at implementing FRR with BGP (and possibly > OSPF) on the ACS VR. > > > > We are looking for requirements, recommendations, ideas, rants, > etc...etc... > > > > Ok! Here we go. > > I think that you mean that the VR will actually route the IPv6 traffic and > for that you need to have a way of getting a subnet routed to the VR. > > BGP is probably you best bet here. Although OSPFv3 technically supports > this it is very badly implemented in Frr for example. > > Now FRR is a very good router and one of the fancy features it supports is > BGP Unnumered. This allows for auto configuration of BGP over a L2 network > when both sides are sending Router Advertisements. This is very easy for > flexible BGP configurations where both sides have dynamic IPs. > > What you want to do is that you get a /56, /48 or something which is > >/64 bits routed to the VR. > > Now you can sub-segment this into separate /64 subnets. You don't want to > go smaller then a /64 is that prevents you from using SLAAC for IPv6 > address configuration. This is how it works for Shared Networks now in > Basic and Advanced Zones. > > FRR can now also send out the Router Advertisements on the downlinks > sending out: > > - DNS servers > - DNS domain > - Prefix (/64) to be used > > There is no need for DHCPv6. You can calculate the IPv6 address the VM > will obtain by using the MAC and the prefix. > > So in short: > > - Using BGP you routed a /48 to the VR > - Now you split this into /64 subnets towards the isolated networks > > Wido > > > Alex Mattioli > > > > > > > > > > -- Regards, Hean Seng
RE: IPV6 in Isolated/VPC networks
Hi Wido, That's pretty much in line with our thoughts, thanks for the input. I believe we agree on the following points then: - FRR with BGP (no OSPF) - Route /48 (or/56) down to the VR - /64 per network - SLACC for IP addressing I believe the next big question is then "on which level of ACS do we manage AS numbers?". I see two options: 1) Private AS number on a per-zone basis 2) Root Admin assigned AS number on a domain/account basis 3) End-user driven AS number on a per network basis (for bring your own AS and IP scenario) Thoughts? Cheers Alex -Original Message- From: Wido den Hollander Sent: 13 July 2021 15:08 To: d...@cloudstack.apache.org; Alex Mattioli Cc: Wei Zhou ; Rohit Yadav ; Gabriel Beims Bräscher Subject: Re: IPV6 in Isolated/VPC networks On 7/7/21 1:16 PM, Alex Mattioli wrote: > Hi all, > @Wei Zhou<mailto:wei.z...@shapeblue.com> @Rohit > Yadav<mailto:rohit.ya...@shapeblue.com> and myself are investigating how to > enable IPV6 support on Isolated and VPC networks and would like your input on > it. > At the moment we are looking at implementing FRR with BGP (and possibly OSPF) > on the ACS VR. > > We are looking for requirements, recommendations, ideas, rants, etc...etc... > Ok! Here we go. I think that you mean that the VR will actually route the IPv6 traffic and for that you need to have a way of getting a subnet routed to the VR. BGP is probably you best bet here. Although OSPFv3 technically supports this it is very badly implemented in Frr for example. Now FRR is a very good router and one of the fancy features it supports is BGP Unnumered. This allows for auto configuration of BGP over a L2 network when both sides are sending Router Advertisements. This is very easy for flexible BGP configurations where both sides have dynamic IPs. What you want to do is that you get a /56, /48 or something which is >/64 bits routed to the VR. Now you can sub-segment this into separate /64 subnets. You don't want to go smaller then a /64 is that prevents you from using SLAAC for IPv6 address configuration. This is how it works for Shared Networks now in Basic and Advanced Zones. FRR can now also send out the Router Advertisements on the downlinks sending out: - DNS servers - DNS domain - Prefix (/64) to be used There is no need for DHCPv6. You can calculate the IPv6 address the VM will obtain by using the MAC and the prefix. So in short: - Using BGP you routed a /48 to the VR - Now you split this into /64 subnets towards the isolated networks Wido > Alex Mattioli > > > >
