Hi,
See my inline responses:
Op 11-08-2021 om 14:26 schreef Rohit Yadav:
Hi all,
Thanks for your feedback and ideas, I've gone ahead with discussing them
with Alex and came up with a PoC/design which can be implemented in the
following phases:
* Phase1: implement ipv6 support in isolated networks and VPC with
static routing
* Phase2: discuss and implement support for dynamic routing (TBD)
For Phase1 here's the high-level proposal:
* IPv6 address management:
* At the zone level root-admin specifies a /64 public range that
will be used for VRs, then they can add a /48, or /56 IPv6 range for guest
networks (to be used by isolated networks and VPC tiers)
* On creation of any IPv6 enabled isolated network or VPC tier,
from the /48 or /56 block a /64 network is allocated/used
* We assume SLAAC and autoconfiguration, no DHCPv6 in the zone
(discuss: is privacy a concern, can privacy extensions rfc4941 of slaac be
explored?)
Privacy Extensions are only a concern for client devices which roam
between different IPv6 networks.
If you IPv6 address of a client keeps the same suffix (MAC based) and
switches network then only the prefix (/64) will change.
This way a network like Google, Facebook, etc could track your device
moving from network to network if they only look at the last 64-bits of
the IPv6 address.
For servers this is not a problem as you already know in which network
they are.
* Network offerings: root-admin can create new network offerings
(with VPC too) that specifies a network stack option:
* ipv4 only (default, for backward compatibility all
networks/offerings post-upgrade migrate to this option)
* ipv4-and-ipv6
* ipv6-only (this can be phase 1.b)
* A new routing option: static (phase1), dynamic (phase2, with
multiple sub-options such as ospf/bgp etc...)
This means that the network admin will need to statically route the IPv6
subnet to the VR's outside IPv6 address, for example, on a JunOS router:
set routing-options rib inet6.0 static route 2001:db8:500::/48 next-hop
2001:db8:100::50
I'm assuming that 2001:db8:100::50 is the address of the VR on the
outside (/64) network. In reality this will probably be a longer
address, but this is for just the example.
* VR changes:
* VR gets its guest and public nics set to inet6 auto
* For each /64 allocated to guest network and VPC tiers, radvd
is configured to do RA
radvd is fine, but looking at phase 2 with dynamic routing you might
already want to look into FRRouting. FRR can also advertise RAs while
not doing any routing.
interface ens4
no ipv6 nd suppress-ra
ipv6 nd prefix 2001:db8:500::/64
ipv6 nd rdnss 2001:db8:400::53 2001:db8:200::53
See: http://docs.frrouting.org/en/latest/ipv6.html
* Firewall: a new ipv6 zone/chain is created for ipv6 where ipv6
firewall rules (ACLs, ingress, egress) are implemented; ACLs between VPC
tiers are managed/implemented by ipv6 firewall on VR
Please take a look at the existing security_group.py script which
implements RFC4890
https://datatracker.ietf.org/doc/html/rfc4890
ICMPv6 is a vital part of IPv6 and certain packets should always be
allowed.
* It is assumed that static routes are created on the core/main
router by the admin or automated using some scripts/tools; for this
CloudStack will announce events with details of /64 networks and VR's
public IPv6 address that can be consumed by a rabbitmq/message bus client
(for example), or a custom cron job or script as part of orchestration.
(this wouldn't be necessary for dynamic routing bgp with phase2)\\
You would only need to announce the /48 or /56 allocated to the VR,
that's all. You don't need to inform the upstream router about the /64
subnets created within that larger subnet.
* Guest Networking: With SLAAC, it's easy for CloudStack to
calculate allocate and use a /64 and determine the IPv6 address of VR nics
and guest VM nics
* A user create an isolated network/VPC with an offering that is
ipv6 enabled
* A user can manage firewall for the IPv6 address/guest nics;
there'll be no port forward and LB feature though for IPv6
* A users can run workloads in the guest VMs that listen on
publically routable ipv6 addresses
* Usage/billing etc continue to work, no change needed
Network layout:
[core/ISP router] -> [VR] -> [guest netwokr or VPC tier on a VLAN] ->
[guest VMs/nics]
*core/ISP router needs static routes to be added (manually or
automated), assumes a /48 or /56 configured for the zone
Thoughts, feedback?
Looks doable!
Side-note: It would be very cool if you could use parts of this
implementation to also route /48, /56, or /60 subnets to individual VMs
in Shared networks.
Why? This allows for running Docker containers with native IPv6 inside
the VM or running a (Open)VPN server inside a VM which then assigns
public IPv6 addresses to clients connected.
Instead of routing the subnet to a VR we route the subnet to a single
instance in a shared network.
If we could then also move these subnets between Instances easily one
can quickly migrate to a different instance while keeping the same IPv6
subnet.
Wido
Proof-of-concept commentary: here's what I did to test the idea:
* Created an isolated network and deployed a VM in my home lab
The VR running on KVM has following nics
eth0 - guest network
eth1 - link local
eth2 - public network
* I setup a custom openwrt router on a RPi4 to serve as a toy-core
router where I create a wan6 IPv6 tunnel using tunnel broker and I got a
/48 allocated. My configuration looks like:
/48 - 2001:470:ed36::/48 (allocated by tunnel broker)
/64 - 2001:470:36:3e2::/64 (default allocated by)
I create a LAN ipv6 (public network for CloudStack VR): at subnet/prefix
0:
LAN IPv6 address: 2001:470:ed36:0::1/64
Address mode: SLAAC+stateless DHCP (no dhcpv6)
*
*
In the isolated VR, I enabled ipv6 as:
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.accept_ra = 1
net.ipv6.conf.all.accept_redirects = 1
net.ipv6.conf.all.autoconf = 1
Set up a IPv6 nameserver/dns in /etc/resolve.conf
And configured the nics:
echo iface eth0 inet6 auto >> /etc/network/interfaces
echo iface eth2 inet6 auto >> /etc/network/interfaces
/etc/init.d/networking restart
Next, restart ACS isolated network without cleanup to have it
reconfigure IPv4 nics, firewall, NAT etc
*
Next, I created a /64 network for the isolated guest network on eth0 of
VR using radvd:
# cat /etc/radvd.conf
interface eth0
{
AdvSendAdvert on;
MinRtrAdvInterval 5;
MaxRtrAdvInterval 15;
prefix 2001:470:ed36:1::/64
{
AdvOnLink on;
AdvAutonomous on;
};
};
systemctl restart radvd
All guest VMs nics and VR's eth0 gets IPv6 address (SLAAC) in this
...:1::/64 network
* Finally I added a static route in toy core-router for the new /64
IPv6 range in the isolated network
2001:470:ed36:1::/64 via <public IPv6 address of the VR on eth2> dev
<local lan nic>
*
... and I enabled firewall rules to allow any traffic to pass for the
new /64 network
And voila all done! I create a domain AAAA record that points to my
guest VM IPv6 address a test webserver on
http://ipv6-isolated-ntwk-demo.yadav.cloud/
(Note: I'll get rid of the tunnel and request a new /48 block after a
few days, sharing this solely for testing purposes)
Regards.
