On Mon, Jan 13, 2014 at 02:39:32pm +0100, Jose A. Lopes wrote: > On Fri, Jan 10, 2014 at 05:50:32PM +0200, Vangelis Koukis wrote: > [snip] > > In a similar note, who will be responsible for setting up the DHCP > > server? It could be the administrator's responsibility, but then if it > > is Ganeti the entity which picks the MAC addresses and IPs for the guest > > side of the TAP interfaces, how will this DHCP server be notified, so as > > to only server the correct IP addresses to specific MAC addresses? > > > > Also, if it is the administrator's responsibility, then perhaps the > > admin should be able to set up standard ifup hooks, like for every > > other interface of an instance. But in the following examples, you > > specifically set script=no,downscript=no. > > > > Another possibility would be for Ganeti to come prepackaged with its > > own, embedded DHCP server just for serving requests on the TAPs used for > > the communication mechanism. We've been using snf-nfdhcpd > > (https://code.grnet.gr/projects/snf-nfdhcpd) for just that in > > production. > > > > Actually, in previous conversation Guido had asked us to document how to > > set it up with Ganeti, and merge the resulting docs with the Ganeti > > upstream. Perhaps it would make sense to combine the effort now, and use > > snf-nfdhcpd as an embedded DHCP server with Ganeti. Sorry for not having > > documented it earlier. > > And by the way, just out of curiosity, what were the reasons that lead > you to implement your own DHCP server? Have considered other servers? > Have you tried, for example, dnsmasq? >
Hello Jose,
I'll try to summarize here the main principles behind snf-nfdhcpd,
and its differences compared to dnsmasq or similar servers.
snf-nfdhcpd is based on NFQUEUE. The administrator can enable processing
of DHCP requests on individual TAP interfaces by injecting snf-nfdhcpd
in the processing pipeline for IP packets dynamically.
The daemon runs on the host and is controlled by manipulating files under its
state directory, /var/lib/nfdhcpd by default (configurable). Creation of a new
file under this directory ("binding file") instructs the daemon to answer DHCP
requests on the specified TAP interface. Please see below for an example
binding file.
A simple scenario is:
a) snf-nfdhcpd starts. Upon initialization, it creates an NFQUEUE (e.g., 42,
configurable), and listens on it for incoming DHCP requests. It also begins to
watch its state directory, /var/lib/nfdhcpd via inotify().
b) A new VM gets created, let's assume its NIC has address mac0, lives on TAP
interface tap0, and is to receive IP address ip0 via DHCP.
c) Someone (e.g., a Ganeti KVM ifup script, or in our case snf-network,
see http://code.grnet.gr/projects/snf-network) creates a new binding
file informing snf-nfdhcpd that it is to reply to DHCP requests from MAC
mac0 on TAP interface tap0, and include IP ip0 in the DHCP reply.
d) The administrator injects snf-nfdhcpd in the processing pipeline for packets
coming from tap0, using iptables. This can happen for every TAP
interface, e.g.:
# iptables -t mangle -A PREROUTING -i tap+ -m udp -p udp --dport 67 -j NFQUEUE
--queue-num 42
or for individual TAP interfaces.
e) From now on, whenever a DHCP request is sent out by the VM, the
iptables rule will forward the packet to nfdhcpd, which will consult
its bindings database, find the entry for tap0, verify the source MAC,
and inject a DHCP reply for the corresponding IP address into tap0.
This has various advantages compared to dnsmasq or similar servers:
a) The DHCP service can be activated dynamically, per-interface,
by manipulating iptables accordingly. There is no need to restart
the daemon, or edit (potentially read-only) configuration files,
you only need to drop a file under /var/lib/nfdhcpd.
b) There is no interference to existing DHCP servers listening to port
67. Everything happens directly via NFQUEUE.
c) The host doesn't even need to have an IP address on the interfaces
where DHCP replies are served, making it invisible to the VMs. This
may be beneficial from a security perspective. Similarly, it doesn't
matter if the TAP interface is bridged or routed.
d) MAC addresses are bound on TAP interfaces. Requests coming from
unrelated TAP interfaces are ignored, and packet processing happens
as if snf-nfdhcpd didn't exist in the first place.
e) snf-nfdhcpd is written in pure Python and uses scapy for packet
processing. This has proved super-useful when trying to troubleshooting
networking problems in production.
Example snf-nfdhcpd binding file:
A binding file in snf-nfdhcpd's state directory is named after the
physical interface where the daemon is to receive incoming DHCP requests
from, and defines at least the following variables:
* INDEV: The logical interface where the packet is received on. For
bridged setups, the bridge interface, e.g., br0. Otherwise, same as
the file name.
* MAC: The MAC address where the DHCP request must be originating from
* IP: The IPv4 address to be returned in DHCP replies
* SUBNET: The IPv4 subnet to be returned in DHCP replies in CIDR form
* GATEWAY: The IPv4 gateway to be returned in DHCP replies
Please see package snf-network for an example Ganeti KVM ifup script
which exercises all of the above, plus IPv6 autoconfiguration:
https://code.grnet.gr/projects/snf-network/repository/revisions/master/entry/common.sh#L153
We are in the process of integrating the above description as
documentation for snf-nfdhcpd.
Looking forward to your comments,
Thanks,
Vangelis.
signature.asc
Description: Digital signature
