Updated proposal, based on feedback the last time around.
dme.
IPv4 Network Configuration Enhancements for Xen Guest Domains
=============================================================
Author: David Edmondson <[EMAIL PROTECTED]>
Date: 2007/03/14 11:58:58
$Id: xen-ip-config.org,v 1.4 2007/03/14 11:58:49 dme Exp $
Table of Contents
=================
1 Introduction
1.1 Further Work
2 Configuration Options
2.1 Specifying Configuration Options
2.2 Kernel Command-line Arguments
2.2.1 Examples
3 Accessing the Configuration Data in a Guest Domain
4 Non-Diskless Systems
4.1 Interaction with sysidtool(1M)
5 Diskless Systems
5.1 Ordering of Configuration Sources
6 Oustanding Issues
6.1 Multiple Network Interfaces
7 /sbin/devprop
8 Interaction with other projects
9 Interface Table
1 Introduction
##############
The Xen hypervisor and the port of Solaris to it are described in
PSARC/2006/260. That case is considered required pre-reading for
this.
The IPv4 configuration attributes for a network interface in Solaris
are traditionally managed in three ways:
- Static configuration files in /etc used by the /network/physical SMF
service (this includes DHCP for diskfull systems),
- DHCP sourced information for diskless systems, typically obtained by
firmware and provided to the bootloader and Solaris kernel during
boot,
- RARP sourced information for diskless systems, typically obtained by
the Solaris kernel during boot.
The first and last of these mechanisms work for guest domains today.
The second mechanism is specific to systems booting on hardware and
hence not currently relevant to Solaris on Xen.
As part of the configuration details for a guest domain it is possible
and common to specify a limited set of IPv4 attributes. These details
are passed to the kernel booting in the guest domain as command line
arguments. The set of attributes supported is defined by the
pre-existing hypervisor configuration tools.
This proposal describes how the current Solaris IPv4 configuration
mechanisms will be enhanced to observe and, as appropriate, use the
configuration details provided to the kernel.
1.1 Further Work
^^^^^^^^^^^^^^^^
Future projects may include:
- Enhancing the hypervisor control tools (specifically the domain
builder) to perform DHCP based attribute discovery (see RFE
6528791).
- Improving the interaction between the hypervisor control tools and
other components that may use a DHCP server in the control domain to
manage local virtual machines.
- Enhancing the hypervisor control tools and associated mechanism to
support the specification of IPv6 attributes.
None of the possible future projects are included in this proposal,
and it is believed that the proposal places no restriction on their
possible implementation at a future date.
2 Configuration Options
#######################
2.1 Specifying Configuration Options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Guest domain configuration files are fragments of Python[1] code.
Certain variable names are "well known" to the configuration
infrastructure and can be set to control the creation of the guest
domain.
In this context the configuration variables of concern are:
| Variable | Type | Meaning | Examples
|
|------------+--------+-----------------------------------+-----------------------------|
| ip | string | IPv4 address of the domain | "10.6.39.40"
|
| netmask | string | netmask of the domain | "255.255.255.0"
|
| gateway | string | default router for the domain | "10.6.39.1"
|
| dhcp | string | is DHCP enabled? | "on" "off"
|
| vif | list | network interfaces specifications | see below
|
| nfs_root | string | pathname of NFS root filesystem |
"pushrod:/export/root/xpv1" |
| nfs_server | string | IPv4 address of the NFS server | "10.6.39.80"
|
This set of variables are defined by the implementation of the
hypervisor control tools, which are External.
The "vif" specification controls how many[2] inter-domain network
interfaces are available in the guest domain and some configuration
information for those network interfaces. The "vif" specification
takes the form of a Python list with an element for each network
interface available to the guest domain. For example:
: vif = [ 'mac="aa:0:1:2:3:4"' ]
indicates that a single network interface is provided to the domain
(as the list contains a single element) and the MAC address of that
interface is "aa:0:1:2:3:4".
Similarly:
: vif = [ '', '' ]
indicates two network interfaces provided to the domain. Both of
these interfaces have automatically generated MAC addresses.
The "vif" specification allows other configuration items per-network
interface that are not relevant to this case.
The hypervisor control tools use the configuration variables during the
creation of the guest domain and also construct two command line
arguments that are passed to the booted kernel.
2.2 Kernel Command-line Arguments
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The configuration details specified during domain creation are used to
construct two command line arguments for the kernel being booted in
the guest domain. Each of the arguments takes the form of
name, the equals sign ("=") and a colon separated list of settings.
The two parameters of relevance are "ip" and "nfsroot". Each is
followed by a fixed number of settings, seven for "ip" and three for
"nfsroot". The meaning of each setting is defined by its' position.
For "ip":
| Setting | Meaning |
|---------+---------------------------------------------------|
| 1 | IPv4 address |
| 2 | NFS server (ignored in favour of "nfsroot" below) |
| 3 | default router |
| 4 | netmask |
| 5 | hostname |
| 6 | network interface name |
| 7 | host configuration protocol |
For "nfsroot":
| Setting | Meaning |
|---------+------------------------------------|
| 1 | NFS server IPv4 address |
| 2 | NFS server name |
| 3 | root filesystem path on NFS server |
If no information is provided for a setting it is simply left blank.
2.2.1 Examples
~~~~~~~~~~~~~~
: ip=10.6.39.40:10.6.39.80::255.255.255.0::eth0:off
| Setting | Value |
|-----------------------------+---------------|
| IPv4 address | 10.6.39.40 |
| NFS server | 10.6.39.80 |
| default router | none |
| netmask | 255.255.255.0 |
| hostname | none |
| network interface name | eth0 |
| host configuration protocol | off |
: nfsroot=10.6.39.80:pushrod:/export/root/xpv1
| Setting | Value |
|-------------------------+-------------------|
| NFS server IPv4 address | 10.6.39.80 |
| NFS server name | pushrod |
| root filesystem path | /export/root/xpv1 |
3 Accessing the Configuration Data in a Guest Domain
####################################################
During early startup the Solaris kernel creates boot properties for
all command line arguments provided by the hypervisor guest domain
builder. Each command line argument has "xpv-" prefixed to generate a
boot property name. Hence the following system properties will be
created by the example above (output extracted from "prtconf -v"):
: name='xpv-ip' type=string items=1
: value='10.6.39.40:10.6.39.80::255.255.255.0::eth0:off'
: name='xpv-nfsroot' type=string items=1
: value='10.6.39.80:pushrod:/export/root/xpv1'
The "xpv-ip" and "xpv-nfsroot" properties are further decomposed and
used to set other system properties. Where possible system properties
already used by Solaris are set to an appropriate value:
| System property | Meaning | Existing property? |
|-------------------+-----------------------------+--------------------|
| host-ip | IPv4 address | yes |
| router-ip | default router | yes |
| subnet-mask | netmask | yes |
| network-interface | network interface name | yes[3] |
| xpv-hcp | host configuration protocol | no |
| fstype | root filesystem type | yes |
| server-ip | NFS server IPv4 address | yes |
| server-name | NFS server name | yes |
| server-path | root filesystem path | yes |
All of these system properties can be accessed by the normal property
lookup routines (see ddi_prop_lookup(9F)).
A new utility is provided to allow the examination of system
properties from within shell scripts (/sbin/devprop).
4 Non-Diskless Systems
######################
The configuration of network interfaces in non-diskless systems is
typically the responsibility of the /network/physical service. This
service has been enhanced such that in the absence of any of the
existing configuration mechanisms (e.g. files in /etc) the system
properties generated from the guest domain configuration are examined.
If the "network-interface" property is set then the "xpv-hcp" property
is examined. Three possible values for "xpv-hcp" are understood:
"dhcp" - DHCPv4 based configuration,
"off" - static IP address configuration,
empty - static IP address configuration.
A setting of "dhcp" results in the specified network interface being
plumbed and, presuming that this is successful, the DHCP agent is
started to manage the interface.
For the "off" and empty settings three further system properties are
examined - host-ip, subnet-mask and router-ip. If host-ip and
subnet-mask are present then the specified network interface is
plumbed, the IP address and netmask are set and the interface is
marked "up". If router-ip is also set then a default route via that
host is added to the system.
4.1 Interaction with sysidtool(1M)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When a freshly installed guest domain is first booted the
sysidtool(1M) framework will offer to store configuration data for any
network interfaces available to the guest domain. Any IPv4
configuration attributes specified here would prevent the use of
similar attributes specified in the guest domain configuration files.
5 Diskless Systems
##################
Diskless systems rely on kernel-based configuration of a network
interface to allow them to reach the NFS server which provides their
root filesystem. As such it is not possible to rely on changes to the
/network/physical service.
The diskless root mechanism is enhanced to examine system properties
in a manner similar to that described for non-diskless systems with
the following differences:
- the "xpv-hcp" property is ignored and no DHCP based mechanism is
available - the address must be statically configured,
- all of the following properties _must_ be specified:
host-ip, subnet-mask, router-ip, server-path, server-name,
server-ip
Providing all of the required properties are present, they are used to
configure the network interface and mount the root filesystem from the
NFS server.
5.1 Ordering of Configuration Sources
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The current diskless boot mechanism prefers DHCP parameters over RARP.
This project will cause the system to prefer system properties over
DHCP parameters.
6 Oustanding Issues
###################
6.1 Multiple Network Interfaces
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The guest domain configuration file allows an administrator to
indicate that multiple network interfaces should be provided to the
guest domain but only allows the configuration details for the first
interface to be declared.
7 /sbin/devprop
###############
: Usage: devprop [-n device-path] [-vq] [-{b|i|l|s}] [property [...]]
devprop provides easy access to properties from the device tree for
administrators and scripting language developers (most commonly shell
scripts).
devprop will be delivered on all Solaris platforms.
8 Interaction with other projects
#################################
The Network Auto-Magic (NWAM) project[4] aims to:
"...simplify and automate network configuration on Solaris."
The current description[5] of NWAM suggests that the information
provided in the hypervisor guest domain configuration would be input
to, or perhaps an automatically created, link-layer profile.
The details of this interaction should be discussed further with the
NWAM community.
9 Interface Table
#################
| Interface | Stability |
Comments |
|-----------------------------------------------------------+-------------+-----------------------------|
| /sbin/devprop | Committed |
location |
| devprop [-n device-path] [-vq] [-{bils}] [property [...]] | Volatile | CLI
options, output |
| kernel command line arguments | Uncommitted |
Hypervisor tools may change |
| system properties beginning "xpv-" | Uncommitted |
Hypervisor tools may change |
Footnotes:
[1] http://www.python.org
[2] Earlier versions of hypervisor tools supported a "nics" variable which
set the number of network interfaces available to the guest
domain. This use of this parameter is discouraged and it will
cease to work in a future revision of the tools.
[3] The hypervisor tools have a default network interface name of "eth0".
This string, if seen, is translated into "xnf0" as a usability
improvement. A message is output reporting that the translation
has taken place.
[4] http://www.opensolaris.org/os/project/nwam/
[5] http://www.opensolaris.org/os/project/nwam/architecture/
_______________________________________________
xen-discuss mailing list
xen-discuss@opensolaris.org
