Thanks for writing this up, Guru! Since we do have integrators that are not
porting this to hypervisors, I took a crack at making it less
hypervisor-specific. I've added my version below. What do you think?
By the way, we're accumulating a lot of docs in the root directory. What do
people think about moving them to a "docs" directory or something? Also, we
use a number of number of different capitalization styles. If this is the
preferred style, should we think about renaming ones like "PORTING" to
"Porting"?
--Justin
-=-=-=-=-=-=-=-=-=-=-
Integration Guide for Centralized Control
=========================================
This document describes how to integrate Open vSwitch onto a new
platform to expose the state of the switch and attached devices for
centralized control. (If you are looking to port the switching
components of Open vSwitch to a new platform, please see the PORTING
document.) The focus of this guide is on hypervisors, but many of the
interfaces are useful for hardware switches, as well. The XenServer
integration is the most mature implementation, so most of the examples
are drawn from it.
The externally visible interface to this integration is
platform-agnostic. We encourage anyone who integrates Open vSwitch to
use the same interface, because keeping a uniform interface means that
controllers require less customization for individual platforms (and
perhaps no customization at all).
Integration centers around the Open vSwitch database and mostly involves
the 'external_ids' columns in several of the tables. These columns are
not interpreted by Open vSwitch itself. Instead, they provide
information to a controller that permits the controller to associate a
database record with a more meaningful entity. The main job of the
integrator, then, is to ensure that these values are correctly populated
and maintained.
An integrator sets the columns in the database by talking to the
ovsdb-server daemon. A few of the columns can be set during startup by
calling the ovs-ctl tool from inside the startup scripts. The
'xenserver/etc_init.d_openvswitch' script provides examples of its use,
and the ovs-ctl(8) manpage contains complete documentation. At runtime,
ovs-vsctl can be be used to set columns in the database. The script
'xenserver/etc_xensource_scripts_vif' contains examples of its use, and
ovs-vsctl(8) manpage contains complete documentation.
Python and C bindings to the database are provided if deeper integration
with a program are needed. The XenServer ovs-xapi-sync daemon
('xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync') provides an
example of using the Python bindings. More information on the python
bindings is available at 'python/ovs/db/idl.py'. Information on the C
bindings is available at 'lib/ovsdb-idl.h'.
The following diagram shows how integration scripts fit into the Open vSwitch
architecture:
+----------------------------------------+
| Controller Cluster +
+----------------------------------------+
|
|
+----------------------------------------------------------+
| | |
| +--------------+---------------+ |
| | | |
| +-------------------+ +------------------+ |
| | ovsdb-server |-----------| ovs-vswitchd | |
| +-------------------+ +------------------+ |
| | | |
| +---------------------+ | |
| | Integration scripts | | |
| | (ex: ovs-xapi-sync) | | |
| +---------------------+ | |
| | Userspace |
|----------------------------------------------------------|
| | Kernel |
| | |
| +---------------------+ |
| | OVS Kernel Module | |
| +---------------------+ |
+----------------------------------------------------------+
A description of the most relevant fields for integration follows. By
setting these values, controllers are able to understand the network and
manage it more dynamically and precisely. For more details about the
database and each individual column, please refer to the
ovs-vswitchd.conf.db(5) manpage.
Open_vSwitch table
------------------
The Open_vSwitch table is used to describe the switch as a whole. The
'system_type' and 'system_version' columns are used to identify the
platform to the controller. The 'external_ids:system-id' key is used
to uniquely identify the physical host. In XenServer, the system-id
will likely be the same as the UUID returned by 'xe host-list'. This key
allows controllers to distinguish between multiple hypervisors.
Most of this configuration can be done with the ovs-ctl command at
startup. For example:
ovs-ctl --system-type="XenServer" --system-version="6.0.0-50762p" \
--system-id="${UUID}" "${other_options}" start
Alternatively, the ovs-vsctl command may be used to set a particular
value at runtime. For example:
ovs-vsctl set open_vswitch . external-ids:system-id='"${UUID}"'
The 'other_config:enable-statistics' key may be set to "true" to have OVS
populate the database with statistics (e.g., number of CPUs, memory,
system load) for the controller's use.
Bridge table
------------
The Bridge table is used to describe individual bridges within an Open
vSwitch instance. The 'external-ids:bridge-id' key is used to uniquely
identify a particular bridge. In XenServer, this will likely be the
same as the UUID returned by 'xe network-list' for that particular
bridge.
For example, to set the identifier for bridge "br0", the following
command can be used:
ovs-vsctl set Bridge br0 external-ids:bridge-id='"${UUID}"'
The MAC address of the bridge may be manually configured by setting it
with the "other_config:hwaddr" key. For example:
ovs-vsctl set Bridge br0 other_config:hw_addr="01:23:45:67:89:0a"
Interface table
---------------
The Interface table is used to describe an interface under the control
of Open vSwitch. The 'external_ids' column contains keys that are used
to provide additional information about the interface:
attached-mac
This field contains the MAC address of the device attached to
the interface. On a hypervisor, this is the MAC address of the
interface as seen inside a VM. Typically in Linux-based
hypervisors, a tap interface is created and the VM sees this as
a physical NIC with a valid MAC address. The VM’s kernel writes
L2 frames to this interface and the frame makes it to the
bridge.
iface-id
This field uniquely identifies the interface. In hypervisors,
this allows the controller to follow VM network interfaces as
VMs migrate. A well-chosen identifier should also allow an
administrator or a controller to associate the interface with
the corresponding object in the VM management system. For
example, the Open vSwitch integration with XenServer by default
uses the XenServer assigned UUID for a VIF record as the
iface-id.
iface-status
In a hypervisor, there are situations where there are multiple
interface choices for a single virtual ethernet interface inside
a VM. An example for this situation is VIF and tap interfaces
in the XenServer. XenServer uses VIF interfaces for
para-virtual enabled VMs and tap interfaces for regular VMs.
But only one interface can be used at a time. This external-id
can have the value 'active' or 'inactive' to indicate the
interface that is currently under use. One may also use this
external-id for more complicated strategies such as VM
migration.
vm-id
This field uniquely identifies the VM to which this interface
belongs. A single VM may have multiple interfaces attached to
it.
As in the previous tables, the ovs-vsctl command may be used to
configure the values. For example, to set the 'iface-id' on eth0, the
following command can be used:
ovs-vsctl set Interface eth0 external-ids:iface-id='"${UUID}"'
On Apr 25, 2012, at 10:27 AM, Gurucharan Shetty wrote:
> The goal of this document is to give some information to
> the integrators that plan to use OpenvSwitch in multiple
> hypervisors running multiple VMs.
>
> Signed-off-by: Gurucharan Shetty <[email protected]>
> ---
> IntegrationGuide | 143 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 files changed, 143 insertions(+), 0 deletions(-)
> create mode 100644 IntegrationGuide
>
> diff --git a/IntegrationGuide b/IntegrationGuide
> new file mode 100644
> index 0000000..0565943
> --- /dev/null
> +++ b/IntegrationGuide
> @@ -0,0 +1,143 @@
> +Open vSwitch integrates smoothly with XenServer. The externally
> +visible interface to this integration is hypervisor-agnostic. We
> +encourage anyone who integrates Open vSwitch with a hypervisor to use
> +the same interface, because keeping a uniform interface means that
> +controllers require less customization for individual hypervisor
> +platforms (perhaps no customization at all).
> +
> +This document describes the suggested Open vSwitch interface to
> +hypervisors and the rationale behind it. The XenServer integration is
> +the most mature implementation, so most of the examples are drawn from
> +it.
> +
> +Hypervisor integration centers around the Open vSwitch database and in
> +particular the 'external-ids' column in several of its tables. Each
> +'external-ids' column contains string key-value pairs. Nothing inside
> +Open vSwitch itself interprets these key-value pairs. Instead, they
> +provide information to a controller that permits the controller to
> +associate a database record with a more meaningful entity. The main
> +job of the integrator, then, is to ensure that the various
> +'external-ids' are correctly populated and maintained.
> +
> +An integrator can set the columns in the database by talking to the
> +ovsdb-server daemon. Few of the columns can be set during the system startup
> +by calling the ovs-ctl tool from inside the startup scripts. You can look at
> +xenserver/etc_init.d_openvswitch as an example or refer to the ovs-ctl(8)
> +manpage. ovs-vsctl can be also be used to set the columns in the database.
> +You can look at xenserver/etc_xensource_scripts_vif as an example or refer to
> +the ovs-vsctl(8) man page.
> +
> +Using the Python or C bindings provided by Open vSwitch is another way. An
> +example of using the python bindings is the ovs-xapi-sync daemon in the
> +XenServer. You can look at this code in
> +xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync. More information for
> +the python bindings is available at python/ovs/db/idl.py. Information for
> the
> +C bindings is available at lib/ovsdb-idl.h.
> +
> +The following diagram shows how integration scripts fit into the Open vSwitch
> +architecture.
> +
> + +-----------------------+ +-----------------+
> + |Integration scripts | | Kernel |
> + |ex: ovs-xapi-sync | +-----------------+
> + +-----------------------+ |
> + | |
> + | |
> + +-----------------------+ +-----------------+
> + | ovsdb-server |<----------->| ovs-vswitchd |
> + +-----------------------+ +-----------------+
> + | |
> + | |
> + +-----------------------+ |
> + | Controller |---------------------|
> + +-----------------------+
> +
> +
> +For more details about the database and each individual column, please
> +refer to the ovs-vswitchd.conf.db(5) manpage.
> +
> +The 'external-ids' column in the following tables, if set with appropriate
> +key-value pairs by the integration scripts, can be used by the controllers to
> +create, remove, migrate or control the VMs in a more efficient manner.
> +
> +Open_vSwitch table
> +------------------
> +The 'external_ids' column in the 'Open_vSwitch' table is used to set
> +multiple key-value pairs with information about the host/hypervisor running
> +Open vSwitch. One example for the key is 'system-id' which can uniquely
> +identify the hypervisor.
> +
> +You should ideally set this in a start up script using ovs-ctl.
> +ovs-ctl --system-id="${UUID}" "${other_options}" start
> +
> +You can also set this using the following ovs-vsctl command.
> +ovs-vsctl set open_vswitch . external-ids:system-id='"${UUID}"'
> +
> +In XenServer, this will likely be the same as the UUID returned by
> +'xe host-list' and is set in one of the startup scripts. This key helps
> +controllers distinguish among multiple hypervisors.
> +
> +You can set the 'other_config' column with a key-value pair of
> +'enable-statistics="true"'. This causes OVS to populate the database with
> +hypervisor statistics, such as the number of CPUs, memory, system load, etc.,
> +for the controller's use.
> +
> +You can also set some appropriate values to the 'system_type' and
> +'system_version' columns through ovs-ctl tool that will let you differentiate
> +between different host software types and versions.
> +
> +Bridge table
> +------------
> +The 'external_ids' column in the 'Bridge' table is used to set multiple
> +key-value pairs with information about a particular bridge. One example for
> +the key is 'bridge-id' which uniquely identifies a bridge. In XenServer,
> +this will likely be the same as the UUID returned by 'xe network-list' for
> +that particular bridge. A controller uses this unique key to identify a
> +bridge, get its name, and then run some commands for that particular bridge.
> +ex:
> +ovs-vsctl -- --columns=name find bridge external-ids:bridge-id='"${UUID}"'
> +
> +You can also set the 'other_config' column with a key-value pair of
> +'hwaddr="xx:xx:xx:xx:xx:xx"'. This field should have the MAC address of the
> +bridge.
> +
> +Interface table
> +---------------
> +The 'external_ids' column in the 'Interface' table is used to set multiple
> +key-value pairs with information about a particular interface. Some of the
> +examples for the keys are:
> +
> +attached-mac:
> +This field contains the MAC address of this interface as seen inside a VM.
> +Typically in Linux based hypervisors, a tap interface is created and the VM
> +sees this as a physical NIC with a valid MAC address. The VM’s kernel writes
> +L2 frames to this interface and the frame makes it to the bridge.
> +
> +iface-id:
> +This field uniquely identifies the interface, to allow the controller to
> +follow VM network interfaces as VMs migrate around a collection of
> hypervisors.
> +A well-chosen identifier should also allow an administrator or a controller
> to
> +associate the interface with the corresponding object in the VM management
> +system. For example, the Open vSwitch integration with XenServer by default
> +uses the XenServer assigned UUID for a VIF record as iface-id.
> +
> +iface-status:
> +There are situations where there are multiple interface choices in the
> +hypervisor for a single virtual ethernet interface inside a VM. An example
> +for this situation is vif and tap interfaces in the XenServer. XenServer
> uses
> +vif interfaces for para-virtual enabled VMs and tap interfaces for regular
> +VMs. But only one interface can be used at a time. This external id can
> have
> +the value 'active' or 'inactive' to indicate the interface that is currently
> +under use. One can probably also use this external id for more complicated
> +strategies like VM migration.
> +
> +vm-id:
> +This field uniquely identifies the VM to which this interface belongs.
> +A single VM can have multiple interfaces attached to it. This value will
> allow
> +an admin or controller to configure or add flows to the bridge based on the
> VMs
> +to which they belong.
> +
> +An administrator/controller can use the above identifiers to get the 'ofport'
> +associated with this interface and then manage an OpenFlow flow.
> +Ex: ovs-vsctl -- --columns=ofport find interface \
> +external-ids:attached-mac='"92:a8:d3:65:3f:33"'
> --
> 1.7.2.5
>
> _______________________________________________
> dev mailing list
> [email protected]
> http://openvswitch.org/mailman/listinfo/dev
_______________________________________________
dev mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/dev
