Hello,
Here is an update on the issue of not being able to capture packets flowing
between two VMs on the same host.
Isolating the problem:
It has been confirmed that the TaaS OVS driver cannot capture packets
ingressing (flowing into) a VM's vNIC under the following conditions:
- The sender port and the receiver port (which is being monitored) reside on
the same host.
- The sender port and the receiver port are attached to the same (virtual)
network.
We do not have a problem in the above situation for traffic egressing (flowing
out) of the VM's vNIC. We also do not have a problem if the sender port and
receiver port are on different hosts, or if the sender and receiver ports
belong to different (virtual) networks.
Trafffic Capture Technique:
The implementation uses the following methods to capture packets flowing into
and out of a VM's vNIC.
Egress Side (flowing out of the vNIC):
The 'in_port=<OVS-Port-ID>' check is used to identify packets coming in from
the port associated with the VM's vNIC. This allows us to capture all packets
flowing out of the vNIC under question. No problems here.
Ingress Side (flowing into the vNIC):
The 'dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address>' check is used to
identify packets destined for the port associated with the VM's vNIC.
The VLAN tag check was put in place because Neutron port MAC addresses are
required to be unique only within a (virtual) network. Now each instance port
in br-int gets tagged with a VLAN id that is the same for all ports on that
host, which are associated with a particular (virtual) network. Ports belonging
to different (virtual) networks therefore get tagged with different VLAN ids.
The combination of "VLAN tag + destination MAC address" was felt to be a
sufficient check to ensure that we are truly dealing with traffic heading out
of the port being monitored.
Root Cause of the problem:
The br-int bridge operates using the legacy (or normal) mode of operation. We
have observed that under these circumstances, OVS does not add VLAN tags to
packets, as long as they are flowing within the bridge. Packets escaping from
br-int into br-tun (via the patch ports), however, get tagged as expected when
they arrive in br-tun.
Due to this behavior, our ingress side flow (described above) fails to capture
packets flowing into a VM's vNIC, when it is originating from another port on
the same host and on the same virtual network.
Possible Solutions:
OVS documentation (refer: Open vSwitch FAQ) seems to recommend not using the
normal operating mode, but handling VLANs explicitly, if flows related to VLAN
ids are going to be used. As mentioned above, to ensure that we correctly
handle the Neutron requirement that port MAC addresses are unique only within a
virtual network, we need to add VLAN related checks to the TaaS flows. However,
br-int today relies on the normal operation mode for most of its work.
This has left us now in a quandary situation. Here is a proposal to move us
forward:
1. As a temporary fix, we can convert the TaaS ingress side flow in
br-int from
dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address>
to
dl_dst=<Port-MAC-Address>
This will obviously mean that we no longer support the notion that Neutron port
MAC addresses are unique only within a virtual network. However, given that the
probability of two port MAC addresses on a host being the same is low, this
should suffice for the short term.
[We will also disable the flows used to capture broadcast/multicast traffic
flowing into a VM's vNIC]
2. Implement VLAN handling explicitly in br-int and thereby move away
from the normal operating mode. This will be a core Neutron change and not
limited to just TaaS. However, I have the feeling that there will be other
projects (outside of TaaS) that will sooner or later have a need to detect
traffic flowing into a vNIC and they will also run into the same issue we are
presently facing.
3. Go back to the (current) TaaS ingress side flow
(dl_vlan=<Port-VLAN-tag>, dl_dst=<Port-MAC-Address>) when (2) is implemented.
We will then be able to once again support the Neutron requirement that port
MAC addresses need be unique only within a (virtual) network.
[We will re-enable the flows used to capture broadcast/multicast traffic
flowing into a VM's vNIC]
Thoughts /comments are welcome.
Thanks and regards,
Anil
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
