Read that IBM article you referenced, it made a lot of things clear to me. Sent from my iPhone
On Aug 17, 2012, at 5:54 AM, "Matthew Patton" <mpat...@inforelay.com> wrote: > 'cloudVirBr' is hard-coded in > /usr/lib[64]/cloud/agent/scripts/vm/network/vnet/modifyvlan.sh in function > addVlan(). > > What I don't understand is how the script is even working. It does > > vconfig add ethN V > brctl addbr brV > brctl addif brV ethN.V > > Furthermore, in scripts/storage/qcow2/modifyvlan.sh (anyone looked at why > there are 3 different copies of this shell script?!?) it does the same thing > but using bonds. > > But when I run thru the sequence by hand (eth or bond) it doesn't pass > traffic. The only way I get it to work is: > > brctl addbr br > brctl addif br ethN > vconfig add ethN V > ifconfig ethN.V <ipaddr> <netmask> > > And I can define as many ethN.[1..9] as I like with all different VLAN IDs > and attach them to the single bridge. My way also works when adding a bond to > the bridge and adding as many VLAN tagged bonds as I like. > > But the guest isn't receiving response packets to things like ARP. When I > sniff bond0 on the hypervisor I see the VLAN tag in the frame AS LONG AS in > the guest I configure ethN.V; as would be logical. The ARP reply also gets as > far as bond0 on the way back. Using tcpdump on interface 'vnet0' doesn't show > any packets going in either direction though they are transiting bond0 just > fine. I can see the ARP replies on bond0.N which is again as expected. > > If I can sniff bond0, how come I can't sniff vnet0? > There is something odd in the 'brctl showmacs <bridge>' output which probably > explains why vnet0 sees no packets. > > The guest's MAC is 52:54:00:c9:8d:d1. It's on port #2, marked local=no and > eventually ages out. But there is another MAC on port 2 that is the same > except it starts with 'fe'. This one is marked local=yes and doesn't age out. > It seems this is the MAC given to the host-side of the tap. So I can > understand it being semi-permanent. So then why is the the guest's > (?private?) MAC address showing up in the bridge? If the ARP response packets > have the wrong (guest's private MAC) in them (and they do) that would explain > why it can't return to the guest since it "missed" the on-ramp that is the > hypervisor's end of the TUN interface. > > > Looks like a mistake in the tap code that's forgetting to rewrite the MAC > value? Maybe because there is a VLAN tag in the frame? > > > At first there were no "QEMU networks" defined since I had undefined the > default one since I wasn't interested in NAT. > [root@kvm1b ~]# virsh net-list > Name State Autostart > ----------------------------------------- > > But seeing as that might be a problem I defined one for bridging. > <network> > <name>bridge-shared</name> > <uuid>0824e08b-bb0e-452a-e2c9-dcca52af2341</uuid> > <forward mode='bridge'/> > <bridge name='shared' /> > </network> > > > Now I have a network > [root@kvm1b networks]# virsh net-list > Name State Autostart > ----------------------------------------- > bridge-shared active yes > > > This is the interface section from my guest's XML. > <interface type='bridge'> > <mac address='52:54:00:c9:8d:d1'/> > <source bridge='shared'/> > <model type='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0'/> > </interface> > > [root@kvm1b ~]# virsh domiflist test > Interface Type Source Model MAC > ------------------------------------------------------- > vnet0 bridge shared virtio 52:54:00:c9:8d:d1 > > [root@kvm1b networks]# virsh iface-list > Name State MAC Address > -------------------------------------------- > bond0.39 active 00:25:90:4c:ba:92 > bond0.398 active 00:25:90:4c:ba:92 > eth2 active 90:e2:ba:04:f6:28 > eth3 active 90:e2:ba:04:f6:29 > lo active 00:00:00:00:00:00 > shared active 00:25:90:4c:ba:92 > > and for good measure > > [root@kvm1b networks]# virsh iface-dumpxml shared > <interface type='bridge' name='shared'> > <protocol family='ipv6'> > <ip address='fe80::225:90ff:fe4c:ba92' prefix='64'/> > </protocol> > <bridge> > <interface type='bond' name='bond0'> > <bond> > <interface type='ethernet' name='eth0'> > <mac address='00:25:90:4c:ba:92'/> > </interface> > <interface type='ethernet' name='eth1'> > <mac address='00:25:90:4c:ba:92'/> > </interface> > </bond> > </interface> > <interface type='ethernet' name='vnet0'> > <mac address='fe:54:00:c9:8d:d1'/> > </interface> > </bridge> > </interface> > > [root@kvm1b networks]# virsh domifstat test vnet0 > vnet0 rx_bytes 44592 > vnet0 rx_packets 847 > vnet0 rx_errs 0 > vnet0 rx_drop 0 > vnet0 tx_bytes 2150 > vnet0 tx_packets 35 > vnet0 tx_errs 0 > vnet0 tx_drop 0 > > So you'd think something was getting back to the guest. I just don't know > what. tcpdump running inside the guest on eth0 shows nothing at all, just > like sniffing vnet0. > > The MAC hitting the wire on the hypervisor's bond0 is still the guest's > private MAC and it seems the response packets don't know where to go once > they come back to the bridge. > > > On a different note I ran across this: > http://publib.boulder.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liaat/liaatbptap.htm > So QEMU has it's own bastard redefinition of the term "VLAN"? WTF!
