Re: Ethernet card locking up when acting as virtual bridge
Hi there, On 11/10/2017 07:24 PM, Andrew W wrote: > That turned out to be the Cisco switch on the other end which is an > ESW500 series Small Business Switch (i.e web gui only no IOS CLI). On > there there is a 'Smartports Wizard' which allows you to set a 'role' > for each port and unless it is set to 'Access Point' I got the > strange behaviour. > > As the virtual bridge and wifi access point are pretty much the same > thing in principle I tried changing the port role to 'Access Point' > as well and it seems to have solved the issue. Ive no idea what that > is changing in the switch config, and when I asked on the Cisco forum > I couldnt get an answer but it seems to be the cause of the trouble. I presume that if a port is not set in "Access point' mode the switch will assume that packets coming from that port will always come from the same MAC address, and since the VMs you have all have differing addresses this causes the switch to silently drop packets from a different MAC address. And depending on how they actually implemented that in detail the behavior may be quite weird. Regards, Christian
Re: Ethernet card locking up when acting as virtual bridge
On 09/11/2017 19:52, Michael Stone wrote: I'd wonder if it's reached the point tha that the hardware is failing. I was a bit premature in thinking it was fixed! Tried 3 different 3Com cards now plus a Linksys with a Via chipset but got identical behavour on all. Back to the original config: did you know you could add the built-in adapter to the bridge, give your IP address to the bridge rather than directly to the built-in interface, and use the VMs the same way you are now without a second NIC? Yes but as I was experiementing I wanted to keep it separate so if I screwed up I could at least still SSH into the host. The weird thing is I have 3 VMs, all started with the same commands just the made up MAC address and virtual bridge port numbers differing. One of them has never given a problem it always works, the other two seem to alternate, one "goes offline" the other comes online, then vice versa, yet if I use VNC to bring up the console for the VM it is still running ok and I cant see any log messages on it indicating that the network has gone down. The arping command is indicating that when a VM goes offline it is not responding to ARP requests over the network yet it does from another VM and can be connected to perfectly from another VM I then had a brainwave, I had a similar issue with a Cisco wifi access point where some wifi clients could connect ok, others would connect but then there was no traffic flowing. That turned out to be the Cisco switch on the other end which is an ESW500 series Small Business Switch (i.e web gui only no IOS CLI). On there there is a 'Smartports Wizard' which allows you to set a 'role' for each port and unless it is set to 'Access Point' I got the strange behaviour. As the virtual bridge and wifi access point are pretty much the same thing in principle I tried changing the port role to 'Access Point' as well and it seems to have solved the issue. Ive no idea what that is changing in the switch config, and when I asked on the Cisco forum I couldnt get an answer but it seems to be the cause of the trouble.
Re: Ethernet card locking up when acting as virtual bridge
On Thu, Nov 09, 2017 at 12:55:05PM -0500, Dan Ritter wrote: On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote: On 11/08/17 02:54, Andrew Wood wrote: > 3Com Etherlink Model 3C905C That card is *old* -- it brings back memories. :-) And, 3Com is gone. Is there any FOSS support for 3Com stuff? This is one of the classic fast-ethernet cards, supported by the original Beowulf cluster implementation: http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html One would expect that any bugs left in the kernel driver (net/ethernet/3com/3c59x.ko) would be hammered flat by now. I'd wonder if it's reached the point tha that the hardware is failing. At any rate, a generic realtek gigabit card (free after rebate!) will run rings around what was a good card in the late 90s but functionally obsolete today. Back to the original config: did you know you could add the built-in adapter to the bridge, give your IP address to the bridge rather than directly to the built-in interface, and use the VMs the same way you are now without a second NIC? Mike Stone
Re: Ethernet card locking up when acting as virtual bridge
On 11/09/17 09:55, Dan Ritter wrote: On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote: On 11/08/17 02:54, Andrew Wood wrote: 3Com Etherlink Model 3C905C That card is *old* -- it brings back memories. :-) And, 3Com is gone. Is there any FOSS support for 3Com stuff? This is one of the classic fast-ethernet cards, supported by the original Beowulf cluster implementation: http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html One would expect that any bugs left in the kernel driver (net/ethernet/3com/3c59x.ko) would be hammered flat by now. Kernel changes can and do break device drivers -- I wrote a driver for Linux 2.4.18 back in the day; it broke on 2.4.19. The grim reality is that everything has to be re-validated on every kernel release. kernel.org says it's had four changes in the last year. Good -- the OP's hardware *is* supported on Linux. :-) So, the next place to look is configuration settings. It looks like Christian Seiler provided some good advice. David
Re: Ethernet card locking up when acting as virtual bridge
On Wed, Nov 08, 2017 at 03:21:00PM -0800, David Christensen wrote: > On 11/08/17 02:54, Andrew Wood wrote: > > 3Com Etherlink Model 3C905C > > That card is *old* -- it brings back memories. :-) And, 3Com is gone. Is > there any FOSS support for 3Com stuff? This is one of the classic fast-ethernet cards, supported by the original Beowulf cluster implementation: http://webhome.phy.duke.edu/~rgb/brahma/Resources/beowulf/linux/drivers/vortex.html One would expect that any bugs left in the kernel driver (net/ethernet/3com/3c59x.ko) would be hammered flat by now. kernel.org says it's had four changes in the last year. -dsr-
Re: Ethernet card locking up when acting as virtual bridge
On Wed, Nov 8, 2017 at 5:21 PM, David Christensen wrote: > On 11/08/17 02:54, Andrew Wood wrote: > >> 3Com Etherlink Model 3C905C >> > > That card is *old* -- it brings back memories. :-) And, 3Com is gone. Is > there any FOSS support for 3Com stuff? You know you're getting old when they put photos of your old network gear on webpages as if they were museum pieces, eg. https://en.wikipedia.org/wiki/3Com David > >
Re: Ethernet card locking up when acting as virtual bridge
On 11/08/17 02:54, Andrew Wood wrote: 3Com Etherlink Model 3C905C That card is *old* -- it brings back memories. :-) And, 3Com is gone. Is there any FOSS support for 3Com stuff? Intel supports FOSS on their products, which means their products are much more likely to work correctly on Debian GNU/Linux (and others). So, I buy Intel stuff. David
Re: Ethernet card locking up when acting as virtual bridge
On 08/11/2017 14:59, Christian Seiler wrote: Is is possible for you to try a static IP on this interface and see if that solves your problem? Ive cleared the dhcp on br1 (and not assigned a static, left it with no IP) and so far its working OK I will leave it a couple of days and see if it has indeed fixed the problem Thanks Christian for your comprehensive reply and help. Regards Andrew
Re: Ethernet card locking up when acting as virtual bridge
Am 2017-11-08 11:54, schrieb Andrew Wood: My configuration is below. Initially it worked fine, except that once in a while the card would seemingly 'lock up' i.e no VMs could get network access but unplugging and replugging the Cat 5 cable fixed it. Recently however the issue has been occuring more and more, sometimes some VMs can get network access and not others. Ive tried swapping the card for an identical model but its made no difference. Im not sure if it something to do with my config (maybe the made up MAC addresses?) or if its a bug in the cards driver or firmware. Config is below eth0 is the hosts Realtek port, enp3s0 is the 3Com used for the VMs iface br1 inet dhcp No idea if this is the problem or not, but you are using DHCP for the bridge interface. Maybe the DHCP client's management of the bridge interface interferes with this? (DHCP clients like to 'take over' an interface and may set flags that don't work properly.) I don't think I've ever used a DHCP client on a bridge before other than for short testing purposes. Is is possible for you to try a static IP on this interface and see if that solves your problem? If that doesn't help: - The Linux bridge interface will always use the numerically lowest (in a Big-Endian sense) MAC address that's configured as the MAC address for the bridge. If you're doing virtualization, you should stick to MAC addresses that are very high for this reason, so that the network card's own address always wins out. Many virtualization tools ensure that the MAC addresses they generate start with 0xfe for this reason. (This might also confuse DHCP clients btw. if the MAC address of the interface changes behind their back.) - When the problem occurs: do the VMs at least see each other? If that doesn't even work, your problem isn't your network card, but something else. - Anything in the kernel log? If not, is there maybe a debug option for your specific network driver you could enable so that something is added to the logs? - Try running 'ip l' and 'ip a' both when the problem occurs and when it doesn't occur and see if there's any difference in the output of either of these. - You said that unplugging and replugging the cable made it work again - maybe the link sensing between your network card and the switch / whatever you have on the other side is broken for some reason? Could you try to force the right speed / duplex settings via ethtool and see if that helps? - Maybe your network card supports various offloading features (such as TCP checksums) but when used as a switch they don't always work properly - you could try disabling them (also via ethtool, see the manpage). - You said that you've tried replacing the network card - have you tried replacing the cable? I've had some weird experiences with broken network cables. (For example, very strange intermittent packet loss in some instances, which cause very weird effects in higher-level protocols.) Regards, Christian
Ethernet card locking up when acting as virtual bridge
Im trying to use a 3Com Etherlink Model 3C905C to provide network access for some virtual machines running under QEMU. The machine has a Realtek Gigabit Ethernet controller on the motherboard which I use solely for the hosts network interface. I've added a 3Com PCI card to act as the interface solely for the VMs allowing them to have IPs on the real network rather than using QEMUs NAT. My configuration is below. Initially it worked fine, except that once in a while the card would seemingly 'lock up' i.e no VMs could get network access but unplugging and replugging the Cat 5 cable fixed it. Recently however the issue has been occuring more and more, sometimes some VMs can get network access and not others. Ive tried swapping the card for an identical model but its made no difference. Im not sure if it something to do with my config (maybe the made up MAC addresses?) or if its a bug in the cards driver or firmware. Config is below eth0 is the hosts Realtek port, enp3s0 is the 3Com used for the VMs The host OS is StretchAm I doing something wrong or is this a bug? Many thanks ==/etc/network/interfaces: # The loopback network interfaceauto loiface lo inet loopback auto eth0iface eth0 inet static address 192.168.253.201 netmask 255.255.255.0 broadcast 192.168.253.255 dns-nameservers 192.168.253.254 up /sbin/route add default gw 192.168.253.254 dev eth0 auto enp3s0auto br1iface br1 inet dhcp bridge_ports enp3s0 bridge_stp off bridge_fd 0 bridge_maxwait 0== The script I use to start QEMU contains lines like this for each VM: #host debian9test vnc on port 5902, give it virtual hub port br1p1tunctl -u root -t br1p1 #create TAP device br1p1brctl addif br1 br1p1 #add br1p1 as a port on bridge br1kvm -hda /var/qemuvm/debian9test.img -m 1024 -usb \-net nic,macaddr=52:54:00:00:00:02 -net tap,ifname=br1p1 -vnc 192.168.253.201:2 & #host bitsafe vnc on port 5904, give it virtual hub port br1p2tunctl -u root -t br1p2 #create TAP device br1p2brctl addif br1 br1p2 # add br1p2 as a port on bridge br1kvm -hda /dev/disk/by-id/ata-TOSHIBA_HDWJ110_37K1P2AET \-hdb /dev/disk/by-id/ata-TOSHIBA_HDWJ110_37UNTL5IT \-m 1024 -usb -net nic,macaddr=52:54:00:00:00:04 \-net tap,ifname=br1p2 -vnc 192.168.253.201:4 & #host gatekeeper (RADIUS) vnc on port 5905, give it virtual hub port br1p3tunctl -u root -t br1p3 #create TAP device br1p3brctl addif br1 br1p3 #add br1p3 as a port on bridge br1kvm -hda /var/qemuvm/debian-gatekeeper.img \-m 1024 -usb -net nic,macaddr=52:54:00:00:00:05 \-net tap,ifname=br1p3 -vnc 192.168.253.201:5 & ==Output from ifconfig: br1: flags=4163 mtu 1500 inet 192.168.253.80 netmask 255.255.255.0 broadcast 192.168.253.255 inet6 fe80::250:daff:fe3b:9a4d prefixlen 64 scopeid 0x20 ether 00:50:da:3b:9a:4d txqueuelen 1000 (Ethernet) RX packets 22633 bytes 1224397 (1.1 MiB) RX errors 0 dropped 9 overruns 0 frame 0 TX packets 159 bytes 18064 (17.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 br1p1: flags=4163 mtu 1500 ether 76:73:87:89:3c:f1 txqueuelen 1000 (Ethernet) RX packets 1391 bytes 97413 (95.1 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 22130 bytes 1514391 (1.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 br1p2: flags=4163 mtu 1500 ether 82:e0:2d:0d:d8:26 txqueuelen 1000 (Ethernet) RX packets 4416 bytes 337769 (329.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 25436 bytes 1728395 (1.6 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 br1p3: flags=4163 mtu 1500 ether 26:67:8d:04:5c:b9 txqueuelen 1000 (Ethernet) RX packets 329 bytes 38930 (38.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 22583 bytes 1553522 (1.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp3s0: flags=4163 mtu 1500 ether 00:50:da:3b:9a:4d txqueuelen 1000 (Ethernet) RX packets 27632 bytes 2188088 (2.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 6303 bytes 493295 (481.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 20 base 0x5000 eth0: flags=4163 mtu 1500 inet 192.168.253.201 netmask 255.255.255.0 broadcast 192.168.253.255 inet6 fe80::21f:d0ff:fe08:8bc6 prefixlen 64 scopeid 0x20 ether 00:1f:d0:08:8b:c6 txqueuelen 1000 (Ethernet) RX packets 22949 bytes 20198348 (19.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24046 bytes 1880515 (1.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10
