Re: veb(4) exceeds 1514 byte frame size while bridge(4) doesn't?
On Tue, Mar 23, 2021 at 08:24:56PM +1000, David Gwynne wrote:
> it looks like ure(4) hardware doesn't strip the fcs before pushing it to
> the host over usb, but the ure(4) driver doesn't strip it.
>
> this usually isn't a huge deal because layers like ip just ignore
> the extra bytes. bridge(4) was ok with this because it actually
> parses ip packets and removes the extra bytes. veb(4) does a lot less
> (by design) so it just lets the fcs on the end of ure packets go out to
> other nics.
>
> from what i can tell, ure should remove the fcs. that's what this diff
> does. can you try it?
This fixed the problem without introducing any noticeable other effects,
thanks!
> cheers,
> dlg
>
> Index: if_ure.c
> ===
> RCS file: /cvs/src/sys/dev/usb/if_ure.c,v
> retrieving revision 1.21
> diff -u -p -r1.21 if_ure.c
> --- if_ure.c 14 Oct 2020 23:47:55 - 1.21
> +++ if_ure.c 23 Mar 2021 10:18:54 -
> @@ -1896,10 +1896,17 @@ ure_rxeof(struct usbd_xfer *xfer, void *
> ifp->if_ierrors++;
> goto done;
> }
> + if (pktlen < ETHER_MIN_LEN) {
> + DPRINTF(("ethernet frame is too short\n"));
> + ifp->if_ierrors++;
> + goto done;
> + }
>
> total_len -= roundup(pktlen, URE_RX_BUF_ALIGN);
> buf += sizeof(rxhdr);
>
> + /* trim fcs */
> + pktlen -= ETHER_CRC_LEN;
> m = m_devget(buf, pktlen, ETHER_ALIGN);
> if (m == NULL) {
> DPRINTF(("unable to allocate mbuf for next packet\n"));
>
--
Jurjen Oskam
Re: veb(4) exceeds 1514 byte frame size while bridge(4) doesn't?
On Tue, Mar 23, 2021 at 08:24:56PM +1000, David Gwynne wrote:
>
> On Sun, Mar 21, 2021 at 04:24:24PM +0100, Jurjen Oskam wrote:
> > Hi,
> >
> > When trying out veb(4), I ran into a situation where TCP sessions across a
> > veb(4) bridge stalled while the exact same config using bridge(4) worked
> > fine.
> >
> > After some investigation, it seems that veb(4) adds an FCS to the outgoing
> > frame, while bridge(4) doesn't. When this causes the outgoing frame to
> > exceed
> > 1514 bytes, the destination doesn't receive it.
> >
> > I must note that I was using USB NICs, one of them being quite old.
> >
> > Am I doing something wrong, or is the problem in (one of) the NIC(s)?
>
> it looks like ure(4) hardware doesn't strip the fcs before pushing it to
> the host over usb, but the ure(4) driver doesn't strip it.
>
> this usually isn't a huge deal because layers like ip just ignore
> the extra bytes. bridge(4) was ok with this because it actually
> parses ip packets and removes the extra bytes. veb(4) does a lot less
> (by design) so it just lets the fcs on the end of ure packets go out to
> other nics.
>
> from what i can tell, ure should remove the fcs. that's what this diff
> does. can you try it?
You're right, ure(4) should strip the FCS.
ok kevlo@
> cheers,
> dlg
>
> Index: if_ure.c
> ===
> RCS file: /cvs/src/sys/dev/usb/if_ure.c,v
> retrieving revision 1.21
> diff -u -p -r1.21 if_ure.c
> --- if_ure.c 14 Oct 2020 23:47:55 - 1.21
> +++ if_ure.c 23 Mar 2021 10:18:54 -
> @@ -1896,10 +1896,17 @@ ure_rxeof(struct usbd_xfer *xfer, void *
> ifp->if_ierrors++;
> goto done;
> }
> + if (pktlen < ETHER_MIN_LEN) {
> + DPRINTF(("ethernet frame is too short\n"));
> + ifp->if_ierrors++;
> + goto done;
> + }
>
> total_len -= roundup(pktlen, URE_RX_BUF_ALIGN);
> buf += sizeof(rxhdr);
>
> + /* trim fcs */
> + pktlen -= ETHER_CRC_LEN;
> m = m_devget(buf, pktlen, ETHER_ALIGN);
> if (m == NULL) {
> DPRINTF(("unable to allocate mbuf for next packet\n"));
>
Re: veb(4) exceeds 1514 byte frame size while bridge(4) doesn't?
On Sun, Mar 21, 2021 at 04:24:24PM +0100, Jurjen Oskam wrote:
> Hi,
>
> When trying out veb(4), I ran into a situation where TCP sessions across a
> veb(4) bridge stalled while the exact same config using bridge(4) worked fine.
>
> After some investigation, it seems that veb(4) adds an FCS to the outgoing
> frame, while bridge(4) doesn't. When this causes the outgoing frame to exceed
> 1514 bytes, the destination doesn't receive it.
>
> I must note that I was using USB NICs, one of them being quite old.
>
> Am I doing something wrong, or is the problem in (one of) the NIC(s)?
it looks like ure(4) hardware doesn't strip the fcs before pushing it to
the host over usb, but the ure(4) driver doesn't strip it.
this usually isn't a huge deal because layers like ip just ignore
the extra bytes. bridge(4) was ok with this because it actually
parses ip packets and removes the extra bytes. veb(4) does a lot less
(by design) so it just lets the fcs on the end of ure packets go out to
other nics.
from what i can tell, ure should remove the fcs. that's what this diff
does. can you try it?
cheers,
dlg
Index: if_ure.c
===
RCS file: /cvs/src/sys/dev/usb/if_ure.c,v
retrieving revision 1.21
diff -u -p -r1.21 if_ure.c
--- if_ure.c14 Oct 2020 23:47:55 - 1.21
+++ if_ure.c23 Mar 2021 10:18:54 -
@@ -1896,10 +1896,17 @@ ure_rxeof(struct usbd_xfer *xfer, void *
ifp->if_ierrors++;
goto done;
}
+ if (pktlen < ETHER_MIN_LEN) {
+ DPRINTF(("ethernet frame is too short\n"));
+ ifp->if_ierrors++;
+ goto done;
+ }
total_len -= roundup(pktlen, URE_RX_BUF_ALIGN);
buf += sizeof(rxhdr);
+ /* trim fcs */
+ pktlen -= ETHER_CRC_LEN;
m = m_devget(buf, pktlen, ETHER_ALIGN);
if (m == NULL) {
DPRINTF(("unable to allocate mbuf for next packet\n"));
veb(4) exceeds 1514 byte frame size while bridge(4) doesn't?
Hi, When trying out veb(4), I ran into a situation where TCP sessions across a veb(4) bridge stalled while the exact same config using bridge(4) worked fine. After some investigation, it seems that veb(4) adds an FCS to the outgoing frame, while bridge(4) doesn't. When this causes the outgoing frame to exceed 1514 bytes, the destination doesn't receive it. I must note that I was using USB NICs, one of them being quite old. Am I doing something wrong, or is the problem in (one of) the NIC(s)? The configuration of the veb(4) bridge was as follows (full dmesg below): openbsd$ ls -l /etc/hostname.* -rw-r- 1 root wheel 3 Mar 19 10:08 /etc/hostname.axe0 -rw-r- 1 root wheel 3 Mar 19 10:08 /etc/hostname.ure0 -rw-r- 1 root wheel 21 Mar 21 16:16 /etc/hostname.veb0 openbsd$ cat /etc/hostname.axe0 up openbsd$ cat /etc/hostname.ure0 up openbsd$ cat /etc/hostname.veb0 add ure0 add axe0 up The configuration of bridge(4) was exactly the same, I simply renamed hostname.veb0 to hostname.bridge0 and rebooted. On the axe(4) side of the bridge was my Windows 10 PC, on the ure(4) side of the bridge was my home network. On the Windows 10 PC, I used ICMP echo requests to the default gateway to diagnose the problem. Using bridge(4) it went as expected: * When sending an echo request with a payload of 1473 and DF set, Windows wouldn't send the packet: "Packet needs to be fragmented but DF set." * Payload 1472 and DF set: no packet loss Using veb(4), this happened: * Payload 1473, DF set: "Packet needs to be fragmented" (expected) * Payload 1472, 1471, 1470 or 1469 and DF set: 100% packet loss (not expected) * Payload <=1468 and DF set: no packet loss (expected) On the bridge, I ran Wireshark on axe0 (where the Windows PC was attached). I noticed that the echo reply was logged by Wireshark when using bridge(4) as well as when using veb(4). When using bridge(4) the frame had a size of 1514, but when using veb(4) the frame had a size of 1518. Inspecting the differences between the frames with the echo response revealed that veb(4) seemed to insert the FCS (on the right) while bridge(4) doesn't: Frame 55: 1514 bytes on wire (12112 bi | Frame 34: 1518 bytes on wire (12144 bi Interface id: 0 (axe0) Interface id: 0 (axe0) Interface name: axe0 Interface name: axe0 Encapsulation type: Ethernet (1) Encapsulation type: Ethernet (1) Arrival Time: Mar 21, 2021 16:21:0 | Arrival Time: Mar 21, 2021 16:29:3 [Time shift for this packet: 0.000 [Time shift for this packet: 0.000 Epoch Time: 1616340062.761312000 s | Epoch Time: 1616340579.977005000 s [Time delta from previous captured | [Time delta from previous captured [Time delta from previous displaye [Time delta from previous displaye [Time since reference or first fra | [Time since reference or first fra Frame Number: 55 | Frame Number: 34 Frame Length: 1514 bytes (12112 bi | Frame Length: 1518 bytes (12144 bi Capture Length: 1514 bytes (12112 | Capture Length: 1518 bytes (12144 [Frame is marked: False] [Frame is marked: False] [Frame is ignored: False][Frame is ignored: False] [Protocols in frame: eth:ethertype [Protocols in frame: eth:ethertype Ethernet II, Src: Shuttle_f1:11:c1 (80 Ethernet II, Src: Shuttle_f1:11:c1 (80 Destination: Giga-Byt_37:28:2b (18 Destination: Giga-Byt_37:28:2b (18 Address: Giga-Byt_37:28:2b (18 Address: Giga-Byt_37:28:2b (18 ..0. ..0. ...0 ...0 Source: Shuttle_f1:11:c1 (80:ee:73 Source: Shuttle_f1:11:c1 (80:ee:73 Address: Shuttle_f1:11:c1 (80: Address: Shuttle_f1:11:c1 (80: ..0. ..0. ...0 ...0 Type: IPv4 (0x0800) Type: IPv4 (0x0800) > Frame check sequence: 0x13a4f5bc [ > [FCS Status: Unverified] Internet Protocol Version 4, Src: 192. Internet Protocol Version 4, Src: 192. 0100 = Version: 4 0100 = Version: 4 0101 = Header Length: 20 byte 0101 = Header Length: 20 byte Differentiated Services Field: 0x0 Differentiated Services Field: 0x0 00.. = Differentiated Ser 00.. = Differentiated Ser ..00 = Explicit Congestio ..00 = Explicit Congestio Total Length: 1500 Total Length: 1500 Identification: 0xc088 (49288) | Identification: 0x221d (8733) Flags: 0x40, Don't fragment Flags: 0x40, Don't fragment 0...
