IPv4 on ix(4) slow/nothing - 7.4

Mischa Wed, 18 Oct 2023 06:36:11 -0700

Hi All,

Just upgraded a couple of machines to 7.4. smooth as always!!


I am however seeing issues with IPv4, slowness or no throughput at all.

The machines I have upgraded are using an Intel X540-T network card andis connected on 10G.

ix0 at pci5 dev 0 function 0 "Intel X540T" rev 0x01, msix, 16 queues,address b8:ca:3a:62:ee:40ix1 at pci5 dev 0 function 1 "Intel X540T" rev 0x01, msix, 16 queues,address b8:ca:3a:62:ee:42


root@n2:~ # sysctl kern.version

kern.version=OpenBSD 7.4 (GENERIC.MP) #1397: Tue Oct 10 09:02:37 MDT2023

    dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

There are a bunch of VMs running on top of it, as soon as I want tofetch something with ftp, for example, I don't get anything over IPv4,with IPv6 everything is normal.

mischa@www2:~ $ ftp -4https://mirror.openbsd.amsterdam/pub/OpenBSD/7.4/amd64/install74.iso

Trying 46.23.88.18...

Requestinghttps://mirror.openbsd.amsterdam/pub/OpenBSD/7.4/amd64/install74.iso0% | | 512 KB - stalled-^C


A trace on mirror / n2:

n2:~ # tcpdump -i vport880 host 46.23.88.32
tcpdump: listening on vport880, link-type EN10MB

15:16:08.730274 www2.high5.nl.1828 > n2.high5.nl.https: S2182224746:2182224746(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale6,nop,nop,timestamp 2899683458 0> (DF)

15:16:08.730297 arp who-has www2.high5.nl tell n2.high5.nl
15:16:08.731535 arp reply www2.high5.nl is-at fe:51:bb:1e:12:11

15:16:08.731540 n2.high5.nl.https > www2.high5.nl.1828: S633749938:633749938(0) ack 2182224747 win 16384 <mss1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 31299551062899683458> (DF)15:16:08.732017 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1 win 256<nop,nop,timestamp 2899683458 3129955106> (DF)15:16:08.785752 www2.high5.nl.1828 > n2.high5.nl.https: P 1:312(311) ack1 win 256 <nop,nop,timestamp 2899683510 3129955106> (DF)15:16:08.786092 n2.high5.nl.https > www2.high5.nl.1828: P 1:128(127) ack312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786376 n2.high5.nl.https > www2.high5.nl.1828: P 128:134(6) ack312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786396 n2.high5.nl.https > www2.high5.nl.1828: P 134:166(32)ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786455 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448)ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786457 n2.high5.nl.https > www2.high5.nl.1828: .1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 31299551662899683510> (DF)15:16:08.786460 n2.high5.nl.https > www2.high5.nl.1828: P 3062:3803(741)ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786943 www2.high5.nl.1828 > n2.high5.nl.https: . ack 134 win255 <nop,nop,timestamp 2899683510 3129955166> (DF)15:16:08.796534 n2.high5.nl.https > www2.high5.nl.1828: P 3803:4345(542)ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF)15:16:08.796577 n2.high5.nl.https > www2.high5.nl.1828: P 4345:4403(58)ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF)15:16:08.797518 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4345}> (DF)15:16:08.797522 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4403}> (DF)15:16:09.790297 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448)ack 312 win 271 <nop,nop,timestamp 3129956176 2899683526> (DF)15:16:09.790902 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1614 win233 <nop,nop,timestamp 2899684519 3129956176,nop,nop,sack 1 {3803:4403}> (DF)15:16:09.790917 n2.high5.nl.https > www2.high5.nl.1828: .1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 31299561762899684519> (DF)15:16:09.790923 n2.high5.nl.https > www2.high5.nl.1828: P3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 31299561762899684519> (DF)15:16:10.790299 n2.high5.nl.https > www2.high5.nl.1828: .1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 31299571762899684519> (DF)15:16:10.791204 www2.high5.nl.1828 > n2.high5.nl.https: . ack 3062 win233 <nop,nop,timestamp 2899685520 3129957176,nop,nop,sack 1 {3803:4403}> (DF)15:16:10.791223 n2.high5.nl.https > www2.high5.nl.1828: P3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 31299571762899685520> (DF)15:16:10.791692 www2.high5.nl.1828 > n2.high5.nl.https: . ack 4403 win235 <nop,nop,timestamp 2899685520 3129957176> (DF)15:16:10.802647 www2.high5.nl.1828 > n2.high5.nl.https: P 312:318(6) ack4403 win 256 <nop,nop,timestamp 2899685528 3129957176> (DF)15:16:11.000297 n2.high5.nl.https > www2.high5.nl.1828: . ack 318 win271 <nop,nop,timestamp 3129957386 2899685528> (DF)15:16:11.001162 www2.high5.nl.1828 > n2.high5.nl.https: P 318:527(209)ack 4403 win 256 <nop,nop,timestamp 2899685730 3129957386> (DF)15:16:11.001860 n2.high5.nl.https > www2.high5.nl.1828: P 4403:5059(656)ack 527 win 271 <nop,nop,timestamp 3129957386 2899685730> (DF)15:16:11.001989 n2.high5.nl.https > www2.high5.nl.1828: .5059:6507(1448) ack 527 win 271 <nop,nop,timestamp 31299573862899685730> (DF)15:16:11.001992 n2.high5.nl.https > www2.high5.nl.1828: .6507:7955(1448) ack 527 win 271 <nop,nop,timestamp 31299573862899685730> (DF)15:16:11.195431 www2.high5.nl.1828 > n2.high5.nl.https: . ack 5059 win256 <nop,nop,timestamp 2899685924 3129957386> (DF)15:16:11.195447 n2.high5.nl.https > www2.high5.nl.1828: .7955:9403(1448) ack 527 win 271 <nop,nop,timestamp 31299575762899685924> (DF)



Running a trace on www2 I am seeing:

www2:~ # tcpdump -i vio0 host 46.23.88.18
tcpdump: listening on vio0, link-type EN10MB

15:16:08.729974 www2.high5.nl.1828 > n2.high5.nl.https: S2182224746:2182224746(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale6,nop,nop,timestamp 2899683458 0> (DF)

15:16:08.731114 arp who-has www2.high5.nl tell n2.high5.nl
15:16:08.731229 arp reply www2.high5.nl is-at fe:51:bb:1e:12:11

15:16:08.731631 n2.high5.nl.https > www2.high5.nl.1828: S633749938:633749938(0) ack 2182224747 win 16384 <mss1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 31299551062899683458> (DF)15:16:08.731732 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1 win 256<nop,nop,timestamp 2899683458 3129955106> (DF)15:16:08.785371 www2.high5.nl.1828 > n2.high5.nl.https: P 1:312(311) ack1 win 256 <nop,nop,timestamp 2899683510 3129955106> (DF)15:16:08.786219 n2.high5.nl.https > www2.high5.nl.1828: P 1:128(127) ack312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786522 n2.high5.nl.https > www2.high5.nl.1828: P 128:134(6) ack312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786523 n2.high5.nl.https > www2.high5.nl.1828: P 134:166(32)ack 312 win 271 <nop,nop,timestamp 3129955166 2899683510> (DF)15:16:08.786525 truncated-ip - 1665 bytes missing!n2.high5.nl.https >www2.high5.nl.1828: P 166:2138(1972) ack 312 win 271 <nop,nop,timestamp3129955166 2899683510> (DF)15:16:08.786628 www2.high5.nl.1828 > n2.high5.nl.https: . ack 134 win255 <nop,nop,timestamp 2899683510 3129955166> (DF)15:16:08.797085 n2.high5.nl.https > www2.high5.nl.1828: P 3803:4345(542)ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF)15:16:08.797086 n2.high5.nl.https > www2.high5.nl.1828: P 4345:4403(58)ack 312 win 271 <nop,nop,timestamp 3129955176 2899683510> (DF)15:16:08.797207 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4345}> (DF)15:16:08.797208 www2.high5.nl.1828 > n2.high5.nl.https: . ack 166 win256 <nop,nop,timestamp 2899683526 3129955166,nop,nop,sack 1 {3803:4403}> (DF)15:16:09.790498 n2.high5.nl.https > www2.high5.nl.1828: . 166:1614(1448)ack 312 win 271 <nop,nop,timestamp 3129956176 2899683526> (DF)15:16:09.790611 www2.high5.nl.1828 > n2.high5.nl.https: . ack 1614 win233 <nop,nop,timestamp 2899684519 3129956176,nop,nop,sack 1 {3803:4403}> (DF)15:16:09.791036 truncated-ip - 817 bytes missing!n2.high5.nl.https >www2.high5.nl.1828: P 1614:3586(1972) ack 312 win 271 <nop,nop,timestamp3129956176 2899684519> (DF)15:16:10.790743 n2.high5.nl.https > www2.high5.nl.1828: .1614:3062(1448) ack 312 win 271 <nop,nop,timestamp 31299571762899684519> (DF)15:16:10.790872 www2.high5.nl.1828 > n2.high5.nl.https: . ack 3062 win233 <nop,nop,timestamp 2899685520 3129957176,nop,nop,sack 1 {3803:4403}> (DF)15:16:10.791287 n2.high5.nl.https > www2.high5.nl.1828: P3062:4403(1341) ack 312 win 271 <nop,nop,timestamp 31299571762899685520> (DF)15:16:10.791393 www2.high5.nl.1828 > n2.high5.nl.https: . ack 4403 win235 <nop,nop,timestamp 2899685520 3129957176> (DF)15:16:10.802259 www2.high5.nl.1828 > n2.high5.nl.https: P 312:318(6) ack4403 win 256 <nop,nop,timestamp 2899685528 3129957176> (DF)15:16:11.000715 n2.high5.nl.https > www2.high5.nl.1828: . ack 318 win271 <nop,nop,timestamp 3129957386 2899685528> (DF)15:16:11.000830 www2.high5.nl.1828 > n2.high5.nl.https: P 318:527(209)ack 4403 win 256 <nop,nop,timestamp 2899685730 3129957386> (DF)15:16:11.002040 n2.high5.nl.https > www2.high5.nl.1828: P 4403:5059(656)ack 527 win 271 <nop,nop,timestamp 3129957386 2899685730> (DF)15:16:11.002042 truncated-ip - 924 bytes missing!n2.high5.nl.https >www2.high5.nl.1828: . 5059:7031(1972) ack 527 win 271 <nop,nop,timestamp3129957386 2899685730> (DF)15:16:11.195146 www2.high5.nl.1828 > n2.high5.nl.https: . ack 5059 win256 <nop,nop,timestamp 2899685924 3129957386> (DF)15:16:11.195567 n2.high5.nl.https > www2.high5.nl.1828: .7955:9403(1448) ack 527 win 271 <nop,nop,timestamp 31299575762899685924> (DF)



From "What is new" I noticed:

* Enable LRO per default in network drivers. LRO allows to receiveaggregated packets larger than the MTU. Receiving TCP streams becomesmuch faster. Currently only ix(4) and lo(4) devices support LRO, andix(4) is limited to IPv4 and hardware newer than the old 82598 model.


Could this have something to do with it?

Mischa

IPv4 on ix(4) slow/nothing - 7.4

Reply via email to