Re: em performs worse than igb (latency wise) in 12?
On Sat, Apr 6, 2019 at 10:24 PM Graham Menhennitt wrote: > Not that it's at all relevant to the question here, but... > > It does mostly work without em in the 12 kernel - I'm not sure how, but > it does. > > I upgraded to 12-stable via source but didn't add em to my custom > kernel. Most things worked - basic network functionality. But I had > problems with ipfw and igb. Adding em to the kernel fixed them. > FWIW the latest GENERIC kernel includes the iflib, em, etc devices as far as I can tell. I found the new UPDATING entry about iflib "no longer unconditionally compiled into the kernel" a bit confusing... So long as you are including GENERIC it should be the same as 12-RELEASE. > Graham > > On 6/4/19 6:12 am, Kris von Mach wrote: > > On 4/6/2019 2:56 AM, Pete French wrote: > >> Something odd going on there there - I am using 12-STABLE and I have > >> igb just fine, and it attaches to the same hardware that 11 did: > > > > It does work in 12, throughput is great, just that the latency is > > higher than 11. > > > > igb0: flags=8843 metric 0 mtu > > 1500 > > > options=e527bb > > > > > ether 38:ea:a7:8d:c1:6c > > inet 208.72.56.19 netmask 0xfc00 broadcast 208.72.59.255 > > inet6 fe80::3aea:a7ff:fe8d:c16c%igb0 prefixlen 64 scopeid 0x1 > > inet6 2602:ffb8::208:72:56:9 prefixlen 64 > > media: Ethernet autoselect (1000baseT ) > > status: active > > nd6 options=21 > > > >> Do you have a custom kernel, and if so did you see this note in > >> UPDATING? > > > > Yes I do, but it includes all of GENERIC which includes em drivers, > > otherwise it wouldn't even work with the network card. > > > > my custom kernel: > > > > include GENERIC > > ident CUSTOM > > makeoptions WITH_EXTRA_TCP_STACKS=1 > > options TCPHPTS > > options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) > > options IPSTEALTH > > options AHC_REG_PRETTY_PRINT # Print register bitfields in debug > > options AHD_REG_PRETTY_PRINT # Print register bitfields in debug > > device cryptodev > > device aesni > > > > I did try without RACK just in case that was the culprit. > > > > > > ___ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org > " > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On 4/7/2019 6:49 AM, Matthew Macy wrote: On Sat, Apr 6, 2019 at 1:23 PM Michael Butler wrote: I'd be interested to see if substituting the port net/intel-em-kmod has any effect on the issue, I would as well. igb, em, and lem are all the same driver in 12. This makes maintenance a lot easier. However, the older NICs have a lot of errata workarounds that aren't explicitly commented as such. My first guess is this card suffers from one such errata workaround that has been dropped in the update. I've tried net/intel-em-kmod, it actually became worse went from 100 requests/sec to about 90. That makes sense about maintenance. Though I believe i350 is less than 5 years old, HP's 366FLR version is 4. So it's not that old, and at least in gigabit level nic, is one of the best afaik. Is there some other 1gig nic that is recommended for 12? Or is it time to switch to 10gbit? I've heard good things about Chelsio for 10gbit. I went back to 11-Stable for now. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
Not that it's at all relevant to the question here, but... It does mostly work without em in the 12 kernel - I'm not sure how, but it does. I upgraded to 12-stable via source but didn't add em to my custom kernel. Most things worked - basic network functionality. But I had problems with ipfw and igb. Adding em to the kernel fixed them. Graham On 6/4/19 6:12 am, Kris von Mach wrote: On 4/6/2019 2:56 AM, Pete French wrote: Something odd going on there there - I am using 12-STABLE and I have igb just fine, and it attaches to the same hardware that 11 did: It does work in 12, throughput is great, just that the latency is higher than 11. igb0: flags=8843 metric 0 mtu 1500 options=e527bb ether 38:ea:a7:8d:c1:6c inet 208.72.56.19 netmask 0xfc00 broadcast 208.72.59.255 inet6 fe80::3aea:a7ff:fe8d:c16c%igb0 prefixlen 64 scopeid 0x1 inet6 2602:ffb8::208:72:56:9 prefixlen 64 media: Ethernet autoselect (1000baseT ) status: active nd6 options=21 Do you have a custom kernel, and if so did you see this note in UPDATING? Yes I do, but it includes all of GENERIC which includes em drivers, otherwise it wouldn't even work with the network card. my custom kernel: include GENERIC ident CUSTOM makeoptions WITH_EXTRA_TCP_STACKS=1 options TCPHPTS options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) options IPSTEALTH options AHC_REG_PRETTY_PRINT # Print register bitfields in debug options AHD_REG_PRETTY_PRINT # Print register bitfields in debug device cryptodev device aesni I did try without RACK just in case that was the culprit. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On Sat, Apr 6, 2019 at 1:23 PM Michael Butler wrote: > > On 2019-04-06 08:58, Kris von Mach wrote: > > On 4/6/2019 2:56 AM, Pete French wrote: > >> Something odd going on there there - I am using 12-STABLE and I have > >> igb just fine, and it attaches to the same hardware that 11 did: > > > > I ran apache bench, and I got a result of 100 requests/sec on 12-STABLE > > vs 16,000 requests/sec on 11-STABLE. So something is definitely wrong. > > Nothing changed other than going from 11 to 12. > > I'd be interested to see if substituting the port net/intel-em-kmod has > any effect on the issue, I would as well. igb, em, and lem are all the same driver in 12. This makes maintenance a lot easier. However, the older NICs have a lot of errata workarounds that aren't explicitly commented as such. My first guess is this card suffers from one such errata workaround that has been dropped in the update. -M ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On 2019-04-06 08:58, Kris von Mach wrote: > On 4/6/2019 2:56 AM, Pete French wrote: >> Something odd going on there there - I am using 12-STABLE and I have >> igb just fine, and it attaches to the same hardware that 11 did: > > I ran apache bench, and I got a result of 100 requests/sec on 12-STABLE > vs 16,000 requests/sec on 11-STABLE. So something is definitely wrong. > Nothing changed other than going from 11 to 12. I'd be interested to see if substituting the port net/intel-em-kmod has any effect on the issue, imb ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On 4/6/2019 2:56 AM, Pete French wrote: Something odd going on there there - I am using 12-STABLE and I have igb just fine, and it attaches to the same hardware that 11 did: I ran apache bench, and I got a result of 100 requests/sec on 12-STABLE vs 16,000 requests/sec on 11-STABLE. So something is definitely wrong. Nothing changed other than going from 11 to 12. ab on 12-STABLE: ab -n 100 -c 10 http://ip/ This is ApacheBench, Version 2.3 <$Revision: 1843412 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking ip (be patient).done Server Software: nginx Server Hostname: ip Server Port: 80 Document Path: / Document Length: 16247 bytes Concurrency Level: 10 Time taken for tests: 1.013 seconds Complete requests: 100 Failed requests: 0 Total transferred: 1649900 bytes HTML transferred: 1624700 bytes Requests per second: 98.69 [#/sec] (mean) Time per request: 101.331 [ms] (mean) Time per request: 10.133 [ms] (mean, across all concurrent requests) Transfer rate: 1590.07 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 1 Processing: 100 101 0.5 101 102 Waiting: 0 0 0.1 0 1 Total: 100 101 0.6 101 102 Percentage of the requests served within a certain time (ms) 50% 101 66% 101 75% 101 80% 102 90% 102 95% 102 98% 102 99% 102 100% 102 (longest request) ab on 11-STABLE: ab -n 100 -c 10 http://ip/ This is ApacheBench, Version 2.3 <$Revision: 1843412 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking ip (be patient).done Server Software: nginx Server Hostname: ip Server Port: 80 Document Path: / Document Length: 16247 bytes Concurrency Level: 10 Time taken for tests: 0.006 seconds Complete requests: 100 Failed requests: 0 Total transferred: 1651400 bytes HTML transferred: 1624700 bytes Requests per second: 16123.83 [#/sec] (mean) Time per request: 0.620 [ms] (mean) Time per request: 0.062 [ms] (mean, across all concurrent requests) Transfer rate: 260028.27 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 0 0 0.1 0 1 Waiting: 0 0 0.0 0 1 Total: 0 1 0.0 1 1 Percentage of the requests served within a certain time (ms) 50% 1 66% 1 75% 1 80% 1 90% 1 95% 1 98% 1 99% 1 100% 1 (longest request) ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On 4/6/2019 2:56 AM, Pete French wrote: Something odd going on there there - I am using 12-STABLE and I have igb just fine, and it attaches to the same hardware that 11 did: It does work in 12, throughput is great, just that the latency is higher than 11. igb0: flags=8843 metric 0 mtu 1500 options=e527bb ether 38:ea:a7:8d:c1:6c inet 208.72.56.19 netmask 0xfc00 broadcast 208.72.59.255 inet6 fe80::3aea:a7ff:fe8d:c16c%igb0 prefixlen 64 scopeid 0x1 inet6 2602:ffb8::208:72:56:9 prefixlen 64 media: Ethernet autoselect (1000baseT ) status: active nd6 options=21 Do you have a custom kernel, and if so did you see this note in UPDATING? Yes I do, but it includes all of GENERIC which includes em drivers, otherwise it wouldn't even work with the network card. my custom kernel: include GENERIC ident CUSTOM makeoptions WITH_EXTRA_TCP_STACKS=1 options TCPHPTS options SC_KERNEL_CONS_ATTR=(FG_GREEN|BG_BLACK) options IPSTEALTH options AHC_REG_PRETTY_PRINT # Print register bitfields in debug options AHD_REG_PRETTY_PRINT # Print register bitfields in debug device cryptodev device aesni I did try without RACK just in case that was the culprit. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: em performs worse than igb (latency wise) in 12?
On 05/04/2019 16:01, Kris von Mach wrote: I've upgraded from Stable 11 to Stable 12, and noticed that igb has been removed and is now part of em driver. However, the performance seems a lot worse. This is using HP 366FLR which is just HP's version of Intel i350 I believe. Something odd going on there there - I am using 12-STABLE and I have igb just fine, and it attaches to the same hardware that 11 did: ... [petefrench@dilbert ~]$ ssh turpentine Last login: Fri Apr 5 18:52:50 2019 from 2a02:b90:3002:411::6 FreeBSD 12.0-STABLE r343538 GENERIC Baby baby turpentine... [webadmin@turpentine ~]$ ifconfig igb0 igb0: flags=8843 metric 0 mtu 1500 options=e507bb ether ac:1f:6b:46:5e:32 inet 10.32.10.5 netmask 0x broadcast 10.32.255.255 inet6 fe80::ae1f:6bff:fe46:5e32%igb0 prefixlen 64 scopeid 0x1 inet6 2a02:1658:1:2:e550::5 prefixlen 64 media: Ethernet autoselect (1000baseT ) status: active nd6 options=21 Do you have a custom kernel, and if so did you see this note in UPDATING? 20170109: The igb(4), em(4) and lem(4) ethernet drivers are now implemented via IFLIB. If you have a custom kernel configuration that excludes em(4) but you use igb(4), you need to re-add em(4) to your custom configuration. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
