Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-09-18 Thread Josef.Zahner1
Hi Navdeep Any plans to make a pull request for your patch that we have the feature as well in FreeBSD officially? We use your change in prod now, works like a charm… But I don’t see your modification here https://github.com/freebsd/freebsd-src/blob/main/sys/dev/cxgbe Without your patch, we ha

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-13 Thread Josef.Zahner1
It was not my intention to tune before testing. The point was, when we started testing, we have seen that RSS wasn’t working with the Intel E810 NIC out of the box just after the installation of FreeBSD (OPNsense). All load was only on CPU0, no load distribution at all. So we enabled the RSS sys

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-11 Thread Navdeep Parhar
On 7/10/23 2:36 AM, josef.zahn...@swisscom.com wrote: Ok we are getting closer to a solution. But I’m a little bit confused, I’ll try to explain why. I’ve removed the following sysctl values, as you stated that they are not official FreeBSD values: -net.inet.rss.bits="2" -net.inet.rss.enable

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-10 Thread Josef.Zahner1
Ok we are getting closer to a solution. But I’m a little bit confused, I’ll try to explain why. I’ve removed the following sysctl values, as you stated that they are not official FreeBSD values: - net.inet.rss.bits="2" - net.inet.rss.enabled="1" - net.isr.bindthreads="1" - net.isr.maxthreads="-1

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-06 Thread Navdeep Parhar
On Mon, Jul 3, 2023 at 8:48 AM wrote: > > Sorry for the spam, I do see the values with sysctl now. It seems that > FreeBSD always loads the if_cxgbe.ko from /boot/kernel/if_cxgbe.ko. So what > I’ve done is, I renamed the original file and copied the newly compiled > if_cxgbe.ko from /boot/modul

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
;2" net.inet.rss.enabled="1" net.isr.bindthreads="1" net.isr.maxthreads="-1" t5fw_cfg_load="yes" Do I miss anything? Cheers Josef From: Zahner Josef, GSA-REP-LI Date: Monday, 3 July 2023 at 16:50 To: Navdeep Parhar Cc: freebsd-net@freebsd.org Su

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
ver still the same result, the new sysctl values are not visible. Any help would be appreciated. Btw.: reboot has been done. Cheers Josef From: Zahner Josef, GSA-REP-LI Date: Monday, 3 July 2023 at 15:45 To: Navdeep Parhar Cc: freebsd-net@freebsd.org Subject: Re: Chelsio NIC with RSS - Traf

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
Hi Navdeep Nevermind, I found the following command to apply your patch diff: root@fw-94:/sys/dev/cxgbe # patch < /root/chelsio_rss_patch.diff There was an inconsistency with “adapter.h”, your change on line 219 was in my case a few lines further below, so I decided to manually edit those two ch

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-03 Thread Josef.Zahner1
, 2 July 2023 at 03:21 To: Zahner Josef, GSA-REP-LI Cc: freebsd-net@freebsd.org Subject: Re: Chelsio NIC with RSS - Traffic distribution to different Queues Hello, Please try this patch: https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.freebsd.org%2F~np%2Fnorssq.diff&

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-07-01 Thread Navdeep Parhar
Hello, Please try this patch: https://people.freebsd.org/~np/norssq.diff It adds these sysctls to the driver. 1) hw.cxgbe.rsrv_norssq. This is what you originally asked for. 2) hw.cxgbe.rx_budget. This can be used to force the driver's RX to yield periodically. What kind of system (cores, memo

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-29 Thread Josef.Zahner1
Can you tell me which netstat command you have in mind? I tried “netstat -Q”, it shows a few drops but not that much that it would explain the CARP drops. What I can tell you is, that especially CARP on the corresponding server is just sending out packets as long as it is the master box and CPU0

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-27 Thread Navdeep Parhar
On 6/27/23 12:47 AM, josef.zahn...@swisscom.com wrote: We are familiar with «hw.cxgbe.rsrv_noflowq», but as you already stated it’s only for TX direction, it doesn’t help us at all. Okay. Our problem is in fact, that on FreeBSD only CPU0 seems to do the slow protocol (LACP, CARP,…) stuff and

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-27 Thread Josef.Zahner1
We are familiar with «hw.cxgbe.rsrv_noflowq», but as you already stated it’s only for TX direction, it doesn’t help us at all. Our problem is in fact, that on FreeBSD only CPU0 seems to do the slow protocol (LACP, CARP,…) stuff and even though the other CPUs are completely idle, if CPU0 has 100

Re: Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-27 Thread Navdeep Parhar
On Mon, Jun 26, 2023 at 5:58 AM wrote: > > Hi guys > > > > I’m trying to do something similar like in the question here > (https://lists.freebsd.org/pipermail/freebsd-net/2017-March/047378.html). We > have Chelsio NICs (T580-LP-CR). > > > > Our goal is the following: > > RSS Queue 0 -> only cont

Chelsio NIC with RSS - Traffic distribution to different Queues

2023-06-26 Thread Josef.Zahner1
Hi guys I’m trying to do something similar like in the question here (https://lists.freebsd.org/pipermail/freebsd-net/2017-March/047378.html). We have Chelsio NICs (T580-LP-CR). Our goal is the following: * RSS Queue 0 -> only control plane traffic, eg. CARP (IP Protocol 112 – Multicast),