On 6/27/23 12:47 AM, josef.zahn...@swisscom.com wrote:
We are familiar with «hw.cxgbe.rsrv_noflowq», but as you already stated it’s only for TX direction, it doesn’t help us at all.
Okay.
Our problem is in fact, that on FreeBSD only CPU0 seems to do the slow protocol (LACP, CARP,…) stuff and even though the other CPUs are completely idle, if CPU0 has 100% load (which is in fact possible to achieve with one iperf session) you can create a scenario where LACP/CARP stops working due to the load on CPU0. So the idea would be to get rid of the RSS load of CPU0 to ensure that the slow protocols are always working as expected.
Are the LACP/CARP problems due to control packets getting dropped? Or did all the control packets make it in but the rx queue handling is so backlogged that the kernel did not process them in time, resulting in protocol timeouts? Look at the netstat counters for packet drops. If there are none then it must be slow processing. If you see rx drops then try setting hw.cxgbe.cong_drops=1 in loader.conf.
To sum it up, yes I’m in theory able to test a RX patch, as it is only a test setup. However I’m completely new in FreeBSD and we are using it only because our firewall (OPNsense) does use it. FreeBSD came on the ISO with the installation medium. So please explain in detail what we have to do…
My patch will apply to the stock FreeBSD kernel. I'm not familiar with OPNsense development workflow so not sure what to do there. If cong_drops=1 doesn't fix your problem all by itself then combining it with this future-patch really should fix it. I'll post the patch in a day or two.
Regards, Navdeep
Cheers Josef
