On 28.1.2024. 10:44, Marcus Glocker wrote:
> On Sun, Jan 28, 2024 at 12:16:20AM +0100, Hrvoje Popovski wrote:
>
>> On 27.1.2024. 21:01, Marcus Glocker wrote:
>>> On Sat, Jan 27, 2024 at 08:01:09AM +0100, Hrvoje Popovski wrote:
>>>
>>>> On 26.1.2024. 21:56, Marcus Glocker wrote:
>>>>> On Fri, Jan 26, 2024 at 11:41:49AM +0100, Hrvoje Popovski wrote:
>>>>>
>>>>>> I've manage to reproduce TSO em problem on anoter setup, unfortunatly
>>>>>> production.
>>>>>>
>>>>>> Setup is very simple
>>>>>>
>>>>>> em0 - carp <- uplink
>>>>>> em1 - pfsync
>>>>>> ix1 - vlans - carp
>>>>> Would it be possible that you also share an "ifconfig -a hwfeatures" of
>>>>> that box? You can mask the IPs if it's too sensitive.
>>>>>
>>>>> I still try to reproduce the issue here, and for now I can't.
>>>>> Maybe in your full ifconfig output I can see some specifics about your
>>>>> configuration, which makes it more likely to reproduce the issue here.
>>>>>
>>>> Hi,
>>>>
>>>> here's ifconfig from second setup where watchdog is triggered much faster.
>>>> Originally in this setup uplink is ix0, I've change that to em0 to see
>>>> would the problem be same as in other setup and it is, and that's good
>>>> because this is pfsync setup for students and I can do whatever I want
>>>> with it :)
>>> Thanks.
>>>
>>> But still, I can do whatever I want on my em(4) I210 box, carp(4),
>>> vlan(4), creating a lot of traffic, I can't reproduce the watchdog which
>>> you are seeing :-( I'm not sure if this is something related to your
>>> I350.
>>>
>>> Also, I can't understand why the watchdog still triggers when you disable
>>> TSO by setting net.inet.tcp.tso=0.
>>>
>>> Just to rule out that you're receiving a MAXMCLBYTES (65536) packet,
>>> while EM_TSO_SIZE (65535) is one byte less, can you please apply this
>>> diff to -current and test it? I doubt it will make a difference, but
>>> I'm running a bit out of ideas here.
>>
>>
>> Hi,
>>
>> with this diff I'm still getting em watchdog
>>
>> Jan 28 00:14:12 bcbnfw1 /bsd: em0: watchdog: head 120 tail 185 TDH 185
>> TDT 120
>
> Thanks for testing again.
>
> I think we might have a generic problem with TSO with the current em(4)
> code and some chips. Referring to this recent FreeBSD commit.
>
> e1000: disable TSO on lem(4) and em(4):
> Disable TSO on lem(4) and em(4) until a ring stall can be debugged.
> https://github.com/freebsd/freebsd-src/commit/797e480cba8834e584062092c098e60956d28180
>
> Can you try this diff to specifically disable TSO for I350 please?
>
> We will need to discuss internally which way to go. I see those
> options currently:
>
> - Entirely pull out the TSO diff.
> - Leave the TSO code in but disable TSO for now (what FreeBSD did).
> - Leave the TSO code in but disable TSO only for chips we see issues
> with (this diff).
>
Hi,
with this diff I still see TSOv4 and TSOv6 on i350 is this ok ?
em0 watchgod is triggered with or without net.inet.tcp.tso=1/0
em0: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
hwfeatures=31b7<CSUM_IPv4,CSUM_TCPv4,CSUM_UDPv4,VLAN_MTU,VLAN_HWTAGGING,CSUM_TCPv6,CSUM_UDPv6,TSOv4,TSOv6>
hardmtu 9216
lladdr 0c:c4:7a:da:cd:5a
index 3 priority 0 llprio 3
groups: egress
media: Ethernet autoselect (1000baseT full-duplex,master,rxpause)
status: active
em0 at pci7 dev 0 function 0 "Intel I350" rev 0x01: msi, address
Jan 28 13:18:45 bcbnfw1 /bsd: em0: watchdog: head 89 tail 153 TDH 153 TDT 89
Jan 28 13:41:19 bcbnfw1 /bsd: em0: watchdog: head 336 tail 400 TDH 400
TDT 336
Jan 28 13:58:13 bcbnfw1 /bsd: em0: watchdog: head 172 tail 236 TDH 236
TDT 172
>
> Index: if_em.c
> ===================================================================
> RCS file: /cvs/src/sys/dev/pci/if_em.c,v
> diff -u -p -u -p -r1.370 if_em.c
> --- if_em.c 31 Dec 2023 08:42:33 -0000 1.370
> +++ if_em.c 28 Jan 2024 09:30:59 -0000
> @@ -2013,7 +2013,9 @@ em_setup_interface(struct em_softc *sc)
> if (sc->hw.mac_type >= em_82575 && sc->hw.mac_type <= em_i210) {
> ifp->if_capabilities |= IFCAP_CSUM_IPv4;
> ifp->if_capabilities |= IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6;
> - ifp->if_capabilities |= IFCAP_TSOv4 | IFCAP_TSOv6;
> + /* XXX: Enabling TSO on I350 causes watchdogs */
> + if (sc->hw.mac_type != em_i350)
> + ifp->if_capabilities |= IFCAP_TSOv4 | IFCAP_TSOv6;
> }
>
> /*
>
