Re: [Bug 209423] WARN_ON_ONCE() at rtl8169_tso_csum_v2()
On Tue, 19 Jan 2021 14:54:31 +0100 Eric Dumazet wrote: > On Tue, Jan 19, 2021 at 1:40 PM Juerg Haefliger > wrote: > > > > > I seem to have stumbled over the same or a similar issue with a Raspberry Pi > > 3B+ running 5.11-rc4 and using the on-board lan78xx USB NIC. The Pi is used > > as a gateway. If I enable IP forwarding on the Pi and pound on eth0 [1], I > > get tons of the below warnings after a couple of seconds: > > > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.744157] skb len=54 > > headroom=5194 headlen=54 tailroom=10816 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.744157] > > mac=(5194,14) net=(5208,20) trans=5228 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.744157] > > shinfo(txflags=0 nr_frags=0 gso(size=1448 type=0 segs=1)) > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.744157] csum(0xe505 > > ip_summed=0 complete_sw=0 valid=0 level=0) > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.744157] hash(0x0 > > sw=0 l4=0) proto=0x0800 pkttype=0 iif=2 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.774147] dev > > name=eth0 feat=0x0x01114b09 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.779355] skb linear: > > : e0 28 6d 9e b9 22 b8 27 eb 3e ab fb 08 00 45 00 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.787365] skb linear: > > 0010: 00 28 00 00 40 00 3f 06 41 d0 c0 a8 63 84 02 14 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.795266] skb linear: > > 0020: d3 bf ed 3e 01 bb d4 0f 88 7e 00 00 00 00 50 04 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.803168] skb linear: > > 0030: 00 00 6a 58 00 00 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.808384] > > [ cut here ] > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.813200] lan78xx: > > caps=(0x01114b09, 0x) > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.819717] WARNING: > > CPU: 0 PID: 0 at net/core/dev.c:3197 skb_warn_bad_offload+0x84/0x100 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.828190] Modules > > linked in: > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.831354] CPU: 0 PID: > > 0 Comm: swapper/0 Not tainted 5.11.0-rc4 #103 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.838009] Hardware > > name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT) > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.844478] pstate: > > 6005 (nZCv daif -PAN -UAO -TCO BTYPE=--) > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.850685] pc : > > skb_warn_bad_offload+0x84/0x100 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.855464] lr : > > skb_warn_bad_offload+0x84/0x100 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.860242] sp : > > 800010003850 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.863665] x29: > > 800010003850 x28: 7a96fb196290 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.869160] x27: > > 7a96c5958300 x26: 0001 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.874654] x25: > > a73eee323000 x24: 7a96ee84b000 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.880148] x23: > > a73eee7f4f00 x22: > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.885642] x21: > > a73eee0327e0 x20: 7a96ee84b000 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.891136] x19: > > 7a96c5958300 x18: 0010 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.896630] x17: > > x16: > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.902123] x15: > > ad55 x14: 0010 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.907617] x13: > > x12: a73eedd9d950 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.913109] x11: > > a73eee885de0 x10: a73eee86dda0 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.918603] x9 : > > a73eecf2f45c x8 : 00017fe8 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.924097] x7 : > > c000efff x6 : 0003 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.929590] x5 : > > x4 : > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abfb kernel: [ 1914.935081] x3 : > > 0100 x2 : 1000 > > Jan 19 07:55:22 rpi-3b-plus-rev1d3-abf
Re: [Bug 209423] WARN_ON_ONCE() at rtl8169_tso_csum_v2()
On Thu, 8 Oct 2020 20:50:28 +0200
Eric Dumazet wrote:
> On Thu, Oct 8, 2020 at 8:42 PM Heiner Kallweit wrote:
> >
> > On 08.10.2020 19:15, Eric Dumazet wrote:
> > > On Thu, Oct 8, 2020 at 6:37 PM Heiner Kallweit
> > > wrote:
> > >>
> > >> On 02.10.2020 13:48, Eric Dumazet wrote:
> > >>> On Fri, Oct 2, 2020 at 1:09 PM Heiner Kallweit
> > >>> wrote:
> >
> > On 02.10.2020 10:46, Eric Dumazet wrote:
> > > On Fri, Oct 2, 2020 at 10:32 AM Eric Dumazet
> > > wrote:
> > >>
> > >>
> > >>
> > >> On 10/2/20 10:26 AM, Eric Dumazet wrote:
> > >>> On Thu, Oct 1, 2020 at 10:34 PM Heiner Kallweit
> > >>> wrote:
> >
> > I have a problem with the following code in ndo_start_xmit() of
> > the r8169 driver. A user reported the WARN being triggered due
> > to gso_size > 0 and gso_type = 0. The chip supports TSO(6).
> > The driver is widely used, therefore I'd expect much more such
> > reports if it should be a common problem. Not sure what's special.
> > My primary question: Is it a valid use case that gso_size is
> > greater than 0, and no SKB_GSO_ flag is set?
> > Any hint would be appreciated.
> >
> >
> > >>>
> > >>> Maybe this is not a TCP packet ? But in this case GSO should have
> > >>> taken place.
> > >>>
> > >>> You might add a
> > >>> pr_err_once("gso_type=%x\n", shinfo->gso_type);
> > >>>
> > >
> > >>
> > >> Ah, sorry I see you already printed gso_type
> > >>
> > >> Must then be a bug somewhere :/
> > >
> > >
> > > napi_reuse_skb() does :
> > >
> > > skb_shinfo(skb)->gso_type = 0;
> > >
> > > It does _not_ clear gso_size.
> > >
> > > I wonder if in some cases we could reuse an skb while gso_size is not
> > > zero.
> > >
> > > Normally, we set it only from dev_gro_receive() when the skb is queued
> > > into GRO engine (status being GRO_HELD)
> > >
> > Thanks Eric. I'm no expert that deep in the network stack and just
> > wonder
> > why napi_reuse_skb() re-initializes less fields in shinfo than
> > __alloc_skb().
> > The latter one does a
> > memset(shinfo, 0, offsetof(struct skb_shared_info, dataref));
> >
> > >>>
> > >>> memset() over the whole thing is more expensive.
> > >>>
> > >>> Here we know the prior state of some fields, while __alloc_skb() just
> > >>> got a piece of memory with random content.
> > >>>
> > What I can do is letting the affected user test the following.
> >
> > diff --git a/net/core/dev.c b/net/core/dev.c
> > index 62b06523b..8e75399cc 100644
> > --- a/net/core/dev.c
> > +++ b/net/core/dev.c
> > @@ -6088,6 +6088,7 @@ static void napi_reuse_skb(struct napi_struct
> > *napi, struct sk_buff *skb)
> >
> > skb->encapsulation = 0;
> > skb_shinfo(skb)->gso_type = 0;
> > + skb_shinfo(skb)->gso_size = 0;
> > skb->truesize = SKB_TRUESIZE(skb_end_offset(skb));
> > skb_ext_reset(skb);
> >
> > >>>
> > >>> As I hinted, this should not be needed.
> > >>>
> > >>> For debugging purposes, I would rather do :
> > >>>
> > >>> BUG_ON(skb_shinfo(skb)->gso_size);
> > >>>
> > >>
> > >> We did the following for debugging:
> > >>
> > >> diff --git a/net/core/dev.c b/net/core/dev.c
> > >> index 62b06523b..4c943b774 100644
> > >> --- a/net/core/dev.c
> > >> +++ b/net/core/dev.c
> > >> @@ -3491,6 +3491,9 @@ static netdev_features_t gso_features_check(const
> > >> struct sk_buff *skb,
> > >> {
> > >> u16 gso_segs = skb_shinfo(skb)->gso_segs;
> > >>
> > >> + if (!skb_shinfo(skb)->gso_type)
> > >> + skb_warn_bad_offload(skb);
> > >
> > > You also want to get a stack trace here, to give us the call graph.
> > >
> >
> > Here it comes, full story is in
> > https://bugzilla.kernel.org/show_bug.cgi?id=209423
> >
> >
> > [236222.967498] [ cut here ]
> > [236222.967508] r8169: caps=(0x010041b2, 0x)
> > [236222.967668] WARNING: CPU: 0 PID: 0 at net/core/dev.c:3184
> > skb_warn_bad_offload+0x72/0xe0
> > [236222.967691] Modules linked in: tcp_diag udp_diag raw_diag inet_diag
> > unix_diag tun nft_nat nft_masq nft_objref nf_conntrack_netbios_ns
> > nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
> > nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
> > nft_chain_nat ip_set_hash_net ip6table_nat ip6table_mangle ip6table_raw
> > ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6
> > nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security ip_set nf_tables
> > nfnetlink ip6table_filter ip6_tables iptable_filter sunrpc vfat fat
> > snd_hda_codec_realtek snd_hda_codec_generic edac_mce_amd ledtrig_audio
> > kvm_amd snd_hda_codec_hdmi ccp snd_hda_intel snd_intel_ds
Re: lan78xx: /sys/class/net/eth0/carrier stuck at 1
On Tue, 3 Nov 2020 16:27:23 +0100
Andrew Lunn wrote:
> On Tue, Nov 03, 2020 at 01:47:12PM +0100, Juerg Haefliger wrote:
> > On Fri, 23 Oct 2020 15:05:19 +0200
> > Andrew Lunn wrote:
> >
> > > On Fri, Oct 23, 2020 at 08:29:59AM +0200, Juerg Haefliger wrote:
> > > > On Wed, 21 Oct 2020 21:35:48 +0200
> > > > Andrew Lunn wrote:
> > > >
> > > > > On Wed, Oct 21, 2020 at 05:00:53PM +0200, Juerg Haefliger wrote:
> > > > > > Hi,
> > > > > >
> > > > > > If the lan78xx driver is compiled into the kernel and the network
> > > > > > cable is
> > > > > > plugged in at boot, /sys/class/net/eth0/carrier is stuck at 1 and
> > > > > > doesn't
> > > > > > toggle if the cable is unplugged and replugged.
> > > > > >
> > > > > > If the network cable is *not* plugged in at boot, all seems to work
> > > > > > fine.
> > > > > > I.e., post-boot cable plugs and unplugs toggle the carrier flag.
> > > > > >
> > > > > > Also, everything seems to work fine if the driver is compiled as a
> > > > > > module.
> > > > > >
> > > > > > There's an older ticket for the raspi kernel [1] but I've just
> > > > > > tested this
> > > > > > with a 5.8 kernel on a Pi 3B+ and still see that behavior.
> > > > >
> > > > > Hi Jürg
> > > >
> > > > Hi Andrew,
> > > >
> > > >
> > > > > Could you check if a different PHY driver is being used when it is
> > > > > built and broken vs module or built in and working.
> > > > >
> > > > > Look at /sys/class/net/eth0/phydev/driver
> > > >
> > > > There's no such file.
> > >
> > > I _think_ that means it is using genphy, the generic PHY driver, not a
> > > specific vendor PHY driver? What does
> > >
> > > /sys/class/net/eth0/phydev/phy_id contain.
> >
> > There is no directory /sys/class/net/eth0/phydev.
>
> [Goes and looks at the code]
>
> The symbolic link is only created if the PHY is connected to the MAC
> if the MAC has been registered with the core first. lan78xx does it
> the other way around:
>
> ret = lan78xx_phy_init(dev);
> if (ret < 0)
> goto out4;
>
> ret = register_netdev(netdev);
> if (ret != 0) {
> netif_err(dev, probe, netdev, "couldn't register the
> device\n");
> goto out5;
> }
>
> The register dump you show below indicates an ID of 007c132, which
> fits the drivers drivers/net/phy/microchip.c : "Microchip
> LAN88xx". Any mention of that in dmesg, do you see the module loaded?
It's built-in but is being used (as tracing shows).
> >
> > > > Given that all works fine as long as the cable is unplugged at boot
> > > > points
> > > > more towards a race at boot or incorrect initialization sequence or
> > > > something.
> > >
> > > Could be. Could you run
> > >
> > > mii-tool -vv eth0
> >
> > Hrm. Running that command unlocks the carrier flag and it starts toggling on
> > cable unplug/plug. First invocation:
> >
> > $ sudo mii-tool -vv eth0
> > Using SIOCGMIIPHY=0x8947
> > eth0: negotiated 1000baseT-FD flow-control, link ok
> > registers for MII PHY 1:
> > 1040 79ed 0007 c132 05e1 cde1 000f
> > 0200 0800 3000
> > 0088 3200 0004
> > 0040 a000 a000 a035
> > product info: vendor 00:01:f0, model 19 rev 2
> > basic mode: autonegotiation enabled
> > basic status: autonegotiation complete, link ok
> > capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
> > advertising: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD
> > 10baseT-HD flow-control
> > link partner: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD
> > 10baseT-HD flow-control
> >
> > Subsequent invocation:
> >
> > $ sudo mii-tool -vv eth0
> > Using SIOCGMIIPHY=0x8947
> > eth0: negotiated 1000baseT-FD flow-control, link ok
> > registers for MII PHY 1:
> > 1040 79ed 0007 c132 05e1 cde1 000d
> > 0200 0800 3000
Re: lan78xx: /sys/class/net/eth0/carrier stuck at 1
On Fri, 23 Oct 2020 15:05:19 +0200 Andrew Lunn wrote: > On Fri, Oct 23, 2020 at 08:29:59AM +0200, Juerg Haefliger wrote: > > On Wed, 21 Oct 2020 21:35:48 +0200 > > Andrew Lunn wrote: > > > > > On Wed, Oct 21, 2020 at 05:00:53PM +0200, Juerg Haefliger wrote: > > > > Hi, > > > > > > > > If the lan78xx driver is compiled into the kernel and the network cable > > > > is > > > > plugged in at boot, /sys/class/net/eth0/carrier is stuck at 1 and > > > > doesn't > > > > toggle if the cable is unplugged and replugged. > > > > > > > > If the network cable is *not* plugged in at boot, all seems to work > > > > fine. > > > > I.e., post-boot cable plugs and unplugs toggle the carrier flag. > > > > > > > > Also, everything seems to work fine if the driver is compiled as a > > > > module. > > > > > > > > There's an older ticket for the raspi kernel [1] but I've just tested > > > > this > > > > with a 5.8 kernel on a Pi 3B+ and still see that behavior. > > > > > > Hi Jürg > > > > Hi Andrew, > > > > > > > Could you check if a different PHY driver is being used when it is > > > built and broken vs module or built in and working. > > > > > > Look at /sys/class/net/eth0/phydev/driver > > > > There's no such file. > > I _think_ that means it is using genphy, the generic PHY driver, not a > specific vendor PHY driver? What does > > /sys/class/net/eth0/phydev/phy_id contain. There is no directory /sys/class/net/eth0/phydev. $ ls /sys/class/net/eth0/ addr_assign_type broadcastcarrier_down_count dev_port duplex ifalias link_mode netdev_group phys_port_name proto_down statisticstype addr_len carrier carrier_up_countdeviceflags ifindex mtu operstate phys_switch_id queues subsystem uevent address carrier_changes dev_id dormant gro_flush_timeout iflink name_assign_type phys_port_id power speed tx_queue_len > > Given that all works fine as long as the cable is unplugged at boot points > > more towards a race at boot or incorrect initialization sequence or > > something. > > Could be. Could you run > > mii-tool -vv eth0 Hrm. Running that command unlocks the carrier flag and it starts toggling on cable unplug/plug. First invocation: $ sudo mii-tool -vv eth0 Using SIOCGMIIPHY=0x8947 eth0: negotiated 1000baseT-FD flow-control, link ok registers for MII PHY 1: 1040 79ed 0007 c132 05e1 cde1 000f 0200 0800 3000 0088 3200 0004 0040 a000 a000 a035 product info: vendor 00:01:f0, model 19 rev 2 basic mode: autonegotiation enabled basic status: autonegotiation complete, link ok capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD advertising: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control link partner: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control Subsequent invocation: $ sudo mii-tool -vv eth0 Using SIOCGMIIPHY=0x8947 eth0: negotiated 1000baseT-FD flow-control, link ok registers for MII PHY 1: 1040 79ed 0007 c132 05e1 cde1 000d 0200 0800 3000 0088 3200 0004 0040 a000 a035 product info: vendor 00:01:f0, model 19 rev 2 basic mode: autonegotiation enabled basic status: autonegotiation complete, link ok capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD advertising: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control link partner: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control In the first invocation, register 0x1a shows a pending link-change interrupt (0xa000) which wasn't serviced (and cleared) for some reason. Dumping the registers cleared that interrupt bit and things start working correctly afterwards. Nor sure yet why that first interrupt is ignored. ...Juerg > in the good and bad case. > >Andrew > pgpEo77BWLfwu.pgp Description: OpenPGP digital signature
Re: lan78xx: /sys/class/net/eth0/carrier stuck at 1
On Wed, 21 Oct 2020 21:35:48 +0200 Andrew Lunn wrote: > On Wed, Oct 21, 2020 at 05:00:53PM +0200, Juerg Haefliger wrote: > > Hi, > > > > If the lan78xx driver is compiled into the kernel and the network cable is > > plugged in at boot, /sys/class/net/eth0/carrier is stuck at 1 and doesn't > > toggle if the cable is unplugged and replugged. > > > > If the network cable is *not* plugged in at boot, all seems to work fine. > > I.e., post-boot cable plugs and unplugs toggle the carrier flag. > > > > Also, everything seems to work fine if the driver is compiled as a module. > > > > There's an older ticket for the raspi kernel [1] but I've just tested this > > with a 5.8 kernel on a Pi 3B+ and still see that behavior. > > Hi Jürg Hi Andrew, > Could you check if a different PHY driver is being used when it is > built and broken vs module or built in and working. > > Look at /sys/class/net/eth0/phydev/driver There's no such file. > I'm wondering if in the builtin case, it is using genphy, but with > modules it uses a more specific vendor driver. Given that all works fine as long as the cable is unplugged at boot points more towards a race at boot or incorrect initialization sequence or something. ...Juerg > Andrew pgpul9eCxBRmA.pgp Description: OpenPGP digital signature
lan78xx: /sys/class/net/eth0/carrier stuck at 1
Hi, If the lan78xx driver is compiled into the kernel and the network cable is plugged in at boot, /sys/class/net/eth0/carrier is stuck at 1 and doesn't toggle if the cable is unplugged and replugged. If the network cable is *not* plugged in at boot, all seems to work fine. I.e., post-boot cable plugs and unplugs toggle the carrier flag. Also, everything seems to work fine if the driver is compiled as a module. There's an older ticket for the raspi kernel [1] but I've just tested this with a 5.8 kernel on a Pi 3B+ and still see that behavior. ...Juerg [1] https://github.com/raspberrypi/firmware/issues/1100 pgp9e4yvmV6Ey.pgp Description: OpenPGP digital signature
Re: [ovs-dev] openvswitch crash on i386
On Tue, 5 Mar 2019 11:58:42 -0800 Joe Stringer wrote: > On Tue, Mar 5, 2019 at 2:12 AM Christian Ehrhardt > wrote: > > > > On Tue, Mar 5, 2019 at 10:58 AM Juerg Haefliger > > wrote: > > > > > > Hi, > > > > > > Running the following commands in a loop will crash an i386 5.0 kernel > > > typically within a few iterations: > > > > > > ovs-vsctl add-br test > > > ovs-vsctl del-br test > > > > > > [ 106.215748] BUG: unable to handle kernel paging request at e8a35f3b > > > [ 106.216733] #PF error: [normal kernel read fault] > > > [ 106.217464] *pdpt = 19a76001 *pde = > > > [ 106.218346] Oops: [#1] SMP PTI > > > [ 106.218911] CPU: 0 PID: 2050 Comm: systemd-udevd Tainted: G > > > E 5.0.0 #25 > > > [ 106.220103] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > > > BIOS 1.11.1-1ubuntu1 04/01/2014 > > > [ 106.221447] EIP: kmem_cache_alloc_trace+0x7a/0x1b0 > > > [ 106.222178] Code: 01 00 00 8b 07 64 8b 50 04 64 03 05 28 61 e8 d2 8b > > > 08 89 4d ec 85 c9 0f 84 03 01 00 00 8b 45 ec 8b 5f 14 8d 4a 01 8b 37 01 > > > c3 <33> 1b 33 9f b4 00 00 00 64 0f c7 0e 75 cb 8b 75 ec 8b 47 14 0f 18 > > > [ 106.224752] EAX: e8a35f3b EBX: e8a35f3b ECX: 869f EDX: 869e > > > [ 106.225683] ESI: d2e96ef0 EDI: da401a00 EBP: d9b85dd0 ESP: d9b85db0 > > > [ 106.226662] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: > > > 00010282 > > > [ 106.227710] CR0: 80050033 CR2: e8a35f3b CR3: 185b8000 CR4: 06f0 > > > [ 106.228703] DR0: DR1: DR2: DR3: > > > [ 106.229604] DR6: fffe0ff0 DR7: 0400 > > > [ 106.230114] Call Trace: > > > [ 106.230525] ? kernfs_fop_open+0xb4/0x390 > > > [ 106.231176] kernfs_fop_open+0xb4/0x390 > > > [ 106.231856] ? security_file_open+0x7c/0xc0 > > > [ 106.232562] do_dentry_open+0x131/0x370 > > > [ 106.233229] ? kernfs_fop_write+0x180/0x180 > > > [ 106.233905] vfs_open+0x25/0x30 > > > [ 106.234432] path_openat+0x2fd/0x1450 > > > [ 106.235084] ? cp_new_stat64+0x115/0x140 > > > [ 106.235754] ? cp_new_stat64+0x115/0x140 > > > [ 106.236427] do_filp_open+0x6a/0xd0 > > > [ 106.237026] ? cp_new_stat64+0x115/0x140 > > > [ 106.237748] ? strncpy_from_user+0x3d/0x180 > > > [ 106.238539] ? __alloc_fd+0x36/0x120 > > > [ 106.239256] do_sys_open+0x175/0x210 > > > [ 106.239955] sys_openat+0x1b/0x20 > > > [ 106.240596] do_fast_syscall_32+0x7f/0x1e0 > > > [ 106.241313] entry_SYSENTER_32+0x6b/0xbe > > > [ 106.242017] EIP: 0xb7fae871 > > > [ 106.242559] Code: 8b 98 58 cd ff ff 89 c8 85 d2 74 02 89 0a 5b 5d c3 > > > 8b 04 24 c3 8b 14 24 c3 8b 34 24 c3 8b 3c 24 c3 51 52 55 89 e5 0f 34 cd > > > 80 <5d> 5a 59 c3 90 90 90 90 8d 76 00 58 b8 77 00 00 00 cd 80 90 8d 76 > > > [ 106.245551] EAX: ffda EBX: ff9c ECX: bffdcb60 EDX: 00088000 > > > [ 106.246651] ESI: EDI: b7f9e000 EBP: 00088000 ESP: bffdc970 > > > [ 106.247706] DS: 007b ES: 007b FS: GS: 0033 SS: 007b EFLAGS: > > > 0246 > > > [ 106.248851] Modules linked in: openvswitch(E) > > > [ 106.249621] CR2: e8a35f3b > > > [ 106.250218] ---[ end trace 6a8d05679a59cda7 ]--- > > > > > > I've bisected this down to the following commit that seems to have > > > introduced > > > the issue: > > > > > > commit 120645513f55a4ac5543120d9e79925d30a0156f (refs/bisect/bad) > > > Author: Jarno Rajahalme > > > Date: Fri Apr 21 16:48:06 2017 -0700 > > > > > > openvswitch: Add eventmask support to CT action. > > > > > > Add a new optional conntrack action attribute OVS_CT_ATTR_EVENTMASK, > > > which can be used in conjunction with the commit flag > > > (OVS_CT_ATTR_COMMIT) to set the mask of bits specifying which > > > conntrack events (IPCT_*) should be delivered via the Netfilter > > > netlink multicast groups. Default behavior depends on the system > > > configuration, but typically a lot of events are delivered. This can > > > be > > > very chatty for the NFNLGRP_CONNTRACK_UPDATE group, even if only some > > > types of events are of interest. > > > > > > Netfilter core init_conntrack() adds the event cache extension, so we > > > only need to set the ctmask value. However, if the system is > > > c
openvswitch crash on i386
Hi, Running the following commands in a loop will crash an i386 5.0 kernel typically within a few iterations: ovs-vsctl add-br test ovs-vsctl del-br test [ 106.215748] BUG: unable to handle kernel paging request at e8a35f3b [ 106.216733] #PF error: [normal kernel read fault] [ 106.217464] *pdpt = 19a76001 *pde = [ 106.218346] Oops: [#1] SMP PTI [ 106.218911] CPU: 0 PID: 2050 Comm: systemd-udevd Tainted: GE 5.0.0 #25 [ 106.220103] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.1-1ubuntu1 04/01/2014 [ 106.221447] EIP: kmem_cache_alloc_trace+0x7a/0x1b0 [ 106.222178] Code: 01 00 00 8b 07 64 8b 50 04 64 03 05 28 61 e8 d2 8b 08 89 4d ec 85 c9 0f 84 03 01 00 00 8b 45 ec 8b 5f 14 8d 4a 01 8b 37 01 c3 <33> 1b 33 9f b4 00 00 00 64 0f c7 0e 75 cb 8b 75 ec 8b 47 14 0f 18 [ 106.224752] EAX: e8a35f3b EBX: e8a35f3b ECX: 869f EDX: 869e [ 106.225683] ESI: d2e96ef0 EDI: da401a00 EBP: d9b85dd0 ESP: d9b85db0 [ 106.226662] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010282 [ 106.227710] CR0: 80050033 CR2: e8a35f3b CR3: 185b8000 CR4: 06f0 [ 106.228703] DR0: DR1: DR2: DR3: [ 106.229604] DR6: fffe0ff0 DR7: 0400 [ 106.230114] Call Trace: [ 106.230525] ? kernfs_fop_open+0xb4/0x390 [ 106.231176] kernfs_fop_open+0xb4/0x390 [ 106.231856] ? security_file_open+0x7c/0xc0 [ 106.232562] do_dentry_open+0x131/0x370 [ 106.233229] ? kernfs_fop_write+0x180/0x180 [ 106.233905] vfs_open+0x25/0x30 [ 106.234432] path_openat+0x2fd/0x1450 [ 106.235084] ? cp_new_stat64+0x115/0x140 [ 106.235754] ? cp_new_stat64+0x115/0x140 [ 106.236427] do_filp_open+0x6a/0xd0 [ 106.237026] ? cp_new_stat64+0x115/0x140 [ 106.237748] ? strncpy_from_user+0x3d/0x180 [ 106.238539] ? __alloc_fd+0x36/0x120 [ 106.239256] do_sys_open+0x175/0x210 [ 106.239955] sys_openat+0x1b/0x20 [ 106.240596] do_fast_syscall_32+0x7f/0x1e0 [ 106.241313] entry_SYSENTER_32+0x6b/0xbe [ 106.242017] EIP: 0xb7fae871 [ 106.242559] Code: 8b 98 58 cd ff ff 89 c8 85 d2 74 02 89 0a 5b 5d c3 8b 04 24 c3 8b 14 24 c3 8b 34 24 c3 8b 3c 24 c3 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d 76 00 58 b8 77 00 00 00 cd 80 90 8d 76 [ 106.245551] EAX: ffda EBX: ff9c ECX: bffdcb60 EDX: 00088000 [ 106.246651] ESI: EDI: b7f9e000 EBP: 00088000 ESP: bffdc970 [ 106.247706] DS: 007b ES: 007b FS: GS: 0033 SS: 007b EFLAGS: 0246 [ 106.248851] Modules linked in: openvswitch(E) [ 106.249621] CR2: e8a35f3b [ 106.250218] ---[ end trace 6a8d05679a59cda7 ]--- I've bisected this down to the following commit that seems to have introduced the issue: commit 120645513f55a4ac5543120d9e79925d30a0156f (refs/bisect/bad) Author: Jarno Rajahalme Date: Fri Apr 21 16:48:06 2017 -0700 openvswitch: Add eventmask support to CT action. Add a new optional conntrack action attribute OVS_CT_ATTR_EVENTMASK, which can be used in conjunction with the commit flag (OVS_CT_ATTR_COMMIT) to set the mask of bits specifying which conntrack events (IPCT_*) should be delivered via the Netfilter netlink multicast groups. Default behavior depends on the system configuration, but typically a lot of events are delivered. This can be very chatty for the NFNLGRP_CONNTRACK_UPDATE group, even if only some types of events are of interest. Netfilter core init_conntrack() adds the event cache extension, so we only need to set the ctmask value. However, if the system is configured without support for events, the setting will be skipped due to extension not being found. Signed-off-by: Jarno Rajahalme Reviewed-by: Greg Rose Acked-by: Joe Stringer Signed-off-by: David S. Miller Reverting that commit from 5.0 makes the problem go away. I'm not able to reproduce the crash on x86_64. ...Juerg pgpvzXPQ_6Z0y.pgp Description: OpenPGP digital signature
Re: [PATCH net v2 0/3] Tunneling fixes
> This series fixes a problem that was reported where encapsulated packets
> do not have their encapsulation offload markers stripped off when being
> decapsulated. This causes a significant performance drop if the packets
> are later retransmitted.
>
> Fixing this revealed two other bugs which are also addressed as prerequisites:
> * GRO can aggregate packets for multiple layers of encapsulation which the
>stack cannot properly handle.
> * IPIP packets which are combined by GRO are not marked properly with their
>GSO type.
>
> Note that this is based off the net-next tree as the current target for
> bug fixes.
I need to backport this series to the 4.4 kernel to fix a performance issue
we're seeing. The series
applies but commit a09a4c8dd1ec (tunnels: Remove encapsulation offloads on
decap) breaks compilation
when CONFIG_IPV6_SIT is enabled. This is because the patch uses
iptunnel_pull_header() whose usage
changed with commit 7f290c94352e (iptunnel: scrub packet in
iptunnel_pull_header) which is not in 4.4.
7f290c94352e seems to be a cleanup patch which also requires c9e78efb6f66
(vxlan: move vxlan device
lookup before iptunnel_pull_header) and potentially others. Rather than pulling
in a slew of cleanup
patches, I was wondering if the following from commit a09a4c8dd1ec can be
rewritten without using
the 'new' iptunnel_pull_header() function:
diff --git a/net/ipv6/sit.c b/net/ipv6/sit.c
index f45b8ffc2840..83384308d032 100644
--- a/net/ipv6/sit.c
+++ b/net/ipv6/sit.c
@@ -681,14 +681,16 @@ static int ipip6_rcv(struct sk_buff *skb)
skb->mac_header = skb->network_header;
skb_reset_network_header(skb);
IPCB(skb)->flags = 0;
- skb->protocol = htons(ETH_P_IPV6);
+ skb->dev = tunnel->dev;
if (packet_is_spoofed(skb, iph, tunnel)) {
tunnel->dev->stats.rx_errors++;
goto out;
}
- __skb_tunnel_rx(skb, tunnel->dev, tunnel->net);
+ if (iptunnel_pull_header(skb, 0, htons(ETH_P_IPV6),
+ !net_eq(tunnel->net, dev_net(tunnel->dev
+ goto out;
Thanks
...Juerg
> v2: No code changes, just additional information in commit messages and
> a new cover letter.
>
> Jesse Gross (3):
> ipip: Properly mark ipip GRO packets as encapsulated.
> tunnels: Don't apply GRO to multiple layers of encapsulation.
> tunnels: Remove encapsulation offloads on decap.
>
> include/linux/netdevice.h | 4 ++--
> include/net/ip_tunnels.h | 16
> net/core/dev.c| 2 +-
> net/ipv4/af_inet.c| 24 ++--
> net/ipv4/fou.c| 13 +++--
> net/ipv4/gre_offload.c| 5 +
> net/ipv4/ip_tunnel_core.c | 3 ++-
> net/ipv4/udp_offload.c| 6 +++---
> net/ipv6/ip6_offload.c| 15 ++-
> net/ipv6/sit.c| 6 --
> 10 files changed, 80 insertions(+), 14 deletions(-)
signature.asc
Description: OpenPGP digital signature
Re: [PATCH RFC 0/4] xfs: Transmit flow steering
As Rick states, this fixes a performance issue with the 4.4 kernel for us. Tested-by: Juerg Haefliger On 09/28/2016 05:13 PM, Rick Jones wrote: > > Here is a quick look at performance tests for the result of trying the > prototype fix for the packet reordering problem with VMs sending over > an XPS-configured NIC. In particular, the Emulex/Avago/Broadcom > Skyhawk. The fix was applied to a 4.4 kernel. > > Before: 3884 Mbit/s > After: 8897 Mbit/s > > That was from a VM on a node with a Skyhawk and 2 E5-2640 processors > to baremetal E5-2640 with a BE3. Physical MTU was 1500, the VM's > vNIC's MTU was 1400. Systems were HPE ProLiants in OS Control Mode > for power management, with the "performance" frequency governor > loaded. An OpenStack Mitaka setup with Distributed Virtual Router. > > We had some other NIC types in the setup as well. XPS was also > enabled on the ConnectX3-Pro. It was not enabled on the 82599ES (a > function of the kernel being used, which had it disabled from the > first reports of XPS negatively affecting VM traffic at the beginning > of the year) > > Average Mbit/s From NIC type To Bare Metal BE3: > NIC Type, > CPU on VM HostBeforeAfter > > ConnectX-3 Pro,E5-2670v39224 9271 > BE3, E5-26409016 9022 > 82599, E5-2640 9192 9003 > BCM57840, E5-2640 9213 9153 > Skyhawk, E5-26403884 8897 > > For completeness: > Average Mbit/s To NIC type from Bare Metal BE3: > NIC Type, > CPU on VM HostBeforeAfter > > ConnectX-3 Pro,E5-2670v39322 9144 > BE3, E5-26409074 9017 > 82599, E5-2640 8670 8564 > BCM57840, E5-2640 2468 * 7979 > Skyhawk, E5-26408897 9269 > > * This is the Busted bnx2x NIC FW GRO implementation issue. It was > not visible in the "After" because the system was setup to disable > the NIC FW GRO by the time it booted on the fix kernel. > > Average Transactions/s Between NIC type and Bare Metal BE3: > NIC Type, > CPU on VM HostBeforeAfter > > ConnectX-3 Pro,E5-2670v3 12421 12612 > BE3, E5-26408178 8484 > 82599, E5-2640 8499 8549 > BCM57840, E5-2640 8544 8560 > Skyhawk, E5-26408537 8701 > > happy benchmarking, > > Drew Balliet > Jeurg Haefliger > rick jones > > The semi-cooked results with additional statistics: > > 554M - BE3 > 544+M - ConnectX-3 Pro > 560M - 82599ES > 630M - BCM57840 > 650M - Skyhawk > > (substitute is simply replacing a system name with the model of NIC and CPU) > Bulk To (South) and From (North) VM, Before: > $ ../substitute.sh > vxlan_554m_control_performance_gvnr_dvr_northsouth_stream.log | > ~/netperf2_trunk/doc/examples/parse_single_stream.py -r -5 -f 1 -f 3 -f 4 -f > 7 -f 8 > Field1,Field3,Field4,Field7,Field8,Min,P10,Median,Average,P90,P99,Max,Count > North,560M,E5-2640,554FLB,E5-2640,8148.090,9048.830,9235.400,9192.868,9315.980,9338.845,9339.500,113 > North,630M,E5-2640,554FLB,E5-2640,8909.980,9113.238,9234.750,9213.140,9299.442,9336.206,9337.830,47 > North,544+M,E5-2670v3,554FLB,E5-2640,9013.740,9182.546,9229.620,9224.025,9264.036,9299.206,9301.970,99 > North,650M,E5-2640,554FLB,E5-2640,3187.680,3393.724,3796.160,3884.765,4405.096,4941.391,4956.300,129 > North,554M,E5-2640,554FLB,E5-2640,8700.930,8855.768,9026.030,9016.061,9158.846,9213.687,9226.150,135 > South,554FLB,E5-2640,560M,E5-2640,7754.350,8193.114,8718.540,8670.612,9026.436,9262.355,9285.010,113 > South,554FLB,E5-2640,630M,E5-2640,1897.660,2068.290,2514.430,2468.323,2787.162,2942.934,2957.250,53 > South,554FLB,E5-2640,544+M,E5-2670v3,9298.260,9314.432,9323.220,9322.207,9328.324,9330.704,9331.080,100 > South,554FLB,E5-2640,650M,E5-2640,8407.050,8907.136,9304.390,9206.776,9321.320,9325.347,9326.410,103 > South,554FLB,E5-2640,554M,E5-2640,7844.900,8632.530,9199.385,9074.535,9308.070,9319.224,9322.360,132 > 0 too-short lines ignored. > > Bulk To (South) and From (North) VM, After: > > $ ../substitute.sh > vxlan_554m_control_performance_gvnr_xpsfix_dvr_northsouth_stream.log | > ~/netperf2_trunk/doc/examples/parse_single_stream.py -r -5 -f 1 -f 3 -f 4 -f > 7 -f 8 > Field1,Field3,Field4,Field7,Field8,Min,P10,Median,Average,P90,P99,Max,Count > North,560M,E5-2640,554FLB,E5-2640,7576.790,8213.890,9182.870,9003.190,9295.975,9315.878,9318.160,36 > North,630M,E5
[PATCH stable 4.4] tipc: move linearization of buffers to generic code
From: Jon Paul Maloy
commit c7cad0d6f70cd4ce8644ffe528a4df1cdc2e77f5 upstream.
In commit 5cbb28a4bf65c7e4 ("tipc: linearize arriving NAME_DISTR
and LINK_PROTO buffers") we added linearization of NAME_DISTRIBUTOR,
LINK_PROTOCOL/RESET and LINK_PROTOCOL/ACTIVATE to the function
tipc_udp_recv(). The location of the change was selected in order
to make the commit easily appliable to 'net' and 'stable'.
We now move this linearization to where it should be done, in the
functions tipc_named_rcv() and tipc_link_proto_rcv() respectively.
Reviewed-by: Ying Xue
Signed-off-by: Jon Maloy
Signed-off-by: David S. Miller
Signed-off-by: Juerg Haefliger
---
This commit fixes an issue with nodes not joining a cluster over TIPC
after reboots. Upstream TIPC recommends to include this commit in the
stable kernel:
https://sourceforge.net/p/tipc/mailman/message/35368150/
---
net/tipc/link.c | 2 ++
net/tipc/name_distr.c | 1 +
net/tipc/udp_media.c | 5 -
3 files changed, 3 insertions(+), 5 deletions(-)
diff --git a/net/tipc/link.c b/net/tipc/link.c
index 91aea071ab27..72268eac4ec7 100644
--- a/net/tipc/link.c
+++ b/net/tipc/link.c
@@ -1262,6 +1262,8 @@ static int tipc_link_proto_rcv(struct tipc_link *l,
struct sk_buff *skb,
/* fall thru' */
case ACTIVATE_MSG:
+ skb_linearize(skb);
+ hdr = buf_msg(skb);
/* Complete own link name with peer's interface name */
if_name = strrchr(l->name, ':') + 1;
diff --git a/net/tipc/name_distr.c b/net/tipc/name_distr.c
index c07612bab95c..f51c8bdbea1c 100644
--- a/net/tipc/name_distr.c
+++ b/net/tipc/name_distr.c
@@ -397,6 +397,7 @@ void tipc_named_rcv(struct net *net, struct sk_buff_head
*inputq)
spin_lock_bh(&tn->nametbl_lock);
for (skb = skb_dequeue(inputq); skb; skb = skb_dequeue(inputq)) {
+ skb_linearize(skb);
msg = buf_msg(skb);
mtype = msg_type(msg);
item = (struct distr_item *)msg_data(msg);
diff --git a/net/tipc/udp_media.c b/net/tipc/udp_media.c
index 70c03271b798..6af78c6276b4 100644
--- a/net/tipc/udp_media.c
+++ b/net/tipc/udp_media.c
@@ -48,7 +48,6 @@
#include
#include "core.h"
#include "bearer.h"
-#include "msg.h"
/* IANA assigned UDP port */
#define UDP_PORT_DEFAULT 6118
@@ -224,10 +223,6 @@ static int tipc_udp_recv(struct sock *sk, struct sk_buff
*skb)
{
struct udp_bearer *ub;
struct tipc_bearer *b;
- int usr = msg_user(buf_msg(skb));
-
- if ((usr == LINK_PROTOCOL) || (usr == NAME_DISTRIBUTOR))
- skb_linearize(skb);
ub = rcu_dereference_sk_user_data(sk);
if (!ub) {
--
2.9.3
[PATCH] net/ixgbe: Allow resetting VF admin mac to zero
The VF administrative mac addresses (stored in the PF driver) are
initialized to zero when the PF driver starts up.
These addresses may be modified in the PF driver through ndo calls
initiated by iproute2 or libvirt.
While we allow the PF/host to change the VF admin mac address from zero
to a valid unicast mac, we do not allow restoring the VF admin mac to
zero. We currently only allow changing this mac to a different unicast mac.
This leads to problems when libvirt scripts are used to deal with
VF mac addresses, and libvirt attempts to revoke the mac so this
host will not use it anymore.
Fix this by allowing resetting a VF administrative MAC back to zero.
Implementation and commit message shamelessly stolen from:
commit 6e5224224faa ("net/mlx4_core: Allow resetting VF admin mac to zero")
Signed-off-by: Juerg Haefliger
---
drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
index c5caacd..d075387 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c
@@ -1280,7 +1280,7 @@ void ixgbe_ping_all_vfs(struct ixgbe_adapter *adapter)
int ixgbe_ndo_set_vf_mac(struct net_device *netdev, int vf, u8 *mac)
{
struct ixgbe_adapter *adapter = netdev_priv(netdev);
- if (!is_valid_ether_addr(mac) || (vf >= adapter->num_vfs))
+ if (is_multicast_ether_addr(mac) || (vf >= adapter->num_vfs))
return -EINVAL;
adapter->vfinfo[vf].pf_set_mac = true;
dev_info(&adapter->pdev->dev, "setting MAC %pM on VF %d\n", mac, vf);
--
2.8.1
