Re: [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all
On 14/12/18 - 11:05:30, David Miller wrote: > From: Christoph Paasch > Date: Fri, 14 Dec 2018 07:52:26 -0800 > > > I don't think this patch needs to be backported. I haven't hit this panic on > > v4.14.x (although, I only have been using v4.14.79 for now - not yet the > > latest one). And the commits that seem to have introduced the issue haven't > > been backported either AFAICS. That being said, I don't think a backport > > would cause any side-effects. > > > > Or have you seen reports of this panic in the stable branches? > > > > (same for the other stable branches v4.4,... - cfr., the other mails > > regarding this patch here) > > I was being ultra paranoid when I decided to submit this to 4.14 too, > and as Eric says it does not harm. Sounds good to me. I just wanted to point out that I didn't reproduce the issue nor tested the patch on the older stable branches. Christoph
Re: [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all
From: Christoph Paasch Date: Fri, 14 Dec 2018 07:52:26 -0800 > I don't think this patch needs to be backported. I haven't hit this panic on > v4.14.x (although, I only have been using v4.14.79 for now - not yet the > latest one). And the commits that seem to have introduced the issue haven't > been backported either AFAICS. That being said, I don't think a backport > would cause any side-effects. > > Or have you seen reports of this panic in the stable branches? > > (same for the other stable branches v4.4,... - cfr., the other mails > regarding this patch here) I was being ultra paranoid when I decided to submit this to 4.14 too, and as Eric says it does not harm.
Re: [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all
Hello Greg,
On 14/12/18 - 12:59:22, Greg Kroah-Hartman wrote:
> 4.14-stable review patch. If anyone has any objections, please let me know.
>
> --
>
> From: Christoph Paasch
>
> [ Upstream commit 9410d386d0a829ace9558336263086c2fbbe8aed ]
>
> __qdisc_drop_all() accesses skb->prev to get to the tail of the
> segment-list.
>
> With commit 68d2f84a1368 ("net: gro: properly remove skb from list")
> the skb-list handling has been changed to set skb->next to NULL and set
> the list-poison on skb->prev.
>
> With that change, __qdisc_drop_all() will panic when it tries to
> dereference skb->prev.
>
> Since commit 992cba7e276d ("net: Add and use skb_list_del_init().")
> __list_del_entry is used, leaving skb->prev unchanged (thus,
> pointing to the list-head if it's the first skb of the list).
> This will make __qdisc_drop_all modify the next-pointer of the list-head
> and result in a panic later on:
>
> [ 34.501053] general protection fault: [#1] SMP KASAN PTI
> [ 34.501968] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.20.0-rc2.mptcp #108
> [ 34.502887] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> 0.5.1 01/01/2011
> [ 34.504074] RIP: 0010:dev_gro_receive+0x343/0x1f90
> [ 34.504751] Code: e0 48 c1 e8 03 42 80 3c 30 00 0f 85 4a 1c 00 00 4d 8b 24
> 24 4c 39 65 d0 0f 84 0a 04 00 00 49 8d 7c 24 38 48 89 f8 48 c1 e8 03 <42> 0f
> b6 04 30 84 c0 74 08 3c 04
> [ 34.507060] RSP: 0018:8883af507930 EFLAGS: 00010202
> [ 34.507761] RAX: 0007 RBX: 8883970b2c80 RCX:
> 111072e165a6
> [ 34.508640] RDX: 111075867008 RSI: 8883ac338040 RDI:
> 0038
> [ 34.509493] RBP: 8883af5079d0 R08: 8883970b2d40 R09:
> 0062
> [ 34.510346] R10: 0034 R11: R12:
>
> [ 34.511215] R13: R14: dc00 R15:
> 8883ac338008
> [ 34.512082] FS: () GS:8883af50()
> knlGS:
> [ 34.513036] CS: 0010 DS: ES: CR0: 80050033
> [ 34.513741] CR2: 55ccc3e9d020 CR3: 0003abf32000 CR4:
> 06e0
> [ 34.514593] Call Trace:
> [ 34.514893]
> [ 34.515157] napi_gro_receive+0x93/0x150
> [ 34.515632] receive_buf+0x893/0x3700
> [ 34.516094] ? __netif_receive_skb+0x1f/0x1a0
> [ 34.516629] ? virtnet_probe+0x1b40/0x1b40
> [ 34.517153] ? __stable_node_chain+0x4d0/0x850
> [ 34.517684] ? kfree+0x9a/0x180
> [ 34.518067] ? __kasan_slab_free+0x171/0x190
> [ 34.518582] ? detach_buf+0x1df/0x650
> [ 34.519061] ? lapic_next_event+0x5a/0x90
> [ 34.519539] ? virtqueue_get_buf_ctx+0x280/0x7f0
> [ 34.520093] virtnet_poll+0x2df/0xd60
> [ 34.520533] ? receive_buf+0x3700/0x3700
> [ 34.521027] ? qdisc_watchdog_schedule_ns+0xd5/0x140
> [ 34.521631] ? htb_dequeue+0x1817/0x25f0
> [ 34.522107] ? sch_direct_xmit+0x142/0xf30
> [ 34.522595] ? virtqueue_napi_schedule+0x26/0x30
> [ 34.523155] net_rx_action+0x2f6/0xc50
> [ 34.523601] ? napi_complete_done+0x2f0/0x2f0
> [ 34.524126] ? kasan_check_read+0x11/0x20
> [ 34.524608] ? _raw_spin_lock+0x7d/0xd0
> [ 34.525070] ? _raw_spin_lock_bh+0xd0/0xd0
> [ 34.525563] ? kvm_guest_apic_eoi_write+0x6b/0x80
> [ 34.526130] ? apic_ack_irq+0x9e/0xe0
> [ 34.526567] __do_softirq+0x188/0x4b5
> [ 34.527015] irq_exit+0x151/0x180
> [ 34.527417] do_IRQ+0xdb/0x150
> [ 34.527783] common_interrupt+0xf/0xf
> [ 34.528223]
>
> This patch makes sure that skb->prev is set to NULL when entering
> netem_enqueue.
>
> Cc: Prashant Bhole
> Cc: Tyler Hicks
> Cc: Eric Dumazet
> Fixes: 68d2f84a1368 ("net: gro: properly remove skb from list")
> Suggested-by: Eric Dumazet
> Signed-off-by: Christoph Paasch
> Reviewed-by: Eric Dumazet
> Signed-off-by: David S. Miller
> Signed-off-by: Greg Kroah-Hartman
I don't think this patch needs to be backported. I haven't hit this panic on
v4.14.x (although, I only have been using v4.14.79 for now - not yet the
latest one). And the commits that seem to have introduced the issue haven't
been backported either AFAICS. That being said, I don't think a backport
would cause any side-effects.
Or have you seen reports of this panic in the stable branches?
(same for the other stable branches v4.4,... - cfr., the other mails
regarding this patch here)
Cheers,
Christoph
> ---
> net/sched/sch_netem.c |3 +++
> 1 file changed, 3 insertions(+)
>
> --- a/net/sched/sch_netem.c
> +++ b/net/sched/sch_netem.c
> @@ -436,6 +436,9 @@ static int netem_enqueue(struct sk_buff
> int count = 1;
> int rc = NET_XMIT_SUCCESS;
>
> + /* Do not fool qdisc_drop_all() */
> + skb->prev = NULL;
> +
> /* Random duplication */
> if (q->duplicate && q->duplicate >= get_crandom(&q->dup_cor))
> ++count;
>
>
Re: [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all
On 14/12/18 - 07:52:26, Christoph Paasch wrote:
> Hello Greg,
>
> On 14/12/18 - 12:59:22, Greg Kroah-Hartman wrote:
> > 4.14-stable review patch. If anyone has any objections, please let me know.
> >
> > --
> >
> > From: Christoph Paasch
> >
> > [ Upstream commit 9410d386d0a829ace9558336263086c2fbbe8aed ]
> >
> > __qdisc_drop_all() accesses skb->prev to get to the tail of the
> > segment-list.
> >
> > With commit 68d2f84a1368 ("net: gro: properly remove skb from list")
> > the skb-list handling has been changed to set skb->next to NULL and set
> > the list-poison on skb->prev.
> >
> > With that change, __qdisc_drop_all() will panic when it tries to
> > dereference skb->prev.
> >
> > Since commit 992cba7e276d ("net: Add and use skb_list_del_init().")
> > __list_del_entry is used, leaving skb->prev unchanged (thus,
> > pointing to the list-head if it's the first skb of the list).
> > This will make __qdisc_drop_all modify the next-pointer of the list-head
> > and result in a panic later on:
> >
> > [ 34.501053] general protection fault: [#1] SMP KASAN PTI
> > [ 34.501968] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.20.0-rc2.mptcp
> > #108
> > [ 34.502887] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> > 0.5.1 01/01/2011
> > [ 34.504074] RIP: 0010:dev_gro_receive+0x343/0x1f90
> > [ 34.504751] Code: e0 48 c1 e8 03 42 80 3c 30 00 0f 85 4a 1c 00 00 4d 8b
> > 24 24 4c 39 65 d0 0f 84 0a 04 00 00 49 8d 7c 24 38 48 89 f8 48 c1 e8 03
> > <42> 0f b6 04 30 84 c0 74 08 3c 04
> > [ 34.507060] RSP: 0018:8883af507930 EFLAGS: 00010202
> > [ 34.507761] RAX: 0007 RBX: 8883970b2c80 RCX:
> > 111072e165a6
> > [ 34.508640] RDX: 111075867008 RSI: 8883ac338040 RDI:
> > 0038
> > [ 34.509493] RBP: 8883af5079d0 R08: 8883970b2d40 R09:
> > 0062
> > [ 34.510346] R10: 0034 R11: R12:
> >
> > [ 34.511215] R13: R14: dc00 R15:
> > 8883ac338008
> > [ 34.512082] FS: () GS:8883af50()
> > knlGS:
> > [ 34.513036] CS: 0010 DS: ES: CR0: 80050033
> > [ 34.513741] CR2: 55ccc3e9d020 CR3: 0003abf32000 CR4:
> > 06e0
> > [ 34.514593] Call Trace:
> > [ 34.514893]
> > [ 34.515157] napi_gro_receive+0x93/0x150
> > [ 34.515632] receive_buf+0x893/0x3700
> > [ 34.516094] ? __netif_receive_skb+0x1f/0x1a0
> > [ 34.516629] ? virtnet_probe+0x1b40/0x1b40
> > [ 34.517153] ? __stable_node_chain+0x4d0/0x850
> > [ 34.517684] ? kfree+0x9a/0x180
> > [ 34.518067] ? __kasan_slab_free+0x171/0x190
> > [ 34.518582] ? detach_buf+0x1df/0x650
> > [ 34.519061] ? lapic_next_event+0x5a/0x90
> > [ 34.519539] ? virtqueue_get_buf_ctx+0x280/0x7f0
> > [ 34.520093] virtnet_poll+0x2df/0xd60
> > [ 34.520533] ? receive_buf+0x3700/0x3700
> > [ 34.521027] ? qdisc_watchdog_schedule_ns+0xd5/0x140
> > [ 34.521631] ? htb_dequeue+0x1817/0x25f0
> > [ 34.522107] ? sch_direct_xmit+0x142/0xf30
> > [ 34.522595] ? virtqueue_napi_schedule+0x26/0x30
> > [ 34.523155] net_rx_action+0x2f6/0xc50
> > [ 34.523601] ? napi_complete_done+0x2f0/0x2f0
> > [ 34.524126] ? kasan_check_read+0x11/0x20
> > [ 34.524608] ? _raw_spin_lock+0x7d/0xd0
> > [ 34.525070] ? _raw_spin_lock_bh+0xd0/0xd0
> > [ 34.525563] ? kvm_guest_apic_eoi_write+0x6b/0x80
> > [ 34.526130] ? apic_ack_irq+0x9e/0xe0
> > [ 34.526567] __do_softirq+0x188/0x4b5
> > [ 34.527015] irq_exit+0x151/0x180
> > [ 34.527417] do_IRQ+0xdb/0x150
> > [ 34.527783] common_interrupt+0xf/0xf
> > [ 34.528223]
> >
> > This patch makes sure that skb->prev is set to NULL when entering
> > netem_enqueue.
> >
> > Cc: Prashant Bhole
> > Cc: Tyler Hicks
> > Cc: Eric Dumazet
> > Fixes: 68d2f84a1368 ("net: gro: properly remove skb from list")
> > Suggested-by: Eric Dumazet
> > Signed-off-by: Christoph Paasch
> > Reviewed-by: Eric Dumazet
> > Signed-off-by: David S. Miller
> > Signed-off-by: Greg Kroah-Hartman
>
> I don't think this patch needs to be backported. I haven't hit this panic on
> v4.14.x (although, I only have been using v4.14.79 for now - not yet the
> latest one). And the commits that seem to have introduced the issue haven't
> been backported either AFAICS. That being said, I don't think a backport
> would cause any side-effects.
>
> Or have you seen reports of this panic in the stable branches?
>
> (same for the other stable branches v4.4,... - cfr., the other mails
> regarding this patch here)
To clarify: Backport to v4.19.x is needed. The backport to v4.14 and older
should not be needed.
Christoph
>
>
> Cheers,
> Christoph
>
>
>
>
> > ---
> > net/sched/sch_netem.c |3 +++
> > 1 file changed, 3 insertions(+)
> >
> > --- a/net/sched/sch_netem.c
> > +++ b/net/sched/sch_netem.c
> > @@ -436,6 +436,9 @@ static int netem_enqueue(struct sk_buff
> >
Re: [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all
On Fri, Dec 14, 2018 at 7:54 AM Christoph Paasch wrote: > To clarify: Backport to v4.19.x is needed. The backport to v4.14 and older > should not be needed. Sure, but this does not cause any harm. No further action needed :)
