Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, At 06/04/2018 07:17 PM, Thomas Gleixner wrote: On Mon, 4 Jun 2018, Dou Liyang wrote: Here, why didn't we avoid this cleanup by diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index a75de0792942..0cc59646755f 100644 --- a/arch/x86/kernel/apic/vector.c +++

Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, At 06/04/2018 07:17 PM, Thomas Gleixner wrote: On Mon, 4 Jun 2018, Dou Liyang wrote: Here, why didn't we avoid this cleanup by diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index a75de0792942..0cc59646755f 100644 --- a/arch/x86/kernel/apic/vector.c +++

Re: WARNING and PANIC in irq_matrix_free

On Mon, 4 Jun 2018, Dou Liyang wrote: > Here, why didn't we avoid this cleanup by > > diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c > index a75de0792942..0cc59646755f 100644 > --- a/arch/x86/kernel/apic/vector.c > +++ b/arch/x86/kernel/apic/vector.c > @@ -821,6 +821,9

Re: WARNING and PANIC in irq_matrix_free

On Mon, 4 Jun 2018, Dou Liyang wrote: > Here, why didn't we avoid this cleanup by > > diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c > index a75de0792942..0cc59646755f 100644 > --- a/arch/x86/kernel/apic/vector.c > +++ b/arch/x86/kernel/apic/vector.c > @@ -821,6 +821,9

Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, Sorry to ask the questions at this series, my mailbox was kicked out of the mailing list a few days ago, and didn't receive the e-mail. please see below At 05/29/2018 04:09 AM, Thomas Gleixner wrote: On Mon, 28 May 2018, Song Liu wrote: This doesn't fix the issue with bnxt. Here

Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, Sorry to ask the questions at this series, my mailbox was kicked out of the mailing list a few days ago, and didn't receive the e-mail. please see below At 05/29/2018 04:09 AM, Thomas Gleixner wrote: On Mon, 28 May 2018, Song Liu wrote: This doesn't fix the issue with bnxt. Here

Re: WARNING and PANIC in irq_matrix_free

Song, On Tue, 29 May 2018, Song Liu wrote: > > On May 29, 2018, at 1:35 AM, Thomas Gleixner wrote: > >> Maybe we cannot enable all trace points under irq_vectors/ and irq_matrix. > > > > Right. Sorry, I forgot to say that we only need the irq_vectors ones which > > are related to vector

Re: WARNING and PANIC in irq_matrix_free

Song, On Tue, 29 May 2018, Song Liu wrote: > > On May 29, 2018, at 1:35 AM, Thomas Gleixner wrote: > >> Maybe we cannot enable all trace points under irq_vectors/ and irq_matrix. > > > > Right. Sorry, I forgot to say that we only need the irq_vectors ones which > > are related to vector

Re: WARNING and PANIC in irq_matrix_free

> On May 29, 2018, at 1:35 AM, Thomas Gleixner wrote: > > On Mon, 28 May 2018, Song Liu wrote: >>> On May 28, 2018, at 1:09 PM, Thomas Gleixner wrote: >>> >>> On Mon, 28 May 2018, Song Liu wrote: This doesn't fix the issue with bnxt. Here is a trace with this patch: >>> >>> That's

Re: WARNING and PANIC in irq_matrix_free

> On May 29, 2018, at 1:35 AM, Thomas Gleixner wrote: > > On Mon, 28 May 2018, Song Liu wrote: >>> On May 28, 2018, at 1:09 PM, Thomas Gleixner wrote: >>> >>> On Mon, 28 May 2018, Song Liu wrote: This doesn't fix the issue with bnxt. Here is a trace with this patch: >>> >>> That's

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Song Liu wrote: > > On May 28, 2018, at 1:09 PM, Thomas Gleixner wrote: > > > > On Mon, 28 May 2018, Song Liu wrote: > >> This doesn't fix the issue with bnxt. Here is a trace with this patch: > > > > That's just the backtrace which is not really helpful. The real question is

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Song Liu wrote: > > On May 28, 2018, at 1:09 PM, Thomas Gleixner wrote: > > > > On Mon, 28 May 2018, Song Liu wrote: > >> This doesn't fix the issue with bnxt. Here is a trace with this patch: > > > > That's just the backtrace which is not really helpful. The real question is

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Song Liu wrote: > This doesn't fix the issue with bnxt. Here is a trace with this patch: That's just the backtrace which is not really helpful. The real question is what leads to this scenaria. What I was asking for is to enable the irq_vector and irq_matrix trace points

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Song Liu wrote: > This doesn't fix the issue with bnxt. Here is a trace with this patch: That's just the backtrace which is not really helpful. The real question is what leads to this scenaria. What I was asking for is to enable the irq_vector and irq_matrix trace points

Re: WARNING and PANIC in irq_matrix_free

> On May 28, 2018, at 3:53 AM, Thomas Gleixner wrote: > > On Fri, 25 May 2018, Song Liu wrote: >> On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: >>> On Wed, 23 May 2018, Tariq Toukan wrote: I have your patch merged into my internal branch, it prints the following: [

Re: WARNING and PANIC in irq_matrix_free

> On May 28, 2018, at 3:53 AM, Thomas Gleixner wrote: > > On Fri, 25 May 2018, Song Liu wrote: >> On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: >>> On Wed, 23 May 2018, Tariq Toukan wrote: I have your patch merged into my internal branch, it prints the following: [

Re: WARNING and PANIC in irq_matrix_free

> On May 28, 2018, at 7:27 AM, Thomas Gleixner wrote: > > On Mon, 28 May 2018, Tariq Toukan wrote: >> On 28/05/2018 1:53 PM, Thomas Gleixner wrote: >>> On Fri, 25 May 2018, Song Liu wrote: On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: > On Wed, 23 May 2018, Tariq

Re: WARNING and PANIC in irq_matrix_free

> On May 28, 2018, at 7:27 AM, Thomas Gleixner wrote: > > On Mon, 28 May 2018, Tariq Toukan wrote: >> On 28/05/2018 1:53 PM, Thomas Gleixner wrote: >>> On Fri, 25 May 2018, Song Liu wrote: On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: > On Wed, 23 May 2018, Tariq

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Tariq Toukan wrote: > On 28/05/2018 1:53 PM, Thomas Gleixner wrote: > > On Fri, 25 May 2018, Song Liu wrote: > > > On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner > > > wrote: > > > > On Wed, 23 May 2018, Tariq Toukan wrote: > > > > > I have your patch

Re: WARNING and PANIC in irq_matrix_free

On Mon, 28 May 2018, Tariq Toukan wrote: > On 28/05/2018 1:53 PM, Thomas Gleixner wrote: > > On Fri, 25 May 2018, Song Liu wrote: > > > On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner > > > wrote: > > > > On Wed, 23 May 2018, Tariq Toukan wrote: > > > > > I have your patch merged into my

Re: WARNING and PANIC in irq_matrix_free

On Fri, 25 May 2018, Song Liu wrote: > On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: > > On Wed, 23 May 2018, Tariq Toukan wrote: > >> I have your patch merged into my internal branch, it prints the following: > >> > >> [ 4898.226258] Trying to clear prev_vector: 0 >

Re: WARNING and PANIC in irq_matrix_free

On Fri, 25 May 2018, Song Liu wrote: > On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: > > On Wed, 23 May 2018, Tariq Toukan wrote: > >> I have your patch merged into my internal branch, it prints the following: > >> > >> [ 4898.226258] Trying to clear prev_vector: 0 > >> [ 4898.226439]

Re: WARNING and PANIC in irq_matrix_free

On 28/05/2018 1:53 PM, Thomas Gleixner wrote: On Fri, 25 May 2018, Song Liu wrote: On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: On Wed, 23 May 2018, Tariq Toukan wrote: I have your patch merged into my internal branch, it prints the following: [ 4898.226258]

Re: WARNING and PANIC in irq_matrix_free

On 28/05/2018 1:53 PM, Thomas Gleixner wrote: On Fri, 25 May 2018, Song Liu wrote: On Wed, May 23, 2018 at 1:49 AM, Thomas Gleixner wrote: On Wed, 23 May 2018, Tariq Toukan wrote: I have your patch merged into my internal branch, it prints the following: [ 4898.226258] Trying to clear

Re: WARNING and PANIC in irq_matrix_free

Adding Broadcom developers to the thread. Hi Michael, Vasundhara, and Andy, Could you please help look into the case I found? ethtool crashes the system for both net/master and net-next/master. Thanks, Song On Fri, May 25, 2018 at 1:10 PM, Song Liu wrote: > Hi, > > We

Re: WARNING and PANIC in irq_matrix_free

Adding Broadcom developers to the thread. Hi Michael, Vasundhara, and Andy, Could you please help look into the case I found? ethtool crashes the system for both net/master and net-next/master. Thanks, Song On Fri, May 25, 2018 at 1:10 PM, Song Liu wrote: > Hi, > > We are seeing something

Re: WARNING and PANIC in irq_matrix_free

Hi, We are seeing something probably related. We run ethtool on a system with Broadcom NIC to increase number of combined queues. [root@ ~]# ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 9 TX: 8 Other: 0

Re: WARNING and PANIC in irq_matrix_free

Hi, We are seeing something probably related. We run ethtool on a system with Broadcom NIC to increase number of combined queues. [root@ ~]# ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 9 TX: 8 Other: 0

Re: WARNING and PANIC in irq_matrix_free

On Wed, 23 May 2018, Tariq Toukan wrote: > On 19/05/2018 2:20 PM, Thomas Gleixner wrote: > > On Fri, 18 May 2018, Dmitry Safonov wrote: > > > I'm not entirely sure that it's the same fault, but at least backtrace > > > looks resembling. > > > > Yes, it's similar, but not the same issue. I'll

Re: WARNING and PANIC in irq_matrix_free

On Wed, 23 May 2018, Tariq Toukan wrote: > On 19/05/2018 2:20 PM, Thomas Gleixner wrote: > > On Fri, 18 May 2018, Dmitry Safonov wrote: > > > I'm not entirely sure that it's the same fault, but at least backtrace > > > looks resembling. > > > > Yes, it's similar, but not the same issue. I'll

Re: WARNING and PANIC in irq_matrix_free

On 19/05/2018 2:20 PM, Thomas Gleixner wrote: On Fri, 18 May 2018, Dmitry Safonov wrote: I'm not entirely sure that it's the same fault, but at least backtrace looks resembling. Yes, it's similar, but not the same issue. I'll stare are the code ... Thanks, tglx We still see the

Re: WARNING and PANIC in irq_matrix_free

On 19/05/2018 2:20 PM, Thomas Gleixner wrote: On Fri, 18 May 2018, Dmitry Safonov wrote: I'm not entirely sure that it's the same fault, but at least backtrace looks resembling. Yes, it's similar, but not the same issue. I'll stare are the code ... Thanks, tglx We still see the

Re: WARNING and PANIC in irq_matrix_free

On Fri, 18 May 2018, Dmitry Safonov wrote: > I'm not entirely sure that it's the same fault, but at least backtrace > looks resembling. Yes, it's similar, but not the same issue. I'll stare are the code ... Thanks, tglx

Re: WARNING and PANIC in irq_matrix_free

On Fri, 18 May 2018, Dmitry Safonov wrote: > I'm not entirely sure that it's the same fault, but at least backtrace > looks resembling. Yes, it's similar, but not the same issue. I'll stare are the code ... Thanks, tglx

Re: WARNING and PANIC in irq_matrix_free

2018-05-18 23:41 GMT+01:00 Dmitry Safonov <0x7f454...@gmail.com>: > What I did - was running x86 selftests and ssh'ing at the same moment. > Will try it again if it'll fire by any chance. No, I've tried a couple of times (after reboot as it's ONCE), but it looks hard to reproduce by hands without

Re: WARNING and PANIC in irq_matrix_free

2018-05-18 23:41 GMT+01:00 Dmitry Safonov <0x7f454...@gmail.com>: > What I did - was running x86 selftests and ssh'ing at the same moment. > Will try it again if it'll fire by any chance. No, I've tried a couple of times (after reboot as it's ONCE), but it looks hard to reproduce by hands without

Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, Tariq, 2018-02-20 18:11 GMT+00:00 Thomas Gleixner : > On Tue, 20 Feb 2018, Tariq Toukan wrote: > >> Hi Thomas, >> >> We started seeing new issues in our net-device daily regression tests. >> They are related to patch [1] introduced in kernel 4.15-rc1. >> >> We

Re: WARNING and PANIC in irq_matrix_free

Hi Thomas, Tariq, 2018-02-20 18:11 GMT+00:00 Thomas Gleixner : > On Tue, 20 Feb 2018, Tariq Toukan wrote: > >> Hi Thomas, >> >> We started seeing new issues in our net-device daily regression tests. >> They are related to patch [1] introduced in kernel 4.15-rc1. >> >> We frequently see a warning

Re: WARNING and PANIC in irq_matrix_free

On 22/02/2018 11:38 PM, Thomas Gleixner wrote: On Wed, 21 Feb 2018, Tariq Toukan wrote: On 20/02/2018 8:18 PM, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Tariq Toukan wrote: Is there CPU hotplugging in play? No. Ok. I'll come back to you

Re: WARNING and PANIC in irq_matrix_free

On 22/02/2018 11:38 PM, Thomas Gleixner wrote: On Wed, 21 Feb 2018, Tariq Toukan wrote: On 20/02/2018 8:18 PM, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Tariq Toukan wrote: Is there CPU hotplugging in play? No. Ok. I'll come back to you

Re: WARNING and PANIC in irq_matrix_free

On Wed, 21 Feb 2018, Tariq Toukan wrote: > On 20/02/2018 8:18 PM, Thomas Gleixner wrote: > > On Tue, 20 Feb 2018, Thomas Gleixner wrote: > > > On Tue, 20 Feb 2018, Tariq Toukan wrote: > > > > > > Is there CPU hotplugging in play? > > No. Ok. > > > > > > I'll come back to you tomorrow with a

Re: WARNING and PANIC in irq_matrix_free

On Wed, 21 Feb 2018, Tariq Toukan wrote: > On 20/02/2018 8:18 PM, Thomas Gleixner wrote: > > On Tue, 20 Feb 2018, Thomas Gleixner wrote: > > > On Tue, 20 Feb 2018, Tariq Toukan wrote: > > > > > > Is there CPU hotplugging in play? > > No. Ok. > > > > > > I'll come back to you tomorrow with a

Re: WARNING and PANIC in irq_matrix_free

On 20/02/2018 8:18 PM, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Tariq Toukan wrote: Is there CPU hotplugging in play? No. I'll come back to you tomorrow with a plan how to debug that after staring into the code some more. Do you have a

Re: WARNING and PANIC in irq_matrix_free

On 20/02/2018 8:18 PM, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Thomas Gleixner wrote: On Tue, 20 Feb 2018, Tariq Toukan wrote: Is there CPU hotplugging in play? No. I'll come back to you tomorrow with a plan how to debug that after staring into the code some more. Do you have a

Re: WARNING and PANIC in irq_matrix_free

On Tue, 20 Feb 2018, Thomas Gleixner wrote: > On Tue, 20 Feb 2018, Tariq Toukan wrote: > > Is there CPU hotplugging in play? > > I'll come back to you tomorrow with a plan how to debug that after staring > into the code some more. Do you have a rough idea what the test case is doing? Thanks,

Re: WARNING and PANIC in irq_matrix_free

On Tue, 20 Feb 2018, Thomas Gleixner wrote: > On Tue, 20 Feb 2018, Tariq Toukan wrote: > > Is there CPU hotplugging in play? > > I'll come back to you tomorrow with a plan how to debug that after staring > into the code some more. Do you have a rough idea what the test case is doing? Thanks,

Re: WARNING and PANIC in irq_matrix_free

On Tue, 20 Feb 2018, Tariq Toukan wrote: > Hi Thomas, > > We started seeing new issues in our net-device daily regression tests. > They are related to patch [1] introduced in kernel 4.15-rc1. > > We frequently see a warning in dmesg [2]. Repro is not consistent, we tried to > narrow it down to

Re: WARNING and PANIC in irq_matrix_free

On Tue, 20 Feb 2018, Tariq Toukan wrote: > Hi Thomas, > > We started seeing new issues in our net-device daily regression tests. > They are related to patch [1] introduced in kernel 4.15-rc1. > > We frequently see a warning in dmesg [2]. Repro is not consistent, we tried to > narrow it down to

WARNING and PANIC in irq_matrix_free

Hi Thomas, We started seeing new issues in our net-device daily regression tests. They are related to patch [1] introduced in kernel 4.15-rc1. We frequently see a warning in dmesg [2]. Repro is not consistent, we tried to narrow it down to a smaller run but couldn't. In addition, sometimes

WARNING and PANIC in irq_matrix_free

Hi Thomas, We started seeing new issues in our net-device daily regression tests. They are related to patch [1] introduced in kernel 4.15-rc1. We frequently see a warning in dmesg [2]. Repro is not consistent, we tried to narrow it down to a smaller run but couldn't. In addition, sometimes