Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, Jan 22, 2016 at 8:46 AM, Christoph Lameter wrote: > > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if it was already set before. [..] Ok, I am assuming this is in Andrew's queue already, but this bug

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, Jan 22, 2016 at 8:46 AM, Christoph Lameter wrote: > > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if it was already set before. [..] Ok, I am assuming this is in Andrew's queue

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri 22-01-16 10:46:14, Christoph Lameter wrote: > On Fri, 22 Jan 2016, Michal Hocko wrote: > > > Could you repost the patch with the updated description? > > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 22 Jan 2016, Michal Hocko wrote: > Could you repost the patch with the updated description? Subject: vmstat: Remove BUG_ON from vmstat_update If we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri 22-01-16 10:07:01, Christoph Lameter wrote: > On Fri, 22 Jan 2016, Michal Hocko wrote: > > > Wouldn't it be much more easier and simply get rid of the VM_BUG_ON? > > What is the point of keeping it in the first place. The code can > > perfectly cope with the race. > > Ok then lets do

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 22 Jan 2016, Michal Hocko wrote: > Wouldn't it be much more easier and simply get rid of the VM_BUG_ON? > What is the point of keeping it in the first place. The code can > perfectly cope with the race. Ok then lets do that.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu 21-01-16 11:38:46, Christoph Lameter wrote: > On Thu, 21 Jan 2016, Michal Hocko wrote: > > > It goes like this: > > CPU0: CPU1 > > vmstat_update > > cpumask_test_and_set_cpu (0->1) > > [...] > >

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, Jan 21, 2016 at 11:08 PM, Christoph Lameter wrote: > Subject: vmstat: Queue work before clearing cpu_stat_off > > There is a race between vmstat_shepherd and quiet_vmstat() because > the responsibility for checking for counter updates changes depending > on the state of teh bit in

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, Jan 21, 2016 at 11:08 PM, Christoph Lameter wrote: > Subject: vmstat: Queue work before clearing cpu_stat_off > > There is a race between vmstat_shepherd and quiet_vmstat() because > the responsibility for checking for counter updates changes depending > on the state of

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 22 Jan 2016, Michal Hocko wrote: > Wouldn't it be much more easier and simply get rid of the VM_BUG_ON? > What is the point of keeping it in the first place. The code can > perfectly cope with the race. Ok then lets do that.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri 22-01-16 10:46:14, Christoph Lameter wrote: > On Fri, 22 Jan 2016, Michal Hocko wrote: > > > Could you repost the patch with the updated description? > > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu 21-01-16 11:38:46, Christoph Lameter wrote: > On Thu, 21 Jan 2016, Michal Hocko wrote: > > > It goes like this: > > CPU0: CPU1 > > vmstat_update > > cpumask_test_and_set_cpu (0->1) > > [...] > >

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 22 Jan 2016, Michal Hocko wrote: > Could you repost the patch with the updated description? Subject: vmstat: Remove BUG_ON from vmstat_update If we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri 22-01-16 10:07:01, Christoph Lameter wrote: > On Fri, 22 Jan 2016, Michal Hocko wrote: > > > Wouldn't it be much more easier and simply get rid of the VM_BUG_ON? > > What is the point of keeping it in the first place. The code can > > perfectly cope with the race. > > Ok then lets do

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 21 Jan 2016, Michal Hocko wrote: > It goes like this: > CPU0: CPU1 > vmstat_update > cpumask_test_and_set_cpu (0->1) > [...] > vmstat_shepherd > >

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu 21-01-16 09:45:12, Christoph Lameter wrote: > On Thu, 21 Jan 2016, Michal Hocko wrote: [...] > > The vmstat update might be still waiting for its timer, idle mode started > > and kick vmstat_update which might cpumask_test_and_set_cpu. Once the > > idle terminates and the originally

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 21 Jan 2016, Michal Hocko wrote: > > > Since 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and > > > shut down on idle") quiet_vmstat might update cpu_stat_off and mark a > > > particular cpu to be handled by vmstat_shepherd. This might trigger > > > a VM_BUG_ON in

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 15:57:43, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > > > On Wed 20-01-16 09:55:22, Christoph Lameter wrote: > > [...] > > > Subject: vmstat: Remove BUG_ON from vmstat_update > > > > > > If we detect that there is nothing to do just set the flag and do

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 15:57:43, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > > > On Wed 20-01-16 09:55:22, Christoph Lameter wrote: > > [...] > > > Subject: vmstat: Remove BUG_ON from vmstat_update > > > > > > If we detect that there is nothing to do just set the flag and do

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 21 Jan 2016, Michal Hocko wrote: > > > Since 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and > > > shut down on idle") quiet_vmstat might update cpu_stat_off and mark a > > > particular cpu to be handled by vmstat_shepherd. This might trigger > > > a VM_BUG_ON in

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu 21-01-16 09:45:12, Christoph Lameter wrote: > On Thu, 21 Jan 2016, Michal Hocko wrote: [...] > > The vmstat update might be still waiting for its timer, idle mode started > > and kick vmstat_update which might cpumask_test_and_set_cpu. Once the > > idle terminates and the originally

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 21 Jan 2016, Michal Hocko wrote: > It goes like this: > CPU0: CPU1 > vmstat_update > cpumask_test_and_set_cpu (0->1) > [...] > vmstat_shepherd > >

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > On Wed 20-01-16 09:55:22, Christoph Lameter wrote: > [...] > > Subject: vmstat: Remove BUG_ON from vmstat_update > > > > If we detect that there is nothing to do just set the flag and do not check > > if it was already set before. Races really do not

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:55:22, Christoph Lameter wrote: [...] > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if it was already set before. Races really do not matter. If the flag is > set by any code then the

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Sasha Levin wrote: > > As I've mentioned - this reproduces frequently. I'd be happy to add in debug > information into the kernel that might help you reproduce it, but as it seems > like a timing issue, I can't provide a simple reproducer. This isnt really important I think.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/20/2016 10:20 AM, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > >> > On Wed 20-01-16 09:56:26, Sasha Levin wrote: >>> > > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > > I am just reading through this old discussion again because "vmstat: > > > make

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > On Wed 20-01-16 09:56:26, Sasha Levin wrote: > > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > > I am just reading through this old discussion again because "vmstat: > > > make vmstat_updater deferrable again and shut down on idle" which seems > > > to

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:14:06, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > > > [CCing Andrew] > > > > I am just reading through this old discussion again because "vmstat: > > make vmstat_updater deferrable again and shut down on idle" which seems > > to be the culprit AFAIU

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > [CCing Andrew] > > I am just reading through this old discussion again because "vmstat: > make vmstat_updater deferrable again and shut down on idle" which seems > to be the culprit AFAIU has been merged as 0eb77e988032 and I do not see > any follow up

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:56:26, Sasha Levin wrote: > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > I am just reading through this old discussion again because "vmstat: > > make vmstat_updater deferrable again and shut down on idle" which seems > > to be the culprit AFAIU has been merged as

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/20/2016 09:37 AM, Michal Hocko wrote: > I am just reading through this old discussion again because "vmstat: > make vmstat_updater deferrable again and shut down on idle" which seems > to be the culprit AFAIU has been merged as 0eb77e988032 and I do not see > any follow up fix merged to

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/20/2016 09:37 AM, Michal Hocko wrote: > I am just reading through this old discussion again because "vmstat: > make vmstat_updater deferrable again and shut down on idle" which seems > to be the culprit AFAIU has been merged as 0eb77e988032 and I do not see > any follow up fix merged to

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/20/2016 10:20 AM, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > >> > On Wed 20-01-16 09:56:26, Sasha Levin wrote: >>> > > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > > I am just reading through this old discussion again because "vmstat: > > > make

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:14:06, Christoph Lameter wrote: > On Wed, 20 Jan 2016, Michal Hocko wrote: > > > [CCing Andrew] > > > > I am just reading through this old discussion again because "vmstat: > > make vmstat_updater deferrable again and shut down on idle" which seems > > to be the culprit AFAIU

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > On Wed 20-01-16 09:56:26, Sasha Levin wrote: > > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > > I am just reading through this old discussion again because "vmstat: > > > make vmstat_updater deferrable again and shut down on idle" which seems > > > to

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Sasha Levin wrote: > > As I've mentioned - this reproduces frequently. I'd be happy to add in debug > information into the kernel that might help you reproduce it, but as it seems > like a timing issue, I can't provide a simple reproducer. This isnt really important I think.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:56:26, Sasha Levin wrote: > On 01/20/2016 09:37 AM, Michal Hocko wrote: > > I am just reading through this old discussion again because "vmstat: > > make vmstat_updater deferrable again and shut down on idle" which seems > > to be the culprit AFAIU has been merged as

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed 20-01-16 09:55:22, Christoph Lameter wrote: [...] > Subject: vmstat: Remove BUG_ON from vmstat_update > > If we detect that there is nothing to do just set the flag and do not check > if it was already set before. Races really do not matter. If the flag is > set by any code then the

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > [CCing Andrew] > > I am just reading through this old discussion again because "vmstat: > make vmstat_updater deferrable again and shut down on idle" which seems > to be the culprit AFAIU has been merged as 0eb77e988032 and I do not see > any follow up

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Wed, 20 Jan 2016, Michal Hocko wrote: > On Wed 20-01-16 09:55:22, Christoph Lameter wrote: > [...] > > Subject: vmstat: Remove BUG_ON from vmstat_update > > > > If we detect that there is nothing to do just set the flag and do not check > > if it was already set before. Races really do not

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/04/2016 01:05 PM, Christoph Lameter wrote: > On Thu, 24 Dec 2015, Sasha Levin wrote: > >>> Also what workload triggers the BUG()? >> >> Fuzzing with trinity inside a KVM guest. I've attached my config. > > Ok build and bootup works fine after fix from Tetsuo to config. Does not > like my

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 24 Dec 2015, Sasha Levin wrote: > > Also what workload triggers the BUG()? > > Fuzzing with trinity inside a KVM guest. I've attached my config. Ok build and bootup works fine after fix from Tetsuo to config. Does not like my initrd it seems. Is there a root with the tools available

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 01/04/2016 01:05 PM, Christoph Lameter wrote: > On Thu, 24 Dec 2015, Sasha Levin wrote: > >>> Also what workload triggers the BUG()? >> >> Fuzzing with trinity inside a KVM guest. I've attached my config. > > Ok build and bootup works fine after fix from Tetsuo to config. Does not > like my

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 24 Dec 2015, Sasha Levin wrote: > > Also what workload triggers the BUG()? > > Fuzzing with trinity inside a KVM guest. I've attached my config. Ok build and bootup works fine after fix from Tetsuo to config. Does not like my initrd it seems. Is there a root with the tools available

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Builds with your kernel config fail with CHK include/generated/bounds.h CHK include/generated/asm-offsets.h CALLscripts/checksyscalls.sh CHK include/generated/compile.h make[1]: *** No rule to make target 'signing_key.pem', needed by 'certs/signing_key.x509'. Stop.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 24 Dec 2015, Sasha Levin wrote: > >> > [ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 > >> > [ 3462.554836] pending: vmstat_update > > Does that mean that vmstat_update locks up or something that schedules it? > > I think it means that vmstat_update didn't finish

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Thu, 24 Dec 2015, Sasha Levin wrote: > >> > [ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 > >> > [ 3462.554836] pending: vmstat_update > > Does that mean that vmstat_update locks up or something that schedules it? > > I think it means that vmstat_update didn't finish

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Builds with your kernel config fail with CHK include/generated/bounds.h CHK include/generated/asm-offsets.h CALLscripts/checksyscalls.sh CHK include/generated/compile.h make[1]: *** No rule to make target 'signing_key.pem', needed by 'certs/signing_key.x509'. Stop.

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 04:14 PM, Christoph Lameter wrote: > On Mon, 21 Dec 2015, Sasha Levin wrote: > >> > I've also noticed a new warning from the workqueue code which my scripts >> > didn't pick up before: >> > >> > [ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 >> > stuck

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 04:14 PM, Christoph Lameter wrote: > On Mon, 21 Dec 2015, Sasha Levin wrote: > >> > I've also noticed a new warning from the workqueue code which my scripts >> > didn't pick up before: >> > >> > [ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 >> > stuck

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Ran this here but no errors. Need config etc to reproduce. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Ran this here but no errors. Need config etc to reproduce. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Mon, 21 Dec 2015, Sasha Levin wrote: > I've also noticed a new warning from the workqueue code which my scripts > didn't pick up before: > > [ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 > stuck for 54s! > [ 3462.522041] workqueue vmstat: flags=0xc > [

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 03:28 PM, Sasha Levin wrote: > On 12/21/2015 08:08 AM, Christoph Lameter wrote: >> > On Fri, 18 Dec 2015, Sasha Levin wrote: >> > >> > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) >> > Hmmm.. Yes we need to fold the diffs first before disabling the timer >> > otherwise

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 08:08 AM, Christoph Lameter wrote: > On Fri, 18 Dec 2015, Sasha Levin wrote: > >> > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) > Hmmm.. Yes we need to fold the diffs first before disabling the timer > otherwise the shepherd task may intervene. > > Does this patch fix it?

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 18 Dec 2015, Sasha Levin wrote: > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) Hmmm.. Yes we need to fold the diffs first before disabling the timer otherwise the shepherd task may intervene. Does this patch fix it? Subject: quiet_vmstat: Avoid race with shepherd by folding

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 03:28 PM, Sasha Levin wrote: > On 12/21/2015 08:08 AM, Christoph Lameter wrote: >> > On Fri, 18 Dec 2015, Sasha Levin wrote: >> > >> > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) >> > Hmmm.. Yes we need to fold the diffs first before disabling the timer >> > otherwise

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On 12/21/2015 08:08 AM, Christoph Lameter wrote: > On Fri, 18 Dec 2015, Sasha Levin wrote: > >> > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) > Hmmm.. Yes we need to fold the diffs first before disabling the timer > otherwise the shepherd task may intervene. > > Does this patch fix it?

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Mon, 21 Dec 2015, Sasha Levin wrote: > I've also noticed a new warning from the workqueue code which my scripts > didn't pick up before: > > [ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 > stuck for 54s! > [ 3462.522041] workqueue vmstat: flags=0xc > [

Re: mm, vmstat: kernel BUG at mm/vmstat.c:1408!

On Fri, 18 Dec 2015, Sasha Levin wrote: > [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) Hmmm.. Yes we need to fold the diffs first before disabling the timer otherwise the shepherd task may intervene. Does this patch fix it? Subject: quiet_vmstat: Avoid race with shepherd by folding

mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Hi all, I've started seeing the following in the latest -next kernel. [ 531.127489] kernel BUG at mm/vmstat.c:1408! [ 531.128157] invalid opcode: [#1] PREEMPT SMP KASAN [ 531.128872] Modules linked in: [ 531.129324] CPU: 6 PID: 407 Comm: kworker/6:1 Not tainted

mm, vmstat: kernel BUG at mm/vmstat.c:1408!

Hi all, I've started seeing the following in the latest -next kernel. [ 531.127489] kernel BUG at mm/vmstat.c:1408! [ 531.128157] invalid opcode: [#1] PREEMPT SMP KASAN [ 531.128872] Modules linked in: [ 531.129324] CPU: 6 PID: 407 Comm: kworker/6:1 Not tainted