Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hi, Move printk and (some of) MM people to the recipients list. On (01/10/18 09:02), Tejun Heo wrote: [..] > The particular case that we've been seeing regularly in the fleet was > the following scenario. > > 1. Console is IPMI emulated serial console. Super slow. Also >netconsole is in

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hi, Move printk and (some of) MM people to the recipients list. On (01/10/18 09:02), Tejun Heo wrote: [..] > The particular case that we've been seeing regularly in the fleet was > the following scenario. > > 1. Console is IPMI emulated serial console. Super slow. Also >netconsole is in

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 07:43), Tejun Heo wrote: > > > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > > pain. Have it be 100 printks, it needs to be fixed if it is happening. > > And having all printks just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 07:43), Tejun Heo wrote: > > > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > > pain. Have it be 100 printks, it needs to be fixed if it is happening. > > And having all printks just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Peter. On Wed, Jan 24, 2018 at 10:36:07AM +0100, Peter Zijlstra wrote: > On Wed, Jan 10, 2018 at 09:02:23AM -0800, Tejun Heo wrote: > > 1. Console is IPMI emulated serial console. Super slow. Also > >netconsole is in use. > > So my IPMI SoE typically run at 115200 Baud (or higher)

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Peter. On Wed, Jan 24, 2018 at 10:36:07AM +0100, Peter Zijlstra wrote: > On Wed, Jan 10, 2018 at 09:02:23AM -0800, Tejun Heo wrote: > > 1. Console is IPMI emulated serial console. Super slow. Also > >netconsole is in use. > > So my IPMI SoE typically run at 115200 Baud (or higher)

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, Jan 10, 2018 at 09:02:23AM -0800, Tejun Heo wrote: > 1. Console is IPMI emulated serial console. Super slow. Also >netconsole is in use. So my IPMI SoE typically run at 115200 Baud (or higher) and I've not had trouble like that (granted I don't typically trigger OOM storms, but they

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, Jan 10, 2018 at 09:02:23AM -0800, Tejun Heo wrote: > 1. Console is IPMI emulated serial console. Super slow. Also >netconsole is in use. So my IPMI SoE typically run at 115200 Baud (or higher) and I've not had trouble like that (granted I don't typically trigger OOM storms, but they

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 21:52), Steven Rostedt wrote: > On Wed, 24 Jan 2018 11:11:33 +0900 > Sergey Senozhatsky wrote: > > > Please take a look. > > Was there something specific to look at? Not really. Just my previous email, basically. You said "I have to look at the

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 21:52), Steven Rostedt wrote: > On Wed, 24 Jan 2018 11:11:33 +0900 > Sergey Senozhatsky wrote: > > > Please take a look. > > Was there something specific to look at? Not really. Just my previous email, basically. You said "I have to look at the latest code." so I replied. Well,

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 11:11:33 +0900 Sergey Senozhatsky wrote: > Please take a look. Was there something specific to look at? I'm doing a hundred different things at once, and my memory cache keeps getting flushed. -- Steve

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 11:11:33 +0900 Sergey Senozhatsky wrote: > Please take a look. Was there something specific to look at? I'm doing a hundred different things at once, and my memory cache keeps getting flushed. -- Steve

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On (01/23/18 11:24), Steven Rostedt wrote: [..] > > With WQ we don't lockup the kernel, because we flush printk_safe in > > preemptible context. And people are very much expected to fix the > > misbehaving consoles. But that should not be printk_safe problem. > > Right, but now you just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On (01/23/18 11:24), Steven Rostedt wrote: [..] > > With WQ we don't lockup the kernel, because we flush printk_safe in > > preemptible context. And people are very much expected to fix the > > misbehaving consoles. But that should not be printk_safe problem. > > Right, but now you just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Sergey. On Wed, Jan 24, 2018 at 01:01:53AM +0900, Sergey Senozhatsky wrote: > On (01/23/18 10:41), Steven Rostedt wrote: > [..] > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > > pain.

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Sergey. On Wed, Jan 24, 2018 at 01:01:53AM +0900, Sergey Senozhatsky wrote: > On (01/23/18 10:41), Steven Rostedt wrote: > [..] > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > > pain.

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hey, On Tue, Jan 23, 2018 at 11:13:30AM -0500, Steven Rostedt wrote: > From what I understand is that there's an issue with one of the printk > consoles, due to memory pressure or whatnot. Then a printk happens > within a printk recursively. It gets put into the safe buffer and an > irq is sent

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hey, On Tue, Jan 23, 2018 at 11:13:30AM -0500, Steven Rostedt wrote: > From what I understand is that there's an issue with one of the printk > consoles, due to memory pressure or whatnot. Then a printk happens > within a printk recursively. It gets put into the safe buffer and an > irq is sent

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 01:01:53 +0900 Sergey Senozhatsky wrote: > On (01/23/18 10:41), Steven Rostedt wrote: > [..] > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 01:01:53 +0900 Sergey Senozhatsky wrote: > On (01/23/18 10:41), Steven Rostedt wrote: > [..] > > We can have more. But if printk is causing printks, that's a major bug. > > And work queues are not going to fix it, it will just spread out the > > pain. Have it be 100 printks,

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 23 Jan 2018 07:43:47 -0800 Tejun Heo wrote: > So, at least in the case that we were seeing, it isn't that black and > white. printk keeps causing printks but only because printk buffer > flushing is preventing the printk'ing context from making forward > progress. The

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 23 Jan 2018 07:43:47 -0800 Tejun Heo wrote: > So, at least in the case that we were seeing, it isn't that black and > white. printk keeps causing printks but only because printk buffer > flushing is preventing the printk'ing context from making forward > progress. The key problem there

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Tejun On (01/23/18 07:43), Tejun Heo wrote: > Hello, Steven. > > On Tue, Jan 23, 2018 at 10:41:21AM -0500, Steven Rostedt wrote: > > > I don't want to have heuristics in print_safe, I don't want to have a > > > magic > > > number controlled by a user-space visible knob, I don't want to

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Tejun On (01/23/18 07:43), Tejun Heo wrote: > Hello, Steven. > > On Tue, Jan 23, 2018 at 10:41:21AM -0500, Steven Rostedt wrote: > > > I don't want to have heuristics in print_safe, I don't want to have a > > > magic > > > number controlled by a user-space visible knob, I don't want to

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 10:41), Steven Rostedt wrote: [..] > We can have more. But if printk is causing printks, that's a major bug. > And work queues are not going to fix it, it will just spread out the > pain. Have it be 100 printks, it needs to be fixed if it is happening. > And having all printks just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 10:41), Steven Rostedt wrote: [..] > We can have more. But if printk is causing printks, that's a major bug. > And work queues are not going to fix it, it will just spread out the > pain. Have it be 100 printks, it needs to be fixed if it is happening. > And having all printks just

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Tue, Jan 23, 2018 at 10:41:21AM -0500, Steven Rostedt wrote: > > I don't want to have heuristics in print_safe, I don't want to have a magic > > number controlled by a user-space visible knob, I don't want to have the > > first 3 lines of a lockdep splat. > > We can have more.

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Tue, Jan 23, 2018 at 10:41:21AM -0500, Steven Rostedt wrote: > > I don't want to have heuristics in print_safe, I don't want to have a magic > > number controlled by a user-space visible knob, I don't want to have the > > first 3 lines of a lockdep splat. > > We can have more.

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 00:21:30 +0900 Sergey Senozhatsky wrote: > On (01/23/18 09:56), Steven Rostedt wrote: > [..] > > > Why do we even use irq_work for printk_safe? > > > > Why not? > > > > Really, I think you are trying to solve a symptom and not the problem. >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 24 Jan 2018 00:21:30 +0900 Sergey Senozhatsky wrote: > On (01/23/18 09:56), Steven Rostedt wrote: > [..] > > > Why do we even use irq_work for printk_safe? > > > > Why not? > > > > Really, I think you are trying to solve a symptom and not the problem. > > If we are having issues with

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 09:56), Steven Rostedt wrote: [..] > > Why do we even use irq_work for printk_safe? > > Why not? > > Really, I think you are trying to solve a symptom and not the problem. > If we are having issues with irq_work, we are going to have issues with > a work queue. It's just spreading

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 09:56), Steven Rostedt wrote: [..] > > Why do we even use irq_work for printk_safe? > > Why not? > > Really, I think you are trying to solve a symptom and not the problem. > If we are having issues with irq_work, we are going to have issues with > a work queue. It's just spreading

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 23 Jan 2018 15:40:23 +0900 Sergey Senozhatsky wrote: > Why do we even use irq_work for printk_safe? Why not? Really, I think you are trying to solve a symptom and not the problem. If we are having issues with irq_work, we are going to have issues with

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 23 Jan 2018 15:40:23 +0900 Sergey Senozhatsky wrote: > Why do we even use irq_work for printk_safe? Why not? Really, I think you are trying to solve a symptom and not the problem. If we are having issues with irq_work, we are going to have issues with a work queue. It's just spreading

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 15:40), Sergey Senozhatsky wrote: > > Why do we even use irq_work for printk_safe? > ... perhaps because of wq: pool->lock -> printk -> call_console_drivers -> printk -> vprintk_safe -> wq: pool->lock Which is a "many things have gone wrong" type of scenario. Maybe we can

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 15:40), Sergey Senozhatsky wrote: > > Why do we even use irq_work for printk_safe? > ... perhaps because of wq: pool->lock -> printk -> call_console_drivers -> printk -> vprintk_safe -> wq: pool->lock Which is a "many things have gone wrong" type of scenario. Maybe we can

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 15:40), Sergey Senozhatsky wrote: [..] > Why do we even use irq_work for printk_safe? > > Okay... So, how about this. For printk_safe we use system_wq for flushing. > IOW, we flush from a task running exactly on the same CPU which hit printk > recursion, not from IRQ. From

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/23/18 15:40), Sergey Senozhatsky wrote: [..] > Why do we even use irq_work for printk_safe? > > Okay... So, how about this. For printk_safe we use system_wq for flushing. > IOW, we flush from a task running exactly on the same CPU which hit printk > recursion, not from IRQ. From

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On (01/21/18 23:15), Sergey Senozhatsky wrote: [..] > we have printk recursion from console drivers. it's redirected to > printk_safe and we queue an IRQ work to flush the buffer > > printk > console_unlock >call_console_drivers > net_console > printk > printk_save

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On (01/21/18 23:15), Sergey Senozhatsky wrote: [..] > we have printk recursion from console drivers. it's redirected to > printk_safe and we queue an IRQ work to flush the buffer > > printk > console_unlock >call_console_drivers > net_console > printk > printk_save

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/22/18 19:28), Sergey Senozhatsky wrote: > On (01/22/18 17:56), Sergey Senozhatsky wrote: > [..] > > Assume the following, > > But more importantly we are missing another huge thing - console_unlock(). IOW, not every console_unlock() is from vprintk_emit(). We can have console_trylock() ->

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/22/18 19:28), Sergey Senozhatsky wrote: > On (01/22/18 17:56), Sergey Senozhatsky wrote: > [..] > > Assume the following, > > But more importantly we are missing another huge thing - console_unlock(). IOW, not every console_unlock() is from vprintk_emit(). We can have console_trylock() ->

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/22/18 17:56), Sergey Senozhatsky wrote: [..] > Assume the following, But more importantly we are missing another huge thing - console_unlock(). Suppose: console_lock(); << preemption >> printk

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/22/18 17:56), Sergey Senozhatsky wrote: [..] > Assume the following, But more importantly we are missing another huge thing - console_unlock(). Suppose: console_lock(); << preemption >> printk

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/21/18 16:04), Steven Rostedt wrote: [..] > > The problem is that we flush printk_safe right when console_unlock() > > printing > > loop enables local IRQs via printk_safe_exit_irqrestore() [given that IRQs > > were enabled in the first place when the CPU went to console_unlock()]. > > This

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/21/18 16:04), Steven Rostedt wrote: [..] > > The problem is that we flush printk_safe right when console_unlock() > > printing > > loop enables local IRQs via printk_safe_exit_irqrestore() [given that IRQs > > were enabled in the first place when the CPU went to console_unlock()]. > > This

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sun, 21 Jan 2018 23:15:21 +0900 Sergey Senozhatsky wrote: > so fix the console drivers ;) Totally agree! > > > > > just kidding. ok... Darn it! ;-) > the problem is that we flush printk_safe right when console_unlock() printing > loop enables local

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sun, 21 Jan 2018 23:15:21 +0900 Sergey Senozhatsky wrote: > so fix the console drivers ;) Totally agree! > > > > > just kidding. ok... Darn it! ;-) > the problem is that we flush printk_safe right when console_unlock() printing > loop enables local IRQs via

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/20/18 10:49), Steven Rostedt wrote: [..] > > printks from console_unlock()->call_console_drivers() are redirected > > to printk_safe buffer. we need irq_work on that CPU to flush its > > printk_safe buffer. > > So is the issue that we keep triggering this irq work then? Then this >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/20/18 10:49), Steven Rostedt wrote: [..] > > printks from console_unlock()->call_console_drivers() are redirected > > to printk_safe buffer. we need irq_work on that CPU to flush its > > printk_safe buffer. > > So is the issue that we keep triggering this irq work then? Then this >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sat, 20 Jan 2018 16:14:02 +0900 Sergey Senozhatsky wrote: > [..] > > asmlinkage int vprintk_emit(int facility, int level, > > const char *dict, size_t dictlen, > > @@ -1849,6 +1918,17 @@ asmlinkage int vprintk_emit(int facility, int

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sat, 20 Jan 2018 16:14:02 +0900 Sergey Senozhatsky wrote: > [..] > > asmlinkage int vprintk_emit(int facility, int level, > > const char *dict, size_t dictlen, > > @@ -1849,6 +1918,17 @@ asmlinkage int vprintk_emit(int facility, int level, > > > > /* This stops

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sat, 20 Jan 2018 04:19:53 -0800 Tejun Heo wrote: > I'm a bit worried tho because this essentially seems like "detect > recursion, ignore messages" approach. netcons can have a very large > surface for bugs. Suppressing those messages would make them > difficult to debug.

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Sat, 20 Jan 2018 04:19:53 -0800 Tejun Heo wrote: > I'm a bit worried tho because this essentially seems like "detect > recursion, ignore messages" approach. netcons can have a very large > surface for bugs. Suppressing those messages would make them > difficult to debug. For example, all

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Fri, Jan 19, 2018 at 01:20:52PM -0500, Steven Rostedt wrote: > I was thinking about this a bit more, and instead of offloading a > recursive printk, perhaps its best to simply throttle it. Because the > problem may not go away if a printk thread takes over, because the bug > is

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Fri, Jan 19, 2018 at 01:20:52PM -0500, Steven Rostedt wrote: > I was thinking about this a bit more, and instead of offloading a > recursive printk, perhaps its best to simply throttle it. Because the > problem may not go away if a printk thread takes over, because the bug > is

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/19/18 13:20), Steven Rostedt wrote: [..] > I was thinking about this a bit more, and instead of offloading a > recursive printk, perhaps its best to simply throttle it. Because the > problem may not go away if a printk thread takes over, because the bug > is really the printk

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/19/18 13:20), Steven Rostedt wrote: [..] > I was thinking about this a bit more, and instead of offloading a > recursive printk, perhaps its best to simply throttle it. Because the > problem may not go away if a printk thread takes over, because the bug > is really the printk

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Tejun, I was thinking about this a bit more, and instead of offloading a recursive printk, perhaps its best to simply throttle it. Because the problem may not go away if a printk thread takes over, because the bug is really the printk infrastructure filling the printk buffer keeping printk from

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Tejun, I was thinking about this a bit more, and instead of offloading a recursive printk, perhaps its best to simply throttle it. Because the problem may not go away if a printk thread takes over, because the bug is really the printk infrastructure filling the printk buffer keeping printk from

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Thu, 18 Jan 2018 13:31:16 +0900 Sergey Senozhatsky wrote: > d'oh... indeed, I copy-pasted the wrong URL... it should > have been lkml.kernel.org/r/ [and it actually was]. I've learned to do a copy after entering the lkml.kernel.org link into the browser

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Thu, 18 Jan 2018 13:31:16 +0900 Sergey Senozhatsky wrote: > d'oh... indeed, I copy-pasted the wrong URL... it should > have been lkml.kernel.org/r/ [and it actually was]. I've learned to do a copy after entering the lkml.kernel.org link into the browser url, and before hitting enter. The

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed 2018-01-17 12:05:51, Tejun Heo wrote: > Hello, Steven. > > On Wed, Jan 17, 2018 at 12:12:51PM -0500, Steven Rostedt wrote: > > From what I gathered, you said an OOM would trigger, and then the > > network console would not be able to allocate memory and it would > > trigger a printk too,

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed 2018-01-17 12:05:51, Tejun Heo wrote: > Hello, Steven. > > On Wed, Jan 17, 2018 at 12:12:51PM -0500, Steven Rostedt wrote: > > From what I gathered, you said an OOM would trigger, and then the > > network console would not be able to allocate memory and it would > > trigger a printk too,

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 12:05), Tejun Heo wrote: [..] > > This could very well be a great place to force offloading. If a printk > > is called from within a printk, at the same context (normal, softirq, > > irq or NMI), then we should trigger the offloading. > > I was thinking more of a timeout based

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 12:05), Tejun Heo wrote: [..] > > This could very well be a great place to force offloading. If a printk > > is called from within a printk, at the same context (normal, softirq, > > irq or NMI), then we should trigger the offloading. > > I was thinking more of a timeout based

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 12:12), Steven Rostedt wrote: [..] > /* > * Can we actually use the console at this time on this cpu? > @@ -2333,6 +2390,7 @@ void console_unlock(void) > > for (;;) { > struct printk_log *msg; > + bool offload; > size_t ext_len = 0; >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 12:12), Steven Rostedt wrote: [..] > /* > * Can we actually use the console at this time on this cpu? > @@ -2333,6 +2390,7 @@ void console_unlock(void) > > for (;;) { > struct printk_log *msg; > + bool offload; > size_t ext_len = 0; >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 14:04), Petr Mladek wrote: > On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > > On (01/16/18 10:45), Steven Rostedt wrote: > > [..] > > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > > > Especially since Konstantin is working on pulling in all LKML archives, > >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/17/18 14:04), Petr Mladek wrote: > On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > > On (01/16/18 10:45), Steven Rostedt wrote: > > [..] > > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > > > Especially since Konstantin is working on pulling in all LKML archives, > >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Wed, Jan 17, 2018 at 12:12:51PM -0500, Steven Rostedt wrote: > From what I gathered, you said an OOM would trigger, and then the > network console would not be able to allocate memory and it would > trigger a printk too, and cause an infinite amount of printks. Yeah, it falls

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Wed, Jan 17, 2018 at 12:12:51PM -0500, Steven Rostedt wrote: > From what I gathered, you said an OOM would trigger, and then the > network console would not be able to allocate memory and it would > trigger a printk too, and cause an infinite amount of printks. Yeah, it falls

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 12:12:51 -0500 Steven Rostedt wrote: > @@ -2393,15 +2451,20 @@ void console_unlock(void) >* waiter waiting to take over. >*/ > console_lock_spinning_enable(); > + offload = recursion_check_start();

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 12:12:51 -0500 Steven Rostedt wrote: > @@ -2393,15 +2451,20 @@ void console_unlock(void) >* waiter waiting to take over. >*/ > console_lock_spinning_enable(); > + offload = recursion_check_start(); > >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 07:15:09 -0800 Tejun Heo wrote: > It's great that Steven's patches solve a good number of problems. It > is also true that there's a class of problems that it doesn't solve, > which other approaches do. The productive thing to do here is trying > to solve

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 07:15:09 -0800 Tejun Heo wrote: > It's great that Steven's patches solve a good number of problems. It > is also true that there's a class of problems that it doesn't solve, > which other approaches do. The productive thing to do here is trying > to solve the unsolved one

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 14:04:07 +0100 Petr Mladek wrote: > On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > > On (01/16/18 10:45), Steven Rostedt wrote: > > [..] > > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > > > Especially since Konstantin is

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed, 17 Jan 2018 14:04:07 +0100 Petr Mladek wrote: > On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > > On (01/16/18 10:45), Steven Rostedt wrote: > > [..] > > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > > > Especially since Konstantin is working on pulling in

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On Wed, Jan 17, 2018 at 10:12:08AM +0100, Petr Mladek wrote: > IMHO, the bad scenario with OOM was that any printk() called in > the OOM report became console_lock owner and was responsible > for pushing all new messages to the console. There was a possible > livelock because OOM Killer

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, On Wed, Jan 17, 2018 at 10:12:08AM +0100, Petr Mladek wrote: > IMHO, the bad scenario with OOM was that any printk() called in > the OOM report became console_lock owner and was responsible > for pushing all new messages to the console. There was a possible > livelock because OOM Killer

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > On (01/16/18 10:45), Steven Rostedt wrote: > [..] > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > Especially since Konstantin is working on pulling in all LKML archives, > > the above should be denoted as: > > > > Link: >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Wed 2018-01-17 11:18:56, Sergey Senozhatsky wrote: > On (01/16/18 10:45), Steven Rostedt wrote: > [..] > > > [1] https://marc.info/?l=linux-mm=145692016122716 > > > > Especially since Konstantin is working on pulling in all LKML archives, > > the above should be denoted as: > > > > Link: >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 11:44:56, Tejun Heo wrote: > Hello, Steven. > > On Thu, Jan 11, 2018 at 09:55:47PM -0500, Steven Rostedt wrote: > > All I did was start off a work queue on each CPU, and each CPU does one > > printk() followed by a millisecond sleep. No 10,000 printks, nothing > > in an

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 11:44:56, Tejun Heo wrote: > Hello, Steven. > > On Thu, Jan 11, 2018 at 09:55:47PM -0500, Steven Rostedt wrote: > > All I did was start off a work queue on each CPU, and each CPU does one > > printk() followed by a millisecond sleep. No 10,000 printks, nothing > > in an

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 11:13), Petr Mladek wrote: [..] > IMHO, it would make sense if flushing the printk buffer behaves > the same when called either from printk() or from any other path. > I mean that it should be aggressive and allow an effective > hand off. > > It should be safe as long as

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 11:13), Petr Mladek wrote: [..] > IMHO, it would make sense if flushing the printk buffer behaves > the same when called either from printk() or from any other path. > I mean that it should be aggressive and allow an effective > hand off. > > It should be safe as long as

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 11:19), Petr Mladek wrote: [..] > > [1] https://marc.info/?l=linux-mm=145692016122716 > > Fixes: 6b97a20d3a79 ("printk: set may_schedule for some of > > console_trylock() callers") > > Signed-off-by: Sergey Senozhatsky > > Reported-by: Tetsuo Handa

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 11:19), Petr Mladek wrote: [..] > > [1] https://marc.info/?l=linux-mm=145692016122716 > > Fixes: 6b97a20d3a79 ("printk: set may_schedule for some of > > console_trylock() callers") > > Signed-off-by: Sergey Senozhatsky > > Reported-by: Tetsuo Handa > > IMHO, this is a step in the

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 10:45), Steven Rostedt wrote: [..] > > [1] https://marc.info/?l=linux-mm=145692016122716 > > Especially since Konstantin is working on pulling in all LKML archives, > the above should be denoted as: > > Link: >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On (01/16/18 10:45), Steven Rostedt wrote: [..] > > [1] https://marc.info/?l=linux-mm=145692016122716 > > Especially since Konstantin is working on pulling in all LKML archives, > the above should be denoted as: > > Link: >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Thu, Jan 11, 2018 at 09:55:47PM -0500, Steven Rostedt wrote: > All I did was start off a work queue on each CPU, and each CPU does one > printk() followed by a millisecond sleep. No 10,000 printks, nothing > in an interrupt handler. Preemption is disabled while the printk >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

Hello, Steven. On Thu, Jan 11, 2018 at 09:55:47PM -0500, Steven Rostedt wrote: > All I did was start off a work queue on each CPU, and each CPU does one > printk() followed by a millisecond sleep. No 10,000 printks, nothing > in an interrupt handler. Preemption is disabled while the printk >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 16 Jan 2018 15:10:13 +0900 Sergey Senozhatsky wrote: > overall that's very close to what I have in one of my private branches. > console_trylock_spinning() for some reason does not perform really > well on my made-up internal printk torture tests. it

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 16 Jan 2018 15:10:13 +0900 Sergey Senozhatsky wrote: > overall that's very close to what I have in one of my private branches. > console_trylock_spinning() for some reason does not perform really > well on my made-up internal printk torture tests. it seems that I One thing I noticed in

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 16 Jan 2018 13:47:16 +0900 Sergey Senozhatsky wrote: > From: Sergey Senozhatsky > Subject: [PATCH] printk: never set console_may_schedule in console_trylock() > > This patch, basically, reverts commit 6b97a20d3a79

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue, 16 Jan 2018 13:47:16 +0900 Sergey Senozhatsky wrote: > From: Sergey Senozhatsky > Subject: [PATCH] printk: never set console_may_schedule in console_trylock() > > This patch, basically, reverts commit 6b97a20d3a79 ("printk: > set may_schedule for some of console_trylock() callers"). >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 13:47:16, Sergey Senozhatsky wrote: > if you don't mind, let me fix the thing that I broke. > that would be responsible. I believe I also must say the following: > Tetsuo, many thanks for reporting the issues for song long, and > sorry that it took quite a while to revert

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 13:47:16, Sergey Senozhatsky wrote: > if you don't mind, let me fix the thing that I broke. > that would be responsible. I believe I also must say the following: > Tetsuo, many thanks for reporting the issues for song long, and > sorry that it took quite a while to revert

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 11:23:49, Sergey Senozhatsky wrote: > On (01/15/18 15:45), Petr Mladek wrote: > > > I think adding the preempt_disable() would fix printk() but let non > > > printk console_unlock() still preempt. > > > > I would personally remove cond_resched() from console_unlock() > >

Re: [PATCH v5 0/2] printk: Console owner and waiter logic cleanup

On Tue 2018-01-16 11:23:49, Sergey Senozhatsky wrote: > On (01/15/18 15:45), Petr Mladek wrote: > > > I think adding the preempt_disable() would fix printk() but let non > > > printk console_unlock() still preempt. > > > > I would personally remove cond_resched() from console_unlock() > >

  1   2   3   >