Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/11/2012 07:37 PM, Tejun Heo wrote: > Hello, > > On Tue, Dec 11, 2012 at 07:32:13PM +0530, Srivatsa S. Bhat wrote: >> On 12/11/2012 07:17 PM, Tejun Heo wrote: >>> Hello, Srivatsa. >>> >>> On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: This approach (of using

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Tejun Heo
Hello, On Tue, Dec 11, 2012 at 07:32:13PM +0530, Srivatsa S. Bhat wrote: > On 12/11/2012 07:17 PM, Tejun Heo wrote: > > Hello, Srivatsa. > > > > On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: > >> This approach (of using synchronize_sched()) also looks good. It is simple, > >>

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/11/2012 07:17 PM, Tejun Heo wrote: > Hello, Srivatsa. > > On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: >> This approach (of using synchronize_sched()) also looks good. It is simple, >> yet effective, but unfortunately inefficient at the writer side (because >> he'll

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Tejun Heo
Hello, Srivatsa. On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: > This approach (of using synchronize_sched()) also looks good. It is simple, > yet effective, but unfortunately inefficient at the writer side (because > he'll have to wait for a full synchronize_sched()). While

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 10:54 PM, Oleg Nesterov wrote: > On 12/10, Srivatsa S. Bhat wrote: >> >> On 12/10/2012 01:52 AM, Oleg Nesterov wrote: >>> On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > But yes, it is easy to blame somebody else's code ;) And I

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 10:58 PM, Oleg Nesterov wrote: > On 12/10, Srivatsa S. Bhat wrote: >> >> On 12/10/2012 02:43 AM, Oleg Nesterov wrote: >>> Damn, sorry for noise. I missed this part... >>> >>> On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > the latency.

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 11:45 PM, Oleg Nesterov wrote: > On 12/10, Srivatsa S. Bhat wrote: >> >> On 12/10/2012 02:27 AM, Oleg Nesterov wrote: >>> However. If this is true, then compared to preempt_disable/stop_machine >>> livelock is possible. Probably this is fine, we have the same problem with >>>

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 11:45 PM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 02:27 AM, Oleg Nesterov wrote: However. If this is true, then compared to preempt_disable/stop_machine livelock is possible. Probably this is fine, we have the same problem with get_online_cpus().

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 10:58 PM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 02:43 AM, Oleg Nesterov wrote: Damn, sorry for noise. I missed this part... On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: the latency. And I guess something like

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/10/2012 10:54 PM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 01:52 AM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: But yes, it is easy to blame somebody else's code ;) And I can't suggest something

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Tejun Heo
Hello, Srivatsa. On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: This approach (of using synchronize_sched()) also looks good. It is simple, yet effective, but unfortunately inefficient at the writer side (because he'll have to wait for a full synchronize_sched()). While

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/11/2012 07:17 PM, Tejun Heo wrote: Hello, Srivatsa. On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: This approach (of using synchronize_sched()) also looks good. It is simple, yet effective, but unfortunately inefficient at the writer side (because he'll have to wait

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Tejun Heo
Hello, On Tue, Dec 11, 2012 at 07:32:13PM +0530, Srivatsa S. Bhat wrote: On 12/11/2012 07:17 PM, Tejun Heo wrote: Hello, Srivatsa. On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: This approach (of using synchronize_sched()) also looks good. It is simple, yet

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-11 Thread Srivatsa S. Bhat
On 12/11/2012 07:37 PM, Tejun Heo wrote: Hello, On Tue, Dec 11, 2012 at 07:32:13PM +0530, Srivatsa S. Bhat wrote: On 12/11/2012 07:17 PM, Tejun Heo wrote: Hello, Srivatsa. On Tue, Dec 11, 2012 at 06:43:54PM +0530, Srivatsa S. Bhat wrote: This approach (of using synchronize_sched()) also

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: > > On 12/10/2012 02:27 AM, Oleg Nesterov wrote: > > On 12/07, Srivatsa S. Bhat wrote: > >> > >> 4. No deadlock possibilities > >> > >>Per-cpu locking is not the way to go if we want to have relaxed rules > >>for lock-ordering. Because, we can end up in

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: > > On 12/10/2012 02:43 AM, Oleg Nesterov wrote: > > Damn, sorry for noise. I missed this part... > > > > On 12/10, Srivatsa S. Bhat wrote: > >> > >> On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > >>> the latency. And I guess something like kick_all_cpus_sync() is

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: > > On 12/10/2012 01:52 AM, Oleg Nesterov wrote: > > On 12/10, Srivatsa S. Bhat wrote: > >> > >> On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > >> > >>> But yes, it is easy to blame somebody else's code ;) And I can't suggest > >>> something better at least right

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 01:52 AM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: But yes, it is easy to blame somebody else's code ;) And I can't suggest something better at least right now. If I understand

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 02:43 AM, Oleg Nesterov wrote: Damn, sorry for noise. I missed this part... On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: the latency. And I guess something like kick_all_cpus_sync() is too heavy.

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-10 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 02:27 AM, Oleg Nesterov wrote: On 12/07, Srivatsa S. Bhat wrote: 4. No deadlock possibilities Per-cpu locking is not the way to go if we want to have relaxed rules for lock-ordering. Because, we can end up in circular-locking

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 02:27 AM, Oleg Nesterov wrote: > On 12/07, Srivatsa S. Bhat wrote: >> >> 4. No deadlock possibilities >> >>Per-cpu locking is not the way to go if we want to have relaxed rules >>for lock-ordering. Because, we can end up in circular-locking dependencies >>as explained in

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 02:43 AM, Oleg Nesterov wrote: > Damn, sorry for noise. I missed this part... > > On 12/10, Srivatsa S. Bhat wrote: >> >> On 12/10/2012 12:44 AM, Oleg Nesterov wrote: >>> the latency. And I guess something like kick_all_cpus_sync() is "too heavy". >> >> I hadn't considered that.

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 01:52 AM, Oleg Nesterov wrote: > On 12/10, Srivatsa S. Bhat wrote: >> >> On 12/10/2012 12:44 AM, Oleg Nesterov wrote: >> >>> But yes, it is easy to blame somebody else's code ;) And I can't suggest >>> something better at least right now. If I understand correctly, we can not >>>

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
Damn, sorry for noise. I missed this part... On 12/10, Srivatsa S. Bhat wrote: > > On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > > the latency. And I guess something like kick_all_cpus_sync() is "too heavy". > > I hadn't considered that. Thinking of it, I don't think it would help us.. > It

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/07, Srivatsa S. Bhat wrote: > > 4. No deadlock possibilities > >Per-cpu locking is not the way to go if we want to have relaxed rules >for lock-ordering. Because, we can end up in circular-locking dependencies >as explained in https://lkml.org/lkml/2012/12/6/290 OK, but this

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: > > On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > > > But yes, it is easy to blame somebody else's code ;) And I can't suggest > > something better at least right now. If I understand correctly, we can not > > use, say, synchronize_sched() in _cpu_down() path > >

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 12:44 AM, Oleg Nesterov wrote: > On 12/07, Srivatsa S. Bhat wrote: >> >> Per-cpu counters can help solve the cache-line bouncing problem. So we >> actually use the best of both: per-cpu counters (no-waiting) at the reader >> side in the fast-path, and global rwlocks in the slowpath.

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/07, Srivatsa S. Bhat wrote: > > Per-cpu counters can help solve the cache-line bouncing problem. So we > actually use the best of both: per-cpu counters (no-waiting) at the reader > side in the fast-path, and global rwlocks in the slowpath. > > [ Fastpath = no writer is active; Slowpath = a

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/07, Srivatsa S. Bhat wrote: Per-cpu counters can help solve the cache-line bouncing problem. So we actually use the best of both: per-cpu counters (no-waiting) at the reader side in the fast-path, and global rwlocks in the slowpath. [ Fastpath = no writer is active; Slowpath = a writer

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 12:44 AM, Oleg Nesterov wrote: On 12/07, Srivatsa S. Bhat wrote: Per-cpu counters can help solve the cache-line bouncing problem. So we actually use the best of both: per-cpu counters (no-waiting) at the reader side in the fast-path, and global rwlocks in the slowpath. [

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: But yes, it is easy to blame somebody else's code ;) And I can't suggest something better at least right now. If I understand correctly, we can not use, say, synchronize_sched() in _cpu_down() path We can't

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
On 12/07, Srivatsa S. Bhat wrote: 4. No deadlock possibilities Per-cpu locking is not the way to go if we want to have relaxed rules for lock-ordering. Because, we can end up in circular-locking dependencies as explained in https://lkml.org/lkml/2012/12/6/290 OK, but this assumes

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Oleg Nesterov
Damn, sorry for noise. I missed this part... On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: the latency. And I guess something like kick_all_cpus_sync() is too heavy. I hadn't considered that. Thinking of it, I don't think it would help us.. It won't get rid

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 01:52 AM, Oleg Nesterov wrote: On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: But yes, it is easy to blame somebody else's code ;) And I can't suggest something better at least right now. If I understand correctly, we can not use, say,

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 02:43 AM, Oleg Nesterov wrote: Damn, sorry for noise. I missed this part... On 12/10, Srivatsa S. Bhat wrote: On 12/10/2012 12:44 AM, Oleg Nesterov wrote: the latency. And I guess something like kick_all_cpus_sync() is too heavy. I hadn't considered that. Thinking of it, I

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-09 Thread Srivatsa S. Bhat
On 12/10/2012 02:27 AM, Oleg Nesterov wrote: On 12/07, Srivatsa S. Bhat wrote: 4. No deadlock possibilities Per-cpu locking is not the way to go if we want to have relaxed rules for lock-ordering. Because, we can end up in circular-locking dependencies as explained in

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/08/2012 12:01 AM, Tejun Heo wrote: > Hello, Srivatsa. > > On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote: >>> lg_lock doesn't do local nesting and I'm not sure how big a deal that >>> is as I don't know how many should be converted. But if nesting is an >>> absolute

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/07/2012 11:46 PM, Tejun Heo wrote: > Hello, again. > > On Fri, Dec 07, 2012 at 09:57:24AM -0800, Tejun Heo wrote: >> possible. Also, I think the right approach would be auditing each >> get_online_cpus_atomic() callsites and figure out proper locking order >> rather than implementing a

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
Hello, Srivatsa. On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote: > > lg_lock doesn't do local nesting and I'm not sure how big a deal that > > is as I don't know how many should be converted. But if nesting is an > > absolute necessity, it would be much better to implement

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/07/2012 11:27 PM, Tejun Heo wrote: > On Fri, Dec 07, 2012 at 11:08:13PM +0530, Srivatsa S. Bhat wrote: >> 4. No deadlock possibilities >> >>Per-cpu locking is not the way to go if we want to have relaxed rules >>for lock-ordering. Because, we can end up in circular-locking

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
Hello, again. On Fri, Dec 07, 2012 at 09:57:24AM -0800, Tejun Heo wrote: > possible. Also, I think the right approach would be auditing each > get_online_cpus_atomic() callsites and figure out proper locking order > rather than implementing a construct this unusual especially as > hunting down

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
On Fri, Dec 07, 2012 at 11:08:13PM +0530, Srivatsa S. Bhat wrote: > 4. No deadlock possibilities > >Per-cpu locking is not the way to go if we want to have relaxed rules >for lock-ordering. Because, we can end up in circular-locking dependencies >as explained in

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
On Fri, Dec 07, 2012 at 11:08:13PM +0530, Srivatsa S. Bhat wrote: 4. No deadlock possibilities Per-cpu locking is not the way to go if we want to have relaxed rules for lock-ordering. Because, we can end up in circular-locking dependencies as explained in

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
Hello, again. On Fri, Dec 07, 2012 at 09:57:24AM -0800, Tejun Heo wrote: possible. Also, I think the right approach would be auditing each get_online_cpus_atomic() callsites and figure out proper locking order rather than implementing a construct this unusual especially as hunting down the

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/07/2012 11:27 PM, Tejun Heo wrote: On Fri, Dec 07, 2012 at 11:08:13PM +0530, Srivatsa S. Bhat wrote: 4. No deadlock possibilities Per-cpu locking is not the way to go if we want to have relaxed rules for lock-ordering. Because, we can end up in circular-locking dependencies as

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Tejun Heo
Hello, Srivatsa. On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote: lg_lock doesn't do local nesting and I'm not sure how big a deal that is as I don't know how many should be converted. But if nesting is an absolute necessity, it would be much better to implement generic

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/07/2012 11:46 PM, Tejun Heo wrote: Hello, again. On Fri, Dec 07, 2012 at 09:57:24AM -0800, Tejun Heo wrote: possible. Also, I think the right approach would be auditing each get_online_cpus_atomic() callsites and figure out proper locking order rather than implementing a construct

Re: [RFC PATCH v3 1/9] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

2012-12-07 Thread Srivatsa S. Bhat
On 12/08/2012 12:01 AM, Tejun Heo wrote: Hello, Srivatsa. On Fri, Dec 07, 2012 at 11:54:01PM +0530, Srivatsa S. Bhat wrote: lg_lock doesn't do local nesting and I'm not sure how big a deal that is as I don't know how many should be converted. But if nesting is an absolute necessity, it