Re: Problem with freezable workqueues

2007-03-08 Thread Johannes Berg
On Tue, 2007-03-06 at 21:31 +0100, Rafael J. Wysocki wrote: > For now, we have decided to make the workqueues nonfreezable (the patch for > that has already been merged, AFAICT). It isn't in 2.6.21-rc3. > > I wanted to adapt the BUG_ON(block IO not from suspend code) > > patch from suspend2 but

Re: Problem with freezable workqueues

2007-03-06 Thread Rafael J. Wysocki
Hi, On Tuesday, 6 March 2007 23:25, Nigel Cunningham wrote: > On Tue, 2007-03-06 at 21:31 +0100, Rafael J. Wysocki wrote: > > On Tuesday, 6 March 2007 01:30, Johannes Berg wrote: > > > On Tue, 2007-02-27 at 22:51 +0100, Rafael J. Wysocki wrote: > > > > > > > For 2.6.21-rc1 I've invented the appen

Re: Problem with freezable workqueues

2007-03-06 Thread Nigel Cunningham
Hi. On Tue, 2007-03-06 at 21:31 +0100, Rafael J. Wysocki wrote: > Hi, > > On Tuesday, 6 March 2007 01:30, Johannes Berg wrote: > > On Tue, 2007-02-27 at 22:51 +0100, Rafael J. Wysocki wrote: > > > > > For 2.6.21-rc1 I've invented the appended workaround (works for me, > > > waiting for > > > Jo

Re: Problem with freezable workqueues

2007-03-06 Thread Rafael J. Wysocki
Hi, On Tuesday, 6 March 2007 01:30, Johannes Berg wrote: > On Tue, 2007-02-27 at 22:51 +0100, Rafael J. Wysocki wrote: > > > For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting > > for > > Johannes to confirm it works for him too), but I think we need something > > bette

Re: Problem with freezable workqueues

2007-03-06 Thread Johannes Berg
On Tue, 2007-02-27 at 22:51 +0100, Rafael J. Wysocki wrote: > For 2.6.21-rc1 I've invented the appended workaround (works for me, waiting > for > Johannes to confirm it works for him too), but I think we need something > better > for -mm and future kernels. Finally I could get back to this but

Re: Problem with freezable workqueues

2007-02-28 Thread Pavel Machek
On Wed 2007-02-28 23:39:30, Rafael J. Wysocki wrote: > On Wednesday, 28 February 2007 21:35, Oleg Nesterov wrote: > > On 02/28, Rafael J. Wysocki wrote: > > > > > > Okay, but I've just finished the patch that removes the freezability of > > > workqueues (appended), so can we please do this in a se

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 21:35, Oleg Nesterov wrote: > On 02/28, Rafael J. Wysocki wrote: > > > > Okay, but I've just finished the patch that removes the freezability of > > workqueues (appended), so can we please do this in a separate one? > > Please, please, no. This patch is of course co

Re: Problem with freezable workqueues

2007-02-28 Thread Pavel Machek
Hi! > > OK, thanks. > > > > We can (I think) do pretty much the same with some additional complications > > in worker_thread() (check !cpu_online() after try_to_freeze() and break). > > Okay, but I've just finished the patch that removes the freezability of > workqueues (appended), so can we ple

Re: Problem with freezable workqueues

2007-02-28 Thread Johannes Berg
On Wed, 2007-02-28 at 12:14 +1100, Nigel Cunningham wrote: > Controversy is no reason to give in! Nevertheless, I think you're right > - I believe the XFS guys said they fixed the issue that had caused I/O > to be submitted post-freeze. Well, we'll see if it appears again, won't > we? I get to be

Re: Problem with freezable workqueues

2007-02-28 Thread Oleg Nesterov
On 02/28, Rafael J. Wysocki wrote: > > Okay, but I've just finished the patch that removes the freezability of > workqueues (appended), so can we please do this in a separate one? Please, please, no. This patch is of course correct, but it breaks _a lot_ of patches in -mm tree. May I ask you to

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 21:08, Oleg Nesterov wrote: > On 02/28, Rafael J. Wysocki wrote: > > > > On Wednesday, 28 February 2007 20:32, Oleg Nesterov wrote: > > > > > > I am sorry, I lost track of this problem. As for 2.6.21, > > > create_freezeable_workqueue > > > doesn't work and conflict

Re: Problem with freezable workqueues

2007-02-28 Thread Oleg Nesterov
On 02/28, Rafael J. Wysocki wrote: > > On Wednesday, 28 February 2007 20:32, Oleg Nesterov wrote: > > > > I am sorry, I lost track of this problem. As for 2.6.21, > > create_freezeable_workqueue > > doesn't work and conflict with suspend. Why can't we remove it from XFS as > > you > > suggested

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 20:32, Oleg Nesterov wrote: > On 02/28, Rafael J. Wysocki wrote: > > > > > --- workqueue.c.org 2007-02-28 18:32:48.0 +0530 > > > +++ workqueue.c 2007-02-28 18:44:23.0 +0530 > > > @@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(str > >

Re: Problem with freezable workqueues

2007-02-28 Thread Oleg Nesterov
On 02/28, Rafael J. Wysocki wrote: > > > --- workqueue.c.org 2007-02-28 18:32:48.0 +0530 > > +++ workqueue.c 2007-02-28 18:44:23.0 +0530 > > @@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(str > > insert_wq_barrier(cwq, &barr, 1); > > cwq->should

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 14:17, Srivatsa Vaddagiri wrote: > On Wed, Feb 28, 2007 at 12:11:03PM +0100, Rafael J. Wysocki wrote: > > > In addition to thawing worker thread before kthread_stopping it, there > > > are minor changes required in worker threads, to check for > > > is_cpu_offline(bin

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 19:17, Gautham R Shenoy wrote: > On Wed, Feb 28, 2007 at 08:37:26AM +0530, Srivatsa Vaddagiri wrote: > > > > Hmm ..good point. So can we assume that disable/enable_nonboot_cpus() are > > called > > with processes frozen already? > > > > Gautham, you need to take th

Re: Problem with freezable workqueues

2007-02-28 Thread Gautham R Shenoy
On Wed, Feb 28, 2007 at 08:37:26AM +0530, Srivatsa Vaddagiri wrote: > > Hmm ..good point. So can we assume that disable/enable_nonboot_cpus() are > called > with processes frozen already? > > Gautham, you need to take this into account in your patchset! Yup. That would mean making the freezer r

Re: Problem with freezable workqueues

2007-02-28 Thread Gautham R Shenoy
On Wed, Feb 28, 2007 at 12:36:52AM +0100, Johannes Berg wrote: > On Wed, 2007-02-28 at 02:28 +0300, Oleg Nesterov wrote: > > > Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess. > > > >Commit: ed746e3b18f4df18afa3763155972c5835f284c5 > > > Yes? with the patch above, _cpu

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 14:27, Srivatsa Vaddagiri wrote: > On Wed, Feb 28, 2007 at 06:47:21PM +0530, Srivatsa Vaddagiri wrote: > > --- workqueue.c.org 2007-02-28 18:32:48.0 +0530 > > +++ workqueue.c 2007-02-28 18:44:23.0 +0530 > > @@ -718,6 +718,8 @@ static void cleanup_w

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 14:17, Srivatsa Vaddagiri wrote: > On Wed, Feb 28, 2007 at 12:11:03PM +0100, Rafael J. Wysocki wrote: > > > In addition to thawing worker thread before kthread_stopping it, there > > > are minor changes required in worker threads, to check for > > > is_cpu_offline(bin

Re: Problem with freezable workqueues

2007-02-28 Thread Srivatsa Vaddagiri
On Wed, Feb 28, 2007 at 06:47:21PM +0530, Srivatsa Vaddagiri wrote: > --- workqueue.c.org 2007-02-28 18:32:48.0 +0530 > +++ workqueue.c 2007-02-28 18:44:23.0 +0530 > @@ -718,6 +718,8 @@ static void cleanup_workqueue_thread(str > insert_wq_barrier(cwq, &barr, 1)

Re: Problem with freezable workqueues

2007-02-28 Thread Srivatsa Vaddagiri
On Wed, Feb 28, 2007 at 12:11:03PM +0100, Rafael J. Wysocki wrote: > > In addition to thawing worker thread before kthread_stopping it, there > > are minor changes required in worker threads, to check for > > is_cpu_offline(bind_cpu) when they come out of refrigerator and jump to > > wait_to_die if

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 04:51, Srivatsa Vaddagiri wrote: > On Wed, Feb 28, 2007 at 08:31:13AM +0530, Srivatsa Vaddagiri wrote: > > This problem (of kthread_stopping a frozen thread) was there when we > > implemented freezer-based cpu hotplug. We worked around that in the > > callbacks by tha

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 10:10, Srivatsa Vaddagiri wrote: > On Wed, Feb 28, 2007 at 11:48:59AM +0300, Oleg Nesterov wrote: > > On 02/28, Srivatsa Vaddagiri wrote: > > > We can just thaw the worker thread selectively before kthread_stopping > > > them. This will let us freeze all worker thread

Re: Problem with freezable workqueues

2007-02-28 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 02:14, Nigel Cunningham wrote: > Hi. > > On Wed, 2007-02-28 at 01:08 +0100, Rafael J. Wysocki wrote: > > On Wednesday, 28 February 2007 01:01, Johannes Berg wrote: > > > On Wed, 2007-02-28 at 00:57 +0100, Rafael J. Wysocki wrote: > > > > > > > Okay, in that case I'd

Re: Problem with freezable workqueues

2007-02-28 Thread Oleg Nesterov
On 02/28, Srivatsa Vaddagiri wrote: > > On Wed, Feb 28, 2007 at 11:48:59AM +0300, Oleg Nesterov wrote: > > On 02/28, Srivatsa Vaddagiri wrote: > > > We can just thaw the worker thread selectively before kthread_stopping > > > them. This will let us freeze all worker threads (which we want to for >

Re: Problem with freezable workqueues

2007-02-28 Thread Srivatsa Vaddagiri
On Wed, Feb 28, 2007 at 11:48:59AM +0300, Oleg Nesterov wrote: > On 02/28, Srivatsa Vaddagiri wrote: > > We can just thaw the worker thread selectively before kthread_stopping > > them. This will let us freeze all worker threads (which we want to for > > hotplug anyway). > > I am not sure this is

Re: Problem with freezable workqueues

2007-02-28 Thread Pavel Machek
Hi! > > I'm afraid this is racy. We can't touch *cwq, it may be freed. Suppose > > that another thread does destroy_workqueue(), and we thaw that thread > > before cwq->thread. > > Okay, in that case I'd suggest removing create_freezeable_workqueue() and > make all workqueues nonfreezable once ag

Re: Problem with freezable workqueues

2007-02-28 Thread Oleg Nesterov
On 02/28, Srivatsa Vaddagiri wrote: > > On Wed, Feb 28, 2007 at 12:57:35AM +0100, Rafael J. Wysocki wrote: > > > How about other kthread_stop()s ? For example, > > > kernel/softirq.c:cpu_callback() ? > > > > They all are PF_NOFREEZE, I suppose. If we make all workqueues nonfreezable > > (as they

Re: Problem with freezable workqueues

2007-02-27 Thread Srivatsa Vaddagiri
On Wed, Feb 28, 2007 at 08:31:13AM +0530, Srivatsa Vaddagiri wrote: > This problem (of kthread_stopping a frozen thread) was there when we > implemented freezer-based cpu hotplug. We worked around that in the > callbacks by thawing the worker thread first before kthread_stopping it, > which is work

Re: Problem with freezable workqueues

2007-02-27 Thread Srivatsa Vaddagiri
On Wed, Feb 28, 2007 at 12:57:35AM +0100, Rafael J. Wysocki wrote: > > How about other kthread_stop()s ? For example, > > kernel/softirq.c:cpu_callback() ? > > They all are PF_NOFREEZE, I suppose. If we make all workqueues nonfreezable > (as they were before), the problem won't appear. We can j

Re: Problem with freezable workqueues

2007-02-27 Thread Srivatsa Vaddagiri
On Tue, Feb 27, 2007 at 10:51:27PM +0100, Rafael J. Wysocki wrote: > We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm > (there are only two of them, in XFS, but still). Namely, their worker threads > deadlock with workqueue_cpu_callback() that gets called durin

Re: Problem with freezable workqueues

2007-02-27 Thread Nigel Cunningham
Hi. On Wed, 2007-02-28 at 01:08 +0100, Rafael J. Wysocki wrote: > On Wednesday, 28 February 2007 01:01, Johannes Berg wrote: > > On Wed, 2007-02-28 at 00:57 +0100, Rafael J. Wysocki wrote: > > > > > Okay, in that case I'd suggest removing create_freezeable_workqueue() and > > > make all workqueue

Re: Problem with freezable workqueues

2007-02-27 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 01:01, Johannes Berg wrote: > On Wed, 2007-02-28 at 00:57 +0100, Rafael J. Wysocki wrote: > > > Okay, in that case I'd suggest removing create_freezeable_workqueue() and > > make all workqueues nonfreezable once again for 2.6.21 (as far as I know, > > only > > the t

Re: Problem with freezable workqueues

2007-02-27 Thread Johannes Berg
On Wed, 2007-02-28 at 00:57 +0100, Rafael J. Wysocki wrote: > Okay, in that case I'd suggest removing create_freezeable_workqueue() and > make all workqueues nonfreezable once again for 2.6.21 (as far as I know, only > the two XFS workqueues are affected). I think Nigel might object but I forgot

Re: Problem with freezable workqueues

2007-02-27 Thread Johannes Berg
On Wed, 2007-02-28 at 01:00 +0100, Rafael J. Wysocki wrote: > > > How about other kthread_stop()s ? For example, > > > kernel/softirq.c:cpu_callback() ? > > > > I'd they should be affected as well. > > They won't be, if they have PF_NOFREEZE set. Yup, I missed that. johannes signature.asc D

Re: Problem with freezable workqueues

2007-02-27 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 00:36, Johannes Berg wrote: > On Wed, 2007-02-28 at 02:28 +0300, Oleg Nesterov wrote: > > > Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess. > > > >Commit: ed746e3b18f4df18afa3763155972c5835f284c5 > > > Yes? with the patch above, _cpu_dow

Re: Problem with freezable workqueues

2007-02-27 Thread Rafael J. Wysocki
On Wednesday, 28 February 2007 00:28, Oleg Nesterov wrote: > On 02/27, Rafael J. Wysocki wrote: > > > > We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm > > (there are only two of them, in XFS, but still). Namely, their worker > >

Re: Problem with freezable workqueues

2007-02-27 Thread Johannes Berg
On Wed, 2007-02-28 at 02:28 +0300, Oleg Nesterov wrote: > Ugh. I know nothing, nothing, nothing about suspend. I'll try to guess. > >Commit: ed746e3b18f4df18afa3763155972c5835f284c5 > Yes? with the patch above, _cpu_down() called _after_ freeze_processes() ??? perfect :) See also my origina

Re: Problem with freezable workqueues

2007-02-27 Thread Oleg Nesterov
On 02/27, Rafael J. Wysocki wrote: > > We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm > (there are only two of them, in XFS, but still). Namely, their worker threads > deadlock with workqueue_cpu_callback() that gets called during the CPU > hot

Problem with freezable workqueues

2007-02-27 Thread Rafael J. Wysocki
Hi, We have a problem with freezable workqueues in 2.6.21-rc1 and in -mm (there are only two of them, in XFS, but still). Namely, their worker threads deadlock with workqueue_cpu_callback() that gets called during the CPU hotplug, becuase workqueue_cpu_callback() tries to stop these threads