Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-12-02 Thread t...@kernel.org
Hello, Neil. (cc'ing Lai) On Fri, Nov 20, 2020 at 10:07:56AM +1100, NeilBrown wrote: > If the workqueue maintainers are unmovable in the position that a > CM-workitem must not use excessive CPU ever, and so must not call > cond_resched(), then I can take that back to the NFS maintainers and Oh,

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-26 Thread NeilBrown
On Thu, Nov 26 2020, Hillf Danton wrote: > On Fri, 20 Nov 2020 15:33:27 +1100 NeilBrown wrote: >> >>My first idea was to add WQ_CPU_INTENSIVE to the nfsiod workqueue, but >>Trond wondered what was special about NFS. Many filesystems call iput >>from a workqueue, so finding a solution that helps t

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-26 Thread NeilBrown
On Wed, Nov 25 2020, t...@kernel.org wrote: > Hello, > > On Fri, Nov 20, 2020 at 10:23:44AM +1100, NeilBrown wrote: >> On Mon, Nov 09 2020, t...@kernel.org wrote: >> >> >Given that nothing on >> > these types of workqueues can be latency sensiti

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-25 Thread t...@kernel.org
Hello, On Fri, Nov 20, 2020 at 10:23:44AM +1100, NeilBrown wrote: > On Mon, Nov 09 2020, t...@kernel.org wrote: > > >Given that nothing on > > these types of workqueues can be latency sensitive > > This caught my eye and it seems worth drilling

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-19 Thread NeilBrown
On Fri, Nov 20 2020, Hillf Danton wrote: > On Fri, 20 Nov 2020 10:07:56 +1100 NeilBrown wrote: >>On Wed, Nov 18 2020, Hillf Danton wrote: >>> On Wed, 18 Nov 2020 16:11:44 +1100 NeilBrown wrote: On Wed, Nov 18 2020, Hillf Danton wrote: ... I don't think this is a good idea. >>> >>> L

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-19 Thread NeilBrown
On Mon, Nov 09 2020, t...@kernel.org wrote: >Given that nothing on > these types of workqueues can be latency sensitive This caught my eye and it seems worth drilling in to. There is no mention of "latency" in workqueue.rst or workqueue.h. But

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-19 Thread NeilBrown
On Wed, Nov 18 2020, Hillf Danton wrote: > On Wed, 18 Nov 2020 16:11:44 +1100 NeilBrown wrote: >> On Wed, Nov 18 2020, Hillf Danton wrote: >> ... >> I don't think this is a good idea. > > Let me add a few more words. > >> cond_resched() is expected to be called often. Adding all this extra > > Th

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-17 Thread NeilBrown
On Wed, Nov 18 2020, Hillf Danton wrote: > On Wed, 18 Nov 2020 09:16:09 +1100 >> From: NeilBrown >> Date: Mon, 9 Nov 2020 13:37:17 +1100 > > What is the brand of your wall clock? The gap between the Date tag > above and 18 Nov is longer than a week. I guess 'git commit --amend' doesn't update th

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-17 Thread NeilBrown
On Mon, Nov 09 2020, t...@kernel.org wrote: > Hello, > > On Mon, Nov 09, 2020 at 02:11:42PM +, Trond Myklebust wrote: >> That means changing all filesystem code to use cpu-intensive queues. As >> far as I can tell, they all use workqueues (most of them using the >> standard system queue) for f

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread NeilBrown
On Mon, Nov 09 2020, Peter Zijlstra wrote: > On Mon, Nov 09, 2020 at 01:50:40PM +, Trond Myklebust wrote: >> On Mon, 2020-11-09 at 09:00 +0100, Peter Zijlstra wrote: > >> > I'm thinking the real problem is that you're abusing workqueues. Just >> > don't stuff so much work into it that this bec

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread t...@kernel.org
Hello, On Mon, Nov 09, 2020 at 02:11:42PM +, Trond Myklebust wrote: > That means changing all filesystem code to use cpu-intensive queues. As > far as I can tell, they all use workqueues (most of them using the > standard system queue) for fput(), dput() and/or iput() calls. I suppose the ass

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread Peter Zijlstra
On Mon, Nov 09, 2020 at 01:50:40PM +, Trond Myklebust wrote: > On Mon, 2020-11-09 at 09:00 +0100, Peter Zijlstra wrote: > > I'm thinking the real problem is that you're abusing workqueues. Just > > don't stuff so much work into it that this becomes a problem. Or > > rather, > > if you do, don'

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread Trond Myklebust
On Mon, 2020-11-09 at 09:01 -0500, t...@kernel.org wrote: > Hello, > > On Mon, Nov 09, 2020 at 01:50:40PM +, Trond Myklebust wrote: > > > I'm thinking the real problem is that you're abusing workqueues. > > > Just > > > don't stuff so much work into it that this becomes a problem. Or > > > rat

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread t...@kernel.org
Hello, On Mon, Nov 09, 2020 at 01:50:40PM +, Trond Myklebust wrote: > > I'm thinking the real problem is that you're abusing workqueues. Just > > don't stuff so much work into it that this becomes a problem. Or > > rather, > > if you do, don't lie to it about it. > > If we can't use workqueue

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread Trond Myklebust
On Mon, 2020-11-09 at 09:00 +0100, Peter Zijlstra wrote: > On Mon, Nov 09, 2020 at 01:54:59PM +1100, NeilBrown wrote: > > diff --git a/include/linux/sched.h b/include/linux/sched.h > > index 4418f5cb8324..728870965df1 100644 > > --- a/include/linux/sched.h > > +++ b/include/linux/sched.h > > @@ -17

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-09 Thread Peter Zijlstra
On Mon, Nov 09, 2020 at 01:54:59PM +1100, NeilBrown wrote: > diff --git a/include/linux/sched.h b/include/linux/sched.h > index 4418f5cb8324..728870965df1 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -1784,7 +1784,12 @@ static inline int test_tsk_need_resched(struct > ta

Re: [PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-08 Thread Michal Hocko
On Mon 09-11-20 13:54:59, Neil Brown wrote: > > If a worker task for a normal (bound, non-CPU-intensive) calls > cond_resched(), this allows other non-workqueue processes to run, but > does *not* allow other workqueue workers to run. This is because > workqueue will only attempt to run one task a

[PATCH rfc] workqueue: honour cond_resched() more effectively.

2020-11-08 Thread NeilBrown
If a worker task for a normal (bound, non-CPU-intensive) calls cond_resched(), this allows other non-workqueue processes to run, but does *not* allow other workqueue workers to run. This is because workqueue will only attempt to run one task at a time on each CPU, unless the current task actually