Re: INFO: rcu detected stall in ext4_write_checks

2019-07-23 Thread Dmitry Vyukov
On Mon, Jul 22, 2019 at 12:03 PM Dmitry Vyukov wrote: > > On Mon, Jul 15, 2019 at 3:46 PM Peter Zijlstra wrote: > > > > On Mon, Jul 15, 2019 at 03:33:11PM +0200, Dmitry Vyukov wrote: > > > On Mon, Jul 15, 2019 at 3:29 PM Peter Zijlstra > > > wrote: > > > > > > > > On Sun, Jul 14, 2019 at 11:49:

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-22 Thread Dmitry Vyukov
On Mon, Jul 15, 2019 at 3:46 PM Peter Zijlstra wrote: > > On Mon, Jul 15, 2019 at 03:33:11PM +0200, Dmitry Vyukov wrote: > > On Mon, Jul 15, 2019 at 3:29 PM Peter Zijlstra wrote: > > > > > > On Sun, Jul 14, 2019 at 11:49:15AM -0700, Paul E. McKenney wrote: > > > > On Sun, Jul 14, 2019 at 05:48:00

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Paul E. McKenney
On Mon, Jul 15, 2019 at 03:39:38PM +0200, Peter Zijlstra wrote: > On Mon, Jul 15, 2019 at 06:01:01AM -0700, Paul E. McKenney wrote: > > Title: Making SCHED_DEADLINE safe for kernel kthreads > > > > Abstract: > > > > Dmitry Vyukov's testing work identified some (ab)uses of sched_setattr() > > that

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Paul E. McKenney
On Mon, Jul 15, 2019 at 03:46:51PM +0200, Peter Zijlstra wrote: > On Mon, Jul 15, 2019 at 03:33:11PM +0200, Dmitry Vyukov wrote: > > On Mon, Jul 15, 2019 at 3:29 PM Peter Zijlstra wrote: > > > > > > On Sun, Jul 14, 2019 at 11:49:15AM -0700, Paul E. McKenney wrote: > > > > On Sun, Jul 14, 2019 at 0

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Peter Zijlstra
On Mon, Jul 15, 2019 at 03:33:11PM +0200, Dmitry Vyukov wrote: > On Mon, Jul 15, 2019 at 3:29 PM Peter Zijlstra wrote: > > > > On Sun, Jul 14, 2019 at 11:49:15AM -0700, Paul E. McKenney wrote: > > > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > > > But short term I don't see a

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Peter Zijlstra
On Mon, Jul 15, 2019 at 06:01:01AM -0700, Paul E. McKenney wrote: > Title: Making SCHED_DEADLINE safe for kernel kthreads > > Abstract: > > Dmitry Vyukov's testing work identified some (ab)uses of sched_setattr() > that can result in SCHED_DEADLINE tasks starving RCU's kthreads for > extended tim

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Dmitry Vyukov
On Mon, Jul 15, 2019 at 3:29 PM Peter Zijlstra wrote: > > On Sun, Jul 14, 2019 at 11:49:15AM -0700, Paul E. McKenney wrote: > > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > > But short term I don't see any other solution than stop testing > > > sched_setattr because it does n

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Dmitry Vyukov
On Mon, Jul 15, 2019 at 3:01 PM Paul E. McKenney wrote: > > On Sun, Jul 14, 2019 at 08:10:27PM -0700, Paul E. McKenney wrote: > > On Sun, Jul 14, 2019 at 12:29:51PM -0700, Paul E. McKenney wrote: > > > On Sun, Jul 14, 2019 at 03:05:22PM -0400, Theodore Ts'o wrote: > > > > On Sun, Jul 14, 2019 at 0

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Peter Zijlstra
On Sun, Jul 14, 2019 at 11:49:15AM -0700, Paul E. McKenney wrote: > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > But short term I don't see any other solution than stop testing > > sched_setattr because it does not check arguments enough to prevent > > system misbehavior. Whic

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Peter Zijlstra
On Sat, Jul 06, 2019 at 06:16:55PM -0700, Paul E. McKenney wrote: > 4.SCHED_DEADLINE treats the other three scheduling classes as each > having a period, deadline, and a modest CPU consumption budget > for the members of the class in aggregate. But this has to have > been dis

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-15 Thread Paul E. McKenney
On Sun, Jul 14, 2019 at 08:10:27PM -0700, Paul E. McKenney wrote: > On Sun, Jul 14, 2019 at 12:29:51PM -0700, Paul E. McKenney wrote: > > On Sun, Jul 14, 2019 at 03:05:22PM -0400, Theodore Ts'o wrote: > > > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > > > But short term I don'

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-14 Thread Paul E. McKenney
On Sun, Jul 14, 2019 at 12:29:51PM -0700, Paul E. McKenney wrote: > On Sun, Jul 14, 2019 at 03:05:22PM -0400, Theodore Ts'o wrote: > > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > > But short term I don't see any other solution than stop testing > > > sched_setattr because it

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-14 Thread Paul E. McKenney
On Sun, Jul 14, 2019 at 03:05:22PM -0400, Theodore Ts'o wrote: > On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > > But short term I don't see any other solution than stop testing > > sched_setattr because it does not check arguments enough to prevent > > system misbehavior. Which i

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-14 Thread Theodore Ts'o
On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > But short term I don't see any other solution than stop testing > sched_setattr because it does not check arguments enough to prevent > system misbehavior. Which is a pity because syzkaller has found some > bad misconfigurations that

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-14 Thread Paul E. McKenney
On Sun, Jul 14, 2019 at 05:48:00PM +0300, Dmitry Vyukov wrote: > On Sun, Jul 7, 2019 at 4:17 AM Paul E. McKenney wrote: > > > > > I suppose RCU could take the dueling-banjos approach and use > > > > > increasingly > > > > > aggressive scheduler policies itself, up to and including > > > > > SCHE

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-14 Thread Dmitry Vyukov
On Sun, Jul 7, 2019 at 4:17 AM Paul E. McKenney wrote: > > > > I suppose RCU could take the dueling-banjos approach and use > > > > increasingly > > > > aggressive scheduler policies itself, up to and including > > > > SCHED_DEADLINE, > > > > until it started getting decent forward progress. Ho

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-06 Thread Paul E. McKenney
On Sat, Jul 06, 2019 at 11:03:11AM -0700, Paul E. McKenney wrote: > On Sat, Jul 06, 2019 at 11:02:26AM -0400, Theodore Ts'o wrote: > > On Fri, Jul 05, 2019 at 11:16:31PM -0700, Paul E. McKenney wrote: > > > I suppose RCU could take the dueling-banjos approach and use increasingly > > > aggressive s

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-06 Thread Paul E. McKenney
On Sat, Jul 06, 2019 at 11:02:26AM -0400, Theodore Ts'o wrote: > On Fri, Jul 05, 2019 at 11:16:31PM -0700, Paul E. McKenney wrote: > > I suppose RCU could take the dueling-banjos approach and use increasingly > > aggressive scheduler policies itself, up to and including SCHED_DEADLINE, > > until it

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-06 Thread Theodore Ts'o
On Fri, Jul 05, 2019 at 11:16:31PM -0700, Paul E. McKenney wrote: > I suppose RCU could take the dueling-banjos approach and use increasingly > aggressive scheduler policies itself, up to and including SCHED_DEADLINE, > until it started getting decent forward progress. However, that > sounds like

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Paul E. McKenney
On Sat, Jul 06, 2019 at 12:28:01AM -0400, Theodore Ts'o wrote: > On Fri, Jul 05, 2019 at 12:10:55PM -0700, Paul E. McKenney wrote: > > > > Exactly, so although my patch might help for CONFIG_PREEMPT=n, it won't > > help in your scenario. But looking at the dmesg from your URL above, > > I see the

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Theodore Ts'o
On Fri, Jul 05, 2019 at 12:10:55PM -0700, Paul E. McKenney wrote: > > Exactly, so although my patch might help for CONFIG_PREEMPT=n, it won't > help in your scenario. But looking at the dmesg from your URL above, > I see the following: I just tested with CONFIG_PREEMPT=n % grep CONFIG_PREEMPT /

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Paul E. McKenney
On Fri, Jul 05, 2019 at 05:48:31PM +0200, Dmitry Vyukov wrote: > On Fri, Jul 5, 2019 at 5:17 PM Paul E. McKenney wrote: > > > > On Fri, Jul 05, 2019 at 03:24:26PM +0200, Dmitry Vyukov wrote: > > > On Thu, Jun 27, 2019 at 12:47 AM Theodore Ts'o wrote: > > > > > > > > More details about what is goi

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Dmitry Vyukov
On Fri, Jul 5, 2019 at 5:17 PM Paul E. McKenney wrote: > > On Fri, Jul 05, 2019 at 03:24:26PM +0200, Dmitry Vyukov wrote: > > On Thu, Jun 27, 2019 at 12:47 AM Theodore Ts'o wrote: > > > > > > More details about what is going on. First, it requires root, because > > > one of that is required is u

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Amir Goldstein
> Does the (untested, probably does not even build) patch shown below help? > This patch assumes that the kernel was built with CONFIG_PREEMPT=n. > And that I found all the tight loops on the do_sendfile() code path. > I *think* you have. FWIW, it would have been nicer for sendfile(2) and copy_fi

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Paul E. McKenney
On Fri, Jul 05, 2019 at 03:24:26PM +0200, Dmitry Vyukov wrote: > On Thu, Jun 27, 2019 at 12:47 AM Theodore Ts'o wrote: > > > > More details about what is going on. First, it requires root, because > > one of that is required is using sched_setattr (which is enough to > > shoot yourself in the foo

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Dmitry Vyukov
On Thu, Jun 27, 2019 at 12:47 AM Theodore Ts'o wrote: > > More details about what is going on. First, it requires root, because > one of that is required is using sched_setattr (which is enough to > shoot yourself in the foot): > > sched_setattr(0, {size=0, sched_policy=0x6 /* SCHED_??? */, sched

Re: INFO: rcu detected stall in ext4_write_checks

2019-07-05 Thread Dmitry Vyukov
On Wed, Jun 26, 2019 at 8:43 PM Theodore Ts'o wrote: > > On Wed, Jun 26, 2019 at 10:27:08AM -0700, syzbot wrote: > > Hello, > > > > syzbot found the following crash on: > > > > HEAD commit:abf02e29 Merge tag 'pm-5.2-rc6' of git://git.kernel.org/pu.. > > git tree: upstream > > console out

Re: INFO: rcu detected stall in ext4_write_checks

2019-06-26 Thread Theodore Ts'o
More details about what is going on. First, it requires root, because one of that is required is using sched_setattr (which is enough to shoot yourself in the foot): sched_setattr(0, {size=0, sched_policy=0x6 /* SCHED_??? */, sched_flags=0, sched_nice=0, sched_priority=0, sched_runtime=225179981

Re: INFO: rcu detected stall in ext4_write_checks

2019-06-26 Thread Theodore Ts'o
The reproducer causes similar rcu stalls when using xfs: RSP: 0018:aae8c0953c58 EFLAGS: 0246 ORIG_RAX: ff13 RAX: 0288 RBX: b05a RCX: aae8c0953d50 RDX: 001c RSI: 001c RDI: dcec41772800 RBP: dcec41772800 R08: 0015143

Re: INFO: rcu detected stall in ext4_write_checks

2019-06-26 Thread Theodore Ts'o
On Wed, Jun 26, 2019 at 10:27:08AM -0700, syzbot wrote: > Hello, > > syzbot found the following crash on: > > HEAD commit:abf02e29 Merge tag 'pm-5.2-rc6' of git://git.kernel.org/pu.. > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=1435aaf6a0 > kernel

INFO: rcu detected stall in ext4_write_checks

2019-06-26 Thread syzbot
Hello, syzbot found the following crash on: HEAD commit:abf02e29 Merge tag 'pm-5.2-rc6' of git://git.kernel.org/pu.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=1435aaf6a0 kernel config: https://syzkaller.appspot.com/x/.config?x=e5c77f8090a3b96b da