Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Dave Chinner
On Mon, Aug 24, 2015 at 05:45:35PM -0400, Tejun Heo wrote: > Hello, > > On Mon, Aug 24, 2015 at 11:09:27PM +0200, Jan Kara wrote: > > It is inefficient, yes. But note that 'writeback' and 'dirty' states are > > completely independent. Page can be in any of the !dirty & !writeback, > > That isn't

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, On Mon, Aug 24, 2015 at 05:45:35PM -0400, Tejun Heo wrote: > That isn't true for pages being dirtied through set_page_dirty(). > It's guaranteed that a dirty inode remains on one of the b_* lists > till there's no dirty page and writeback is complete. I got confused here. inodes get remov

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, Dave. On Tue, Aug 25, 2015 at 08:27:20AM +1000, Dave Chinner wrote: > > I'm still a bit confused. What prevents the following from happening? > > > > 1. io completion of last dirty page of an inode and work item for > >xfs_setfilesize() is queued. > > > > 2. inode removed from dirty

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Dave Chinner
On Mon, Aug 24, 2015 at 02:10:38PM -0400, Tejun Heo wrote: > Hello, Dave. > > On Fri, Aug 21, 2015 at 09:04:51AM +1000, Dave Chinner wrote: > > > Maybe I'm misunderstanding the code but all xfs_writepage() calls are > > > from unbound workqueues - the writeback workers - while > > > xfs_setfilesiz

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, On Mon, Aug 24, 2015 at 11:09:27PM +0200, Jan Kara wrote: > It is inefficient, yes. But note that 'writeback' and 'dirty' states are > completely independent. Page can be in any of the !dirty & !writeback, That isn't true for pages being dirtied through set_page_dirty(). It's guaranteed th

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Jan Kara
On Mon 24-08-15 15:32:42, Tejun Heo wrote: > Hello, Jan. > > On Mon, Aug 24, 2015 at 09:08:47PM +0200, Jan Kara wrote: > > Inode may contain writeback pages (but not dirty pages) without being on > > any of the dirty lists. That is correct. Josef Bacik had patches to create > > Hmmm... Can you pl

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, Jan. On Mon, Aug 24, 2015 at 09:08:47PM +0200, Jan Kara wrote: > Inode may contain writeback pages (but not dirty pages) without being on > any of the dirty lists. That is correct. Josef Bacik had patches to create Hmmm... Can you please expand on how / why that happens? It's kinda weird

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Jan Kara
On Mon 24-08-15 13:11:44, Tejun Heo wrote: > Hello, > > On Mon, Aug 24, 2015 at 10:51:50AM -0400, Tejun Heo wrote: > > > Bah, I see the problem and indeed it was introduced by commit > > > e79729123f639 > > > "writeback: don't issue wb_writeback_work if clean". The problem is that > > > we bail o

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, Dave. On Fri, Aug 21, 2015 at 09:04:51AM +1000, Dave Chinner wrote: > > Maybe I'm misunderstanding the code but all xfs_writepage() calls are > > from unbound workqueues - the writeback workers - while > > xfs_setfilesize() are from bound workqueues, so I wondered why that > > was and looke

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, On Mon, Aug 24, 2015 at 10:51:50AM -0400, Tejun Heo wrote: > > Bah, I see the problem and indeed it was introduced by commit e79729123f639 > > "writeback: don't issue wb_writeback_work if clean". The problem is that > > we bail out of sync_inodes_sb() if there is no dirty IO. Which is wrong

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Tejun Heo
Hello, Jan. On Mon, Aug 24, 2015 at 11:19:59AM +0200, Jan Kara wrote: > > which shows unmount being the next writeback event queued and > > executed after the IO completions have come in (that missed the > > log). What is missing is the specific queue/exec events for > > sync_sb_inodes() from the

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Jan Kara
On Mon 24-08-15 16:24:25, Dave Chinner wrote: > On Mon, Aug 24, 2015 at 11:18:16AM +0800, Eryu Guan wrote: > > On Mon, Aug 24, 2015 at 11:11:23AM +1000, Dave Chinner wrote: > > > > > > Eryu, can you change the way you run the event trace to be: > > > > > > $ sudo trace-cmd -o ./check > > > >

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Dave Chinner
On Mon, Aug 24, 2015 at 04:34:37PM +0800, Eryu Guan wrote: > On Mon, Aug 24, 2015 at 04:24:25PM +1000, Dave Chinner wrote: > > On Mon, Aug 24, 2015 at 11:18:16AM +0800, Eryu Guan wrote: > > > On Mon, Aug 24, 2015 at 11:11:23AM +1000, Dave Chinner wrote: > > > > > > > > Eryu, can you change the way

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-24 Thread Eryu Guan
On Mon, Aug 24, 2015 at 04:24:25PM +1000, Dave Chinner wrote: > On Mon, Aug 24, 2015 at 11:18:16AM +0800, Eryu Guan wrote: > > On Mon, Aug 24, 2015 at 11:11:23AM +1000, Dave Chinner wrote: > > > > > > Eryu, can you change the way you run the event trace to be: > > > > > > $ sudo trace-cmd -o ./

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-23 Thread Dave Chinner
On Mon, Aug 24, 2015 at 11:18:16AM +0800, Eryu Guan wrote: > On Mon, Aug 24, 2015 at 11:11:23AM +1000, Dave Chinner wrote: > > > > Eryu, can you change the way you run the event trace to be: > > > > $ sudo trace-cmd -o ./check > > > > rather than running the trace as a background operation el

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-23 Thread Eryu Guan
On Mon, Aug 24, 2015 at 11:11:23AM +1000, Dave Chinner wrote: > > Eryu, can you change the way you run the event trace to be: > > $ sudo trace-cmd -o ./check > > rather than running the trace as a background operation elsewhere? > Maybe that will give better results. The results are here ht

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-23 Thread Dave Chinner
On Sat, Aug 22, 2015 at 12:46:09PM +0800, Eryu Guan wrote: > On Sat, Aug 22, 2015 at 10:30:25AM +1000, Dave Chinner wrote: > > On Fri, Aug 21, 2015 at 06:20:53PM +0800, Eryu Guan wrote: > [snip] > > > > Eryu, can you try again, this time manually specifying the writeback > > tracepoints so you exc

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-21 Thread Eryu Guan
On Sat, Aug 22, 2015 at 10:30:25AM +1000, Dave Chinner wrote: > On Fri, Aug 21, 2015 at 06:20:53PM +0800, Eryu Guan wrote: [snip] > > Eryu, can you try again, this time manually specifying the writeback > tracepoints so you exclude the really noisy ones? You can also drop > the xfs_file_buffered_w

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-21 Thread Dave Chinner
On Fri, Aug 21, 2015 at 06:20:53PM +0800, Eryu Guan wrote: > On Wed, Aug 19, 2015 at 07:56:11AM +1000, Dave Chinner wrote: > > On Tue, Aug 18, 2015 at 12:54:39PM -0700, Tejun Heo wrote: > [snip] > > > > I'd suggest looking at some of the XFS tracepoints during the test: > > > > tracepoint

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-21 Thread Eryu Guan
On Wed, Aug 19, 2015 at 07:56:11AM +1000, Dave Chinner wrote: > On Tue, Aug 18, 2015 at 12:54:39PM -0700, Tejun Heo wrote: [snip] > > I'd suggest looking at some of the XFS tracepoints during the test: > > tracepointtrigger > xfs_file_buffered_write once per writ

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-20 Thread Dave Chinner
On Thu, Aug 20, 2015 at 09:55:37AM -0700, Tejun Heo wrote: > Hello, Eryu. Thanks a lot for the trace. > > So, this is from the end of the trace from the failed test. > > ... > kworker/u8:1-1563 [002] 22016.987530: xfs_writepage:dev 253:6 > ino 0xef64fe pgoff 0x9ff000 size 0xa0

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-20 Thread Tejun Heo
Hello, Eryu. Thanks a lot for the trace. So, this is from the end of the trace from the failed test. ... kworker/u8:1-1563 [002] 22016.987530: xfs_writepage:dev 253:6 ino 0xef64fe pgoff 0x9ff000 size 0xa0 offset 0 length 0 delalloc 1 unwritten 0 kworker/2:1-49[002] 220

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-20 Thread Eryu Guan
On Thu, Aug 20, 2015 at 02:12:24PM +0800, Eryu Guan wrote: > On Wed, Aug 19, 2015 at 07:56:11AM +1000, Dave Chinner wrote: > > On Tue, Aug 18, 2015 at 12:54:39PM -0700, Tejun Heo wrote: > > > Hello, > > > > > > On Tue, Aug 18, 2015 at 10:47:18AM -0700, Tejun Heo wrote: > > > > Hmm... the only poss

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-19 Thread Eryu Guan
On Wed, Aug 19, 2015 at 07:56:11AM +1000, Dave Chinner wrote: > On Tue, Aug 18, 2015 at 12:54:39PM -0700, Tejun Heo wrote: > > Hello, > > > > On Tue, Aug 18, 2015 at 10:47:18AM -0700, Tejun Heo wrote: > > > Hmm... the only possibility I can think of is tot_write_bandwidth > > > being zero when it

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-18 Thread Dave Chinner
On Tue, Aug 18, 2015 at 12:54:39PM -0700, Tejun Heo wrote: > Hello, > > On Tue, Aug 18, 2015 at 10:47:18AM -0700, Tejun Heo wrote: > > Hmm... the only possibility I can think of is tot_write_bandwidth > > being zero when it shouldn't be. I've been staring at the code for a > > while now but nothi

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-18 Thread Tejun Heo
Hello, On Tue, Aug 18, 2015 at 10:47:18AM -0700, Tejun Heo wrote: > Hmm... the only possibility I can think of is tot_write_bandwidth > being zero when it shouldn't be. I've been staring at the code for a > while now but nothing rings a bell. Time for another debug patch, I > guess. So, I can n

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-18 Thread Tejun Heo
On Tue, Aug 18, 2015 at 11:16:03AM +0200, Jan Kara wrote: > On Mon 17-08-15 16:02:54, Tejun Heo wrote: > > Hello, Jan. > > > > On Fri, Aug 14, 2015 at 01:14:09PM +0200, Jan Kara wrote: > > > So the patch looks good to me. But the fact that is fixes Eryu's problem > > > means there is something fis

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-18 Thread Jan Kara
On Mon 17-08-15 16:02:54, Tejun Heo wrote: > Hello, Jan. > > On Fri, Aug 14, 2015 at 01:14:09PM +0200, Jan Kara wrote: > > So the patch looks good to me. But the fact that is fixes Eryu's problem > > means there is something fishy going on. Either inodes get wrongly attached > > Seriously, it sho

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-17 Thread Damien Wyart
> > I had an unstable system when running latest Linus tree with Tejun's > > patch applied on top. Nothing fishy in the logs after rebooting without > > the patch, but remote access with ssh when patch applied did not work > > (as if /home partition could not be read). This system has / as ext4 and

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-17 Thread Tejun Heo
Hello, Jan. On Fri, Aug 14, 2015 at 01:14:09PM +0200, Jan Kara wrote: > So the patch looks good to me. But the fact that is fixes Eryu's problem > means there is something fishy going on. Either inodes get wrongly attached Seriously, it shouldn't affect size syncing or xfs but then again my under

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-17 Thread Tejun Heo
Hello, Damien. On Fri, Aug 14, 2015 at 05:14:01PM +0200, Damien Wyart wrote: > I had an unstable system when running latest Linus tree with Tejun's > patch applied on top. Nothing fishy in the logs after rebooting without > the patch, but remote access with ssh when patch applied did not work > (a

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-14 Thread Damien Wyart
> On Thu 13-08-15 18:44:15, Tejun Heo wrote: > > e79729123f63 ("writeback: don't issue wb_writeback_work if clean") > > updated writeback path to avoid kicking writeback work items if there > > are no inodes to be written out; unfortunately, the avoidance logic > > was too aggressive and made sync_

Re: [PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-14 Thread Jan Kara
Hello, On Thu 13-08-15 18:44:15, Tejun Heo wrote: > e79729123f63 ("writeback: don't issue wb_writeback_work if clean") > updated writeback path to avoid kicking writeback work items if there > are no inodes to be written out; unfortunately, the avoidance logic > was too aggressive and made sync_

[PATCH block/for-linus] writeback: fix syncing of I_DIRTY_TIME inodes

2015-08-13 Thread Tejun Heo
e79729123f63 ("writeback: don't issue wb_writeback_work if clean") updated writeback path to avoid kicking writeback work items if there are no inodes to be written out; unfortunately, the avoidance logic was too aggressive and made sync_inodes_sb() skip I_DIRTY_TIME inodes. This patch fixes the br