Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-25 Thread Dave Chinner
On Wed, Sep 25, 2019 at 11:15:30AM +0300, Konstantin Khlebnikov wrote: > On 25/09/2019 10.18, Dave Chinner wrote: > > On Tue, Sep 24, 2019 at 12:00:17PM +0300, Konstantin Khlebnikov wrote: > > > On 24/09/2019 10.39, Dave Chinner wrote: > > > > On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Kh

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-25 Thread Theodore Y. Ts'o
On Wed, Sep 25, 2019 at 05:18:54PM +1000, Dave Chinner wrote: > > > ANd, really such strict writebehind behaviour is going to cause all > > > sorts of unintended problesm with filesystems because there will be > > > adverse interactions with delayed allocation. We need a substantial > > > amount of

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-25 Thread Konstantin Khlebnikov
On 25/09/2019 10.18, Dave Chinner wrote: On Tue, Sep 24, 2019 at 12:00:17PM +0300, Konstantin Khlebnikov wrote: On 24/09/2019 10.39, Dave Chinner wrote: On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Khlebnikov wrote: On 23/09/2019 17.52, Tejun Heo wrote: Hello, Konstantin. On Fri, Sep

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-25 Thread Dave Chinner
On Tue, Sep 24, 2019 at 12:08:04PM -0700, Linus Torvalds wrote: > On Tue, Sep 24, 2019 at 12:39 AM Dave Chinner wrote: > > > > Stupid question: how is this any different to simply winding down > > our dirty writeback and throttling thresholds like so: > > > > # echo $((100 * 1000 * 1000)) > /proc/

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-25 Thread Dave Chinner
On Tue, Sep 24, 2019 at 12:00:17PM +0300, Konstantin Khlebnikov wrote: > On 24/09/2019 10.39, Dave Chinner wrote: > > On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Khlebnikov wrote: > > > On 23/09/2019 17.52, Tejun Heo wrote: > > > > Hello, Konstantin. > > > > > > > > On Fri, Sep 20, 2019 a

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-24 Thread Linus Torvalds
On Tue, Sep 24, 2019 at 12:39 AM Dave Chinner wrote: > > Stupid question: how is this any different to simply winding down > our dirty writeback and throttling thresholds like so: > > # echo $((100 * 1000 * 1000)) > /proc/sys/vm/dirty_background_bytes Our dirty_background stuff is very questionab

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-24 Thread Konstantin Khlebnikov
On 21/09/2019 02.05, Linus Torvalds wrote: On Fri, Sep 20, 2019 at 12:35 AM Konstantin Khlebnikov wrote: This patch implements write-behind policy which tracks sequential writes and starts background writeback when file have enough dirty pages. Apart from a spelling error ("contigious"), my

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-24 Thread Konstantin Khlebnikov
On 24/09/2019 10.39, Dave Chinner wrote: On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Khlebnikov wrote: On 23/09/2019 17.52, Tejun Heo wrote: Hello, Konstantin. On Fri, Sep 20, 2019 at 10:39:33AM +0300, Konstantin Khlebnikov wrote: With vm.dirty_write_behind 1 or 2 files are written e

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-24 Thread Dave Chinner
On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Khlebnikov wrote: > On 23/09/2019 17.52, Tejun Heo wrote: > > Hello, Konstantin. > > > > On Fri, Sep 20, 2019 at 10:39:33AM +0300, Konstantin Khlebnikov wrote: > > > With vm.dirty_write_behind 1 or 2 files are written even faster and > > > > Is

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-23 Thread Konstantin Khlebnikov
On 23/09/2019 18.36, Jens Axboe wrote: On 9/20/19 5:10 PM, Linus Torvalds wrote: On Fri, Sep 20, 2019 at 4:05 PM Linus Torvalds wrote: Now, I hear you say "those are so small these days that it doesn't matter". And maybe you're right. But particularly for slow media, triggering good streamin

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-23 Thread Jens Axboe
On 9/20/19 5:10 PM, Linus Torvalds wrote: > On Fri, Sep 20, 2019 at 4:05 PM Linus Torvalds > wrote: >> >> >> Now, I hear you say "those are so small these days that it doesn't >> matter". And maybe you're right. But particularly for slow media, >> triggering good streaming write behavior has been

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-23 Thread Tejun Heo
Hello, On Mon, Sep 23, 2019 at 06:06:46PM +0300, Konstantin Khlebnikov wrote: > On 23/09/2019 17.52, Tejun Heo wrote: > >Hello, Konstantin. > > > >On Fri, Sep 20, 2019 at 10:39:33AM +0300, Konstantin Khlebnikov wrote: > >>With vm.dirty_write_behind 1 or 2 files are written even faster and > > > >I

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-23 Thread Konstantin Khlebnikov
On 23/09/2019 17.52, Tejun Heo wrote: Hello, Konstantin. On Fri, Sep 20, 2019 at 10:39:33AM +0300, Konstantin Khlebnikov wrote: With vm.dirty_write_behind 1 or 2 files are written even faster and Is the faster speed reproducible? I don't quite understand why this would be. Writing to disk

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-23 Thread Tejun Heo
Hello, Konstantin. On Fri, Sep 20, 2019 at 10:39:33AM +0300, Konstantin Khlebnikov wrote: > With vm.dirty_write_behind 1 or 2 files are written even faster and Is the faster speed reproducible? I don't quite understand why this would be. > during copying amount of dirty memory always stays arou

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-22 Thread kbuild test robot
Hi Konstantin, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on linus/master] [cannot apply to v5.3 next-20190919] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to sp

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-20 Thread Linus Torvalds
On Fri, Sep 20, 2019 at 4:05 PM Linus Torvalds wrote: > > > Now, I hear you say "those are so small these days that it doesn't > matter". And maybe you're right. But particularly for slow media, > triggering good streaming write behavior has been a problem in the > past. Which reminds me: the wri

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-20 Thread Linus Torvalds
On Fri, Sep 20, 2019 at 12:35 AM Konstantin Khlebnikov wrote: > > This patch implements write-behind policy which tracks sequential writes > and starts background writeback when file have enough dirty pages. Apart from a spelling error ("contigious"), my only reaction is that I've wanted this for

Re: [PATCH v2] mm: implement write-behind policy for sequential file writes

2019-09-20 Thread Konstantin Khlebnikov
Script for trivial demo in attachment $ bash test_writebehind.sh SIZE 3,2Gdummy vm.dirty_write_behind = 0 COPY real0m3.629s user0m0.016s sys 0m3.613s Dirty: 3254552 kB SYNC real0m31.953s user0m0.002s sys 0m0.000s vm.dirty_write_behind = 1 COPY real0m32