Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Jens Axboe
On 06/20/2017 06:56 AM, Christoph Hellwig wrote: > On Tue, Jun 20, 2017 at 06:51:34AM -0600, Jens Axboe wrote: >> That sounds fragile... Is it really worth it to avoid stealing three >> bits in the inode? Do we have buffer_heads attached everywhere, that >> sounds surprising to me. > > I'm not con

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Christoph Hellwig
On Tue, Jun 20, 2017 at 06:51:34AM -0600, Jens Axboe wrote: > That sounds fragile... Is it really worth it to avoid stealing three > bits in the inode? Do we have buffer_heads attached everywhere, that > sounds surprising to me. I'm not concerned about saving a few bits - we'll use those elsewhere

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Jens Axboe
On 06/20/2017 06:47 AM, Christoph Hellwig wrote: > On Tue, Jun 20, 2017 at 06:45:45AM -0600, Jens Axboe wrote: >> To the page? Where do you want to find room for that? > > In the page we don't. In the buffer_head we have plenty (as will > the replacement for it if I ever get time to get back to t

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Christoph Hellwig
On Tue, Jun 20, 2017 at 06:45:45AM -0600, Jens Axboe wrote: > To the page? Where do you want to find room for that? In the page we don't. In the buffer_head we have plenty (as will the replacement for it if I ever get time to get back to that)

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Jens Axboe
On 06/20/2017 06:43 AM, Christoph Hellwig wrote: > On Tue, Jun 20, 2017 at 06:43:10AM -0600, Jens Axboe wrote: >>> Question: IFF we can get the per-file hints to propagate to the >>> writeback code (which I think is pretty easy for ext4/xfs/bdev, not >>> sure about btrfs) do we even need the per-i

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Christoph Hellwig
On Tue, Jun 20, 2017 at 06:43:10AM -0600, Jens Axboe wrote: > > Question: IFF we can get the per-file hints to propagate to the > > writeback code (which I think is pretty easy for ext4/xfs/bdev, not > > sure about btrfs) do we even need the per-inode ones at all? > > How is that possible, the fi

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Jens Axboe
On 06/20/2017 02:57 AM, Christoph Hellwig wrote: > On Mon, Jun 19, 2017 at 02:33:41PM -0600, Jens Axboe wrote: >> That doesn't work, in case it's cleared, or for checking whether it has >> been set or not. Oh well, I added a NOT_SET variant for this. See below >> for an incremental that adds suppor

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-20 Thread Christoph Hellwig
On Mon, Jun 19, 2017 at 02:33:41PM -0600, Jens Axboe wrote: > That doesn't work, in case it's cleared, or for checking whether it has > been set or not. Oh well, I added a NOT_SET variant for this. See below > for an incremental that adds support for file write hints as well. Use > the file write h

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 02:33 PM, Jens Axboe wrote: > On 06/19/2017 01:10 PM, Jens Axboe wrote: >> On 06/19/2017 01:00 PM, Jens Axboe wrote: >>> On 06/19/2017 12:58 PM, Christoph Hellwig wrote: On Mon, Jun 19, 2017 at 10:02:09AM -0600, Jens Axboe wrote: > Actually, one good use case is O_DIRECT on a

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 01:10 PM, Jens Axboe wrote: > On 06/19/2017 01:00 PM, Jens Axboe wrote: >> On 06/19/2017 12:58 PM, Christoph Hellwig wrote: >>> On Mon, Jun 19, 2017 at 10:02:09AM -0600, Jens Axboe wrote: Actually, one good use case is O_DIRECT on a block device. Since I'm not a huge fan of

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 01:00 PM, Jens Axboe wrote: > On 06/19/2017 12:58 PM, Christoph Hellwig wrote: >> On Mon, Jun 19, 2017 at 10:02:09AM -0600, Jens Axboe wrote: >>> Actually, one good use case is O_DIRECT on a block device. Since I'm >>> not a huge fan of having per-call hints that is only useful for a

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 12:58 PM, Christoph Hellwig wrote: > On Mon, Jun 19, 2017 at 10:02:09AM -0600, Jens Axboe wrote: >> Actually, one good use case is O_DIRECT on a block device. Since I'm >> not a huge fan of having per-call hints that is only useful for a >> single case, how about we add the hints to t

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Christoph Hellwig
On Mon, Jun 19, 2017 at 10:02:09AM -0600, Jens Axboe wrote: > Actually, one good use case is O_DIRECT on a block device. Since I'm > not a huge fan of having per-call hints that is only useful for a > single case, how about we add the hints to the struct file as well? > For buffered IO, just grab i

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 08:56 AM, Jens Axboe wrote: > On 06/19/2017 12:27 AM, Christoph Hellwig wrote: >> On Sat, Jun 17, 2017 at 01:59:47PM -0600, Jens Axboe wrote: >>> Add four flags for the pwritev2(2) system call, allowing an application >>> to give the kernel a hint about what on-media life times can be

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-19 Thread Jens Axboe
On 06/19/2017 12:27 AM, Christoph Hellwig wrote: > On Sat, Jun 17, 2017 at 01:59:47PM -0600, Jens Axboe wrote: >> Add four flags for the pwritev2(2) system call, allowing an application >> to give the kernel a hint about what on-media life times can be >> expected from a given write. >> >> The inte

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-18 Thread Christoph Hellwig
On Sat, Jun 17, 2017 at 01:59:47PM -0600, Jens Axboe wrote: > Add four flags for the pwritev2(2) system call, allowing an application > to give the kernel a hint about what on-media life times can be > expected from a given write. > > The intent is for these values to be relative to each other, no

[PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-17 Thread Jens Axboe
Add four flags for the pwritev2(2) system call, allowing an application to give the kernel a hint about what on-media life times can be expected from a given write. The intent is for these values to be relative to each other, no absolute meaning should be attached to these flag names. Set aside 3

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 12:02 PM, Christoph Hellwig wrote: > On Fri, Jun 16, 2017 at 09:59:38AM -0600, Jens Axboe wrote: Did you see v5? It adds enum write_hint and passes it all the way down, until we transform them into rq/bio flags. >>> >>> Yes. But with all the way down I mean all the way down

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Christoph Hellwig
On Fri, Jun 16, 2017 at 09:59:38AM -0600, Jens Axboe wrote: > >> Did you see v5? It adds enum write_hint and passes it all the way down, > >> until we transform them into rq/bio flags. > > > > Yes. But with all the way down I mean all the way down to the driver :) > > Only missing part is the re

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Christoph Hellwig
On Fri, Jun 16, 2017 at 10:14:01AM -0600, Jens Axboe wrote: > So that would look like the below change on top of the current v5. I > skipped the callers, since those are all easy > s/bio_op_write_hint/write_hint_to_opf changes. That's better. But my idead was to actually go back to req_hints and

[PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
Add four flags for the pwritev2(2) system call, allowing an application to give the kernel a hint about what on-media life times can be expected from a given write. The intent is for these values to be relative to each other, no absolute meaning should be attached to these flag names. Set aside 3

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 09:59 AM, Jens Axboe wrote: > On 06/16/2017 09:52 AM, Christoph Hellwig wrote: >> On Fri, Jun 16, 2017 at 08:35:07AM -0600, Jens Axboe wrote: Agreed. In fact I'd go a little further: we should have a u16 hints; that goes all the way down from fcntl to the

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 09:52 AM, Christoph Hellwig wrote: > On Fri, Jun 16, 2017 at 08:35:07AM -0600, Jens Axboe wrote: >>> Agreed. In fact I'd go a little further: we should have a >>> >>> u16 hints; >>> >>> that goes all the way down from fcntl to the driver, right now >>> we'll allocate the first 3

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Christoph Hellwig
On Fri, Jun 16, 2017 at 08:35:07AM -0600, Jens Axboe wrote: > > Agreed. In fact I'd go a little further: we should have a > > > > u16 hints; > > > > that goes all the way down from fcntl to the driver, right now > > we'll allocate the first 3 bits for the write lifetime hints (2.5, > > so w

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 08:35 AM, Jens Axboe wrote: >> If I look at the mess for allocating the streams I think we need it >> to bubble down. That way the device can allocate the stream at time >> of the fcntl and we can keep the low level driver very simple. > > Mess? The NVMe code seems pretty straight f

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 01:30 AM, Christoph Hellwig wrote: > On Thu, Jun 15, 2017 at 09:23:02AM -0600, Jens Axboe wrote: >> And then I remembered why fadvise _sucks_. It returns the error value >> directly. So 0 is success > 0 is some error. That does not work well >> for adding a set/get interface. Addition

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Jens Axboe
On 06/16/2017 01:33 AM, Christoph Hellwig wrote: > On Thu, Jun 15, 2017 at 08:21:18AM -0600, Jens Axboe wrote: >> I'd still very much like to stick to the same four hints, and not > > Agreed. In fact I'd go a little further: we should have a > > u16 hints; > > that goes all the way down

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Christoph Hellwig
On Thu, Jun 15, 2017 at 08:21:18AM -0600, Jens Axboe wrote: > I'd still very much like to stick to the same four hints, and not Agreed. In fact I'd go a little further: we should have a u16 hints; that goes all the way down from fcntl to the driver, right now we'll allocate the first 3

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-16 Thread Christoph Hellwig
On Thu, Jun 15, 2017 at 09:23:02AM -0600, Jens Axboe wrote: > And then I remembered why fadvise _sucks_. It returns the error value > directly. So 0 is success > 0 is some error. That does not work well > for adding a set/get interface. Additionally, with fadvise, we have > to overload either 'offs

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-15 Thread Jens Axboe
On 06/15/2017 08:21 AM, Jens Axboe wrote: > On 06/15/2017 02:19 AM, Christoph Hellwig wrote: >> I think Darrick has a very valid concern here - using RWF_* flags >> to affect inode or fd-wide state is extremely counter productive. >> >> Combined with the fact that the streams need a special setup i

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-15 Thread Jens Axboe
On 06/15/2017 02:19 AM, Christoph Hellwig wrote: > I think Darrick has a very valid concern here - using RWF_* flags > to affect inode or fd-wide state is extremely counter productive. > > Combined with the fact that the streams need a special setup in NVMe > I'm tempted to say that the interface

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-15 Thread Al Viro
On Wed, Jun 14, 2017 at 09:15:03PM -0700, Darrick J. Wong wrote: > > + */ > > +#define RWF_WRITE_LIFE_SHIFT 4 > > +#define RWF_WRITE_LIFE_MASK0x00f0 /* 4 bits of stream > > ID */ > > +#define RWF_WRITE_LIFE_SHORT (1 << RWF_WRITE_LIFE_SHIFT) > > +#def

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-15 Thread Christoph Hellwig
I think Darrick has a very valid concern here - using RWF_* flags to affect inode or fd-wide state is extremely counter productive. Combined with the fact that the streams need a special setup in NVMe I'm tempted to say that the interface really should be fadvise or similar, which would keep the s

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Jens Axboe
On 06/14/2017 10:15 PM, Darrick J. Wong wrote: >> diff --git a/fs/read_write.c b/fs/read_write.c >> index 47c1d4484df9..9cb2314efca3 100644 >> --- a/fs/read_write.c >> +++ b/fs/read_write.c >> @@ -678,7 +678,7 @@ static ssize_t do_iter_readv_writev(struct file *filp, >> struct iov_iter *iter, >>

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Darrick J. Wong
On Wed, Jun 14, 2017 at 09:45:05PM -0600, Jens Axboe wrote: > Add four flags for the pwritev2(2) system call, allowing an application > to give the kernel a hint about what on-media life times can be > expected from a given write. > > The intent is for these values to be relative to each other, no

[PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Jens Axboe
Add four flags for the pwritev2(2) system call, allowing an application to give the kernel a hint about what on-media life times can be expected from a given write. The intent is for these values to be relative to each other, no absolute meaning should be attached to these flag names. Define IOCB

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Jens Axboe
On 06/14/2017 02:26 PM, Christoph Hellwig wrote: > On Wed, Jun 14, 2017 at 01:05:27PM -0600, Jens Axboe wrote: >> Add four flags for the pwritev2(2) system call, allowing an application >> to give the kernel a hint about what on-media life times can be >> expected from a given write. >> >> The inte

Re: [PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Christoph Hellwig
On Wed, Jun 14, 2017 at 01:05:27PM -0600, Jens Axboe wrote: > Add four flags for the pwritev2(2) system call, allowing an application > to give the kernel a hint about what on-media life times can be > expected from a given write. > > The intent is for these values to be relative to each other, no

[PATCH 04/11] fs: add support for allowing applications to pass in write life time hints

2017-06-14 Thread Jens Axboe
Add four flags for the pwritev2(2) system call, allowing an application to give the kernel a hint about what on-media life times can be expected from a given write. The intent is for these values to be relative to each other, no absolute meaning should be attached to these flag names. Define IOCB