On Wed, 26 Sep 2018 17:49:09 -0400
"Theodore Y. Ts'o" wrote:
> On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> > In almost all cases you don't care so you wouldn't use it. In those cases
> > where it might matter it's almost always the case that a reader won't
> > consume it before it
On Thu, Sep 27, 2018 at 08:43:10AM -0400, Jeff Layton wrote:
>
> Basically, the problem (as I see it) is that we can end up evicting
> uncleanable data from the cache before you have a chance to call fsync,
> and that means that the results of a read after a write are not
> completely reliable.
P
On Tue, 2018-09-25 at 18:30 -0400, Theodore Y. Ts'o wrote:
> On Tue, Sep 25, 2018 at 12:41:18PM -0400, Jeff Layton wrote:
> > That's all well and good, but still doesn't quite solve the main concern
> > with all of this. It's suppose we have this series of events:
> >
> > open file r/w
> > write 1
On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> > And I think that's fine. The only way we can make any guarantees is
> > if we do what Alan suggested, which is to imply that a read on a dirty
> > page *block* until the the page is successfully written back. This
> > would destroy per
On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> In almost all cases you don't care so you wouldn't use it. In those cases
> where it might matter it's almost always the case that a reader won't
> consume it before it hits the media.
>
> That's why I suggested having an fbarrier() so yo
> And I think that's fine. The only way we can make any guarantees is
> if we do what Alan suggested, which is to imply that a read on a dirty
> page *block* until the the page is successfully written back. This
> would destroy performance.
In almost all cases you don't care so you wouldn't use
On Tue, Sep 25, 2018 at 07:35:11PM +0200, Adam Borowski wrote:
> Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:
>
> char path[PATH_MAX];
> fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
> S_IRUSR | S_IWU
On Tue, Sep 25, 2018 at 12:41:18PM -0400, Jeff Layton wrote:
> That's all well and good, but still doesn't quite solve the main concern
> with all of this. It's suppose we have this series of events:
>
> open file r/w
> write 1024 bytes to offset 0
>
> read 1024 bytes from offset 0
>
> Open, wri
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S. One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a
On Tue, 2018-09-25 at 11:46 -0400, Theodore Y. Ts'o wrote:
> On Tue, Sep 25, 2018 at 07:15:34AM -0400, Jeff Layton wrote:
> > Linux has dozens of filesystems and they all behave differently in this
> > regard. A catastrophic failure (paradoxically) makes things simpler for
> > the fs developer, but
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.
Would it. If you open a file unlink it and write to it and then have a
l
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> (Especially since you can get most of the functionality by
> using some naming convention for files that in the process of being
> written, and then teach some program that is regularly scanning the
> entire file system, such as up
On Tue, Sep 25, 2018 at 07:15:34AM -0400, Jeff Layton wrote:
> Linux has dozens of filesystems and they all behave differently in this
> regard. A catastrophic failure (paradoxically) makes things simpler for
> the fs developer, but even on local filesystems isolated errors can
> occur. It's also n
On Tue, 2018-09-25 at 00:30 +0100, Alan Cox wrote:
> > write()
> > kernel attempts to write back page and fails
> > page is marked clean and evicted from the cache
> > read()
> >
> > Now your write is gone and there were no calls between the write and
> > read.
> >
> > The question we still need
> write()
> kernel attempts to write back page and fails
> page is marked clean and evicted from the cache
> read()
>
> Now your write is gone and there were no calls between the write and
> read.
>
> The question we still need to answer is this:
>
> When we attempt to write back some data from
> The other thing which you seem to be assuming is that applications
> which care about precious data won't use fsync(2). And in general,
> it's been fairly well known for decades that if you care about your
> data, you have to use fsync(2) or O_DIRECT writes; and you *must*
> check the error retu
On Thu, 6 Sep 2018 11:17:18 +0200
Rogier Wolff wrote:
> On Thu, Sep 06, 2018 at 12:57:09PM +1000, Dave Chinner wrote:
> > On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
>
> > > And this has worked for years because
> > > the kernel caches stuff from inodes and data-blocks. If yo
On Thu, Sep 06, 2018 at 12:57:09PM +1000, Dave Chinner wrote:
> On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
> > And this has worked for years because
> > the kernel caches stuff from inodes and data-blocks. If you suddenly
> > write stuff to harddisk at 10ms for each seek between
On Wed, Sep 5, 2018 at 4:09 PM 焦晓冬 wrote:
>
> On Tue, Sep 4, 2018 at 11:44 PM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > > >
> > > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > > On Tue, Sep 4, 2
On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
> On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> > There is no requirement for a filesystem to flush data on close().
>
> And you can't start doing things like that.
Of course we can. And we do.
We've been doing target
On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> There is no requirement for a filesystem to flush data on close(). In
> fact, most local filesystems do not. NFS does, but that's because it has
> to in order to provide close-to-open cache consistency semantics.
And these days even NF
On Wed, Sep 05, 2018 at 04:09:42PM +0800, 焦晓冬 wrote:
> Well, since the reader application and the writer application are reading
> a same file, they are indeed related. The reader here is expecting
> to read the lasted data the writer offers, not any data available. The
> reader is surely not expec
On Wed, Sep 05, 2018 at 08:07:25AM -0400, Austin S. Hemmelgarn wrote:
> On 2018-09-05 04:37, 焦晓冬 wrote:
> >At this point, the /bin/sh may be partially old and partially new. Execute
> >this corrupted bin is also dangerous though.
> But the system may still be usable in that state, while returning
On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> There is no requirement for a filesystem to flush data on close().
And you can't start doing things like that. In some weird cases, you
might have an application open-write-close files at a much higher rate
than what a harddisk can han
On 2018-09-05 04:37, 焦晓冬 wrote:
On Wed, Sep 5, 2018 at 4:04 PM Rogier Wolff wrote:
On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
Rogier Wolff - 05.09.18, 09:08:
So when a mail queuer puts mail the mailq files and the mail processor
can get them out of there intact, nobo
On Wed, 2018-09-05 at 09:37 +0200, Martin Steigerwald wrote:
> Jeff Layton - 04.09.18, 17:44:
> > > - If the following read() could be served by a page in memory, just
> > > returns the data. If the following read() could not be served by a
> > > page in memory and the inode/address_space has a wri
On Wed, 2018-09-05 at 16:24 +0800, 焦晓冬 wrote:
> On Wed, Sep 5, 2018 at 4:18 AM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields
Rogier Wolff - 05.09.18, 10:04:
> On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> > Rogier Wolff - 05.09.18, 09:08:
> > > So when a mail queuer puts mail the mailq files and the mail
> > > processor can get them out of there intact, nobody is going to
> > > notice. (I know ma
On Wed, Sep 5, 2018 at 4:04 PM Rogier Wolff wrote:
>
> On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> > Rogier Wolff - 05.09.18, 09:08:
> > > So when a mail queuer puts mail the mailq files and the mail processor
> > > can get them out of there intact, nobody is going to not
On Wed, Sep 5, 2018 at 4:18 AM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > > Well, I think the point was that in the a
On Tue, Sep 4, 2018 at 11:44 PM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > >
> > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff
> > > > wrote:
> > > >
> > >
On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> Rogier Wolff - 05.09.18, 09:08:
> > So when a mail queuer puts mail the mailq files and the mail processor
> > can get them out of there intact, nobody is going to notice. (I know
> > mail queuers should call fsync and report er
Rogier Wolff - 05.09.18, 09:08:
> So when a mail queuer puts mail the mailq files and the mail processor
> can get them out of there intact, nobody is going to notice. (I know
> mail queuers should call fsync and report errors when that fails, but
> there are bound to be applications where calling
Jeff Layton - 04.09.18, 17:44:
> > - If the following read() could be served by a page in memory, just
> > returns the data. If the following read() could not be served by a
> > page in memory and the inode/address_space has a writeback error
> > mark, returns EIO. If there is a writeback error on
On Tue, Sep 04, 2018 at 11:44:20AM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > >
> > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > That is certainly not possible to be done. But at least, shall we r
On Tue, Sep 04, 2018 at 01:35:34PM -0700, Vito Caputo wrote:
> On Tue, Sep 04, 2018 at 04:18:18PM -0400, Jeff Layton wrote:
> > On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > > On Tue, Sep 04, 2018 at 12:12:03PM -0
On Tue, Sep 04, 2018 at 01:35:34PM -0700, Vito Caputo wrote:
> Implement a new class of swap space for backing dirty pages which fail
> to write back. Pages in this space survive reboots, essentially backing
> the implicit commitment POSIX establishes in the face of asynchronous
> writeback errors
On Tue, Sep 04, 2018 at 04:18:18PM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > > Well, I think the point was that
On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > Well, I think the point was that in the above examples you'd prefer that
> > > the read just fail--no
On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > Well, I think the point was that in the above examples you'd prefer that
> > the read just fail--no need to keep the data. A bit marking the file
> > (or even the en
On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
>
> Well, I think the point was that in the above examples you'd prefer that
> the read just fail--no need to keep the data. A bit marking the file
> (or even the entire filesystem) unreadable would satisfy posix, I guess.
> Whethe
On Tue, Sep 04, 2018 at 11:44:20AM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > A practical and concrete example may be,
> > A disk cleaner program that first searches for garbage files that won't be
> > used
> > anymore and save the list in a file (open()-write()-
On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff
> > > wrote:
> > >
> > > ...
> > > > >
> > > > > Jlayton's patch is simple but wonderful i
On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
> >
> > ...
> > > >
> > > > Jlayton's patch is simple but wonderful idea towards correct error
> > > > reporting. It seems one crucial thing
On Tue, 2018-09-04 at 16:58 +0800, 焦晓冬 wrote:
> On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
>
> ...
> > >
> > > Jlayton's patch is simple but wonderful idea towards correct error
> > > reporting. It seems one crucial thing is still here to be fixed. Does
> > > anyone have some idea?
> > >
On Tue, 2018-09-04 at 13:42 +0800, 焦晓冬 wrote:
> Hi,
>
> After reading several writeback error handling articles from LWN, I
> begin to be upset about writeback error handling.
>
> Jlayton's patch is simple but wonderful idea towards correct error
> reporting. It seems one crucial thing is still h
On Tue, Sep 4, 2018 at 5:29 PM Rogier Wolff wrote:
>
> On Tue, Sep 04, 2018 at 04:58:59PM +0800, 焦晓冬 wrote:
>
> > As for suggestion, maybe the error flag of inode/mapping, or the entire
> > inode
> > should not be evicted if there was an error. That hopefully won't take much
> > memory. On extrem
On Tue, Sep 04, 2018 at 04:58:59PM +0800, 焦晓冬 wrote:
> As for suggestion, maybe the error flag of inode/mapping, or the entire inode
> should not be evicted if there was an error. That hopefully won't take much
> memory. On extreme conditions, where too much error inode requires staying
> in memor
On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
...
> >
> > Jlayton's patch is simple but wonderful idea towards correct error
> > reporting. It seems one crucial thing is still here to be fixed. Does
> > anyone have some idea?
> >
> > The crucial thing may be that a read() after a successful
On Tue, Sep 04, 2018 at 02:32:28PM +0800, 焦晓冬 wrote:
> Hi,
>
> After reading several writeback error handling articles from LWN, I
> begin to be upset about writeback error handling.
>
> Jlayton's patch is simple but wonderful idea towards correct error
> reporting. It seems one crucial thing is
Hi,
After reading several writeback error handling articles from LWN, I
begin to be upset about writeback error handling.
Jlayton's patch is simple but wonderful idea towards correct error
reporting. It seems one crucial thing is still here to be fixed. Does
anyone have some idea?
The crucial th
51 matches
Mail list logo
