On Wed, 26 Sep 2018 17:49:09 -0400
"Theodore Y. Ts'o" wrote:
> On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> > In almost all cases you don't care so you wouldn't use it. In those cases
> > where it might matter it's almost always the case that a reader won't
> > consume it before it
On Thu, Sep 27, 2018 at 08:43:10AM -0400, Jeff Layton wrote:
>
> Basically, the problem (as I see it) is that we can end up evicting
> uncleanable data from the cache before you have a chance to call fsync,
> and that means that the results of a read after a write are not
> completely reliable.
P
On Tue, 2018-09-25 at 18:30 -0400, Theodore Y. Ts'o wrote:
> On Tue, Sep 25, 2018 at 12:41:18PM -0400, Jeff Layton wrote:
> > That's all well and good, but still doesn't quite solve the main concern
> > with all of this. It's suppose we have this series of events:
> >
> > open file r/w
> > write 1
On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> > And I think that's fine. The only way we can make any guarantees is
> > if we do what Alan suggested, which is to imply that a read on a dirty
> > page *block* until the the page is successfully written back. This
> > would destroy per
On Wed, Sep 26, 2018 at 07:10:55PM +0100, Alan Cox wrote:
> In almost all cases you don't care so you wouldn't use it. In those cases
> where it might matter it's almost always the case that a reader won't
> consume it before it hits the media.
>
> That's why I suggested having an fbarrier() so yo
> And I think that's fine. The only way we can make any guarantees is
> if we do what Alan suggested, which is to imply that a read on a dirty
> page *block* until the the page is successfully written back. This
> would destroy performance.
In almost all cases you don't care so you wouldn't use
On Tue, Sep 25, 2018 at 07:35:11PM +0200, Adam Borowski wrote:
> Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:
>
> char path[PATH_MAX];
> fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
> S_IRUSR | S_IWU
On Tue, Sep 25, 2018 at 12:41:18PM -0400, Jeff Layton wrote:
> That's all well and good, but still doesn't quite solve the main concern
> with all of this. It's suppose we have this series of events:
>
> open file r/w
> write 1024 bytes to offset 0
>
> read 1024 bytes from offset 0
>
> Open, wri
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S. One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a
On Tue, 2018-09-25 at 11:46 -0400, Theodore Y. Ts'o wrote:
> On Tue, Sep 25, 2018 at 07:15:34AM -0400, Jeff Layton wrote:
> > Linux has dozens of filesystems and they all behave differently in this
> > regard. A catastrophic failure (paradoxically) makes things simpler for
> > the fs developer, but
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.
Would it. If you open a file unlink it and write to it and then have a
l
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> (Especially since you can get most of the functionality by
> using some naming convention for files that in the process of being
> written, and then teach some program that is regularly scanning the
> entire file system, such as up
On Tue, Sep 25, 2018 at 07:15:34AM -0400, Jeff Layton wrote:
> Linux has dozens of filesystems and they all behave differently in this
> regard. A catastrophic failure (paradoxically) makes things simpler for
> the fs developer, but even on local filesystems isolated errors can
> occur. It's also n
On Tue, 2018-09-25 at 00:30 +0100, Alan Cox wrote:
> > write()
> > kernel attempts to write back page and fails
> > page is marked clean and evicted from the cache
> > read()
> >
> > Now your write is gone and there were no calls between the write and
> > read.
> >
> > The question we still need
> write()
> kernel attempts to write back page and fails
> page is marked clean and evicted from the cache
> read()
>
> Now your write is gone and there were no calls between the write and
> read.
>
> The question we still need to answer is this:
>
> When we attempt to write back some data from
> The other thing which you seem to be assuming is that applications
> which care about precious data won't use fsync(2). And in general,
> it's been fairly well known for decades that if you care about your
> data, you have to use fsync(2) or O_DIRECT writes; and you *must*
> check the error retu
On Thu, 6 Sep 2018 11:17:18 +0200
Rogier Wolff wrote:
> On Thu, Sep 06, 2018 at 12:57:09PM +1000, Dave Chinner wrote:
> > On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
>
> > > And this has worked for years because
> > > the kernel caches stuff from inodes and data-blocks. If yo
On Thu, Sep 06, 2018 at 12:57:09PM +1000, Dave Chinner wrote:
> On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
> > And this has worked for years because
> > the kernel caches stuff from inodes and data-blocks. If you suddenly
> > write stuff to harddisk at 10ms for each seek between
On Wed, Sep 5, 2018 at 4:09 PM 焦晓冬 wrote:
>
> On Tue, Sep 4, 2018 at 11:44 PM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > > >
> > > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > > On Tue, Sep 4, 2
On Wed, Sep 05, 2018 at 02:07:46PM +0200, Rogier Wolff wrote:
> On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> > There is no requirement for a filesystem to flush data on close().
>
> And you can't start doing things like that.
Of course we can. And we do.
We've been doing target
On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> There is no requirement for a filesystem to flush data on close(). In
> fact, most local filesystems do not. NFS does, but that's because it has
> to in order to provide close-to-open cache consistency semantics.
And these days even NF
On Wed, Sep 05, 2018 at 04:09:42PM +0800, 焦晓冬 wrote:
> Well, since the reader application and the writer application are reading
> a same file, they are indeed related. The reader here is expecting
> to read the lasted data the writer offers, not any data available. The
> reader is surely not expec
On Wed, Sep 05, 2018 at 08:07:25AM -0400, Austin S. Hemmelgarn wrote:
> On 2018-09-05 04:37, 焦晓冬 wrote:
> >At this point, the /bin/sh may be partially old and partially new. Execute
> >this corrupted bin is also dangerous though.
> But the system may still be usable in that state, while returning
On Wed, Sep 05, 2018 at 06:55:15AM -0400, Jeff Layton wrote:
> There is no requirement for a filesystem to flush data on close().
And you can't start doing things like that. In some weird cases, you
might have an application open-write-close files at a much higher rate
than what a harddisk can han
On 2018-09-05 04:37, 焦晓冬 wrote:
On Wed, Sep 5, 2018 at 4:04 PM Rogier Wolff wrote:
On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
Rogier Wolff - 05.09.18, 09:08:
So when a mail queuer puts mail the mailq files and the mail processor
can get them out of there intact, nobo
On Wed, 2018-09-05 at 09:37 +0200, Martin Steigerwald wrote:
> Jeff Layton - 04.09.18, 17:44:
> > > - If the following read() could be served by a page in memory, just
> > > returns the data. If the following read() could not be served by a
> > > page in memory and the inode/address_space has a wri
On Wed, 2018-09-05 at 16:24 +0800, 焦晓冬 wrote:
> On Wed, Sep 5, 2018 at 4:18 AM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields
Rogier Wolff - 05.09.18, 10:04:
> On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> > Rogier Wolff - 05.09.18, 09:08:
> > > So when a mail queuer puts mail the mailq files and the mail
> > > processor can get them out of there intact, nobody is going to
> > > notice. (I know ma
On Wed, Sep 5, 2018 at 4:04 PM Rogier Wolff wrote:
>
> On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> > Rogier Wolff - 05.09.18, 09:08:
> > > So when a mail queuer puts mail the mailq files and the mail processor
> > > can get them out of there intact, nobody is going to not
On Wed, Sep 5, 2018 at 4:18 AM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > > Well, I think the point was that in the a
On Tue, Sep 4, 2018 at 11:44 PM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > >
> > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff
> > > > wrote:
> > > >
> > >
On Wed, Sep 05, 2018 at 09:39:58AM +0200, Martin Steigerwald wrote:
> Rogier Wolff - 05.09.18, 09:08:
> > So when a mail queuer puts mail the mailq files and the mail processor
> > can get them out of there intact, nobody is going to notice. (I know
> > mail queuers should call fsync and report er
Rogier Wolff - 05.09.18, 09:08:
> So when a mail queuer puts mail the mailq files and the mail processor
> can get them out of there intact, nobody is going to notice. (I know
> mail queuers should call fsync and report errors when that fails, but
> there are bound to be applications where calling
Jeff Layton - 04.09.18, 17:44:
> > - If the following read() could be served by a page in memory, just
> > returns the data. If the following read() could not be served by a
> > page in memory and the inode/address_space has a writeback error
> > mark, returns EIO. If there is a writeback error on
On Tue, Sep 04, 2018 at 11:44:20AM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> > >
> > > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > > That is certainly not possible to be done. But at least, shall we r
On Tue, Sep 04, 2018 at 01:35:34PM -0700, Vito Caputo wrote:
> On Tue, Sep 04, 2018 at 04:18:18PM -0400, Jeff Layton wrote:
> > On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > > On Tue, Sep 04, 2018 at 12:12:03PM -0
On Tue, Sep 04, 2018 at 01:35:34PM -0700, Vito Caputo wrote:
> Implement a new class of swap space for backing dirty pages which fail
> to write back. Pages in this space survive reboots, essentially backing
> the implicit commitment POSIX establishes in the face of asynchronous
> writeback errors
On Tue, Sep 04, 2018 at 04:18:18PM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> > On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > > Well, I think the point was that
On Tue, 2018-09-04 at 14:54 -0400, J. Bruce Fields wrote:
> On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> > On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > > Well, I think the point was that in the above examples you'd prefer that
> > > the read just fail--no
On Tue, Sep 04, 2018 at 06:23:48PM +0200, Rogier Wolff wrote:
> On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
> > Well, I think the point was that in the above examples you'd prefer that
> > the read just fail--no need to keep the data. A bit marking the file
> > (or even the en
On Tue, Sep 04, 2018 at 12:12:03PM -0400, J. Bruce Fields wrote:
>
> Well, I think the point was that in the above examples you'd prefer that
> the read just fail--no need to keep the data. A bit marking the file
> (or even the entire filesystem) unreadable would satisfy posix, I guess.
> Whethe
On Tue, Sep 04, 2018 at 11:44:20AM -0400, Jeff Layton wrote:
> On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> > A practical and concrete example may be,
> > A disk cleaner program that first searches for garbage files that won't be
> > used
> > anymore and save the list in a file (open()-write()-
On Tue, 2018-09-04 at 22:56 +0800, 焦晓冬 wrote:
> On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
> >
> > On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff
> > > wrote:
> > >
> > > ...
> > > > >
> > > > > Jlayton's patch is simple but wonderful i
On Tue, Sep 4, 2018 at 7:09 PM Jeff Layton wrote:
>
> On Tue, 2018-09-04 at 16:58 +0800, Trol wrote:
> > On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
> >
> > ...
> > > >
> > > > Jlayton's patch is simple but wonderful idea towards correct error
> > > > reporting. It seems one crucial thing
On Tue, 2018-09-04 at 16:58 +0800, 焦晓冬 wrote:
> On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
>
> ...
> > >
> > > Jlayton's patch is simple but wonderful idea towards correct error
> > > reporting. It seems one crucial thing is still here to be fixed. Does
> > > anyone have some idea?
> > >
On Tue, 2018-09-04 at 13:42 +0800, 焦晓冬 wrote:
> Hi,
>
> After reading several writeback error handling articles from LWN, I
> begin to be upset about writeback error handling.
>
> Jlayton's patch is simple but wonderful idea towards correct error
> reporting. It seems one crucial thing is still h
On Tue, Sep 4, 2018 at 5:29 PM Rogier Wolff wrote:
>
> On Tue, Sep 04, 2018 at 04:58:59PM +0800, 焦晓冬 wrote:
>
> > As for suggestion, maybe the error flag of inode/mapping, or the entire
> > inode
> > should not be evicted if there was an error. That hopefully won't take much
> > memory. On extrem
On Tue, Sep 04, 2018 at 04:58:59PM +0800, 焦晓冬 wrote:
> As for suggestion, maybe the error flag of inode/mapping, or the entire inode
> should not be evicted if there was an error. That hopefully won't take much
> memory. On extreme conditions, where too much error inode requires staying
> in memor
On Tue, Sep 4, 2018 at 3:53 PM Rogier Wolff wrote:
...
> >
> > Jlayton's patch is simple but wonderful idea towards correct error
> > reporting. It seems one crucial thing is still here to be fixed. Does
> > anyone have some idea?
> >
> > The crucial thing may be that a read() after a successful
On Tue, Sep 04, 2018 at 02:32:28PM +0800, 焦晓冬 wrote:
> Hi,
>
> After reading several writeback error handling articles from LWN, I
> begin to be upset about writeback error handling.
>
> Jlayton's patch is simple but wonderful idea towards correct error
> reporting. It seems one crucial thing is
50 matches
Mail list logo
