Eric Sandeen wrote: > Jim Meyering wrote: >> Eric Sandeen mentioned that dd's O_DIRECT-exposing code >> didn't always work. The problem is that the kernel imposes >> draconian restrictions on the size of buffer that one may >> write to an FD opened with O_DIRECT. Currently, at least on >> ext4 and with a recent linux kernel, that buffer size must >> be a multiple of 512, which happens to be dd's default buffer size. > > (there are alignment restrictions too, just FWIW) > ...
Yes, we should be ok on that front, with page-aligned buffers. >> Note also that the code makes no attempt to determine the >> appropriate block size. So if you chose obs=42, you lose. >> The adventurous user who experiments with oflags=direct >> must also select a block size that works with O_DIRECT >> and the destination file system. > > I think that's fine, docs can say "subject to size and alignment > requirements of the kernel and filesystem" or something I think. I mentioned size, because that can (and may have to) be changed via the obs, ibs, bs options. I didn't want to mention alignment because dd users can't influence that. ... > I suppose it would be nice to at least make that last buffered IO > synchronous if possible, or sync the file afterwards. > POSIX_FADV_DONTNEED for extra credit? :) Good idea. What do you think of this? It feels slightly excessive to make 3 additional syscalls just to preserve some semantics of O_DIRECT for that final buffer. ...but I like it. >From ff159a605e5bc10fe871109f66cba5ee410c9138 Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyer...@redhat.com> Date: Fri, 7 Aug 2009 16:21:35 +0200 Subject: [PATCH] dd: preserve semantics of O_DIRECT even for final block * src/dd.c: Include "ignore-value.h" (iwrite): When disabling O_DIRECT, try to compensate via POSIX_FADV_DONTNEED and fsync. Suggested by Eric Sandeen. --- src/dd.c | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/src/dd.c b/src/dd.c index 43ad718..d9e4c85 100644 --- a/src/dd.c +++ b/src/dd.c @@ -29,6 +29,7 @@ #include "fd-reopen.h" #include "gethrxtime.h" #include "human.h" +#include "ignore-value.h" #include "long-options.h" #include "quote.h" #include "quotearg.h" @@ -843,6 +844,22 @@ iwrite (int fd, char const *buf, size_t size) if (fcntl (STDOUT_FILENO, F_SETFL, old_flags & ~O_DIRECT) != 0) error (0, errno, _("failed to turn off O_DIRECT: %s"), quote (output_file)); + + /* Since we have just turned off O_DIRECT for the final write, + here we try to preserve some of its semantics. First, use + posix_fadvise to tell the system not to pollute the buffer + cache with this data. Don't bother to diagnose lseek or + posix_fadvise failure. */ +#ifdef POSIX_FADV_DONTNEED + off_t off = lseek (STDOUT_FILENO, 0, SEEK_CUR); + if (0 <= off) + ignore_value (posix_fadvise (STDOUT_FILENO, + off, 0, POSIX_FADV_DONTNEED)); +#endif + + /* Attempt to ensure that that final block is committed + to disk as quickly as possible. */ + conversions_mask |= C_FSYNC; } while (total_written < size) -- 1.6.4.115.g33d49