Re: dd conv=fsync
Alexander Bluhm wrote: > We should not implement the magic which device allows fsync(2) into > dd(1). Just do what the user says, if it is nonsense, give him an > error. sure. ok.
Re: dd conv=fsync
On Fri, Feb 15, 2019 at 01:28:15PM +0100, Alexander Bluhm wrote: > We should not implement the magic which device allows fsync(2) into > dd(1). Just do what the user says, if it is nonsense, give him an > error. Seems reasonable to me. > > I know this wording is similar to fsync(2), but I think I'd prefer just a > > reference. The output may not be permanent storage. I agree with tedu. OK kn
Re: dd conv=fsync
On Thu, Feb 14, 2019 at 05:20:13PM -0500, Ted Unangst wrote:
> What does gnu dd do if the output is stdout? Is it an error?
On Linux it fails with an error.
linux$ yes | dd count=4 bs=1 conv=fsync
y
y
dd: fsync failed for 'standard output': Invalid argument
4+0 records in
4+0 records out
4 bytes copied, 0.0004733 s, 8.5 kB/s
linux$ echo $?
1
On OpenBSD calling fsync(2) with a terminal works fine, but it fails
with a pipe.
openbsd$ yes | dd count=4 bs=1 conv=fsync
y
y
4+0 records in
4+0 records out
4 bytes transferred in 0.001 secs (2400 bytes/sec)
openbsd$ yes | dd count=4 bs=1 conv=fsync | cat
dd: fsync stdout: Invalid argument
4+0 records in
4+0 records out
4 bytes transferred in 0.000 secs (4796 bytes/sec)
y
y
> Should there be an fstat check to make sure
> it's a regular file?
We should not implement the magic which device allows fsync(2) into
dd(1). Just do what the user says, if it is nonsense, give him an
error.
> I know this wording is similar to fsync(2), but I think I'd prefer just a
> reference. The output may not be permanent storage. What about
>
> Call
> .Xr fsync 2
> on the output file before exiting.
Sure, new diff.
ok?
bluhm
Index: bin/dd/args.c
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/args.c,v
retrieving revision 1.30
diff -u -p -r1.30 args.c
--- bin/dd/args.c 25 Jul 2018 15:09:48 - 1.30
+++ bin/dd/args.c 14 Feb 2019 15:05:18 -
@@ -274,6 +274,7 @@ static const struct conv {
{ "ascii", C_ASCII,C_EBCDIC, e2a_POSIX },
{ "block", C_BLOCK,C_UNBLOCK, NULL },
{ "ebcdic", C_EBCDIC, C_ASCII,a2e_POSIX },
+ { "fsync", C_FSYNC,0, NULL },
{ "ibm",C_EBCDIC, C_ASCII,a2ibm_POSIX },
{ "lcase", C_LCASE,C_UCASE,NULL },
{ "osync", C_OSYNC,C_BS, NULL },
Index: bin/dd/dd.1
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.1,v
retrieving revision 1.33
diff -u -p -r1.33 dd.1
--- bin/dd/dd.1 17 Aug 2016 21:23:01 - 1.33
+++ bin/dd/dd.1 15 Feb 2019 12:17:29 -
@@ -203,6 +203,10 @@ is a slightly different mapping, which i
.At V
.Cm ibm
value.
+.It Cm fsync
+Call
+.Xr fsync 2
+on the output file before exiting.
.It Cm lcase
Transform uppercase characters into lowercase characters.
.It Cm noerror
Index: bin/dd/dd.c
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.c,v
retrieving revision 1.25
diff -u -p -r1.25 dd.c
--- bin/dd/dd.c 23 Jul 2018 23:09:37 - 1.25
+++ bin/dd/dd.c 14 Feb 2019 18:13:55 -
@@ -347,6 +347,10 @@ dd_close(void)
}
if (out.dbcnt)
dd_out(1);
+ if (ddflags & C_FSYNC) {
+ if (fsync(out.fd) == -1)
+ err(1, "fsync %s", out.name);
+ }
}
void
Index: bin/dd/dd.h
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.h,v
retrieving revision 1.8
diff -u -p -r1.8 dd.h
--- bin/dd/dd.h 13 Aug 2017 02:06:42 - 1.8
+++ bin/dd/dd.h 14 Feb 2019 15:05:18 -
@@ -96,3 +96,4 @@ typedef struct {
#defineC_STATUS0x20
#defineC_NOXFER0x40
#defineC_NOINFO0x80
+#defineC_FSYNC 0x100
Re: dd conv=fsync
On Thu, Feb 14, 2019 at 4:21 PM Ted Unangst wrote: > Alexander Bluhm wrote: > > GNU dd has the conv=fsync feature which does an fsync(2) after final > > write to output. I find this useful for write performance measurement > > through the file system without buffer cache optimization. Others > > may like the reliable storage guarantee of fsync(2). > > > > Do we want dd conv=fsync in OpenBSD? > > What does gnu dd do if the output is stdout? Is it an error? > > With this patch, I get dd: fsync stdout: Invalid argument > gnu dd prints something similar to stderr, and returns exit code 1. dd: fsync failed for 'standard output': Invalid argument 3+1 records in 3+1 records out 1784 bytes (1.8 kB, 1.7 KiB) copied, 9.4827e-05 s, 18.8 MB/s > after the transfer is complete. Should there be an fstat check to make sure > it's a regular file? > > > +.It Cm fsync > > +Write output to permanent storage device before finishing. > > I know this wording is similar to fsync(2), but I think I'd prefer just a > reference. The output may not be permanent storage. What about > > Call > .Xr fsync 2 > on the output file before exiting. > > >
Re: dd conv=fsync
Alexander Bluhm wrote: > GNU dd has the conv=fsync feature which does an fsync(2) after final > write to output. I find this useful for write performance measurement > through the file system without buffer cache optimization. Others > may like the reliable storage guarantee of fsync(2). > > Do we want dd conv=fsync in OpenBSD? What does gnu dd do if the output is stdout? Is it an error? With this patch, I get dd: fsync stdout: Invalid argument after the transfer is complete. Should there be an fstat check to make sure it's a regular file? > +.It Cm fsync > +Write output to permanent storage device before finishing. I know this wording is similar to fsync(2), but I think I'd prefer just a reference. The output may not be permanent storage. What about Call .Xr fsync 2 on the output file before exiting.
dd conv=fsync
Hi,
GNU dd has the conv=fsync feature which does an fsync(2) after final
write to output. I find this useful for write performance measurement
through the file system without buffer cache optimization. Others
may like the reliable storage guarantee of fsync(2).
Do we want dd conv=fsync in OpenBSD?
bluhm
Index: bin/dd/args.c
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/args.c,v
retrieving revision 1.30
diff -u -p -r1.30 args.c
--- bin/dd/args.c 25 Jul 2018 15:09:48 - 1.30
+++ bin/dd/args.c 14 Feb 2019 15:05:18 -
@@ -274,6 +274,7 @@ static const struct conv {
{ "ascii", C_ASCII,C_EBCDIC, e2a_POSIX },
{ "block", C_BLOCK,C_UNBLOCK, NULL },
{ "ebcdic", C_EBCDIC, C_ASCII,a2e_POSIX },
+ { "fsync", C_FSYNC,0, NULL },
{ "ibm",C_EBCDIC, C_ASCII,a2ibm_POSIX },
{ "lcase", C_LCASE,C_UCASE,NULL },
{ "osync", C_OSYNC,C_BS, NULL },
Index: bin/dd/dd.1
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.1,v
retrieving revision 1.33
diff -u -p -r1.33 dd.1
--- bin/dd/dd.1 17 Aug 2016 21:23:01 - 1.33
+++ bin/dd/dd.1 14 Feb 2019 18:12:03 -
@@ -203,6 +203,8 @@ is a slightly different mapping, which i
.At V
.Cm ibm
value.
+.It Cm fsync
+Write output to permanent storage device before finishing.
.It Cm lcase
Transform uppercase characters into lowercase characters.
.It Cm noerror
Index: bin/dd/dd.c
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.c,v
retrieving revision 1.25
diff -u -p -r1.25 dd.c
--- bin/dd/dd.c 23 Jul 2018 23:09:37 - 1.25
+++ bin/dd/dd.c 14 Feb 2019 18:13:55 -
@@ -347,6 +347,10 @@ dd_close(void)
}
if (out.dbcnt)
dd_out(1);
+ if (ddflags & C_FSYNC) {
+ if (fsync(out.fd) == -1)
+ err(1, "fsync %s", out.name);
+ }
}
void
Index: bin/dd/dd.h
===
RCS file: /data/mirror/openbsd/cvs/src/bin/dd/dd.h,v
retrieving revision 1.8
diff -u -p -r1.8 dd.h
--- bin/dd/dd.h 13 Aug 2017 02:06:42 - 1.8
+++ bin/dd/dd.h 14 Feb 2019 15:05:18 -
@@ -96,3 +96,4 @@ typedef struct {
#defineC_STATUS0x20
#defineC_NOXFER0x40
#defineC_NOINFO0x80
+#defineC_FSYNC 0x100
