In this great article [1] there is a brief section on buffered
output to files. Also in this thread [2] I was advised to use
explicitly buffered output for maximum performance. This left me
perplexed: surely any high-level routines already use buffered
IO, no?
[1]
http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/
[2]
http://forum.dlang.org/post/ydgmzhlspvvvrbeem...@forum.dlang.org
In order to get a better understanding of the issue, I decided to
do a bit of simple testing with kind with the kind of file
writing I have to do very often: millions of shortish formatted
data records. My simple test program is here:
http://dpaste.dzfl.pl/329023e651c4.
Time to write 10 million formatted records (average of 10 runs)
using format ("%d %d %0.8f", i, i*13, i/1e7) with different
methods:
GDC DMD
f0: 12.0s 15.2s std.stdio.writef
f1: 10.0s 13.6s 1Mb buffer std.array.appender;
std.stdio.rawWrite
f2: 10.3s 13.6s 300Mb buffer std.outbuffer; std.file.write
f3: 4.9s 5.0s std.stdc.fprintf
f4: 4.8s 4.8s 30b buffer std.stdc.sprintf (no file output)
* gdc (Debian 5.2.1-17) 5.2.1 20150911 :: -O3 -frelease
-march=native
* dmd DMD64 D Compiler v2.068.1 :: -O -release -inline
Some observations:
* the two double-buffered output approaches (f1 and f2) are
roughly the same
* the performance gain from the extra buffering was not huge
(about 20%)
* good old fprintf seems to be my best bet :-(
These results may be very specific to my conditions, but I throw
them out there just in case others may be interested.