Am 31.08.2011, 21:52 Uhr, schrieb Andrei Alexandrescu
<seewebsiteforem...@erdani.org>:
On 08/31/2011 02:15 PM, Marco Leise wrote:
[snip]
If I understand the above correctly, the same amount of seeking goes
around in both cases. The only difference is that requests come at a
faster pace, as they should.
Andrei
That's not what I wanted to say. Let me put it like this: If you read
the file in one call to read and the write the whole thing from your
buffer you have only 2 of these 'long' seeks.
Yes. It is fair to assume that the asynchronous case uses buffers of the
same length as the synchronous case. Then the grand total number of
seeks will be the same or better for the asynchronous version (better if
the OS handles concurrent seek requests cleverly with some elevator
algorithm).
It's really simple. The synchronous version is the worst case because it
needs a seek for read and a seek for write in each epoch. (One epoch =
one buffer is read and written.) The asynchronous version still needs to
do seeks for reading and writing, but in the same amount, and issues
them at a faster rate.
Practically you wont use a
2 GB buffer for a 2 GB file though, but I assume that it would be the
fastest copy mode from and to the same HDD, whereas the multi-threaded
approach would make the OS switch between writing some data for the
writer thread and reading some data for the reader thread, probably
several times per second. And each time a seek is involved.
That is the case whether or not the requests are coming from the same
thread or not.
(Due to IO scheduler optimizations it wont end up like this though. The
OS will detect your read pattern (linear) and read a sane amount of data
ahead and disk access will generally be optimized to reduce seek times.)
This will happen all the more if you have multiple threads.
You clearly have a good expertise on OS, I/O, and related matters, and I
am honoured by your presence and by the value you're adding to this
group. However, in this particular argument you're only firing blanks.
Are you sure you have a case?
Andrei
I guess that happens when you use Gentoo. I'm not an OS expert, I find it
interesting what ideas get implemented into an OS, chip or language to
make it faster, easier or safer. So these things tend to stick with me. I
think you are an expert in matters of parallel execution. So I'm
interested in a cp program written by you, that employs these
asynchronous reads/writes in two threads.
In your first post you claimed your algorithm will be faster because the
OS performs synchronous flushes, now you say it will be faster because the
OS does delay the flush and reorders commands. If you stayed at your point
it would be easier for me to refute you. ^^
Your idea that the OS will optimize your multi-threaded calls in a way it
could not do in a single-threaded application sounds interesting. It would
turn the copy operation into the single-threaded version, but this time
inside the kernel where small reads and writes are merged to bigger ones,
hopefully of optimal size. So it is like saying "multi-threading results
in the better single-threading" and we both made a point. This is all
theory though until we have a test program. I think it is more likely that
you 1. add thread-switch overhead and 2. these thread switches cut your
running read/write into smaller pieces, adding more seeks. In any case, a
single-threaded program with large enough buffers to mask the seeking
delay (not Gigabytes) is faster than a multi-threaded version. I just
can't say for sure anymore if there is some minimal buffer size at which
the multi-threaded approach is faster, because of the OS merging things
into larger buffers behind the scenes.
- Marco
