Brad Roberts wrote:
BCS wrote:
Hello Daniel,
http://dobbscodetalk.com/index.php?option=com_myblog&show=Multithreade
d-I-O.html&Itemid=29
-- Daniel
P.S. "It really seems barbaric in our digital age that we all have
motors and levers and gears and spinning things in our boxes." Those
damned spinning things!
Some sort of fixed size thread pool might work better than
thread-per-file after the files are loaded as the overhead of threads
isn't going to do you any good if you have many more threads than you
have cores
Careful.. that rule of thumb applies to compute bound use of threads.
In this example, it's parallelization to achieve file io activity
overlapping with compute activity. If we were talking about physical
disks, I'd say that one thread dedicated to "while(filetoread)
open/read/close" would be sufficient. Two would just add competing
seeks. But the picture likely changes once the files are in page cache.
And SSD's are closer to memory than spindles, so it's unclear what the
'right' threading model there is too. Regardless, with the model of
most compilers today, as long as the data is ready by the time the main
thread is ready for it, it's good enough.
It's fun to envision a really seriously multi-threaded compiler, but,
like for any app, it requires a lot more care.. and dmd would need some
cleanup to not use global state, but not a whole lot from the quick
glances I've made at the source. The devil, as always, is in the
details. There's enough to do and enough bugs to fix already that I
can't see it as worth the investment right now, though.
Later,
Brad
Compilers seem like they'd be easier to parallelize than other
applications... but I think the last version of DSSS launches as many
DMD processes as there are cores anyway.
