BCS wrote: > Hello Daniel, > >> http://dobbscodetalk.com/index.php?option=com_myblog&show=Multithreade >> d-I-O.html&Itemid=29 >> >> -- Daniel >> >> P.S. "It really seems barbaric in our digital age that we all have >> motors and levers and gears and spinning things in our boxes." Those >> damned spinning things! >> > > Some sort of fixed size thread pool might work better than > thread-per-file after the files are loaded as the overhead of threads > isn't going to do you any good if you have many more threads than you > have cores >
Careful.. that rule of thumb applies to compute bound use of threads. In this example, it's parallelization to achieve file io activity overlapping with compute activity. If we were talking about physical disks, I'd say that one thread dedicated to "while(filetoread) open/read/close" would be sufficient. Two would just add competing seeks. But the picture likely changes once the files are in page cache. And SSD's are closer to memory than spindles, so it's unclear what the 'right' threading model there is too. Regardless, with the model of most compilers today, as long as the data is ready by the time the main thread is ready for it, it's good enough. It's fun to envision a really seriously multi-threaded compiler, but, like for any app, it requires a lot more care.. and dmd would need some cleanup to not use global state, but not a whole lot from the quick glances I've made at the source. The devil, as always, is in the details. There's enough to do and enough bugs to fix already that I can't see it as worth the investment right now, though. Later, Brad
