On Wednesday, 21 November 2018 at 10:56:02 UTC, Walter Bright
wrote:
Wouldn't it be awesome to have the lexing/parsing of the
imports all done in parallel?
From my testing lexing/parsing takes small amount of build time
so running it in parallel might be small gain. We should consider
running in parallel more heavy hitting features like CTFE and
templates.
Since we are in wish land here is my wishes. Currently D reads
the all files that are passed in command line before starting
lexing/parsing, but in principle we could start lexing/parsing
after first file is read. In fact we could start after first
file`s first line is read. Out of all operation before semantic
pass, reading from hard disk should be the slowest so it might be
possible to decode utf-8, lex and parse at the speed of reading
from hard disk. If we run these steps in different thread on the
same core with SMT we could better use core`s resources. Reading
file with kernel, decoding UTF-8 with vector instructions and
lexing/parsing with scalar operations while all communication is
done trough L1 and L2 cache.
I thought about using memory mapped files to unblock file reading
as a first step but lack of good documentation about mmf and lack
of thorough understanding of front end made me postpone this
modification. Its a change with little benefit.
The main difficulty in getting that to work is dealing with the
shared string table.
At begging of parsing a thread could get a read-only shared slice
of string table. All strings not in table are put in local string
table. After parsing tables are merged and shared slice is
updated so new thread could start with bigger table. this assumes
that table is not sorted
