Atleast from my experience with this, its dmd thats actually taking the time.
I can't glean this from looking at the code, but are you recompiling the entire program? Web development is a perfect fit for incremental compilation, so when changing one web page's template or view code, you should need to compile only one .d file.
ld and gold seem to support incremental linking, that could help speed things up as well.
