On Tuesday, 20 December 2016 at 20:51:54 UTC, Andrei Alexandrescu
wrote:
I've asked Joakim about this via email just now, likely other
folks also know the answer:
1. I found these PRs related to local imports:
https://github.com/dlang/phobos/pull/4361
https://github.com/dlang/phobos/pull/4365
https://github.com/dlang/phobos/pull/4370
https://github.com/dlang/phobos/pull/4373
https://github.com/dlang/phobos/pull/4379
https://github.com/dlang/phobos/pull/4392
https://github.com/dlang/phobos/pull/4467
Are there more? Is there a central point for them (such as a
bugzilla issue)?
Ilya lists a lot more above, he did most of the work. No central
point that I know of.
2. I see you've done a bunch of work in the area. Where, in your
estimate, are we on the spectrum of making imports local
without a
major reorganization? Any low-hanging fruit to look at, or have
Joakim,
Ilya, Jack and others made a pass through most/all modules
already?
There is more to be done, but my guess would be 80-90% done for
Phobos. Ilya scoped a lot, but usually left druntime imports
alone. Top-level module imports mostly don't use selective
imports yet, because Martin wanted to hold off till he was sure
the symbol leak bug was fixed.
On Tuesday, 20 December 2016 at 22:08:38 UTC, Andrei Alexandrescu
wrote:
On 12/20/2016 03:46 AM, Joakim wrote:
I didn't just say "eh:" I gave evidence for why I think the
problem is
minimal and asked why it's so important to scope those last
3-4 imported
modules too, which you didn't answer.
You have asked for a smoking gun, and one has been found.
I have just uploaded a major update that carefully analyzes the
improvements brought about by switching to local imports in the
D Standard Library. Please refer to the section "Workaround:
Are Local Imports Good Enough?" and Appendix A:
https://github.com/dlang/DIPs/pull/51/files
https://github.com/dlang/DIPs/blob/91baecedcfe7cb75ac22e66478722ec797ebb901/DIPs/DIP1005.md
Thanks for this analysis of the remaining dependency graph, it is
worth looking at. Allow me to poke some holes in it.
To begin with, the amount of scoping that has been done is
overstated, if you simply count scoped imports and compare it to
module-level imports. Each module-level import has to be
replicated multiple times for each local scope, especially in
unittest blocks. A better number is more like 20-30%, as I
pointed out 4 out of 13 modules remain at top-level in std.array.
Using that metric, a 3-4X reduction in top-level imports has led
to at least a 2.2x improvement in imported files, so the effort
has been more meaningful than you conclude.
Second, as I noted above, most top-level imports have not been
made selective yet, because of the symbol leak bug that was
recently fixed by Martin. You will see in my PRs that I only
list those symbols as a comment, because I could not turn those
into selective imports yet. If the compiler is doing its job
right, selective imports should greatly reduce the cost of
importing a module, even if your metric would still show the
module being imported.
Third, checking some of the output from the commands you ran in
your script shows that up to half of the imported modules are
from druntime. I noted earlier that Ilya usually didn't bother
scoping top-level druntime imports, because he perceived their
cost to be low (I scoped them too in the handful I cleaned up,
just for completeness). As far as I know, nobody has bothered to
spend any time scoping druntime, so it would be better if you
filtered out druntime imports from your analysis.
Finally, while it's nice to know the extent of the dependency
graph, what really matters is the _cost_ of each link of the
graph, which is what I keep hammering on. If the cost of links
is small, it doesn't matter how entangled it is. If minimizing
the dependency graph through scoping alone, ie without
implementing this DIP, removes most of the cost, that's all I
care about.
I have noted one example above, where _a single DCD in phobos_,
ie a scoped, selective import, had gigantic costs in terms of
executable size, where entire modules were included because of
it. If that's the case more generally, then _no_ amount of
dependency disentangling will matter, because the cost of single
DCDs is still huge. Perhaps that's just an isolated issue
however, it needs to be investigated.
My point is that the dependency graph matters, but now that we're
getting down to the last entanglements, we need to know the cost
of those last links. Your dependency analysis gives us some
quantitative idea of the size of the remaining graph, but tells
us nothing about the cost of those links. That's what I'm looking
for.
I will spend some time now investigating those costs with sample
code. My request all along has been that you give us some idea
of those costs, if you know the answer already.
