On Sunday, 18 December 2016 at 18:42:36 UTC, Andrei Alexandrescu
wrote:
On 12/18/16 1:03 PM, Joakim wrote:
I largely agree with Dmitry. Ilya refactored several Phobos
modules to
use scoped, selective imports much more, and I pitched in for
some
remaining imports in the largest modules, so that only these
module-level imports remain, ie those necessary for symbols
imported in
template constraints:
std.datetime - https://github.com/dlang/phobos/pull/4373/files
std.uni - https://github.com/dlang/phobos/pull/4365/files
std.string and std.traits -
https://github.com/dlang/phobos/pull/4370/files
Yah, there's been a lot of good work (Jack Stouffer did a lot
as well IIRC) on pushing imports inside. The following searches
should be relevant:
git grep '^\(private \)\?import' | wc -l
yields about 426 top-level import declarations. The number of
indented imports is 4605:
git grep ' *import\W' | wc -l
So we're looking at 10% of imports being problematic. Sadly,
they turn out to make things quite difficult for Phobos. Last
time I looked at the module dependency graph it wasn't a lot
better than it used to before scoped imports. (I don't have it
handy... could anyone please produce it?)
Why do you care _so_ much about the module dependency graph? To
make this question concrete, let's look at an example, the
std.array module you keep mentioning. This is what it looked
like before Ilya scoped as many imports as he could:
https://github.com/dlang/phobos/commit/3fcf723aa498b96de165361b5abb9d3450fdc069#diff-54cf8402b22024ae667d4048a5126f0e
That was a mess, similar to opaque C/C++ code, 13 modules
imported at module-scope were whittled down to 4. You just made
those more specific in a commit related to this DIP, by listing
the actual symbols selectively imported from those four modules:
https://github.com/dlang/phobos/commit/e064d5664f92c4b2f0866c08f6d0290ba66825ed#diff-54cf8402b22024ae667d4048a5126f0e
If I'm looking at the template constraints for any particular
function and see a couple symbols I don't recognize, I don't
think it's a big deal to find the symbols in that list at the top.
In other words, D already allows you to scope most imports. I
don't consider the dozen or two remaining symbols from templaint
constraints and function arguments to provide much overhead.
Rather, I consider the weight of this additional syntax, ie the
cognitive overhead from having to remember and parse more syntax
in my head, to be worse than the remaining dependency reasoning
problem you're trying to solve: the cost outweights the benefit.
Perhaps that's subjective and others may disagree.
Now, there's also the question of purely technical benefits, like
compilation speed or executable bloat. I looked at the latter a
little last summer, after Ilya had cleaned up a lot of the
standard library:
http://forum.dlang.org/thread/gmjqfjoemwtvgqrtd...@forum.dlang.org
I found that commenting out a single scoped, selective import of
"std.string: format" in std.utf led to a 5% decrease in
executable size for "hello world." This is a problem with how
dmd compiles or appends these module dependencies and would
presumably still be there after this DIP, as you would not remove
the dependency.
I think scoped, selective imports have been great at hacking away
at the module dependency graph, as you lay out. It is not clear
what technical costs you see from the remaining few dependencies
and if this DIP is the best way to remove them. I think you
should explain why you want to untangle the remaining dependency
graph, and consider if this DIP is really doing that much.
When I first saw this DIP, like Dmitry I was happy that we
could get rid
of those too, but the more I see these horrible syntax
suggestions for
what is really a minor convenience, I changed my mind.
std.datetime,
the 35k line (17 kloc according to Dscanner) beast of phobos,
only needs
20 or so symbols at module-scope. std.uni- 10k lines, 4.2
kloc- only
needs 17 symbols, all from the three modules Dmitry mentioned.
I don't
think his workaround of splitting up modules is even needed
for such a
low amount of module-level imports.
This paragraph is a good example of a couple of
counterarguments that I think point directly to flaws in the
DIP:
(1) The DIP uses Phobos as an example, so it is applicable
mostly to Phobos. In fact, Phobos is among the systems that
would benefit least from the DIP: it has only druntime as
dependency, and is distributed in its entirety. Many projects
out there list multiple dependencies and may have various
building and distribution policies.
It is not clear how those alternate dependency, building and
distribution policies change the picture. Perhaps you should
cite one of those as an example.
The converse is to believe that working around a problem in
Phobos would render the DIP less useful in general.
The argument is not that Phobos has "worked around" the problem,
but that once you scope everything you can other than template
constraints (which you have said should be standard operating
procedure in the DIP), the problem is minimal.
(2) "I don't like the syntax, hence I don't like the feature."
I see this as a good opportunity for tasteful design, not a
downside of the feature.
The syntax is bad, but even if were beautiful, I don't consider
the addition itself to be worth it. Again, could be subjective.
Maybe there are other issues having to do with symbol
resolution and
dependency encapsulation that are addressed by this DIP, ie the
technical performance of the compiler rather than refactoring
or code
clarity, that I don't fully grasp, but from the first two
points of the
claimed benefits of DCDs, ie ease of reasoning about
dependencies and
refactoring code, I don't think this feature will come
anywhere close to
carrying its own weight.
Does the refactoring in
https://github.com/dlang/phobos/pull/4962 make dependencies
clearer?
I'm fine with your selective import refactoring, which I linked
above:
https://github.com/dlang/phobos/pull/4963
Are you e.g. clear on what you need in order to use Appender?
It is easy to see that it requires isDynamicArray and that is
found in std.traits. It may internally rely on other symbols
that are currently imported from module scope, ie symbols from
template constraints inside the struct Appender that happen to
also be used outside that struct and so are imported at the top
of the module. If you really wanted to make that clear, you
could scope those imports inside the struct too.
Would you want to take this (or another) experiment to another
module and see how it improves its dependency structure?
Ilya and I have already done this, in the PRs I linked above. I
was mildly disappointed back then that I could not get rid of the
remaining module-level imports, because of template constraints.
I was predisposed to favor doing something about it, but I don't
think this DIP is worth it.
As for the third benefit having to do with scalable template
libraries,
I'm not sure I completely understand all the details there,
but I wonder
if those problems aren't an artifact of the way dmd works now
rather
than something that can't be fixed without this DIP.
The DIP now dedicates an entire section to the pluses and
minuses of lazy imports, and concludes that lazy imports would
address scalability if carefully used.
Yes, there might be other ways to lower the technical costs of
the dependency graph too, such as the size issue I mentioned
above.
