Re: Reducing the inter-dependencies (in Phobos and at large)gei

[renar...@yahoo.com]

On Wednesday, 24 April 2013 at 12:03:52 UTC, Dmitry Olshansky 
wrote:
Recently I've struggled again to integrate a module in Phobos 
and the amount of curious forward reference bugs made think 
about a related but not equivalent problem.

Intro
Basically an import graph of Phobos is a rat's nest of mutual 
imports.
With the most of modules being template "toolkits" you 
shouldn't pay for what you don't use. Yet it isn't true in 
general as the module may drag in other modules and with that 
all of static constructors and/or globals related.

Adding even more fun to this stuff - to get a "constraint 
checker" you have to import all other stuff from that module 
(and transitively from all imported by it modules, see 
std.range example below).

One motivating example (though I believe there are much more 
beyond this one):

As some might know regular expression engine can be used for 
both generating sequences for a known pattern and 
matching/finding pieces that do match it in some data.

In fact I have such a functionality in std.regex hidden from 
public interface (not yet complete, wasn't sure of the kind of 
API etc.).

Now the *key* fact:

auto generate(RegEx, rng) (RegEx re, Random rng)
        if(isRegex!RegEx && isUniformRNG!Random)
{
...
}

Now given that innocent signature you have dependency on the 
whole of std.random *including* seeding the default random 
number generator at the program start!

And recall that generating strings while neat is arguably more 
rare need then pattern matching.

Same thing with std.datetime - to get an ability to accept any 
kind of Date as a template parameter (in your API) you have to 
get the whole std.datetime *even if the user never ever calls 
this function* of your module API.

And everyone and their granny depends on full version of 
std.range
that in turn depends on std.algorithm that depends on std.conv 
that depends on std.format(!) and that incidentally on std.uni.
BTW std.uni turns out to be a kind of sink everybody imports 
sooner or later if only for unittests, sadly it's mostly 
imported unconditionally.

And that skipping a full rat's nest to preserve the brains of 
the reader.

After a couple of frustrating evenings dealing with it 
(multiplied by the bogus dmd) I've come up with the idea below.


Solution

First of all no compiler magic required (phew-ew!) :)

Not to mention the 2 obvious facts - smaller modules might 
help, as would guarding by version(unittest) imports used only 
for unit tests.

What we need is to re-arrange the module hierarchy (and we need 
that anyway) so that we split off the "concept" part of modules 
to a separate package.

That would allow modules that need this to use these Duck-typed 
entities (IFF the user ever passes such an entity) can stick 
with importing only the concept part.

Applying that to the current layout would look like:
std.concept.range
std.concept.random
std.concept.* //every other module with any useful isXYZ 
constraint
std.* // stays as is

Any module that has "concept" part then looks like this:

module std.xyz;
import std.concept.xyz;
... //the rest

And then other weakly-dependent modules (i.e. these that are 
satisfied with traits and duck-typed interfaces) can safely 
import std.concept.xyz instead of std.xyz. E.g. std.regex would 
import std.concept.random to get isUniformRNG and rely on duck 
typing thusly described to use it correctly.

The change is backwards compatible and introduces no breakage.
Only clean sugar-free interdependence of modules in Phobos.

Later people (mostly library writers) can use e.g. 
std.concept.range
to avoid pulling full dependency tree in case only constraints 
are needed. The technique can be touted as coding guideline for 
template and duck-type heavy libraries.

Thoughts? Other ideas?