Re: DIP16: Transparently substitute module with package
On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: DIP15 doesn't fix the explicit path problem though. You can't change std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without breaking code. You could make std/algorithm.d publicly import std/alg/* and then DIP15 would allow you to import std.alg to get all of its sub-modules, but you're still forced to use a module to publicly import symbols as part of a migration path, and you can't split a module in place. I think either you or I am missing something. In DIP15, if you define std/algorithm/_.d, and then import std.algorithm, it imports std/algorithm/_.d, which then 1. publicly imports other modules, and 2. aliases symbols to the name std.algorithm.symbol. At least, this is how I understand the intent. It seems equivalent to me to the package.d proposal, it's just using _.d instead of package.d. If you import std.algorithm.sorting, and try and use std.algorithm.sort, yes it will not work. But this does not break existing code (which does not import std.algorithm.sorting), and I find it odd that we want to make std.algorithm.sort work if you don't import std.algorithm. -Steve
Re: DIP16: Transparently substitute module with package
On Monday, April 09, 2012 08:55:27 Steven Schveighoffer wrote: On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: DIP15 doesn't fix the explicit path problem though. You can't change std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without breaking code. You could make std/algorithm.d publicly import std/alg/* and then DIP15 would allow you to import std.alg to get all of its sub-modules, but you're still forced to use a module to publicly import symbols as part of a migration path, and you can't split a module in place. I think either you or I am missing something. In DIP15, if you define std/algorithm/_.d, and then import std.algorithm, it imports std/algorithm/_.d, which then 1. publicly imports other modules, and 2. aliases symbols to the name std.algorithm.symbol. At least, this is how I understand the intent. It seems equivalent to me to the package.d proposal, it's just using _.d instead of package.d. If you import std.algorithm.sorting, and try and use std.algorithm.sort, yes it will not work. But this does not break existing code (which does not import std.algorithm.sorting), and I find it odd that we want to make std.algorithm.sort work if you don't import std.algorithm. Okay. I reread DIP15 again. I guess that I scanned over it too quickly before and/or misremembered it. I had understood that it was proposing that importing std.algorithm where std.algorithm was a package would be the equivalent of importing std.algorithm.* in Java and that there were no extra files involved. So clearly, I've been misunderstanding things here. So, yeah. DIP15 is basically the same as DIP16 except without the std.sort nonsense and the fact that it uses _.d instead of package.d. Using package.d has the advantage of package being a keyword, making it so that no one is going to accidentally create a module that will be treated specially, but it has the downside of likely requiring more special handling by the compiler. I don't really care which we pick though. My main point though, misunderstandings aside, is that it would be _really_ nice to be able to split up a package in place and that without an enhancement of some kind, we can't do that without breaking code. DIP15 appears to fit the bill quite nicely in that regard though. The part of DIP16 which is really bad is the std.sort stuff which. Public importing combined with either the first part of DIP16 or with DIP15 seems to take care of the problem quite nicely. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On 2012-04-07 02:25, Jonathan M Davis wrote: On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote: I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't. import foo.bar.* Is used all over the place in SWT. -- /Jacob Carlborg
Re: DIP16: Transparently substitute module with package
On Saturday, April 07, 2012 18:45:15 Jacob Carlborg wrote: On 2012-04-07 02:25, Jonathan M Davis wrote: On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote: I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't. import foo.bar.* Is used all over the place in SWT. Like I said, some people do like to do it, but Eclipse doesn't like you to, and there are quite a few Java folks who argue that it's bad practice. I forget what the reasons were (maybe increased buld times due to pulling in more symbols or more issues with symbol conflicts - I don't recall), but I wasn't particularly convinced when I heard them. Regardless though, there's a definite contingent against importing with * in the Java community, and it was my understanding that that contigent was the majority of that community, but I don't know. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Saturday, 7 April 2012 at 17:44:20 UTC, Jonathan M Davis wrote Like I said, some people do like to do it, but Eclipse doesn't like you to, and there are quite a few Java folks who argue that it's bad practice. I forget what the reasons were (maybe increased buld times due to pulling in more symbols or more issues with symbol conflicts - I don't recall), but I wasn't particularly convinced when I heard them. Regardless though, there's a definite contingent against importing with * in the Java community, and it was my understanding that that contigent was the majority of that community, but I don't know. - Jonathan M Davis http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-a-java-import-statement-bad Their reasoning sounds more due to various packages reinventing things or being poorly split than actual flaws with package imports themselves. Besides, D already addresses the issues of indicating which one you want, without the horribly long package names that Java has. Seems to me their examples are more like 'import std.*' than 'import std.datetime.*'.
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 23:43, Andrei Alexandrescu a écrit :
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
jmdavisp...@gmx.com wrote:
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:
I don't see how. Just move the code into another module and publicly
import that module from std/algorithm.d. Problem pretty much solved.
The issue is code organization. If you want to split up std.algorithm
(or
std.datetime or whatever) into multiple modules, you have to create a
new
package with a completely different name with no connection to the
original
save for the fact that the original publicly imports it.
My view is that people will not import the smaller modules, they will
only ever import std.algorithm.
I think we should be looking for a solution that not only allows
replacing module - package transparently, but also allows people to
import the newly introduced fine-grained modules.
Andrei
Why not limit name collision to name which make sense ?
For instance, import std.a.b.c is a module. if it refers also to a
function, this import doesn't make any sense, so, even if we have a name
collision, this isn't a big deal (except maybe for reflection ?).
Same goes for std.a.b.c(); which is a function call, and obviously not
the module. Here what I propose to resolve names :
1/ import does always find the .d corresponding file. No exception.
2/ Module a.b.c is in package a, a.b and a.b.c . Any package declaration
in a.b.c match package a.b (one level is removed).
3/ When a name is used in the code and have to be resolved, the
following process occurs :
- The compiler find all stuff that have this name.
- The compiler discard all stuffs that have this name and doesn't make
sense.
- If all remaining items are overload of the same item, then standard
best match rule apply.
- If all remaining items aren't in the same module, or overload or
different items, an error occurs. This is never a problem in the case of
big modules splitted in submodules.
Some examples :
a.d
public import a.b; // import a/b.d
class b {
static void foo() {}
}
a/b.d
public import a.b.c; // import a/b/c.d
void foo(int i) {}
a/b/c.d
void foo() {}
main.d
import a;
foo(); // foo from a/b/c.d
foo(2); // foo from a/b.d
a.foo(); // foo from a/b/c.d
a.b.foo(); // Error, match both a/b.d and a/b/c.d
a.b.foo(2); // foo from a/b.d
a.b.c.foo(); // foo from a/b/c.d
Re: DIP16: Transparently substitute module with package
Le 06/04/2012 01:32, Michel Fortin a écrit : On 2012-04-05 21:43:24 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names? It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place.
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 23:18:07 -0400, Ary Manzana a...@esperanto.org.ar
wrote:
On 4/5/12 10:55 PM, Steven Schveighoffer wrote:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
Starting a new thread from one in announce:
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16
Please comment, after which Walter will approve. Walter's approval
means that he would approve a pull request implementing DIP16 (subject
to regular correctness checks).
Destroy!
BTW, this case makes the part of DIP16 which wants to shortcut fully
qualified names invalid, or at least costly (I posted this code in
another part of the thread, but I thought I'd bring it up higher).
The following is valid code today:
a/b.d:
module a.b;
void foo() {}
struct b
{
static void foo() {}
}
main.d:
import a.b;
void main()
{
a.b.foo();
}
If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo()
the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?
It's a shortcut to the module function foo().
If I replace main.d with:
void main() {
foo();
}
Is foo() the module function foo() from a.b, or is it a shortcut for
a.b.b.foo()? Here you have no doubts. What your mind does it: find a top
level symbol foo in all imported modules. If you find more than one,
error.
That's slightly different, because you must *always* qualify struct b's
foo with a preceeding b.
You must apply the same logic for a.b.foo(). First you search foo in
the a.b symbol. Here you find it: it's the top level function foo in
a.b. Then you stop searching.
However, if you can't find it in the module a.b, you search a top
level symbol foo in all modules that are in package a.b. That's it.
You don't search foo in every possible nesting: just module nesting.
What if a.b is a struct, and it's the only possible match? We don't
search for it? At some point the struct b has to come into play. Or are
you saying we cannot shortcut struct FQN's?
I suppose if we prefer to match modules before types, then name lookup for
fully qualified names only becomes ambiguous with packages allowed to have
their own modules, so it shouldn't affect existing code.
The main issue is, because you can shortcut the FQN, and a chain of
identifiers can have repeated identifiers in them, ambiguity is
possible.
As I said before, it's not a shortcut of the FQN: it's just a shortcut
for the module name.
My example *was* a shortcut for the module name. I did not imply that you
could shortcut the other parts of the FQN.
What about hijacking though? For example:
module a.b
struct c
{
static void foo() {}
}
people now use a.c.foo() to avoid having to type the whole thing
But along comes someone who creates:
module a.c;
void foo() {}
Now, doesn't this usurp a.c.foo() without warning?
-Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote: Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct. That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc. ddoc's output leaves a lot to be desired. The unorganized links at the top suck. Using module order to show symbols instead of category of symbols. How is a doc page ever too big? Even std.datetime loads in a second. It's more that it's too disorganized. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules. As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly. As can the API via public imports. -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 17:43:24 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 4/5/12 4:26 PM, Steven Schveighoffer wrote: On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote: I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. My view is that people will not import the smaller modules, they will only ever import std.algorithm. I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. Of course you *can* do this. I think you mean and allows one to refer to the fine grained module as if it were imported from the top-level module. I don't think that benefit is very great. Why shouldn't we expect people to use the module name they actually imported to refer to a module? If you want selective import, we have: import std.algorithm : sort; I think the real benefit to splitting up the module comes from being able to draw clear lines between which pieces of a large module are related. I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; I keep coming back to std.container. Already it's large, and full of unrelated types. It's only going to get worse. Now, I fully agree that having some way to import a package by itself without having to import all its modules would be ideal (as well as splitting a large module into submodules that live in the same namespace). I think DIP15 has that covered. -Steve
Re: DIP16: Transparently substitute module with package
On 2012-04-06 09:41:27 +, deadalnix deadal...@gmail.com said: Why not limit name collision to name which make sense ? You're proposing that fully qualified names sometime work and sometime do not work based on what you've imported and the context of usage. This makes any FQN conflict hard to detect, because the error will be dependent on the context in which you use the name. You say it'll never happen for modules spitted in submodules, but I disagree. Someday, someone will add a symbol to a submodule without checking every package/module file underneath. Unit tests in the module won't catch the problem because for them to catch the problem the parent module/package has to be imported. Only on certain usages, with the right imports and in the right context will the error occur. In fact, if you add disambiguation based on what makes sense you're making the error more obscure and harder to detect in the first place because sometime it'll occur and sometime not depending on hard to predict conditions. If we allow arbitrary symbols to live inside a package, any conflict in FQN should be caught as soon as possible, ideally as soon as you compile after having added the symbol, and even if you don't use the symbol anywhere. Just like how you get an error today when you try to import a module matching a known package name, you shouldn't be allowed to import a module matching a known symbol name, or create a symbol matching a module fully qualified name. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
On 4/6/12 4:43 AM, deadalnix wrote: Le 06/04/2012 01:32, Michel Fortin a écrit : On 2012-04-05 21:43:24 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names? It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place. One other desirable feature is library distribution. Library writers should be able to offer library xyz as a sheer directory. Then library users should be able to import path.to.xyz and simply get going, or even rename xyz to xyz1 and import path.to.xyz1 without a problem. Andrei
Re: DIP16: Transparently substitute module with package
Le 06/04/2012 16:49, Andrei Alexandrescu a écrit : On 4/6/12 4:43 AM, deadalnix wrote: Le 06/04/2012 01:32, Michel Fortin a écrit : On 2012-04-05 21:43:24 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names? It isn't possible. But as already mentioned, all name doesn't make sense in all situation, so most of the time, disambiguation can be done. Plus, we want to be able to split module when they grow, and in such a situation, collisions will never happen, because all symbols comes from the same module in a first place. One other desirable feature is library distribution. Library writers should be able to offer library xyz as a sheer directory. Then library users should be able to import path.to.xyz and simply get going, or even rename xyz to xyz1 and import path.to.xyz1 without a problem. Andrei Good point. That is an argument for package.d, all.d or _.d .
Re: DIP16: Transparently substitute module with package
On 4/6/12 10:52 AM, deadalnix wrote: Good point. That is an argument for package.d, all.d or _.d . Or [packagename].d, where [packagename] is the name of its sibling folder (std/algorithm.d in the case of std/algorithm/sort.d).
Re: DIP16: Transparently substitute module with package
Le 06/04/2012 20:03, David Gileadi a écrit : On 4/6/12 10:52 AM, deadalnix wrote: Good point. That is an argument for package.d, all.d or _.d . Or [packagename].d, where [packagename] is the name of its sibling folder (std/algorithm.d in the case of std/algorithm/sort.d). No, because in this case, this isn't in the same folder. Still I prefer that opetion because : 1/ You mostly distribute lib as archives, or in a managed way (package manager, build system, etc . . .) 2/ It is simpler as import a.b.c always does the same thing, and other solution introduce complexity in that process, and possible cases of error.
Re: DIP16: Transparently substitute module with package
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote: I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't. I keep coming back to std.container. Already it's large, and full of unrelated types. It's only going to get worse. Now, I fully agree that having some way to import a package by itself without having to import all its modules would be ideal (as well as splitting a large module into submodules that live in the same namespace). I think DIP15 has that covered. DIP15 doesn't fix the explicit path problem though. You can't change std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without breaking code. You could make std/algorithm.d publicly import std/alg/* and then DIP15 would allow you to import std.alg to get all of its sub-modules, but you're still forced to use a module to publicly import symbols as part of a migration path, and you can't split a module in place. DIP15 also less powerful than having a package.d, since it'll automatically import ever public symbol when it may be desirable to have it only import a subset of them or to publicly import something from another module (e.g. as part of a migration path - like moving the benchmarking code in std.datetime to std.benchmark rather than lumping it with the std.datetime stuff when it's split into separate modules). That may or may not be a big deal, but it does mean that DIP15 isn't as powerful as DIP16's package.d solution in that regard. DIP15 also doesn't give a means of documentating a package. By having a package.d file, we have a really good place to provide an overarching description for the package in ddoc. So, while I don't like the second part of DIP16 at all (i.e. the std.sort stuff), I think that the package.d part is a much better way to go. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Friday, April 06, 2012 07:56:41 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote: Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct. That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc. ddoc's output leaves a lot to be desired. The unorganized links at the top suck. Using module order to show symbols instead of category of symbols. How is a doc page ever too big? Even std.datetime loads in a second. It's more that it's too disorganized. There's no question that the links at the top suck, and std.datetime is the poster child for why they suck. The page is too large for how its organized, and it _does_ take significantly longer to load than other Phobos pages, even if it's still usable. There's no question that better links would help significantly though. Regardless, there's enough content there that it would arguably be better served if it were multiple pages. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules. As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly. As can the API via public imports. Not and keep stuff in the same place in the hierarchy. You have to create a completely separate package. The point of the DIP was to make it so that you could do it _in place_ by turning a module like std/algorithm.d into the package std/algorithm/ with modules inside it. Right now, I can go and split up std.datetime into a new package (e.g. std.dtime) and then publicly import it all from std.datetime, but then the new package is completely separate from the module which imports everything. It also makes it look like std.datetime should go away in favor of just keeping std.dtime (the same with std.algorithm if it gets split), which may or may not actually be desirable. Organizationally, it would be much nicer to be able to turn std/datetime.d into std/datetime/, and _that_ is what the DIP is trying to enable. The rest can be done now but not that. You can't split up the API in place. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 02:55, Martin Nowak a écrit : What happen if both pkg.d and pkg/_.d exists ? If it is not in the same path (think -I compiler option). In one case, this is an issue, in the other this isn't. pkg.d would always be a module, hence result in a module/package conflict. We'd need to directly search for a subdirectory to decide whether it's a package. This is true, but why would it be a conflict ? Package already is a tree structure, so it shouldn't cause much trouble. Maybe someone with experience of big Python projects has some valuable insights, but I think one point of adding the file inside a folder is to prevent from unintended pickup of folders. Using eponymous trick, you'd always pick up a file, never a folder. Hence, lookup rules get easier. Plus it is D-ish. I have no doubt that this approach is working in python, and could work in D. This is exactly why this was my first proposal. But this doesn't means that we can't do better. I have tested quite a lot of way to achieve that, in several languages, and this is usually a messy topic, with no single solution.
Re: DIP16: Transparently substitute module with package
On Wed, 04 Apr 2012 20:50:49 -0400, Michel Fortin
michel.for...@michelf.com wrote:
On 2012-04-04 19:48:32 +, Steven Schveighoffer
schvei...@yahoo.com said:
On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr timon.g...@gmx.ch
wrote:
No symbol is resolved until semantic, but I don't think hiding the
module/package symbol if any clashing symbol in the module/any
subpackage exists is a satisfactory solution.
Then we must come up with a way to hide the submodules of a virtual
module.
I tried this, which fails on the current compiler:
import x;
int x;
So it seems we have two choices here:
1. DIP16 needs to get more complex to make package submodules not
accesible as individual modules.
2. Start contextually interpreting identifiers at least in the case of
modules vs. non-modules.
I'd suggest option 2 allows for better backwards compatibility and
more flexibility.
I don't think option 2 is realistic (see my other post).
I saw your other post. I see that a major issue with the way DIP16
intends to shortcut fully qualified names is demonstrated by this simple
example:
a/b/c.d:
module a.b.c;
void foo() {}
struct c
{
static void foo() {}
}
main.d:
import a.b.c;
void main()
{
a.b.c.foo(); // cannot determine what this is. Is it the global
function, or a shortcut FQN to a.b.c.c.foo?
}
This should be counter-case enough to disqualify that idea -- it will
break existing code without ever adding any package.d files. Thanks for
explaining.
I don't think option 1 is an improvement over what we have. I mean, if
you're going to hide the submodules, what is the benefit compared to
just using a different package name for the implementation modules?
The advantage is you can split your logical module into submodules for
maintenance purposes.
You can already refactor std.algorithm this way with no change in the
compiler:
module std.algorithm;
public import std.algorithm_impl.sort;
public import std.algorithm_impl.map;
public import std.algorithm_impl.blah_blah_blah;
…
If we add a language feature, it should be an noticeable improvement
over this situation.
I agree, option 1 is not very convincing.
I think we need a third option.
Here's an idea: we could allow modules having a single symbol with the
same name as the module to behave as if they were the symbol itself,
just like templates behaves. For instance:
module std.algorithm.sort;
void sort(int[] t);
Now you can import std.algorithm.sort and then use the
std.algorithm.sort fully qualified name as if it was a function, even
though it's the module name (std.algorithm.sort.sort would be the
function's name).
Or maybe we could just allow alias sort this at module level? Or is it
allowed already?
I don't like this proposal, simply because this means one function/symbol
per submodule. It should be more flexible than that. But I like the line
of thinking.
Let's re-examine the issue. We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends). But we don't want to break code which currently uses FQN to
access X's symbols.
I think the following might work:
algorithm.d:
import this = std.algorithm_impl.sort;
Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d. If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort. In
essence, sort has *two* FQN, but there are no FQN that refer to more than
one symbol.
I purposely left out the package.d idea because it's orthogonal to this.
-Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix deadal...@gmail.com wrote: If such a syntax is adopted, what would be the point of public imports ? If it is still useful, then your syntax is a better choice, otherwise, we'd better modify public import. Another way to look at it: A public import saves you from having to import dependent modules that you should be importing anyway. A 'this' import treats the other modules as if they were actually part of the imported module in terms of namespace. To give an example, std.range imports std.array. std.array is a module all on its own, and has a specific set of functionality. std.range has a different set of functionality, but you would never want to have std.range imported without also importing std.array. Another example would be a derived class module publicly importing the base class module. On the other hand, something like std.container could have 15 different container types in it. Each of these container types should really live in their own module, in terms of maintenance and separation of private data. For example, std.container.RedBlackTree has no business accessing the private members of std.container.Array. But because they live in the same module, it can. However, we don't want to define std.container.RedBlackTree.RedBlackTree in terms of namespace. So we have one of these combining modules, and everything still lives in the std.container namespace, but we get all the benefits of separating the code into individual modules. -Steve
Re: DIP16: Transparently substitute module with package
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote: No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module. Have you tried it?
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr timon.g...@gmx.ch wrote: On 04/05/2012 02:58 PM, Steven Schveighoffer wrote: No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module. Have you tried it? I just did. OK, what the hell are we arguing about then?! DIP16 is worksforme :) See this part of the spec: http://dlang.org/module.html Read the part on public modules. You may understand why I didn't know about that feature (which seems to work on all the installed compilers I have, back to 2.033). I just read the public import part of TDPL, and there it is, all spelled out quite nicely. I'm going to file a bug against the spec... grrr... I'm now firmly in the we don't need to change anything, just refactor the modules and use public import camp. We don't even need to change ANYTHING in the compiler! Forget everything I said before in this thread ;) I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue. -Steve
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 14:58, Steven Schveighoffer a écrit :
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix deadal...@gmail.com wrote:
Le 05/04/2012 13:46, Steven Schveighoffer a écrit :
I don't like this proposal, simply because this means one
function/symbol per submodule. It should be more flexible than that. But
I like the line of thinking.
Let's re-examine the issue. We need to allow splitting of a module X.d
into pieces for maintenance (or possibly accessibility -- disallowing
friends). But we don't want to break code which currently uses FQN to
access X's symbols.
I think the following might work:
algorithm.d:
import this = std.algorithm_impl.sort;
Which then imports std.algorithm_impl.sort, and effectively aliases all
its symbols into algorithm.d. If std.algorithm_impl.sort defines a
function called sort, then it's also aliased to std.algorithm.sort. In
essence, sort has *two* FQN, but there are no FQN that refer to more
than one symbol.
I purposely left out the package.d idea because it's orthogonal to this.
-Steve
The behavior you described has been proposed for public import, and
have been discussed. This is interesting.
You propose an alternative syntax with is fine and have the advantage
to not disturb already existing public imports, but have the drawback
to create a new syntax, again.
If such a syntax is adopted, what would be the point of public imports
? If it is still useful, then your syntax is a better choice,
otherwise, we'd better modify public import.
No, public imports simply mean that you can view the publicly imported
module. It does *not* add aliases to the importing module.
for example:
foo.d:
module foo;
int x;
bar.d:
module bar;
public import foo;
int y;
main.d:
import bar; // publicly imports foo.
void main()
{
foo.x = 5; // ok, publicly imported via bar (non public, this would be
an error)
bar.y = 5; // ok
bar.x = 5; // error, public import doesn't create new fully qualified
name for foo.x
}
With the system I propose, using the specific technique would make it so
bar.x would also be valid, and would refer to foo.x
I think we need new syntax, or a new language feature, because you don't
want to alter the operation of existing code. Simply changing public
imports would cause lots of existing code to be reinterpreted, possibly
in a way not intended, or that would be ambiguous.
-Steve
I know. I think you answered too fast.
I wasn't stating that public import are the same as what you propose. I
was discussing the fact that both probably answer the same need and that
2 different syntax is something to avoid.
Code wouldn't be broken in unexpected way, because thing will cause
either a compile error (collision of names, but the problem exists with
other solutions too) or is illegal in the current shape of the language.
Re: DIP16: Transparently substitute module with package
On 2012-04-05 11:46:38 +, Steven Schveighoffer schvei...@yahoo.com said: I don't like this proposal, simply because this means one function/symbo l per submodule. It should be more flexible than that. But I like the li ne of thinking. I have the same reserve about it's lack of flexibility too. It would be a nice thing to have for all those libraries having one module per class as it'd reduce the length of the fully qualified name you have to use when you need to disambiguate. But for moving a module like std.algorithm to a package, it's cumbersome. Let's re-examine the issue. We need to allow splitting of a module X.d into pieces for maintenance (or possibly accessibility -- disallowing friends). But we don't want to break code which currently uses FQN to access X's symbols. I think the following might work: algorithm.d: import this = std.algorithm_impl.sort; Which then imports std.algorithm_impl.sort, and effectively aliases all its symbols into algorithm.d. If std.algorithm_impl.sort defines a function called sort, then it's also aliased to std.algorithm.sort. In essence, sort has *two* FQN, but there are no FQN that refer to more tha n one symbol. This is what a public import should do. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
On 4/4/12 10:53 AM, Steven Schveighoffer wrote: On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin michel.for...@michelf.com wrote: Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol. Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c. Andrei
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:26:16 -0400, deadalnix deadal...@gmail.com wrote: Le 05/04/2012 14:58, Steven Schveighoffer a écrit : On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix deadal...@gmail.com wrote: Le 05/04/2012 13:46, Steven Schveighoffer a écrit : No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module. [snip] I know. I think you answered too fast. I wasn't stating that public import are the same as what you propose. I was discussing the fact that both probably answer the same need and that 2 different syntax is something to avoid. Code wouldn't be broken in unexpected way, because thing will cause either a compile error (collision of names, but the problem exists with other solutions too) or is illegal in the current shape of the language. My assertion above is 100% *WRONG*. See this bug I just filed. http://d.puremagic.com/issues/show_bug.cgi?id=7830 In fact public imports today are *exactly* the same as what I proposed. I had no idea. I think we are (well, at least I am) on a wild goose chase here. All that is left to discuss is if/how we want to allow importing of packages. But I think DIP15 already covers that http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 I think DIP16 is worksforme/invalid, and should be closed. -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:35:52 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 4/4/12 10:53 AM, Steven Schveighoffer wrote: On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin michel.for...@michelf.com wrote: Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol. Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c. Stop reading all my posts except for the ones I just posted. I was completely wrong in what I thought the compiler couldn't do, because the spec is lacking. I currently think DIP16 is invalid/worksforme (public imports allows splitting a module into a package). All that is left is how we could specifically import a package with one import statement. DIP15 covers that. But I don't even think we need that, we can fix Phobos without any compiler changes. -Steve
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 16:35, Andrei Alexandrescu a écrit : On 4/4/12 10:53 AM, Steven Schveighoffer wrote: On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin michel.for...@michelf.com wrote: Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol. Interesting. But isn't there an ambiguity when the symbol is not the last one in the chain? Consider: a.b.c.x Could be static member d of class c in module a.b, or module member d in module a.b.c. Andrei The whole point of this thread is to import symbols from submodules into a module. Obviously, this tends to create collisions. This one is hard to solve and should probably be an error. However, such a collision will also appear with DIP16, and DIP16 cause also other type of collisions. So that one is superior - even if not perfect
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:33:14 -0400, Michel Fortin michel.for...@michelf.com wrote: On 2012-04-05 11:46:38 +, Steven Schveighoffer schvei...@yahoo.com said: I don't like this proposal, simply because this means one function/symbo l per submodule. It should be more flexible than that. But I like the li ne of thinking. I have the same reserve about it's lack of flexibility too. It would be a nice thing to have for all those libraries having one module per class as it'd reduce the length of the fully qualified name you have to use when you need to disambiguate. I see the point you are going for. I don't really like the one-class-per-module style as much as how phobos has one module per category. But that's not an excuse not to examine this possibility. I think it's a different issue than what DIP16/DIP15 is trying to solve. But for moving a module like std.algorithm to a package, it's cumbersome. Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them. Let's re-examine the issue. We need to allow splitting of a module X.d into pieces for maintenance (or possibly accessibility -- disallowing friends). But we don't want to break code which currently uses FQN to access X's symbols. I think the following might work: algorithm.d: import this = std.algorithm_impl.sort; Which then imports std.algorithm_impl.sort, and effectively aliases all its symbols into algorithm.d. If std.algorithm_impl.sort defines a function called sort, then it's also aliased to std.algorithm.sort. In essence, sort has *two* FQN, but there are no FQN that refer to more tha n one symbol. This is what a public import should do. I wasn't aware of this. Mostly because the spec omits that feature. I have to retract almost everything I've argued, because we already have the means to fix what I think DIP16 is trying to solve without any compiler changes. -Steve
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 16:47, Steven Schveighoffer a écrit : But for moving a module like std.algorithm to a package, it's cumbersome. Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them. std/internal isn't good. If I just want to import sort, I would have to do std.internal.algorithm.sort, which isn't good. Plus, the package accessibility modifier would be broken with such a pattern. The all.d, package.d, _.d, or, as I propose foldername.d (and submodules into foldername) do not break package accessibility modifier, which is better.
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix deadal...@gmail.com wrote: Le 05/04/2012 16:47, Steven Schveighoffer a écrit : But for moving a module like std.algorithm to a package, it's cumbersome. Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them. std/internal isn't good. If I just want to import sort, I would have to do std.internal.algorithm.sort, which isn't good. import std.algorithm : sort; BTW, this doesn't work today, I'll file a bug. Plus, the package accessibility modifier would be broken with such a pattern. The all.d, package.d, _.d, or, as I propose foldername.d (and submodules into foldername) do not break package accessibility modifier, which is better. I don't really know what you mean, I didn't read that post. Would you mind posting a link or repeating that argument? -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer schvei...@yahoo.com wrote: On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix deadal...@gmail.com wrote: Le 05/04/2012 16:47, Steven Schveighoffer a écrit : Not really. Just move the files to another directory (i.e. std/internal/algorithm) and publicly import them. std/internal isn't good. If I just want to import sort, I would have to do std.internal.algorithm.sort, which isn't good. import std.algorithm : sort; BTW, this doesn't work today, I'll file a bug. Nevermind, it works the same as if sort is defined in a std.algorithm module. I'm kind of surprised the FQN doesn't work though... -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer schvei...@yahoo.com wrote: On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix deadal...@gmail.com wrote: Plus, the package accessibility modifier would be broken with such a pattern. The all.d, package.d, _.d, or, as I propose foldername.d (and submodules into foldername) do not break package accessibility modifier, which is better. I don't really know what you mean, I didn't read that post. Would you mind posting a link or repeating that argument? Also, nevermind :) I realize what you are talking about now. This is a good point. Of course, there are other mechanisms to do this, and DIP15 already proposes an improvement for this. However, at least in the case of phobos, I don't think there's a lot of instances of package accessibility modifier. -Steve
Re: DIP16: Transparently substitute module with package
On 4/5/12 7:43 AM, Steven Schveighoffer wrote: I currently think DIP16 is invalid/worksforme (public imports allows splitting a module into a package). All that is left is how we could specifically import a package with one import statement. Not entirely (I was aware of the way public import works). An issue does exist - there are too many names, i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that. Andrei
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 11:23:12 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 4/5/12 7:43 AM, Steven Schveighoffer wrote: I currently think DIP16 is invalid/worksforme (public imports allows splitting a module into a package). All that is left is how we could specifically import a package with one import statement. Not entirely (I was aware of the way public import works). An issue does exist - there are too many names, i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. Right, but if one only ever imports std.algorithm, who cares what the submodule FQNs are? AIUI, DIP16 also doesn't really reduce the number of names either. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that. Agreed. I think Don mentioned std.math already does something, we may want to look at that model. A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. 2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!). -Steve
Re: DIP16: Transparently substitute module with package
Le 05/04/2012 17:23, Andrei Alexandrescu a écrit : On 4/5/12 7:43 AM, Steven Schveighoffer wrote: I currently think DIP16 is invalid/worksforme (public imports allows splitting a module into a package). All that is left is how we could specifically import a package with one import statement. Not entirely (I was aware of the way public import works). An issue does exist - there are too many names, i.e. the alias pulled in the importing module and also the name being imported. This makes for odd synonyms such as std.algorithm_package.sort.sort being the same as std.algorithm.sort. The other issue is that obviously algorithm_package and algorithm must have distinct names, which makes the scheme a bit awkward at least until we define a compelling convention. I guess we can live with all that. Andrei The first one isn't a problem. It isn't too many names, it is 2 names, and it happen when explicitly told to do so. The second is.
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr timon.g...@gmx.ch wrote: On 04/05/2012 02:58 PM, Steven Schveighoffer wrote: No, public imports simply mean that you can view the publicly imported module. It does *not* add aliases to the importing module. Have you tried it? I just did. OK, what the hell are we arguing about then?! DIP16 is worksforme :) See this part of the spec: http://dlang.org/module.html Read the part on public modules. You may understand why I didn't know about that feature (which seems to work on all the installed compilers I have, back to 2.033). I just read the public import part of TDPL, and there it is, all spelled out quite nicely. I'm going to file a bug against the spec... grrr... I'm now firmly in the we don't need to change anything, just refactor the modules and use public import camp. We don't even need to change ANYTHING in the compiler! Forget everything I said before in this thread ;) I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue. What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code. The package.d portion of the proposal makes it so that you can. Now, public import's current behavior is _almost_ enough to make the second part completely unnecessary. The only change that would be required would be to make it so that std.algorithm.x looks in std/algorithm/package.d if there's no std.algorithm.x module, because otherwise the public imports in package.d would be putting all of the symbols in std.algorithm.package, not std.algorithm (that, and package.d isn't legal at present). So, the whole point of this proposal - to seemlessly allow the transition of a module to a package in place - _does_ require a language/compiler change. But moving stuff from a module to a new package does work already. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote: A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module? What _would_ be valuable and the package.d could provide is an overview of the package. The ddoc comment for the package.d module can become the documentation for the package as a whole. 2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!). Yes. std.datetime will need that once it's split. Without that, much of it can't be split and/or code would have to be needlessly duplicated. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote: A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module? I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation. What _would_ be valuable and the package.d could provide is an overview of the package. The ddoc comment for the package.d module can become the documentation for the package as a whole. 2. deadalnix pointed out that if we come up with a scheme where the package module and its submodules are in the same directory, the package accessibility qualifier can be used (hey look, a use for the package keyword!). Yes. std.datetime will need that once it's split. Without that, much of it can't be split and/or code would have to be needlessly duplicated. Hm.. I just thought of something, as long as the main package module imports everything from the same directory, and doesn't define anything, this isn't an issue. -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr timon.g...@gmx.ch I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue. What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code. But so what? nobody has any code like: import std.algorithm.sort; So who cares where that module goes? I agree it would be ideal to put it there, but I don't think it's strictly necessary. And there is no need for the shortcut for fully qualified names. So, the whole point of this proposal - to seemlessly allow the transition of a module to a package in place - _does_ require a language/compiler change. I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 -Steve
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr timon.g...@gmx.ch I suppose the only thing we don't get is being able to have a module and a package with the same FQN. I don't see that being a major issue. What doesn't work is being able to turn a module into a package with the same name. Right now, we could create a std.alg package with sub-modules containing all of std.algorithm's functionality and change std.algorithm to pubicly import them all, but you can't turn std.algorithm itself into a package without breaking code. But so what? nobody has any code like: import std.algorithm.sort; So who cares where that module goes? I agree it would be ideal to put it there, but I don't think it's strictly necessary. And there is no need for the shortcut for fully qualified names. So, the whole point of this proposal - to seemlessly allow the transition of a module to a package in place - _does_ require a language/compiler change. I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. For instance, with the work that I've done thus far on splitting std.datetime, I've had to create a std.dtime package to hold the modules and have std.datetime pubicly import them. This automatically creates the issue of what the difference between them is (for anyone new to Phobos) and does not indicate their relation at all in the hierarchy. It would be much cleaner to be able to turn std/datetime.d into std/datetime/ with a package.d in it along with the new modules. No, we don't _have_ to do something to make it so that std.algorithm can be turned into std/algorithm/ while still not breaking code. But it would be very nice. Certainly, I don't understand why this DIP would ever have been proposed if Andrei didn't find it valuable. BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 Yes, but having a package.d with the public imports gives you much finer- grained control over what gets imported, and DIP15 doesn't solve the problem of fully qualified uses of std.alorgithm.sort not breaking when std.algorithm.sort gets moved to something like std.algorithm.sorting.d. So, DIP15 doesn't work as a means of seemlessly breaking up a module. It just _mostly_ works as one (since people usually don't fully qualify symbols). Also, package.d would give us a means for documenting a package, which I would very much like to be able to do. Having std.datetime give an overview of the std.dtime package is definitely worse than having a means of having the package document itself - which the package.d file should be able to give us. The package.d portion of DIP16 allows a means of controlling what importing a package would mean, it provides a means of turning a module into package in place, and it potentially provides a way of documenting a package - all of which are valuable. Whether they're valuable enough to merit a language change is obviously up for debate, but certainly, as the designer and primary maintainer of one of the main targets for being split up, I very much like the idea of being able to split up a module in place rather than having to create a new package with a new name with no obvious relation to the original. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote: A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module? I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation. If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. I only see the need to split of a module as the DIP suggests when it's too big, and if the documentation isn't large enough to be an issue, then I don't see how the module itself is going to be too large unless it has a ton of helper stuff, and that stuff can be seemlessy put in another, undocumented module without needing to create a package. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote: I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. My view is that people will not import the smaller modules, they will only ever import std.algorithm. Look at std.bigint, which imports modules from std.internal.math. Nobody ever imports std.internal.math.??? because they just import std.bigint. BTW, importing a directory was already proposed in DIP15. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 Yes, but having a package.d with the public imports gives you much finer- grained control over what gets imported, and DIP15 doesn't solve the problem of fully qualified uses of std.alorgithm.sort not breaking when std.algorithm.sort gets moved to something like std.algorithm.sorting.d. I think you are forgetting that all current code imports std.algorithm, which will register the symbol std.algorithm.sort via public import. No code will break, even if it uses FQN. The only way to break code is to write new import statements. I don't think anyone will do that unknowingly. -Steve
Re: DIP16: Transparently substitute module with package
On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote: A couple issues that still need consideration: 1. If std.algorithm the module becomes std.algorithm the package, what happens with ddoc? We probably *do* need a compiler solution to this. That's assuming that you insist on keeping all of the documentation in one file. That arguably defeats the purpose of splitting up the modules. If there isn't enough in the module to split the documentation, then why do you need to split the module? I thought the whole point was code maintenance? Not documentation splitting... I would have expected people to continue to treat std.algorithm like it was one module, even though it imports several sub-modules for its implementation. If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance. Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules. For example, std.container. Imagine we have a robust set of 15 containers. Why should those all be in one file? They have nothing to do with eachother except they are in the same namespace. Why does RedBlackTree need to be able to access the internals of Array? As the writer of RedBlackTree, I want to be able to test and develop my container without having to worry about the rest of std.container. But I also would like to have the FQN of it to be std.container.RedBlackTree. Public imports allow this *today* without any changes. Note that size isn't so much an issue for me as being able to compartmentalize and develop individual pieces of a module. -Steve
Re: DIP16: Transparently substitute module with package
On 4/5/12 4:26 PM, Steven Schveighoffer wrote: On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote: I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. My view is that people will not import the smaller modules, they will only ever import std.algorithm. I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. Andrei
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote: On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis jmdavisp...@gmx.com If the module isn't large enough to be split for documentation, I find it hard to believe that it needs to be split for maintenance. Why do we ever need to split modules for documentation? Just fix the doc generator so it's not as monolithic. For instance, have one page per class or struct. That may or may not be desirable (certainly in the case of smaller types, I'd argue that it isn't). By doing it on a module basis, you have far more control. But regardless, that would be a major change to ddoc. In either case, the size of the documentation page for a module is currently closely tied to the number of public symbols in the module, so if you have a large API, it can become desirable to split it up simply because the documentation page for it is too large. And by splitting up the API, you fix that problem. Not to mention, if the module is mostly free functions, putting the documentation for each class or struct on its own page doesn't help anyway. So, while that may be a good change to ddoc in at least some cases, it doesn't solve the problem in general. For instance, it would help std.datetime, but it wouldn't help std.algorithm at all. And if all you care about is sub-modules for implementation and want all of the functions in the same module still, then this DIP is pointless. All you have to do is declare undocumented sub-modules which hold the various implementations and have the actual module call them. We already do this sort of thing in Phobos to get around static destructors screaming about circular dependencies. You are starting to see my point :) But I think the issue is not so much that you are splitting the implementation, but splitting up the API into related modules. As I understand it, the entire point of this DIP is to enable splitting up the API cleanly without breaking code. The implementation can already be split up seemlessly. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Thursday, April 05, 2012 16:43:24 Andrei Alexandrescu wrote: On 4/5/12 4:26 PM, Steven Schveighoffer wrote: On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote: I don't see how. Just move the code into another module and publicly import that module from std/algorithm.d. Problem pretty much solved. The issue is code organization. If you want to split up std.algorithm (or std.datetime or whatever) into multiple modules, you have to create a new package with a completely different name with no connection to the original save for the fact that the original publicly imports it. My view is that people will not import the smaller modules, they will only ever import std.algorithm. I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. Yeah. If all we want to do is continue to always import std.algorithm, then the DIP is more or less pointless. It's the splitting of the API among multiple packages while allowing the programmer to either call/import it like he has been or to call/import it from the new module explicitly that the DIP is trying to enable. If we make it possible to split std.algorithm into multiple modules in place, then we avoid breaking code, keep the code organized in the same hierarchy - only more detailed - and allow the programmer to import on either a roughly or finely grained level, depending on which they prefer. And I really like how this could enable us to have package-specific documentation, whereas all documentation is currently module-specific and doesn't enable us to provide a page which gaves an overview of a package. That's not always necessary, but there are times when it would be quite nice (e.g. std.datetime). - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On 2012-04-05 21:43:24 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: I think we should be looking for a solution that not only allows replacing module - package transparently, but also allows people to import the newly introduced fine-grained modules. I think it'd be valuable too. But how do you do that without creating ambiguous fully qualified names? One way would be that by importing a module inside a package, the compiler would check first that the package file contains no symbol with that module name. In other words, if you import std.algorithm.sorting, it'd check that algorithm is not a symbol defined in the std package file (if it exists for std) and that no sorting symbol exists in the std.algorithm package file. That would work. But the problem is that if the package file publicly imports all of its submodules, then you'll need to parse all the imported files to determine if the module you're trying to import conflicts with a package-level symbol. A more practical option would be limit the package files to only two things: 1. a list of modules to import when you import the package (importing the package would be akin importing all modules in this list) 2. a list of symbols aliased in the package's namespace (aliased symbols can be accessed using the package name as a prefix and through selective imports) Since the list of aliases is stored directly in the package file, reading the package file to get the names of each alias is enough to tell there is no conflict with the module name when you're trying to import it. No need to open the other modules the package refers to, or to resolve the symbols the aliases refer to: you only need the name of each alias to verify there is no conflict. For instance, a package file could look like this: package std.algorithm; // modules to import when a file is importing the package // (those are not imported inside this package's namespace) module std.algorithm.sorting; module std.algorithm.mapping; ... // symbols to alias to this package namespace // (importing a module with one of these names is an error) alias std.algorithm.sorting.completeSort completeSort; alias std.algorithm.sorting.isSorted isSorted; alias std.algorithm.sorting.partialSort partialSort; alias std.algorithm.sorting.schwartzSort schwartzSort; alias std.algorithm.sorting.sort sort; ... The drawback is that it's hard to maintain the list of fully-qualified names up to date if you change it in the various modules. But if the goal is only to preserve backward compatibility, creating that list of aliases could be a one-time thing. New symbols would continue to be imported when you import std.algorithm, but they'd not be available directly under std.algorithm: you'd need to use std.algorithm.sorting.newSort if you find yourself in a situation that requires a fully-qualified name. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
Le 03/04/2012 19:44, Martin Nowak a écrit : On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei What about supporting package initalization? I basically proposed that if a submodule of a package was imported, a static import of the package is implicitly added. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15 That is pretty much what was proposed by people in this thread. Basically, it boils down to adding automatically aliases with public imports. I would prefer use pkg.d file instead of pkg/_.d (I though of it a lot recently, and this is what make more sense, even if not my initial proposal). With pkg/package.d or pkg/_.d you can ends up with unnecessary complexity in choosing the file wich is imported, and create error cases. For instance : What happen if both pkg.d and pkg/_.d exists ? If it is not in the same path (think -I compiler option). In one case, this is an issue, in the other this isn't. This file convention solution is superior to the _.d one or package.d one. DIP15 is superior to D16 IMO.
Re: DIP16: Transparently substitute module with package
On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin michel.for...@michelf.com wrote: On 2012-04-02 13:04:31 +, Steven Schveighoffer schvei...@yahoo.com said: On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin The problem is that if .std.algorithm.package contains a sort function and there is also a module called std.algorithm.sort, the fully-qualified name of that 'sort' module wil l become ambiguous. Moreover, whether the fully-qualified name .std.algorithm.sort is ambiguous or not depends on what modules were imported, which is not a very desirable behaviour. So this becomes an error. I don't see this as a major problem. Just don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until y ou want to instantiate one. If you have ambiguous templates in the same module, it'll always be ambiguous irrespective of what you import (and you can blame the module's designer for it). If you have ambiguous templates residing in different modules the symbol will be unambiguous until you've imported the second module (same as overloaded functions). At that point you can disambiguate using the fully-qualified name of the template (or function). Whereas if the fully-qualified name of a module becomes ambiguous because of a symbol in another module, there is no way to disambiguate. All you can do is avoid importing the two conflicting modules together, just like when you encounter two headers trying to define the same symbol in C/C++. How does this happen? The FQN cannot be ambiguous. For example, if we change std/algorithm.d into: std/algorithm/sorting.d which defines sort std/algorithm/finding.d which defines find std/algorithm/package.d which imports sorting.d and finding.d Then how can importing some other module make the FQN ambiguous? The FQN for sort is std.algorithm.sorting.sort. Now, if you define the symbol sort inside std/algorithm/package.d, you have an ambiguity, but so what? If you are responsible for the std.algorithm module, aren't you responsible for all the files in that pseudo-package? I don't see how it changes things from today. Basically, my point is the only ambiguity FQN in the package.d file can create is with other symbols within that same directory, therefore within the same module. All we have to do is adopt the practice that phobos' package.d will only publicly import other modules. It won't define any symbols. Then there will always be an unambiguous FQN. Hm... maybe you mean if you can have std/algorithm/package.d do something like import std.file, which contains another sort function, is that now packaged under std.algorithm.sort? I don't think that's possible in the given DIP. -Steve
Re: DIP16: Transparently substitute module with package
On 2012-04-04 14:08:34 +, Steven Schveighoffer schvei...@yahoo.com said: On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin michel.for...@michelf.com wrote: Whereas if the fully-qualified name of a module becomes ambiguous because of a symbol in another module, there is no way to disambiguate. All you can do is avoid importing the two conflicting modules together, just like when you encounter two headers trying to define the same symbol in C/C++. How does this happen? The FQN cannot be ambiguous. Sure it can if I follow DIP16, because module names can become ambiguous. Let's try this with an example. First, let's define a pretty standard module: std/algorithm/sort.d: module std.algorithm.sort; void sort(T)(T[] array); Here the fully-qualified name of the sort function is .std.algorithm.sort.sort. But according to DIP16's lookup rules, the sort function is also available (if not ambiguous) at: std.sort std.algorithm.sort Question 1: since there is already a module at .std.algorithm.sort, doesn't the module name become ambiguous with the sort function it itself contains? Let's assume the module's name take priority and does not conflict so we can continue. Now we create the package.d file: std/algorithm/package.d: import std.algorithm.sort; And now I write this somewhere in my code: std.algorithm.sort Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? Again, we could decide that the module takes priority. But having symbols take priority over one another is not how D has resolved ambiguities up to now; what D does usually is make it a hard error. If we make it an error the fully-qualified name of anything in the std.algorithm.sort module becomes inaccessible. If we do not make it an error, the module name shadows the function imported in the package. And the problem with shadowing is that it can silently change what code you're calling depending on what you've imported (if you need an example, just ask). You might think I'm trying to split hair in four to find flaws, that no one is going to do things that dumb, but I unfortunately think the problematic pattern is already quite common. How many times have we seen modules containing a class, variable, or function having the same name as the module's name? What should happen when you publicly import those modules in the package.d file? The practice might not be too prevalent in Phobos because modules tend to do a lot of things and are therefore named more generically, but it still happens. For instance: std.array.array std.getopt.getopt std.regex.regex Say you wanted to create a package.d file directly for the whole package std, what should be done for those? -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin michel.for...@michelf.com wrote: On 2012-04-04 14:08:34 +, Steven Schveighoffer schvei...@yahoo.com said: The FQN cannot be ambiguous. Sure it can if I follow DIP16, because module names can become ambiguous. Let's try this with an example. First, let's define a pretty standard module: std/algorithm/sort.d: module std.algorithm.sort; void sort(T)(T[] array); Here the fully-qualified name of the sort function is .std.algorithm.sort.sort. But according to DIP16's lookup rules, the sort function is also available (if not ambiguous) at: std.sort std.algorithm.sort Question 1: since there is already a module at .std.algorithm.sort, doesn't the module name become ambiguous with the sort function it itself contains? OK, but when is it ever valid to refer to a module when the semantic expectations are for something other than a module? I can only think of two places where module names are used, inside an import statement and inside a module statement (three if you count the prefix of a FQN). Maybe I'm missing some case... Let's assume the module's name take priority and does not conflict so we can continue. Now we create the package.d file: std/algorithm/package.d: import std.algorithm.sort; And now I write this somewhere in my code: std.algorithm.sort Question 2: does std.algorithm.sort refer to the std.algorithm.sort *module* or to std.algorithm.package.sort *function* publicly imported from the std.algorithm.sort module? The function. A symbol that is not specified as the module name in import statement or in a module statement is always *not* a module. I think our one saving grace here is that when you want to import a specific symbol from a module, this is not the syntax: import std.stdio.writefln; So there is never any ambiguity as to whether you mean a module identifier or other symbol. You might think I'm trying to split hair in four to find flaws, that no one is going to do things that dumb, but I unfortunately think the problematic pattern is already quite common. How many times have we seen modules containing a class, variable, or function having the same name as the module's name? Tango anyone? :) But yes, I think the issue really becomes, we need to look at context when deciding the semantic meaning of a symbol. I don't think this violates the context-free grammar, because wouldn't this only come into play at the semantic level? Not a compiler writer/hacker, so I don't know. Say you wanted to create a package.d file directly for the whole package std, what should be done for those? No, let's not do that. Ever. :) -Steve
Re: DIP16: Transparently substitute module with package
On 2012-04-04 18:33, Michel Fortin wrote: You might think I'm trying to split hair in four to find flaws, that no one is going to do things that dumb, but I unfortunately think the problematic pattern is already quite common. How many times have we seen modules containing a class, variable, or function having the same name as the module's name? What should happen when you publicly import those modules in the package.d file? I do that all the time in my libraries. -- /Jacob Carlborg
Re: DIP16: Transparently substitute module with package
On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr timon.g...@gmx.ch wrote: On 04/04/2012 07:53 PM, Steven Schveighoffer wrote: OK, but when is it ever valid to refer to a module when the semantic expectations are for something other than a module? I can only think of two places where module names are used, inside an import statement and inside a module statement (three if you count the prefix of a FQN). Maybe I'm missing some case... __traits(allMembers, pack.age.mod.ule); hm... maybe we'd have to have new __traits that would disambiguate, like __traits(allModuleMembers...) This doesn't seem like a huge barrier. Tango anyone? :) But yes, I think the issue really becomes, we need to look at context when deciding the semantic meaning of a symbol. I don't think this violates the context-free grammar, because wouldn't this only come into play at the semantic level? Not a compiler writer/hacker, so I don't know. No symbol is resolved until semantic, but I don't think hiding the module/package symbol if any clashing symbol in the module/any subpackage exists is a satisfactory solution. Then we must come up with a way to hide the submodules of a virtual module. I tried this, which fails on the current compiler: import x; int x; So it seems we have two choices here: 1. DIP16 needs to get more complex to make package submodules not accesible as individual modules. 2. Start contextually interpreting identifiers at least in the case of modules vs. non-modules. I'd suggest option 2 allows for better backwards compatibility and more flexibility. -Steve
Re: DIP16: Transparently substitute module with package
On 2012-04-04 17:53:24 +, Steven Schveighoffer
schvei...@yahoo.com said:
But yes, I think the issue really becomes, we need to look at context
when deciding the semantic meaning of a symbol. I don't think this
violates the context-free grammar, because wouldn't this only come
into play at the semantic level? Not a compiler writer/hacker, so I
don't know.
You'd need a whole lot of context.
Let's say I write:
.a.b.c.d = .e.f.g.h;
Can you tell me which letters are the name of a package/module, and
which are the name of something else? It could be this:
module a.b;
struct c { static int d; }
module e.f.g;
int h;
or it could be this:
module a;
struct b { struct c { struct d { static void opAssign(int); } } }
module e;
struct f { enum g { h = 1 } }
or a multitude of other variants. If you can't know by looking at the
assignment above, how will the parser know?
Fully qualified names really need to refer to a single thing.
Say you wanted to create a package.d file directly for the whole
package std, what should be done for those?
No, let's not do that. Ever. :)
Phobos is just an example. People will want to use package.d files
for their own libraries too, and will complain if it doesn't work.
--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/
Re: DIP16: Transparently substitute module with package
On 2012-04-04 19:48:32 +, Steven Schveighoffer schvei...@yahoo.com said: On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr timon.g...@gmx.ch wrote: No symbol is resolved until semantic, but I don't think hiding the module/package symbol if any clashing symbol in the module/any subpackage exists is a satisfactory solution. Then we must come up with a way to hide the submodules of a virtual module. I tried this, which fails on the current compiler: import x; int x; So it seems we have two choices here: 1. DIP16 needs to get more complex to make package submodules not accesible as individual modules. 2. Start contextually interpreting identifiers at least in the case of modules vs. non-modules. I'd suggest option 2 allows for better backwards compatibility and more flexibility. I don't think option 2 is realistic (see my other post). I don't think option 1 is an improvement over what we have. I mean, if you're going to hide the submodules, what is the benefit compared to just using a different package name for the implementation modules? You can already refactor std.algorithm this way with no change in the compiler: module std.algorithm; public import std.algorithm_impl.sort; public import std.algorithm_impl.map; public import std.algorithm_impl.blah_blah_blah; … If we add a language feature, it should be an noticeable improvement over this situation. I think we need a third option. Here's an idea: we could allow modules having a single symbol with the same name as the module to behave as if they were the symbol itself, just like templates behaves. For instance: module std.algorithm.sort; void sort(int[] t); Now you can import std.algorithm.sort and then use the std.algorithm.sort fully qualified name as if it was a function, even though it's the module name (std.algorithm.sort.sort would be the function's name). Or maybe we could just allow alias sort this at module level? Or is it allowed already? -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
What happen if both pkg.d and pkg/_.d exists ? If it is not in the same path (think -I compiler option). In one case, this is an issue, in the other this isn't. pkg.d would always be a module, hence result in a module/package conflict. We'd need to directly search for a subdirectory to decide whether it's a package. Maybe someone with experience of big Python projects has some valuable insights, but I think one point of adding the file inside a folder is to prevent from unintended pickup of folders.
Re: DIP16: Transparently substitute module with package
On 2012-04-05 00:50:49 +, Michel Fortin michel.for...@michelf.com said: I think we need a third option. Here's an idea: we could allow modules having a single symbol with the same name as the module to behave as if they were the symbol itself, just like templates behaves. For instance: module std.algorithm.sort; void sort(int[] t); Now you can import std.algorithm.sort and then use the std.algorithm.sort fully qualified name as if it was a function, even though it's the module name (std.algorithm.sort.sort would be the function's name). Or maybe we could just allow alias sort this at module level? Or is it allowed already? Forgot to follow through with what import std.algorithm would do. If std.algorithm is a package, then it should only contain modules (which can each pose for a function/class/struct/variable thanks to the feature described above). Importing the package std.algorithm would open the std/algorithm.d file, read a list of package from the file, and proceed to import each of them (just as if you'd have imported each of them separately). Here is an idea for declaring the list of modules to import when you import a package in std/algorithm.d: package std.algorithm; module std.algorithm.sort; module std.algorithm.map; ... (I'm _not_ using a package.d file because that'd incur two lookups: one for std/algorithm.d in case it's a module, and one for std/algorithm/package.d in case it's a package. It also removes the possibility of both files existing at the same time which would be confusing.) This way, modules and packages stay exactly the same in the language as they are now, except that you can now import a package directly instead of importing each module separately. And modules with a single symbol with the same name as the module are treated as if they were that symbol. This allows migrating a module to a package without breaking the existing code base, but the drawback is that each symbol in the module becoming a package need to become its own submodule. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei What about supporting package initalization? I basically proposed that if a submodule of a package was imported, a static import of the package is implicitly added. http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15
Re: DIP16: Transparently substitute module with package
On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin michel.for...@michelf.com wrote: On 2012-03-30 14:46:19 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: Destroy! Since you're asking… One thing the current system avoids is unresolvable symbols. If two symbol name clashes, you just qualify them fully and it'll always be unambiguous. For instance: .std.algorithm.sort(…) Now, if std.algorithm becomes both a module and a package, you could have both a sort function and a sort submodule with no way to distinguish between the two, even when fully qualified. I think this is why D does not allow modules to have the same name as packages. I understand that you try to work around this problem by inventing a .std.algorithm.package scope. Then you make it's content imported automatically inside the .std.algorithm scope for backward compatibility (and convenience). The problem is that if .std.algorithm.package contains a sort function and there is also a module called std.algorithm.sort, the fully-qualified name of that 'sort' module will become ambiguous. Moreover, whether the fully-qualified name .std.algorithm.sort is ambiguous or not depends on what modules were imported, which is not a very desirable behaviour. So this becomes an error. I don't see this as a major problem. Just don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until you want to instantiate one. -Steve
Re: DIP16: Transparently substitute module with package
On Sat, 31 Mar 2012 00:23:32 -0400, Chris NS ibisbase...@gmail.com wrote: I'm pretty impressed with the idea, and look forward to its implementation, but I do have one question. How does this affect (if at all) the implicit friend relationship of declarations? Since package foo.bar is treated as a single module, do the classes in foo/bar/alpha.d and foo/bar/beta.d have access to each other's private members? I'm not sure whether I favor them losing or keeping their friend status. They would lose their friend status. Classes that you need to be friends would have to go into the same submodule, which makes perfect sense to me. -Steve
Re: DIP16: Transparently substitute module with package
On 2012-04-02 13:04:31 +, Steven Schveighoffer schvei...@yahoo.com said: On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin The problem is that if .std.algorithm.package contains a sort function and there is also a module called std.algorithm.sort, the fully-qualified name of that 'sort' module wil l become ambiguous. Moreover, whether the fully-qualified name .std.algorithm.sort is ambiguous or not depends on what modules were imported, which is not a very desirable behaviour. So this becomes an error. I don't see this as a major problem. Just don't name a module sort inside std/algorithm. This is no different than ambiguous templates, which are allowed until y ou want to instantiate one. If you have ambiguous templates in the same module, it'll always be ambiguous irrespective of what you import (and you can blame the module's designer for it). If you have ambiguous templates residing in different modules the symbol will be unambiguous until you've imported the second module (same as overloaded functions). At that point you can disambiguate using the fully-qualified name of the template (or function). Whereas if the fully-qualified name of a module becomes ambiguous because of a symbol in another module, there is no way to disambiguate. All you can do is avoid importing the two conflicting modules together, just like when you encounter two headers trying to define the same symbol in C/C++. With the way D modules were designed, this cannot happen because you can't have a module with the same name as a package. I always thought this was done on purpose, but I might be wrong. Whatever we do, I think it'd be a nice property to preserve that fully-qualified names should always work. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
Chris NS ibisbase...@gmail.com wrote in message news:ugopmohijjcnnrchu...@forum.dlang.org... I'm pretty impressed with the idea, and look forward to its implementation, but I do have one question. How does this affect (if at all) the implicit friend relationship of declarations? Since package foo.bar is treated as a single module, do the classes in foo/bar/alpha.d and foo/bar/beta.d have access to each other's private members? I'm not sure whether I favor them losing or keeping their friend status. There's always the package access specifier.
Re: DIP16: Transparently substitute module with package
On Saturday, 31 March 2012 at 06:39:07 UTC, Nick Sabalausky wrote: Chris NS ibisbase...@gmail.com wrote in message news:ugopmohijjcnnrchu...@forum.dlang.org... I'm pretty impressed with the idea, and look forward to its implementation, but I do have one question. How does this affect (if at all) the implicit friend relationship of declarations? Since package foo.bar is treated as a single module, do the classes in foo/bar/alpha.d and foo/bar/beta.d have access to each other's private members? I'm not sure whether I favor them losing or keeping their friend status. There's always the package access specifier. True, though it bears revisiting in its own right. I've never been completely satisfied with the horizontal visibility, and would have preferred 'package' be defined as visible within current package, *and subpackages* of the current. But now I'm getting a bit off-topic.
Re: DIP16: Transparently substitute module with package
On Sat, 31 Mar 2012 01:46:19 +1100, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment ... We solved this issue in the Euphoria Programming language by introducing the concept of 'public include'. The 'include' directive is similar to DPL's 'import' directive. Normally when a file (module) is included, its public identifiers are *only* visible to the including module and not further module up the include tree. However, if a module is 'public include'd then its public identifiers are set up *as if* they were actually declared in the including module rather than the included module. Thus if Euphoria's module called datetime.e gets too large, we split it into, say two or three new modules and replace the code in datetime.e with corresponding 'public include' statements, thus making it a type of package definition. This means that existing application code does not have to be modified and future code can choose to include either the entire package called datetime.e or individual modules that go into making that package. From the POV of a developer, creating a package out of what was a module is transparent. They do not even know or have to care in order to use it. There are no complicated access rules that can one day trip people up. We made one tweak to the public identifier declaration though to cater for situations in which an identifier which was currently local to a module, but when moved to a new module when the original was transformed to a package, still had to be visible to the code in the package file but invisible to code including the package. We created the 'export' qualifier on an identifier declaration to signal such an item. Original module code ... function to_seconds(datetime x) /* A local function to the module */ public function local_to_gmt(datetime x) /* A publicly exposed function */ New module code ... export function to_seconds(datetime x) /* A function visible to the package file only */ public function local_to_gmt(datetime x) /* A publicly exposed function */ This means that application that include the package datetime.e will not see to_seconds() but code in the datetime.e will see it, and local_to_gmt() will be seen by the application code and the package code. -- Derek Parnell Melbourne, Australia
Re: DIP16: Transparently substitute module with package
On 03/30/2012 11:35 PM, Jonathan M Davis wrote: But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them +1. That is even better than treating the package module specially.
Re: DIP16: Transparently substitute module with package
On Friday, 30 March 2012 at 14:46:16 UTC, Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei Hooray! I was loudly complaining about this issue for years. I'm glad this is finally going to be taken care of. 1. Regarding naming - as others mentioned it's inconvenient for sorting. How about changing the file extension instead of the name? e.g have a algorithm.package instead of algorithm.d? The compiler could even ignore the name of the file as long as it has the proper extension - check for existence of *.package. The compiler should enforce that there's at most one such package file to prevent ambiguities - regardless of the chosen naming scheme. 2. Regarding documentation - I'd encourage (via a D style guideline) putting the overview of the package in this file. A tool such as Ruby's codnar would nicely compliment the reference generated by DDoc. see https://www.ruby-toolbox.com/gems/codnar for details. 3. std.algorithm should be deprecated and eventually removed. All code *is* essentially algorithms and as such importing this module makes as much sense as doing: import code; // generic much? 4. regarding the look-up rules - I'm unsure about this part of the proposal. Others already voiced concerns about issues this might bring.
Re: DIP16: Transparently substitute module with package
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr timon.g...@gmx.ch wrote: On 03/30/2012 11:35 PM, Jonathan M Davis wrote: But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them +1. That is even better than treating the package module specially. That already works, doesn't it?
Re: DIP16: Transparently substitute module with package
Le 30/03/2012 21:33, Andrei Alexandrescu a écrit : On 3/30/12 1:39 PM, Jonathan M Davis wrote: However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Example? Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); In the case of sort, you may know where it's from - particularly since it's so common - but the less well-known the function is, the less likely that is at all obvious where it comes from, and if you're dealing with 3rd party software, then it wouldn't be at all obvious. For instance, how would you know that party.foo is really party.bar.foo? You wouldn't. Why should you? Being so lax about importing could really harm code readibility (and maintainibility, since it increases the odds of name clashes). So, I'm inclined to say that that is a _bad_ idea. Maybe if you produce a solid example, I'd be convinced. Andrei You are reversing the logic. It have to be shown that it is not a problem to do the change. The other way around is flawed logic.
Re: DIP16: Transparently substitute module with package
Le 30/03/2012 23:35, Jonathan M Davis a écrit : On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote: On 3/30/12 1:39 PM, Jonathan M Davis wrote: However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Example? std.sort works because there's only one sort. If there are two, you get a conflict (e.g. if you had std.path.sort which sorted paths in some path-specific manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It would be similar to having to specify std.algorithm.indexOf when you've imported both std.string and std.algorithm. But the problem is when std.path.sort is added _later_. I mentionned that as being a ptotential issue, so this is a +1 . But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them (with package.d being treated as if it had the same name as the package that it's in). That's essentially how it already works except when specifying the full import path for a symbol. And that way, you can specify in std.algorithm.package.d exactly what you want to be imported when std.algorithm is imported (including using : to restrict it to specific symbols in a module), and only those symbols will be treated as if they were part of std.algorithm - both for importing purposes and when specifying the import path when using a symbol. The library maintainer then has control over which symbols get used with which import paths. - Jonathan M Davis I did propose that in another thread, so this is a +1 too.
Re: DIP16: Transparently substitute module with package
On 03/31/2012 03:49 PM, Martin Nowak wrote: On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr timon.g...@gmx.ch wrote: On 03/30/2012 11:35 PM, Jonathan M Davis wrote: But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them +1. That is even better than treating the package module specially. That already works, doesn't it? It already works indeed. My bad.
Re: DIP16: Transparently substitute module with package
However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); You can turn that argument around, std.sort(arr) instead of sort(arr) provides some extra context for disambiguation. You'd be able to do that for public imports in the package.d module anyhow, so it makes sense to support it for implicit packages as well.
Re: DIP16: Transparently substitute module with package
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :
Starting a new thread from one in announce:
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16
Please comment, after which Walter will approve. Walter's approval means
that he would approve a pull request implementing DIP16 (subject to
regular correctness checks).
Destroy!
Andrei
OK, here is an alternative proposal, based on what has been said here,
and in other thread. I think it is a better alternative because it is
almost already working.
1/ you can defined both a.d (d source code) and a (folder) in the same
folder.
2/ import a; will always look for a.d . a folder. a existing isn't an error.
3/ import a.b will look for b.d in a folder. If a.d exists, it isn't an
error.
4/ The package a include what in a.d and everything defined in d files
in the folder a and its subfolders.
5/ What is publicly imported get automatically aliased in the importing
module. See example below if it is not clear :
a.d :
public import a.b;
b.d :
void foo() {}
automatic aliasing in a.d would look like :
public import a.b;
// Automaticaly added aliases :
alias b.foo foo;
And usage from third party code :
import a;
void main() {
foo(); // OK
a.foo(); // OK
a.b.foo(); // OK
}
And with static import :
static import a;
void main() {
foo(); // Error
a.foo(); // OK
a.b.foo(); // OK
}
Re: DIP16: Transparently substitute module with package
On Saturday, March 31, 2012 17:30:36 Timon Gehr wrote: On 03/31/2012 03:49 PM, Martin Nowak wrote: On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr timon.g...@gmx.ch wrote: On 03/30/2012 11:35 PM, Jonathan M Davis wrote: But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them +1. That is even better than treating the package module specially. That already works, doesn't it? It already works indeed. My bad. Does it? I thought that std.range.replace wouldn't work (even though std.range publicly imports std.array), because replace isn't part of std.range. You don't need to import std.array if you import std.range, because std.range does, but you can't specificy it's path as if it were in std.range. I'll have to check... Okay. You're right. std.range.replace _does_ work. So then the only issue is making it so that public imports in std.datetime.package are treated as if they were in std.datetime. That seems like the cleanest solution to me. It goes along quite nicely with making it so that anything in std.datetime.package is imported when importing std.datetime. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit : Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei I was waiting for it :D First, as all.d is already a convention in many D projects, why choose package.d ? I understand this is a keyword, but this will also be painfull for many developer. Second, what the rule of a .d file and a folder existing, but in different path (think -I switch of the compiler). In the first place, is the module declaration really usefull ? About the lookup rule, I understand that if I import a.b and use the function a.b.c.foo , then i must be able to refers to it as a.b.foo, but why a.foo ? It seems to me like going too far in the modification for no benefit (it can only increase the number of collision, and has no benefit I can think of).
Re: DIP16: Transparently substitute module with package
On 03/30/2012 04:46 PM, Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei Those are two proposals. I think the first one should be implemented. I don't really like the second one. 1. It is an over-general solution, because it does not solve a general problem. 2. Library writers lose control. All symbol names in the package are reserved when the package is deployed. It will be a breaking change to introduce a sub-package that happens to have the same name as some symbol defined in any module in the same package hierarchy. 3. symbol lookup is already hard enough to get right, because compile time reflection and conditional code generation can introduce ambiguities and contradictions. DMD does not get it right. It is likely that this change would make fixing this in a general but not-too-conservative way considerably harder. Maybe it would be better to just interpret foo.bar.baz as foo.bar.package.baz if foo.bar is a package that has been imported via the foo.bar.package rewrite? Of course, issue 2 probably would remain.
Re: DIP16: Transparently substitute module with package
On second thought, issue 2 is probably not that much of a problem.
Re: DIP16: Transparently substitute module with package
On 30/03/2012 15:46, Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei The proposal doesn't say what happens when package.d is not found but foo/bar/ exists. Given that a lot of people will just use public import foo.bar.*; in that file, would it make sense for package.d missing to imply import foo.bar.*? That would save typing out every single file in there. Of course it would also be annoying if you wanted to import everything except one file, as you'd then have to type out every single import anyway. The other option is to error, which is probably a more sane option. -- Robert http://octarineparrot.com/
Re: DIP16: Transparently substitute module with package
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). I think package.d should be allowed to specify module. Otherwise, something like /usr/include/d/std/datetime/package.d, what is the assumed package? If module is not useful here, it is not useful anywhere. I think the omission should be strictly anything after the package directory. For example, if you have std/algorithm/package.d and std/algorithm/sorting.d, where package.d imports sorting.d, you can refer to std.algorithm.sorting.sort by omitting sorting, but not by omitting algorithm or std. Other than that, this is a good change. -Steve
Re: DIP16: Transparently substitute module with package
On 3/30/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote:
Destroy!
That means a program that imports std.algorithm may use std.sort
for the symbol std.algorithm.sort.
That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf(foo, foo); - std.string.indexOf
}
That would ease usage of Phobos a little bit. OTOH 'string' already is
a keyword and things might get hairy..
Still this is one of the few proposals I like. My only caveat is the
comment: except the file is not allowed to use the module
declaration.. Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;
Since the module declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.
Re: DIP16: Transparently substitute module with package
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
On 3/30/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote:
Destroy!
That means a program that imports std.algorithm may use std.sort
for the symbol std.algorithm.sort.
That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf(foo, foo); - std.string.indexOf
}
I was actually kind of surprised when I found out this doesn't work. It
seems so natural to resolve ambiguity using as little context as necessary.
That would ease usage of Phobos a little bit. OTOH 'string' already is
a keyword and things might get hairy..
Still this is one of the few proposals I like. My only caveat is the
comment: except the file is not allowed to use the module
declaration.. Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;
Since the module declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.
Re: DIP16: Transparently substitute module with package
On 3/30/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Btw, I bet with the help of hackers like e.g. Kenji Hara we'll have this implemented in a matter of days (if it gets accepted). Compare that to having a C++ committee that would spend the next 5 years writing a 100-page spec on what is or isn't a package, heheh. Didn't Bjarne say on Going Native that he doesn't really know what a module is or should be? He should try D sometime. ;)
Re: DIP16: Transparently substitute module with package
On Friday, March 30, 2012 09:46:19 Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! The first part with package.d seems like a good idea and certainly affects some of what I've been doing with std.datetime. In order to split it right now, you need a new package name, leaving the original for importing everything, and this provides a better way of dealing with that. However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); In the case of sort, you may know where it's from - particularly since it's so common - but the less well-known the function is, the less likely that is at all obvious where it comes from, and if you're dealing with 3rd party software, then it wouldn't be at all obvious. For instance, how would you know that party.foo is really party.bar.foo? You wouldn't. Being so lax about importing could really harm code readibility (and maintainibility, since it increases the odds of name clashes). So, I'm inclined to say that that is a _bad_ idea. I'd propose that we make it so that if a module publicly imports another module, then you could treat it as if it were in that module. So, because std.datetime.package publicly imports std.datetime.systime, you could use std.datetime.SysTime instead of std.datetime.systime.SysTime. The compiler would need to realize that they're exactly the same symbol (we've had bugs relating to importing with : and the like which ended up creating new symbols, and we don't want that here), but that shouldn't be all that hard. That gives you control over which symbols are able to be treated as if they were in a given package rather than affecting everything indiscriminitely (and if you use : with public imports, it should then be possible, to restrict it to the _exact_ set of symbols that you want if you don't want all of the symbols to be treated as if they were in that module). Another question is how this affects the documentation. Does package.d generate a page just like the other modules do? The lack of a module declaration could make that difficult (not impossible, but it would probably require changes to ddoc in addition to the module stuff). Also, does that page get treated in any special manner in how the documentation is laid out, because it's for the package as a whole (probably more of a site question than a ddoc one though)? I'd like to be able to have a page describing the package as a whole for std.datetime in addition to having the individual pages rather than just splitting it up, and leaving the programmer to read each of the individual pages with no overview. And I think that however package.d works, it needs to enable that. - Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Friday, March 30, 2012 20:06:57 Andrej Mitrovic wrote:
On 3/30/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote:
Destroy!
That means a program that imports std.algorithm may use std.sort
for the symbol std.algorithm.sort.
That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf(foo, foo); - std.string.indexOf
}
No, I don't think so. If I understand the proposal correctly, it would enable
std.indexOf (which doesn't help you at all in this case), not string.indexOf.
It's trying to make it so that you can treat a symbol in a sub-module as it
were in a higher module, and string.indexOf doesn't help with that at all.
- Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Friday, March 30, 2012 12:15:44 Brad Anderson wrote:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
On 3/30/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote:
Destroy!
That means a program that imports std.algorithm may use std.sort
for the symbol std.algorithm.sort.
That's quite interesting. Would that also mean that you could do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf(foo, foo); - std.string.indexOf
}
I was actually kind of surprised when I found out this doesn't work. It
seems so natural to resolve ambiguity using as little context as necessary.
It would certainly be desirable in some cases, but I believe that the reason
that it doesn't work is due to the ambiguities that it would create. I'd have
to go dig up old discussions on it though to remember all of the details.
alias is supposed to solve the problem, but it doesn't really work all that
well for it, since private doesn't hide symbols, it only makes them
inaccessible (just like with C++). So, creating aliases in a module causes
problems in other modules that import that module, even if the aliases are
private. There are definitely some folks pushing for private to actually start
hiding symbols (IIRC, there's even a pull request for it), but I don't know
what the odds of convincing Walter are. If/Once that happens, alias will
actually become usable for this sort of situation, and the inability to do
string.indexOf won't be as big a deal.
- Jonathan M Davis
Re: DIP16: Transparently substitute module with package
On Friday, 30 March 2012 at 18:39:44 UTC, Jonathan M Davis wrote: I'd propose that we make it so that if a module publicly imports another module, then you could treat it as if it were in that module. So, because std.datetime.package publicly imports std.datetime.systime, you could use std.datetime.SysTime instead of std.datetime.systime.SysTime. I'm not sure if that's a good idea. I'd prefer a new kind of import statement, perhaps something like: // module std.datetime.package alias import std.datetime.systime; which is similar to a public alias of everything in that module?
Re: DIP16: Transparently substitute module with package
On 3/30/12 1:39 PM, Jonathan M Davis wrote: However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Example? Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); In the case of sort, you may know where it's from - particularly since it's so common - but the less well-known the function is, the less likely that is at all obvious where it comes from, and if you're dealing with 3rd party software, then it wouldn't be at all obvious. For instance, how would you know that party.foo is really party.bar.foo? You wouldn't. Why should you? Being so lax about importing could really harm code readibility (and maintainibility, since it increases the odds of name clashes). So, I'm inclined to say that that is a _bad_ idea. Maybe if you produce a solid example, I'd be convinced. Andrei
Re: DIP16: Transparently substitute module with package
My comments: 1. My first impression was that using foo/bar/package.d instead of foo/bar.d seemed a bit odd and messy. But I realize now that cleverly solves the issue where foo/bar.d would be considered to be inside a different package from foo/bar/*.d. So I like that. Personally, I think I would have gone with foo/bar/_.d as that sorts much, much better, but naming debates can go on forever, and I can live with package.d 2. I don't understand any of this: --- When looking up the symbol foo.bar.baz, currently an exact match is needed. However. when looking up .baz or simply baz, a flexible lookup is used that has many advantages (less verbose, hijacking detection etc). Therefore we think similar flexibility should be imparted to foo.bar.baz, as follows: If a qualified symbol foo.bar.baz appears in code, the compiler considers foo.bar a prefix that sets the starting point of the lookup, and then proceeds with looking up baz from that starting point. That means a program that imports std.algorithm may use std.sort for the symbol std.algorithm.sort. --- I *do* understand a program that imports std.algorithm may use std.sort for the symbol 'std.algorithm.sort', and I think that's a good idea. It solves a problem I hadn't even thought of. But I don't understand that stuff I quoted above. Perhaps you could reword/clarify? 3. Other than that stuff, I'm very much in favor of this. I'll have some of that!
Re: DIP16: Transparently substitute module with package
On Friday, 30 March 2012 at 18:15:57 UTC, Brad Anderson wrote:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
On 3/30/12, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
Destroy!
That means a program that imports std.algorithm may use
std.sort
for the symbol std.algorithm.sort.
That's quite interesting. Would that also mean that you could
do:
import std.algorithm; // has indexOf
import std.string; // has indexOf
void main() {
string.indexOf(foo, foo); - std.string.indexOf
}
I was actually kind of surprised when I found out this doesn't
work. It
seems so natural to resolve ambiguity using as little context
as necessary.
Ya that was the behavior I expected as well. Would be great if it
worked like that. Just back trace the reference until the
ambiguity is resolved.
// -
Also, I'm probably missing something here, but I never understood
why importing a package doesn't work like it does in
Actionscript/Java/others...
import foo.bar.*; // everything
import foo.bar.all; // custom
That makes a lot of sense to me.
Re: DIP16: Transparently substitute module with package
Timon Gehr timon.g...@gmx.ch wrote in message news:jl4jmg$2j1r$1...@digitalmars.com... I don't really like the second one. 1. It is an over-general solution, because it does not solve a general problem. Maybe it would be better to just interpret foo.bar.baz as foo.bar.package.baz if foo.bar is a package that has been imported via the foo.bar.package rewrite? That occurred to me, and I thought about proposing the same thing you're suggesting, but on second thought I wasn't so sure: If I need to disambiguate between std.algorithm.find and foo.bar.baz.find, it might be nice to be able to just say Meh, just...that one in Phobos, ie 'std'. Or Just go with that 'foo' one. I could go either way, really.
Re: DIP16: Transparently substitute module with package
Robert Clipsham rob...@octarineparrot.com wrote in message news:jl4l5t$2m62$1...@digitalmars.com... On 30/03/2012 15:46, Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Destroy! Andrei The proposal doesn't say what happens when package.d is not found but foo/bar/ exists. Given that a lot of people will just use public import foo.bar.*; in that file, would it make sense for package.d missing to imply import foo.bar.*? That would save typing out every single file in there. Of course it would also be annoying if you wanted to import everything except one file, as you'd then have to type out every single import anyway. That would effectively be the same as Java's import foo.* and a lot of people have issues with that. The other option is to error, which is probably a more sane option. That's what I'd suggest doing. Just treat it like importing any other missing package.
Re: DIP16: Transparently substitute module with package
On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote: On 3/30/12 1:39 PM, Jonathan M Davis wrote: However, I'm very nervous about the second part. e.g. std.sort instead of std.algorithm.sort seems like a bad idea to me. It increases the odds of name conflicts for little benefit. Example? std.sort works because there's only one sort. If there are two, you get a conflict (e.g. if you had std.path.sort which sorted paths in some path-specific manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It would be similar to having to specify std.algorithm.indexOf when you've imported both std.string and std.algorithm. But the problem is when std.path.sort is added _later_. All of a sudden, code which used std.sort and worked is now broken. The problem does currently exist in that if we added indexOf to another module - say std.array - then code which imported either std.string or std.algorithm as well as std.array would break with the addition of std.array.indexOf, but your proposal makes it worse. Not only does it provide another way in which adding a function could result in conflicts when existing code is recompiled, but it makes it so that if you add any function anywhere in the _entire standard library_ which has the name as an existing one, you get a conflict (if anyone uses std.x rather than x or the full import path). D does a good job of providing ways to fix name conflicts, but it doesn't do a good job of preventing them when adding new symbols to a library (primarily because it doesn't use static imports by default), and your proposal makes that part of the problem worse. If std.x were to become common practice, then any time that you added a symbol to a library when that symbol was already used by another module, you'd create conflicts (combined with the fact that private doesn't hide symbol names but merely makes them inaccessible, this could result in a lot of symbol name conflicts). Not to mention, it'll make it a lot more confusing to find what modules stuff is actually in if people start doing stuff like std.sort(arr); In the case of sort, you may know where it's from - particularly since it's so common - but the less well-known the function is, the less likely that is at all obvious where it comes from, and if you're dealing with 3rd party software, then it wouldn't be at all obvious. For instance, how would you know that party.foo is really party.bar.foo? You wouldn't. Why should you? Do you know what the foo function does? If you don't, you're going to have to look it up. And if you don't know what module it comes from, you can't do that. You also have to know where foo is from if a foo function is added to another module and causes a conflict, because you're going to have to give the full import path to actually use it. That's currently true with just bare foo as well, but party.foo gives the illusion of specifying where foo is from without actually specifying where it's from. At least right now, if foo is used with its import path, you know that that's actually its import path. Also, what happens if we want to add a module named sort later? The fact that people are using std.sort means that adding std.sort as a module will break code. Granted, it's not very likely that we're going to add a module named sort, but there are plenty of other symbol names that it could happen with. But then again, if we decided to provide a module with all of the major sort algorithms, then maybe we _would_ create a module named std.sort. Just because we don't see a need now doesn't mean that we won't later. In either case, by allowing std.x where x is a symbol in any sub-module of std, you're going to create conflicts any time that you add a module which has the same name as an existing symbol anywhere in the library. Being so lax about importing could really harm code readibility (and maintainibility, since it increases the odds of name clashes). So, I'm inclined to say that that is a _bad_ idea. Maybe if you produce a solid example, I'd be convinced. Well, as I've pointed with a few examples here, your proposal will increase the chances of adding symbol conflicts any time that a symbol is added to a library all just so that you can do std.algorithm.sort instead of std.algorithm.submodule1.sort once sort has been moved to std.algorithm.sumodule1. And we could make it possible to do std.algorithm.sort without adding all of those possible conflicts. The simplest solution would simply be to make it so that if std.algorithm.sort is used, and std.algorithm is a package with a std.algorithm.package module, then the compiler looks in all of the sub-modules of std.algorithm to find sort. That solves the problem right there without increasing the odds of symbol conflicts across the entire library like your
Re: DIP16: Transparently substitute module with package
Andrej Mitrovic andrej.mitrov...@gmail.com wrote in message news:mailman.1240.1333130858.4860.digitalmar...@puremagic.com... Still this is one of the few proposals I like. My only caveat is the comment: except the file is not allowed to use the module declaration.. Wouldn't it be better if we explicitly declared a module as a package instead? In foo\bar\package.d: package foo.bar; Since the module declaration must be on the first line (or second line after shebang), you could special-case DMD to allow the package keyword to be used here. I know D likes to abuse a keyword for multiple things (hello Mr. Static!), but I think we could live with it. Or maybe just require the module name ends with .package
Re: DIP16: Transparently substitute module with package
On Fri, Mar 30, 2012 at 05:35:25PM -0400, Jonathan M Davis wrote: [...] But personally, I like the idea of making it so that publicly imported symbols can be accessed as if they were in the module that publicly imported them (with package.d being treated as if it had the same name as the package that it's in). That's essentially how it already works except when specifying the full import path for a symbol. And that way, you can specify in std.algorithm.package.d exactly what you want to be imported when std.algorithm is imported (including using : to restrict it to specific symbols in a module), and only those symbols will be treated as if they were part of std.algorithm - both for importing purposes and when specifying the import path when using a symbol. The library maintainer then has control over which symbols get used with which import paths. [...] +1. I also have my doubts about the wisdom of std.sort implicitly binding to std.algorithm.sort. It's nice syntactic sugar in the short term, but as Jonathan points out, it can cause headaches in the long term. I say we should hold off on that part of the proposal. T -- Microsoft is to operating systems security ... what McDonalds is to gourmet cooking.
Re: DIP16: Transparently substitute module with package
On 2012-03-30 14:46:19 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: Destroy! Since you're asking… One thing the current system avoids is unresolvable symbols. If two symbol name clashes, you just qualify them fully and it'll always be unambiguous. For instance: .std.algorithm.sort(…) Now, if std.algorithm becomes both a module and a package, you could have both a sort function and a sort submodule with no way to distinguish between the two, even when fully qualified. I think this is why D does not allow modules to have the same name as packages. I understand that you try to work around this problem by inventing a .std.algorithm.package scope. Then you make it's content imported automatically inside the .std.algorithm scope for backward compatibility (and convenience). The problem is that if .std.algorithm.package contains a sort function and there is also a module called std.algorithm.sort, the fully-qualified name of that 'sort' module will become ambiguous. Moreover, whether the fully-qualified name .std.algorithm.sort is ambiguous or not depends on what modules were imported, which is not a very desirable behaviour. So to make sure there is no unresolvable fully-qualified names, when importing std.algorithm.sort the compiler should make sure that no symbol called 'sort' already exist in the .std.algorithm scope (which includes the symbols in .std.algorithm.package and all other packages inside std.algorithm). This is clearly untenable. - - - I recognize the need. If I may, I'll propose something simpler: 1. allow both std/algorithm.d and std/algorithm/sort.d to exist 2. importing std.algorithm.sort will also implicitly import std.algorithm (if it exists) and std (if it exists) 3. if any symbol of std.algorithm clash with the std.algorithm.sort module name, you get an error when importing std.algorithm.sort. Effectively, importing std.algorithm.sort becomes synonymous to importing std, std.algorithm, and std.algorithm.sort. This is what's needed to detect clashes in fully-qualified names. The only issue now (beside a few more imports) is that if std.algorithm imports any of its submodule it becomes a circular import. That's usually fine in D, but not when you have module constructors. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: DIP16: Transparently substitute module with package
On 3/30/12 10:46 PM, Andrei Alexandrescu wrote: Starting a new thread from one in announce: http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16 Please comment, after which Walter will approve. Walter's approval means that he would approve a pull request implementing DIP16 (subject to regular correctness checks). Great. Large modules are my main complaint about D. :-) If I correctly understand the second part (because I couldn't understand the text in the proposal until I read some comments here), then it makes sense. Is it like this? sort(...) - search sort in every module out there std.sort(...) - search sort in every module that's in the std package If both std.algorithm.sort and std.path.sort exist, or something like that, then you would anyway get a clash so you'd have to fully qualify it. But if std.algorithm.sort and foo.bar.sort and you'd import both: import std.algorithm.package; import foo.bar.package; and you'd wanted to use both, then it could be convenient: std.sort(...) foo.sort(...) Though I wonder if this indeed happens a lot. That's why I would wait until there's a real need for it. The main complaint people have is not having a way to import all files in a directory, which is the first point, but I never heard a complaint about the second point. Also, I think it would make sense to change the first part to this: * If the compiler sees a request for importing foo.bar and foo/bar is a directory, then automatically look for the file foo/bar/package.d. *If it doesn't exist, automatically expand the import to import all files under that directory.* If both foo/bar.d and foo/bar/ exists, compilation halts with an error. That way you have convenience and safety. Most of the time people just put in package.d a list of all the files in that directory. Maybe sometimes (not sure) people restrict that list to some modules. And in those cases you can just restrict the list in package.d Please, it's the year 2012. Compilers need to be smarter. Save people some typing time. You save them typing all the imports. But then you make them typing them in that pacakge.d file. Hmm...
Re: DIP16: Transparently substitute module with package
I'm pretty impressed with the idea, and look forward to its implementation, but I do have one question. How does this affect (if at all) the implicit friend relationship of declarations? Since package foo.bar is treated as a single module, do the classes in foo/bar/alpha.d and foo/bar/beta.d have access to each other's private members? I'm not sure whether I favor them losing or keeping their friend status. -- Chris NS
Re: DIP16: Transparently substitute module with package
On Saturday, March 31, 2012 06:23:32 Chris NS wrote: I'm pretty impressed with the idea, and look forward to its implementation, but I do have one question. How does this affect (if at all) the implicit friend relationship of declarations? Since package foo.bar is treated as a single module, do the classes in foo/bar/alpha.d and foo/bar/beta.d have access to each other's private members? I'm not sure whether I favor them losing or keeping their friend status. It wouldn't affect any of that. If something is marked private, then everything within its module has access to it, but nothing else does. If something is marke package, then everything within that package has access to it, but nothing else does. By splitting up foo.bar into foo.bar.alpha and foo.bar.beta, you've created a package. Everything private in either foo.bar.alpha is inaccessible to everything in foo.bar.beta and vice versa. If you want it to be accessible to the other module, then mark it as package. Anything which was private in foo.bar, was an implementation detail of foo.bar, and nothing using foo.bar cared. Anything which is private or package in foo.bar.alpha and foo.bar.beta is an implementation detail, and nothing using anything in foo.bar cares. The whole point of this proposal is to make it seemless for the users of a module when a module is split into a package. And the implementation details of a module are irrelevant to that. It's just the public and protected stuff which matters. - Jonathan M Davis
