Re: A suggestion for modules names / sharing code between projects
On Thursday, 3 March 2016 at 20:05:03 UTC, ag0aep6g wrote: [1] https://en.wikipedia.org/wiki/Reverse_domain_name_notation LOL at the code section of that page.
Re: A suggestion for modules names / sharing code between projects
On 03.03.2016 07:53, Sebastien Alaiwan wrote: However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d". Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.). Yeah, that's a thing. For published libraries one should choose a unique name. I think reverse domain name notation [1] is a convention for exactly that. As far as I know, not many D projects do that, though, if any at all. But it would be a way out if collisions become a problem. Also, dub/code.dlang.org plays a role here, as people may assume that a name is free as long as it's not on the dub list. This works well when dub is indeed the one place to go for libraries. If it isn't, then things may get messy. If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols) Yeah, as far as I can tell, you would have to change the name of one of the libraries, and update all related module declarations and imports. [1] https://en.wikipedia.org/wiki/Reverse_domain_name_notation
Re: A suggestion for modules names / sharing code between projects
On Thursday, 3 March 2016 at 08:39:51 UTC, Laeeth Isharc wrote:
On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien Alaiwan
If, unfortunately, I happen to run into a conflict, i.e my
project uses two unrelated libraries sharing a name for their
top-level namespace, there's no way out for me, right?
(Except relying on shared objects to isolate symbols)
See modules doc page off dlang.org.
import io = std.stdio;
void main()
{ io.writeln("hello!"); // ok, calls std.stdio.writeln
std.stdio.writeln("hello!"); // error, std is undefined
writeln("hello!"); // error, writeln is undefined
}
Thansk, but I was talking about module name collisions, e.g if my
project uses two unrelated libraries, both - sloppily - defining
a module called "geom.utils".
Can this be worked-around using renamed imports?
How is "import myUtils = geom.utils" going to know which of both
"geom.utils" I'm talking about?
As far as I understand, _module_ renaming at import is rather a
tool for defining shorthands, than for disambiguation.
I agree that it might not be a problem in practice - just wanting
to make sure I'm not missing something.
Re: A suggestion for modules names / sharing code between projects
On Thursday, 3 March 2016 at 06:53:33 UTC, Sebastien Alaiwan
wrote:
Hi guys,
thanks a lot for your answers!
On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote:
On 02.03.2016 21:40, Sebastien Alaiwan wrote:
- I only work with separate compilation, using gdc
(Windows-dmd produces OMF, which is a dealbreaker ; but I use
rdmd a lot for small programs).
dmd gives you COFF with -m64 or -m32mscoff (-m32 is the
default is => OMF).
Thanks for the tip, I didn't know that.
I have actually deeper reasons for not using dmd (seemless
cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or
"make CROSS_COMPILE=arm-linux-gnueabi" without any special case
in the Makefile ; and the way its unusual command line
(-oftarget) conflicts with MSYS's path conversion, etc. ).
I don't know the exact reasons why the module system works
like it does. Maybe you're on to something, maybe not. Anyway,
here are some problems/surprises I see with your alternative
idea (which basically means using relative file paths, if I
get you right):
Yes, I think we can say that.
I 100% agree with all the limitations you listed ; which makes
a forced absolute-import system appear to be a lot more
comfortable!
There's also one big limitation I see that you didn't list:
* If you're importing lib_a and lib_b both depending on lib_c,
you need them to expect lib_c to be at the same path/namespace
location. The current module system guarantees this, because
all package names are global.
(And linking with precompiled binaries might become a nightmare)
However, using global package names means you're relying on the
library providers to avoid conflicts, in their package names,
although they might not even know each other. Which basically
implies to always put your name / company name in your package
name, like "import alaiwan.lib_algo.array2d;", or "import
apple.lib_algo.array2d".
Or rename my "lib_algo" to something meaningless / less common,
like "import diamond.array2d" (e.g Derelict, Pegged, imaged,
etc.).
If, unfortunately, I happen to run into a conflict, i.e my
project uses two unrelated libraries sharing a name for their
top-level namespace, there's no way out for me, right?
(Except relying on shared objects to isolate symbols)
See modules doc page off dlang.org.
import io = std.stdio;
void main()
{ io.writeln("hello!"); // ok, calls std.stdio.writeln
std.stdio.writeln("hello!"); // error, std is undefined
writeln("hello!"); // error, writeln is undefined
}
Also static import and scoped imports.
Re: A suggestion for modules names / sharing code between projects
Hi guys, thanks a lot for your answers! On Wednesday, 2 March 2016 at 22:42:18 UTC, ag0aep6g wrote: On 02.03.2016 21:40, Sebastien Alaiwan wrote: - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs). dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF). Thanks for the tip, I didn't know that. I have actually deeper reasons for not using dmd (seemless cross-compilation, e.g "make CROSS_COMPILE=i686-w64-mingw32" or "make CROSS_COMPILE=arm-linux-gnueabi" without any special case in the Makefile ; and the way its unusual command line (-oftarget) conflicts with MSYS's path conversion, etc. ). I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right): Yes, I think we can say that. I 100% agree with all the limitations you listed ; which makes a forced absolute-import system appear to be a lot more comfortable! There's also one big limitation I see that you didn't list: * If you're importing lib_a and lib_b both depending on lib_c, you need them to expect lib_c to be at the same path/namespace location. The current module system guarantees this, because all package names are global. (And linking with precompiled binaries might become a nightmare) However, using global package names means you're relying on the library providers to avoid conflicts, in their package names, although they might not even know each other. Which basically implies to always put your name / company name in your package name, like "import alaiwan.lib_algo.array2d;", or "import apple.lib_algo.array2d". Or rename my "lib_algo" to something meaningless / less common, like "import diamond.array2d" (e.g Derelict, Pegged, imaged, etc.). If, unfortunately, I happen to run into a conflict, i.e my project uses two unrelated libraries sharing a name for their top-level namespace, there's no way out for me, right? (Except relying on shared objects to isolate symbols) I don't think your alternative is obviously superior. Me neither :-) I'm just sharing my doubts about the current system! The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess. Yes, I guess you're right. I'm going to follow your advice and Mike's for my projects, this will allow me to see things more clearly. Anyway, thanks again to all of you for your answers!
Re: A suggestion for modules names / sharing code between projects
On Thursday, 3 March 2016 at 01:47:08 UTC, Mike Parker wrote: The package and module system is very intuitive to me and has been from the beginning, so I'm still not quite sure what it is about it that bothers you. The two major issues you bring up here, the need to import package.module and how to structure your projects, are easily resolved. Am I misunderstanding something? I should also say that I get the impression that you aren't considering the package name as part of the module name. Maybe that's the root of your problem. As I said in a previous post, your desire to use the default package for everything so that you can simply import foo is perfectly fine. The price of doing that, as you point out here, is that you can easily run into conflicts. Packages are the solution to avoiding them and package names are part of the module name, such that module foo is no longer just foo, but mypack.foo. Without a module declaration, the compiler would need some other way to know that C:\\src\libs\mylib\mypack\foo.d is actually mypack.foo and not c.src.lib.mylib.mypack.foo (which again, is why it only uses the file name when no module declaration is given). I can't think of a better way to do it than putting it right there in the source file. Using a command line switch would not be scalable.
Re: A suggestion for modules names / sharing code between projects
On Wednesday, 2 March 2016 at 20:40:39 UTC, Sebastien Alaiwan
wrote:
However, it feels wrong that modules (foo, bar) inside the same
package ("pkg") need to repeat the "absolute" package name when
referencing each other, I mean "import pkg.foo" instead of
"import foo". Not because of the extra typing, but because of
the dependency it introduces. But as I said, maybe my vision is
wrong.
The package name is part of the module name. If you use
thislib.somemod and thatlib.somemod, how would you disambiguate
them without the package name? Even in C and C++, libraries are
often distributed with header files in a sub directory, where the
top-level directory is intended to be added to the include path
and not the subdirecotry, otherwise two includes with the same
name conflict. Some libraries go further and prefix their header
files with a namespace (like sdl_foo.h, sdl_bar.h), because they
get no help from the compiler with this. Making the package name
part of the fully qualified module name helps minimize the
likelihood of such conflicts. It also gives a natural namespace
for conflicting symbols in two modules named foo. You can use
thispack.somemod.foo and thatpack.somemod.foo to disambiguate.
- I only work with separate compilation, using gdc (Windows-dmd
produces OMF, which is a dealbreaker ; but I use rdmd a lot for
small programs).
-m64 or -m32mscoff
- I often mix C code with D, sometimes, C++ code. I compile
them using gcc/g++.
Assuming MinGW, the GDC is definitely your best bet here. The
flags above will get DMD to produce COFF for the MS linker, but
MinGW COFF and MS COFF are not always compatible.
With this flat directory structure, the only way test.d can
import files is by saying, "import stack;", not "import
lib_algo.stack".
(adding ".." to the list of import directories might do the
trick, but this would definitely feel wrong).
It is wrong. You should add ../mylibs :)
Now, let's see a project using this "lib_algo", I call it
"myGame".
$ cd myProjects
$ find
myGame/lib_algo/options.d
myGame/lib_algo/
myGame/lib_algo/array2d.d
myGame/lib_algo/project.mk
myGame/lib_algo/Makefile
myGame/src/main.d
myGame/Makefile
"lib_algo" simply is an external (aka "git submodule"),
pointing to a specific revision of the repository "lib_algo".
The top-level "Makefile" of the projects includes
"lib_algo/project.mk", and all is well, I can compose the list
of source files to compile without having to rely on
"convenience/intermediate" libraries.
Now, if "src/main.d" wants to import "array2d.d", currently, I
must write "import array2d.d", and add "-I myGame/lib_algo" to
my compiler command line.
I said in a previous post that this would not scale.
That's because with this scheme, obviously, I can't have two
modules having the same name, even if they belong to different
libraries.
Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if
"main.d" says "import utils.d", it's ambiguous.
Assuming that all of the libraries you are going to use are set
up the same way, then you can simply pass -ImyGame. Then you can
use lib_algo.utils, lib_geom.utils, and so on. What's the problem
with that?
Clearly, "main.d" needs to say "import lib_algo.utils".
However, if I do this,
I must add a module declaration in "lib_algo/utils.d", and I
must change
"lib_algo/test.d" so it says "import lib_algo.utils".
This is where it begins to feel clumsy. Why should lib_algo
need to be
modified in order to resolve an ambiguity happening in "myGame"
?
You shouldn't have to "change" it. This is how it should be set
up from the beginning. The idea is that you decide on a module
structure as soon as you start your project and then you stick to
it.
The module declaration is necessary when using packages because
otherwise DMD has no idea how far up the path to go to create a
default package name. That's why the it uses only the file name
when no module declaration is present. The package name is the
solution to *prevent* ambiguities, not to resolve them after the
fact. If you don't want lib_algo/util.d and lib_geom/util.d to
conflict, then the use both lib_algo and lib_geom as package
names in module declarations. How else would you propose
resolving the ambiguity?
Moreover, we saw earlier that this modification of the import
done by
"lib_algo/test.d" had consequences on the directory structure
of lib_algo.
In order to be able to build the test executable for lib_algo,
I now need to
have the following structure:
$ find lib_algo
lib_algo/lib_algo/options.d
lib_algo/lib_algo/
lib_algo/lib_algo/array2d.d
lib_algo/test.d
(test.d can be put inside the inner "lib_algo")
Am I the only one to find this counterintuitive - and probably
wrong?
What am I missing there?
No, you don't have to set it up that way. If your top-level
directory is always named the same as your top-level package
name, then you can just use its parent directory.
Re: A suggestion for modules names / sharing code between projects
On 02.03.2016 21:40, Sebastien Alaiwan wrote: - I only work with separate compilation, using gdc (Windows-dmd produces OMF, which is a dealbreaker ; but I use rdmd a lot for small programs). dmd gives you COFF with -m64 or -m32mscoff (-m32 is the default is => OMF). [...] Clearly, "main.d" needs to say "import lib_algo.utils". However, if I do this, I must add a module declaration in "lib_algo/utils.d", and I must change "lib_algo/test.d" so it says "import lib_algo.utils". That's the way you're supposed to do it, yes. One source file <-> one module name. This is where it begins to feel clumsy. Why should lib_algo need to be modified in order to resolve an ambiguity happening in "myGame" ? I don't know the exact reasons why the module system works like it does. Maybe you're on to something, maybe not. Anyway, here are some problems/surprises I see with your alternative idea (which basically means using relative file paths, if I get you right): * You need to be aware of the directory structure when reading/writing code, as `import foo;` may mean different things depending on where you are. And you don't have a module declaration to tell you where you are. * You're not able to copy imports from one file to another anymore. * When moving a module, you have to update all its imports. You can introduce a subtle bug when you miss one, because the old import may still be valid but have a different meaning. * The name of a module depends on the current working directory of the compiler run. You can't change directories between compiler calls and link the object files together. * You can deliberately or accidentally import one source file under two different module names. Two modules from one source file is weird, especially when it happens accidentally. * Can't just throw source files from different packages at the compiler anymore without thinking about their relative locations or the -I switch. [...] Moreover, we saw earlier that this modification of the import done by "lib_algo/test.d" had consequences on the directory structure of lib_algo. In order to be able to build the test executable for lib_algo, I now need to have the following structure: $ find lib_algo lib_algo/lib_algo/options.d lib_algo/lib_algo/ lib_algo/lib_algo/array2d.d lib_algo/test.d (test.d can be put inside the inner "lib_algo") Or, you know: `-I..` :P But changing the directory structure is probably nicer. Am I the only one to find this counterintuitive - and probably wrong? What am I missing there? I don't think your alternative is obviously superior. The current way of having exactly one fixed name per module makes it easy to see what's going on at any point. You have to pay for that by typing a bit more, I guess.
Re: A suggestion for modules names / sharing code between projects
On Wednesday, 2 March 2016 at 08:50:50 UTC, Mike Parker wrote:
I'm curious what sort of system you have in mind. How does the
current system not work for you and what would you prefer to
see?
First, you must know that I've been a C++ programmer for way too
much time ;
and that my way of thinking about modules has probably been -
badly - shaped accordingly :-).
However, it feels wrong that modules (foo, bar) inside the same
package ("pkg") need to repeat the "absolute" package name when
referencing each other, I mean "import pkg.foo" instead of
"import foo". Not because of the extra typing, but because of the
dependency it introduces. But as I said, maybe my vision is wrong.
I'm going to answer your first question, I hope you don't get
bored before the end ;-)
First some generic stuff:
- The libraries I write are source-only, and I never install them
system-wide, as many projects might depend on different revisions
of one library.
- I only work with separate compilation, using gdc (Windows-dmd
produces OMF, which is a dealbreaker ; but I use rdmd a lot for
small programs).
- I often mix C code with D, sometimes, C++ code. I compile them
using gcc/g++.
- My build system is composed of simple dependency-aware
non-recursive makefiles (BTW, this is a great way to keep your
code decoupled, otherwise you might not get feedback that you're
making a dependency mess soon enough!)
This is the best way I found to be able to checkout one project,
type 'make',
and have everything build properly, wether I'm doing it on
GNU/Linux or on Windows (MSYS).
Everything is under version control using GNU Bazaar (which
basically is to git what D is to C++).
I have a small library, which I call "lib_algo", containing
semi-related utilities (containers, topological sort, etc.) which
I use in many projects.
$ cd myLibs
$ find lib_algo -name "*.d" -or -name "*.mk" -or name "Makefile"
lib_algo/options.d
lib_algo/misc.d
lib_algo/pointer.d
lib_algo/set.d
lib_algo/queue.d
lib_algo/algebraic.d
lib_algo/vector.d
lib_algo/bresenham.d
lib_algo/topological.d
lib_algo/fix_array.d
lib_algo/test.d
lib_algo/pool.d
lib_algo/stack.d
lib_algo/list.d
lib_algo/array2d.d
lib_algo/project.mk
lib_algo/Makefile
(I know, there's some overlap with Phobos today)
The project.mk simply contains the list of source files of the
library.
The test.d file actually contains a main function, which I use to
run manual tests of the library functionalities. It's not
included in the project.mk list. And it obviously must import the
files needing to be tested.
With this flat directory structure, the only way test.d can
import files is by saying, "import stack;", not "import
lib_algo.stack".
(adding ".." to the list of import directories might do the
trick, but this would definitely feel wrong).
Now, let's see a project using this "lib_algo", I call it
"myGame".
$ cd myProjects
$ find
myGame/lib_algo/options.d
myGame/lib_algo/
myGame/lib_algo/array2d.d
myGame/lib_algo/project.mk
myGame/lib_algo/Makefile
myGame/src/main.d
myGame/Makefile
"lib_algo" simply is an external (aka "git submodule"), pointing
to a specific revision of the repository "lib_algo".
The top-level "Makefile" of the projects includes
"lib_algo/project.mk", and all is well, I can compose the list of
source files to compile without having to rely on
"convenience/intermediate" libraries.
Now, if "src/main.d" wants to import "array2d.d", currently, I
must write "import array2d.d", and add "-I myGame/lib_algo" to my
compiler command line.
I said in a previous post that this would not scale.
That's because with this scheme, obviously, I can't have two
modules having the same name, even if they belong to different
libraries.
Let's say I have "lib_algo/utils.d" and "lib_geom/utils.d", if
"main.d" says "import utils.d", it's ambiguous.
Clearly, "main.d" needs to say "import lib_algo.utils". However,
if I do this,
I must add a module declaration in "lib_algo/utils.d", and I must
change
"lib_algo/test.d" so it says "import lib_algo.utils".
This is where it begins to feel clumsy. Why should lib_algo need
to be
modified in order to resolve an ambiguity happening in "myGame" ?
Moreover, we saw earlier that this modification of the import
done by
"lib_algo/test.d" had consequences on the directory structure of
lib_algo.
In order to be able to build the test executable for lib_algo, I
now need to
have the following structure:
$ find lib_algo
lib_algo/lib_algo/options.d
lib_algo/lib_algo/
lib_algo/lib_algo/array2d.d
lib_algo/test.d
(test.d can be put inside the inner "lib_algo")
Am I the only one to find this counterintuitive - and probably
wrong?
What am I missing there?
(And please, don't tell me that I need to switch to
dub+git+dmd+D_source_files_only ; I like my tools to be
composable)
Thanks for reading!
Re: A suggestion for modules names / sharing code between projects
On Tuesday, 1 March 2016 at 06:33:06 UTC, Sebastien Alaiwan wrote: However, we still need separate compilation. Otherwise your turnaround time is going to resemble a tractor pulling competition as your project grows. Recompiling everything everytime you change one module is not an option ; Some of the source files I work on in my company require 30s to compile *alone* ; it's easy to reach, especially when the language has features like templates and CTFE (Pegged anyone?). Let me hype DUB a bit. :) DUB provides completely composable dependencies, in a chain (A => B => C) with A not knowing about C. Compile-times are excellent since DUB can do package-wise, file-wise or all-at-once builds from the same description and use caching. Having a declarative build description allows for some goodies like IDE project generation. The package namespace is global, but hierarchical with names like "myorg:mypackage". However, you can't have versionned dependencies for private packages [painlessly] which is quite a big limitation.
Re: A suggestion for modules names / sharing code between projects
On Wednesday, 2 March 2016 at 06:40:53 UTC, Sebastien Alaiwan wrote: Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible. TBH I'm not sure what module directives do now that you mention them.
Re: A suggestion for modules names / sharing code between projects
On Wednesday, 2 March 2016 at 06:40:53 UTC, Sebastien Alaiwan wrote: Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible. Every module you create will have a module name whether you use a module declaration or not. If you don't use one, the compiler uses the file name by default, but *only* the file name. The path on the file system is in no way taken into account (I suppose it could be by adding a flag to the compiler, but it seems to me that would needlessly complicate what is a really simple and easy to use system). A module with no package name is in the default package, i.e. given src/foo.d, using the following declaration: module foo; Is the same as using none at all. If you are content with keeping all of your modules in one or more top-level directory and use no packages, then you can leave out the module declarations and just pass each top-level directory to the compiler with -I. I still think that software entities shouldn't depend upon their own name, or their absolute location in the top-level namespace. This is true for classes, functions/methods, variables, structures ... maybe this rule becomes invalid when third-party packages enter a project? The module names and package names are mangled into the symbol names in the binary. They also serve as namespaces to distinguish conflicting symbols in code. So yes, all of the symbols in a module are dependent on their location in the namespace. It's baked into the language. I'm curious what sort of system you have in mind. How does the current system not work for you and what would you prefer to see?
Re: A suggestion for modules names / sharing code between projects
On Wednesday, 2 March 2016 at 00:16:57 UTC, Guillaume Piolat wrote: Would this work? 1. pick a single module name like module math.optimize; 2. import that module with: import math.optimize; 3. put this module in a hierarchy like that: math/optimize.d 4. pass -I to the compiler However it may clutter your module namespace a bit more. Yeah, this is what I going to do: flat hierarchy, like one 'src' directory and one 'extra' directory in the project root, globally unique package names, passing -Isrc and -Iextra to the compiler, and ... module declaration directives. I was trying to avoid them, but maybe it's not possible. I still think that software entities shouldn't depend upon their own name, or their absolute location in the top-level namespace. This is true for classes, functions/methods, variables, structures ... maybe this rule becomes invalid when third-party packages enter a project?
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote: Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation). I'd like to be able to write, in projectA's main: import internal.math.optimize; This requires me to add, at the beginning of "optimize.d" file, this module definition: module internal.math.optimize; However, this "optimize.d" file is shared between projects, now it's becoming specific to projectA. How am I supposed to share code between projects then? Would this work? 1. pick a single module name like module math.optimize; 2. import that module with: import math.optimize; 3. put this module in a hierarchy like that: math/optimize.d 4. pass -I to the compiler However it may clutter your module namespace a bit more.
Re: A suggestion for modules names / sharing code between projects
Hi Mike, thanks for taking the time to answer. If I understand you correctly, you're advising me to keep my file hierarchy mostly in sync with my module hierarchy (except for one potential leading "src/" or "libs/" that I would add to the import search directories). That's fine with me. However, what's the point of having module declarations directives in the language then? If I understand correctly, the only way to make an interesting use of these directives makes separate compilation impossible. I know dmd is lightning-fast and everything, and not having to manage build dependencies anymore is very attractive. However, we still need separate compilation. Otherwise your turnaround time is going to resemble a tractor pulling competition as your project grows. Recompiling everything everytime you change one module is not an option ; Some of the source files I work on in my company require 30s to compile *alone* ; it's easy to reach, especially when the language has features like templates and CTFE (Pegged anyone?).
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 21:43:31 UTC, Sebastien Alaiwan wrote: Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"? You have to tell the compiler where it is. Is it only possible with separate compilation? Given two modules, src/lib/foo.d and and src/main.d, you can pass them both to the command line and not worry about passing -I to the compiler. dmd src/main.d src/lib/foo.d In this case, foo.d can have a module statement that gives it any name at all. It does not need to match its directory structure: module wicked.cool.mod; This works because you've given the module directly to the compiler and it doesn't need to search for it. If you compile them separately: dmd -c src/lib/foo.d dmd src/main.d foo.obj Now you will see a compiler error. You've given the compiler foo's object file, but it doesn't know where the source file is. When main.d imported wicked.foo.mod, the compiler tried to find wicked/foo/mod.d from its current working directory. No joy. You can test this changing the module declaration in foo.d to lib.foo, then importing lib.foo in main.d. The you can do this: cd src dmd -c lib/foo.d dmd main.d foo.obj Now when it encounters import lib.foo, it will look for lib/foo.d and find it. Now do this: cd .. dmd -c src/lib/foo.d dmd src/main.d foo.obj Again, failure, because it's looking for lib/foo.d. So now you have to tell it where to find lib/foo.d: dmd -Isrc src/main.d foo.obj Now it will look for src/lib/foo.d and find it. This is why you shouldn't muck about with package names. If you want to change the name of the src directory, that's fine. You can simply pass -I to the compiler with the correct value. But if you want to change the name of lib, you are now affecting the compiler's ability to find imports. This becomes a problem with separate compilation (as above), which also means it's a problem when using libraries. If you change a package name for something you're distributing as a library, then all the import statements in client code will need to be changed as well.
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 21:35:48 UTC, Adam D. Ruppe wrote: On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote: What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;". That's easy to automate, just run a find/replace across all the files, or if you want to go really fancy, modify hackerpilot's dfix to do it for you. Yes, I know ; but how can you say there's no redundancy when I have to rely on find/replace to keep all these declarations in sync?
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 21:33:37 UTC, Adam D. Ruppe wrote: On Monday, 29 February 2016 at 21:04:52 UTC, Sebastien Alaiwan wrote: Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"? You have to tell the compiler where it is. Is it only possible with separate compilation? Either way, the module name is *not* optional. (As a reminder, as I said, I'm using separate compilation) meh that's part of your problem, why are you doing it that way? Because it reduces turnaround time (this is something I actually measured on the project I'm currently working on). Having to recompile everything each time I make a modification takes up to 25s on this project. By the way, I'm using gdc. Is it also part of my problem? :-)
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote: What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;". That's easy to automate, just run a find/replace across all the files, or if you want to go really fancy, modify hackerpilot's dfix to do it for you.
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 21:04:52 UTC, Sebastien Alaiwan wrote: Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"? You have to tell the compiler where it is. Either way, the module name is *not* optional. (As a reminder, as I said, I'm using separate compilation) meh that's part of your problem, why are you doing it that way?
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 20:00:48 UTC, Sebastien Alaiwan wrote: On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote: My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having. Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right? Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me. How is it going to work if I want to share heavily templated code, like a container library? Phobos is a library, and templates work with it just fine. You're still importing the .d files. They're just compiled in as part of the library that you link against rather than compiled as part of your project - e.g. pretty much every D program is linked against libphobos, and none of them compile in modules like std.stdio. They just import them. They were compiled as part of libphobos are aside from templates are not compiled as part of your program, just linked. As for templated code, the templates are generated when your code is built, so they're not really linked in in the normal sense, but those modules are still part of the library and not part of your code, so you'd never do something like add std/algorithm/searching.d to the list of files that you're compiling. You'd just import it. If someone wants to hide code, then they'd use .di files and give you those to link against, in which case non-templated code could be hidden, but templated code would still be in those files, because the compiler has to see their source when they're used. And most folks will likely just use .d files, since it's simpler and allows stuff like CTFE and inlining to work. Most of the projects on code.dlang.org are libraries, and you're using dub, they're easy to pull in and link against. You can do the same with your code. - Jonathan M Davis
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 20:59:45 UTC, Adam D. Ruppe wrote: On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote: Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files. Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually) The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name. $ find main.d foo/hello.d $ cat main.d import foo.hello; $ cat foo/hello.d module foo.hello; Ok so now let's say I rename the directory "lib" to "foo". If I don't change the "import lib.hello" to "import foo.hello", how is the compiler going to find "hello.d"? (As a reminder, as I said, I'm using separate compilation)
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 20:05:11 UTC, Sebastien Alaiwan wrote: Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files. Those module declarations aren't redundant - they are virtually required (I think it is a mistake that they aren't explicitly required in all cases, actually) The file layout does not matter to the language itself. Only that module declaration does - it is NOT optional if you want a package name.
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:56:20 UTC, Jesse Phillips wrote: I've used this pattern. ./projectA/lib/math/algo.d ./projectA/lib/math/lcp.d ./projectA/lib/math/optimize.d ./projectA/gui/main.d ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/lib/math/algo.d ./projectB/lib/math/lcp.d ./projectB/lib/math/optimize.d ./projectB/main.d Dub doesn't like me too much, but in general it works. Note that it doesn't matter where your modules live, if they have the declared module name in the file: module math.optimize; If DMD reads this file all your imports should look like: import math.optimize; Even if the file is in app/physics/math. Yeah, I'm using this pattern too at the moment ; Although, I'm trying to avoid having these redundant module declaration directives at the beginning of each of my library files.
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:23:08 UTC, Jonathan M Davis wrote: My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having. Thanks for your answer ; from what I understand, you're basically telling me to only import binary/precompiled libraries, right? Although I had to admit I had never considered this solution, never have the same file be part of multiple projects seems extremely restrictive to me. How is it going to work if I want to share heavily templated code, like a container library?
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote: Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d I've used this pattern. ./projectA/lib/math/algo.d ./projectA/lib/math/lcp.d ./projectA/lib/math/optimize.d ./projectA/gui/main.d ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/lib/math/algo.d ./projectB/lib/math/lcp.d ./projectB/lib/math/optimize.d ./projectB/main.d Dub doesn't like me too much, but in general it works. Note that it doesn't matter where your modules live, if they have the declared module name in the file: module math.optimize; If DMD reads this file all your imports should look like: import math.optimize; Even if the file is in app/physics/math.
Re: A suggestion for modules names / sharing code between projects
On 29.02.2016 20:03, Sebastien Alaiwan wrote: $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation). [...] Clearly, putting everyone back into the same "file namespace" by adding every subdirectory of my project as import paths through the command line is not going to scale. How is not going to scale? You have to add an import path for every location where packages are to be found. How many such locations are you going to have that this is unreasonable? With your example, you'd have -Iinternal in project A, and -Iapp/physics in project B. The module declaration would of course be `module math.optimize;`.
Re: A suggestion for modules names / sharing code between projects
On Monday, 29 February 2016 at 19:03:53 UTC, Sebastien Alaiwan wrote: Clearly, something must be wrong here (and I hope it's me!) I'd be very happy if someone showed me how solve this problem, this has bothered me for ages. My solution would be to never have the same file be part of multiple projects. Share code using libraries rather than just sharing the files. Then modules won't be changing where they are in a project or anything like that. The shared code goes into a library, and whichever projects need it link against that library. You get better modularization and encapsulation that way and completely avoid the problem that you're having. - Jonathan M Davis
A suggestion for modules names / sharing code between projects
Hi all, I've came across the following problem number of times. Let's say I have project A and B, sharing code but having a different tree structure. $ find projectA ./projectA/internal/math/algo.d ./projectA/internal/math/lcp.d ./projectA/internal/math/optimize.d ./projectA/gui/main.d $ find projectB ./projectB/app/render/gfx.d ./projectB/app/render/algo.d ./projectB/app/physics/math/algo.d ./projectB/app/physics/math/lcp.d ./projectB/app/physics/math/optimize.d ./projectB/main.d The directory "math" is shared between projects. It actually is an external/submodule. So it has a standalone existence as a library, and might one day be used by projectC. (In the context of this issue, I'm using separate compilation). I'd like to be able to write, in projectA's main: import internal.math.optimize; This requires me to add, at the beginning of "optimize.d" file, this module definition: module internal.math.optimize; However, this "optimize.d" file is shared between projects, now it's becoming specific to projectA. How am I supposed to share code between projects then? Clearly, putting everyone back into the same "file namespace" by adding every subdirectory of my project as import paths through the command line is not going to scale. After many projects spent thinking of this issue, I came to the conclusion that being able to specify, on the command line, the module name of the module currently being compiled, thus allowing to override it to be inferred to the basename of the file, would solve this problem. As a side-problem related to this one, having to repeat the "absolute" module path to all module declarations/importations is tedious. What if I change the name of the package? Now I might have to change hundreds of module declarations from "module oldname.algo;" to "module newname.algo;". Clearly, something must be wrong here (and I hope it's me!) I'd be very happy if someone showed me how solve this problem, this has bothered me for ages.
