Fixing cyclic import static construction problems
For discussion:
Cyclical Imports
Problem:
a.d
module a;
import b;
static this () { ... }
b.d
module b;
import a;
static this() { ... }
-
Static constructors for a module are only run after static constructors
for all its imports are run. Circular imports, such as the above, are
detected at run time and the program is aborted.
This can in general be solved by moving the static constructor(s) into
a third module, c.d, which does not import a or b. But, people find this
to be unnatural.
Proposed Solution:
Add a pragma,
pragma(cyclic_imports);
This can appear anywhere in a module, and applies globally to that module.
It means that static constructors from imports that are not part of the
cycle
are run first, and that the static constructor for this module may be
run before
the static constructors of other modules that are part of the cycle.
If any static constructors in such a module with the pragma have
the @safe attribute, that is a compile time error.
Re: Fixing cyclic import static construction problems
On 11/28/12 9:34 PM, Walter Bright wrote: For discussion: [snip] I'd say we better finish const, immutable, and shared first. Andrei
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: I'd say we better finish const, immutable, and shared first. If we put off every easy fix until all the hard fixes are done, it means we have longer wait times on everything
Re: Fixing cyclic import static construction problems
Andrei Alexandrescu: I'd say we better finish const, immutable, and shared first. There are few things left to implement for purity (they are listed in Bugzilla), but what's left to do for const and immutable? Bye, bearophile
Re: Fixing cyclic import static construction problems
what's left to do for const and immutable? I guess the answer is too much long, so please ignore the question. Bye, bearophile
Re: Fixing cyclic import static construction problems
On Wednesday, November 28, 2012 22:19:54 Andrei Alexandrescu wrote: > On 11/28/12 9:34 PM, Walter Bright wrote: > > For discussion: > [snip] > > I'd say we better finish const, immutable, and shared first. Both problems need to be addressed, and this one is probably easier. It also has a huge impact on std.benchmark, so I would have thought that you'd be more in favor of it. However, we do have a tendancy to bring up problems like this, discuss them for a while, and then let them be more or less forgotten for a while. It keeps happening with stuff like const ref / auto ref, shared, const and Object, const postblit constructors, @trusted blocks, etc. And we need to actually get them sorted out. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On 11/29/12 03:34, Walter Bright wrote:
> Proposed Solution:
>
> Add a pragma,
>
> pragma(cyclic_imports);
>
> This can appear anywhere in a module, and applies globally to that module.
> It means that static constructors from imports that are not part of the cycle
> are run first, and that the static constructor for this module may be run
> before
> the static constructors of other modules that are part of the cycle.
Bad name. Something like "module_ctors_unordered" would be better (I don't
like that one either, but this way it's at least clear what it does).
The equivalent attribute version could be
module mymodule @unordered_ctors;
which is better, but still has the problem that adding another module ctor
can result is silent breakage. These things can be mixed in, so something
as innocent looking as
import some_lib;
[...]
mixin blah;
can already be a bug. Hence:
static this() @unordered { /*whatever*/ }
and then either enforce that all mod-ctors have this attribute (otherwise
every ctor in that moduile gets treated as ordered), or split them into
two sets (better, but larger change; backward compatible, just requires
a new enough runtime to be used).
Of course it could be also done as
static this() pragma(unordered) { /*whatever*/ }
but this wouldn't really be better, and would require language change (the
fact that you cannot attach a pragma to /anything/ is already a problem,
eg for gcc-specific attributes).
"@unordered" could even be inferred, but I'm not sure how often that would
help in practice. At least w/o making module-level imports invisible inside
module ctors (which would make the deps explicit).
artur
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 07:36:41 Artur Skawina wrote: > On 11/29/12 03:34, Walter Bright wrote: > > Proposed Solution: > > > > Add a pragma, > > > > pragma(cyclic_imports); > > > > This can appear anywhere in a module, and applies globally to that module. > > It means that static constructors from imports that are not part of the > > cycle are run first, and that the static constructor for this module may > > be run before the static constructors of other modules that are part of > > the cycle. > Bad name. Something like "module_ctors_unordered" would be better (I don't > like that one either, but this way it's at least clear what it does). no_cyclic_imports would probably be better given that you're trying to get around a cyclic import, but I don't see what's unclear about cyclic_imports given that that's exactly what the runtime complains about when this problem occurs. > but still has the problem that adding another module ctor > can result is silent breakage. That's actually a really good argument IMHO for having to put the pragma or attribute on every single static constructor in a module. True, it may be a bit annoying, but it would avoid silent breakage. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On 2012-11-29 03:34, Walter Bright wrote: Add a pragma, pragma(cyclic_imports); This can appear anywhere in a module, and applies globally to that module. It means that static constructors from imports that are not part of the cycle are run first, and that the static constructor for this module may be run before the static constructors of other modules that are part of the cycle. I would think that "cyclic_imports" sounds like the static constructors are part of the cycle. -- /Jacob Carlborg
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: On 11/28/12 9:34 PM, Walter Bright wrote: For discussion: [snip] I'd say we better finish const, immutable, and shared first. Andrei +1 Fully agree. Cyclic imports are a minor nuisance that can be easily solvable with better code architecture. Turbo Pascal/Delphi is the only language that I know fully allows cyclic dependencies between modules. So this is not that important for most people. -- Paulo
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 02:34:11 UTC, Walter Bright
wrote:
For discussion:
Cyclical Imports
Problem:
a.d
module a;
import b;
static this () { ... }
b.d
module b;
import a;
static this() { ... }
-
Static constructors for a module are only run after static
constructors
for all its imports are run. Circular imports, such as the
above, are
detected at run time and the program is aborted.
This can in general be solved by moving the static
constructor(s) into
a third module, c.d, which does not import a or b. But, people
find this
to be unnatural.
It is natural. It just doesn't cover important use cases.
Proposed Solution:
Add a pragma,
pragma(cyclic_imports);
This can appear anywhere in a module, and applies globally to
that module.
It means that static constructors from imports that are not
part of the cycle
are run first, and that the static constructor for this module
may be run before
the static constructors of other modules that are part of the
cycle.
If any static constructors in such a module with the pragma have
the @safe attribute, that is a compile time error.
Can we have that implemented in a branch and see how it goes?
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto wrote: On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: On 11/28/12 9:34 PM, Walter Bright wrote: For discussion: [snip] I'd say we better finish const, immutable, and shared first. Andrei +1 Fully agree. Cyclic imports are a minor nuisance that can be easily solvable with better code architecture. Show me please how to solve that problem easily with acceptable results, would you? Turbo Pascal/Delphi is the only language that I know fully allows cyclic dependencies between modules. So this is not that important for most people. -- Paulo
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 12:39:19 Paulo Pinto wrote: > On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei > > Alexandrescu wrote: > > On 11/28/12 9:34 PM, Walter Bright wrote: > >> For discussion: > > [snip] > > > > I'd say we better finish const, immutable, and shared first. > > > > Andrei > > +1 > > Fully agree. > > Cyclic imports are a minor nuisance that can be easily solvable > with better code architecture. > > Turbo Pascal/Delphi is the only language that I know fully allows > cyclic dependencies between modules. So this is not that > important for most people. Basic features in the language require static constructors (e.g. static variables frequently do), and some things just can't be done without them. Andrei's std.benchmark proposal actually doesn't work, because in order to do what it does, in needs to mixin in a static constructor and static destructor (to set up the benchmarking and record it at the end, I believe). That completely falls apart due to cyclical imports. It doesn't actually cause any true circular dependencies, but it's treated as such anyway. A _lot_ of us have run into this problem and a number of us consider it to be a huge design problem in the language. Personally, I'd put it towards the top of design mistakes at this point, and I'm very glad to see Walter actually finally being willing to do something about this. In the past when I've brought up similar solutions, he's been completely opposed to them. Yes, we have other major issues that need to be resolved, and those may very well be of a higher priority, but this is still a very import issue, and it's the sort of issue that can probably be implemented with minimal effort. The main thing is getting Walter to agree to it and how it will be done. Once a decision is made, it wouldn't surprise me if someone like Kenji were able to get it done very quickly. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
Walter Bright wrote: > It means that [...] the static constructor for this > module may be run before This is sufficient only for a simple cycle without any branches or a trivial clique like the one shown. Please recall, that in a clique every member is connected to every other member---in this case by an import statement. It is already not sufficient for a simple clique consisting of three modules, because now orderinh would be given for the remaining two modules. And it is ambiguous if both of the modules in the example given are marked with that pragma. In general a topological sorting has to be specified for all strongly connected components of the graph of imports. This cannot be done within the module, because this would bind the module to that special strongly connected component in that special set of modules. Which in turn would destroy reusability of the module for other programming tasks. Therefore: marking with a pragma isn't a fix for the depicted problem. -manfred
Re: Fixing cyclic import static construction problems
"Walter Bright" wrote in message news:k96hj2$2lus$1...@digitalmars.com... > For discussion: > > Cyclical Imports > > Problem: > > > Proposed Solution: > > Add a pragma, > > pragma(cyclic_imports); > > This can appear anywhere in a module, and applies globally to that module. > It means that static constructors from imports that are not part of the > cycle > are run first, and that the static constructor for this module may be run > before > the static constructors of other modules that are part of the cycle. > I don't think this is sufficient. Imagine a group of modules that really _do_ have a cyclic dependency, and a mixin that adds an independent static this. Ideally you'd be able to mark the mixed-in constructor as independent without tainting the whole module. So just make the pragma apply to declarations, you either mark specific functions (which can then be mixed in) or put `pragma(...):` at the top of your module and you get your behaviour.
Re: Fixing cyclic import static construction problems
On 11/28/12 10:34 PM, Adam D. Ruppe wrote: On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: I'd say we better finish const, immutable, and shared first. If we put off every easy fix until all the hard fixes are done, it means we have longer wait times on everything On the other hand if we work on every easy and unimportant fix we miss big on the important stuff. http://c2.com/cgi/wiki?FourQuadrants Andrei
Re: Fixing cyclic import static construction problems
On 11/29/12 12:25 AM, Jonathan M Davis wrote: On Wednesday, November 28, 2012 22:19:54 Andrei Alexandrescu wrote: On 11/28/12 9:34 PM, Walter Bright wrote: For discussion: [snip] I'd say we better finish const, immutable, and shared first. Both problems need to be addressed, and this one is probably easier. It also has a huge impact on std.benchmark, so I would have thought that you'd be more in favor of it. My perception is that we need to tackle the major issues at this point: process and qualifiers (including shared). However, we do have a tendancy to bring up problems like this, discuss them for a while, and then let them be more or less forgotten for a while. It keeps happening with stuff like const ref / auto ref, shared, const and Object, const postblit constructors, @trusted blocks, etc. And we need to actually get them sorted out. Enhancement requests are the right place for those. Now that we have forum.dlang.org we can link to discussions, too. Andrei
Re: Fixing cyclic import static construction problems
On 11/29/12 12:07 AM, bearophile wrote: what's left to do for const and immutable? I guess the answer is too much long, so please ignore the question. Ars longa vita brevis est. Andrei
Re: Fixing cyclic import static construction problems
On 11/28/12 11:26 PM, bearophile wrote: Andrei Alexandrescu: I'd say we better finish const, immutable, and shared first. There are few things left to implement for purity (they are listed in Bugzilla), but what's left to do for const and immutable? Construction flow and copy conversion. Andrei
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha wrote: On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto wrote: On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: On 11/28/12 9:34 PM, Walter Bright wrote: For discussion: [snip] I'd say we better finish const, immutable, and shared first. Andrei +1 Fully agree. Cyclic imports are a minor nuisance that can be easily solvable with better code architecture. Show me please how to solve that problem easily with acceptable results, would you? You just need to have a better architecture. In 20 years of software development experience I never found a case were this wasn't possible. Maybe you care to provide an example? -- Paulo
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 12:17:49 UTC, Jonathan M Davis wrote: On Thursday, November 29, 2012 12:39:19 Paulo Pinto wrote: On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei Alexandrescu wrote: > On 11/28/12 9:34 PM, Walter Bright wrote: >> For discussion: > [snip] > > I'd say we better finish const, immutable, and shared first. > > Andrei +1 Fully agree. Cyclic imports are a minor nuisance that can be easily solvable with better code architecture. Turbo Pascal/Delphi is the only language that I know fully allows cyclic dependencies between modules. So this is not that important for most people. Basic features in the language require static constructors (e.g. static variables frequently do), and some things just can't be done without them. Andrei's std.benchmark proposal actually doesn't work, because in order to do what it does, in needs to mixin in a static constructor and static destructor (to set up the benchmarking and record it at the end, I believe). That completely falls apart due to cyclical imports. It doesn't actually cause any true circular dependencies, but it's treated as such anyway. A _lot_ of us have run into this problem and a number of us consider it to be a huge design problem in the language. Personally, I'd put it towards the top of design mistakes at this point, and I'm very glad to see Walter actually finally being willing to do something about this. In the past when I've brought up similar solutions, he's been completely opposed to them. Yes, we have other major issues that need to be resolved, and those may very well be of a higher priority, but this is still a very import issue, and it's the sort of issue that can probably be implemented with minimal effort. The main thing is getting Walter to agree to it and how it will be done. Once a decision is made, it wouldn't surprise me if someone like Kenji were able to get it done very quickly. - Jonathan M Davis Maybe I should keep my mouth shut, because actually I don't really use D besides some toy experiments, as there is no place for D in the type of work I currently do, either on the job or privately. I like however to point people to D as a possible C++ successor as way to do some language publicity, and think that these type of discussions might scare D newbies away in what concerns language stability, hence my post. -- Paulo
Re: Fixing cyclic import static construction problems
11/29/2012 4:17 PM, Jonathan M Davis пишет:
On Thursday, November 29, 2012 12:39:19 Paulo Pinto wrote:
On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
Alexandrescu wrote:
On 11/28/12 9:34 PM, Walter Bright wrote:
For discussion:
[snip]
I'd say we better finish const, immutable, and shared first.
Andrei
+1
Fully agree.
Cyclic imports are a minor nuisance that can be easily solvable
with better code architecture.
Turbo Pascal/Delphi is the only language that I know fully allows
cyclic dependencies between modules. So this is not that
important for most people.
Basic features in the language require static constructors (e.g. static
variables frequently do), and some things just can't be done without them.
Andrei's std.benchmark proposal actually doesn't work, because in order to do
what it does, in needs to mixin in a static constructor and static destructor
(to set up the benchmarking and record it at the end, I believe). That
completely falls apart due to cyclical imports.
I do suspect it could be healed by inverting the control flow (or rather
bringing it back).
Basically there is this pattern:
module a;
...
mixin ScheduleForBenchmark;
Same for module b, c, d...
And then compiling all of this with -version benchmark should run the
benchmarks.
It already has another problem: there has to be D main stuffed in there
somehow. It can't be in std.benchmark as it then will have to be
included in phobos(.lib|.a) and it can't be magically turned on/off with
-version switch either unless std.benchmark is passed directly to the
compiler.
The other solution would be: make a separate my_benchmark.d module that
contains:
import a,b,c,d...;
void main(){
runBenchmarks!(a,b,c,d...)();
}
And that's it.
In any case I usually like having some separation between different sets
of modules to benchmark so that I can start them in a 'clean room' scenario.
It doesn't actually cause any
true circular dependencies, but it's treated as such anyway. A _lot_ of us
have run into this problem and a number of us consider it to be a huge design
problem in the language. Personally, I'd put it towards the top of design
mistakes at this point, and I'm very glad to see Walter actually finally being
willing to do something about this. In the past when I've brought up similar
solutions, he's been completely opposed to them.
Yes, we have other major issues that need to be resolved, and those may very
well be of a higher priority, but this is still a very import issue, and it's
the sort of issue that can probably be implemented with minimal effort. The
main thing is getting Walter to agree to it and how it will be done. Once a
decision is made, it wouldn't surprise me if someone like Kenji were able to
get it done very quickly.
- Jonathan M Davis
--
Dmitry Olshansky
Re: Fixing cyclic import static construction problems
On 11/29/12 10:54 AM, Dmitry Olshansky wrote:
Basically there is this pattern:
module a;
...
mixin ScheduleForBenchmark;
Same for module b, c, d...
Correct.
And then compiling all of this with -version benchmark should run the
benchmarks.
It already has another problem: there has to be D main stuffed in there
somehow. It can't be in std.benchmark as it then will have to be
included in phobos(.lib|.a) and it can't be magically turned on/off with
-version switch either unless std.benchmark is passed directly to the
compiler.
The other solution would be: make a separate my_benchmark.d module that
contains:
import a,b,c,d...;
void main(){
runBenchmarks!(a,b,c,d...)();
}
That's workable. I'm hoping, however, to make benchmarks as easy and as
trivial to define as unittests.
Andrei
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 15:18:11 UTC, Paulo Pinto wrote:
On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha
wrote:
On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto
wrote:
On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
Alexandrescu wrote:
On 11/28/12 9:34 PM, Walter Bright wrote:
For discussion:
[snip]
I'd say we better finish const, immutable, and shared first.
Andrei
+1
Fully agree.
Cyclic imports are a minor nuisance that can be easily
solvable with better code architecture.
Show me please how to solve that problem easily with
acceptable results, would you?
You just need to have a better architecture.
In 20 years of software development experience I never found a
case were this wasn't possible.
That's an argument from authority, sorry.
Maybe you care to provide an example?
The general problem is constructing global data structures based
on data introspected at compile-time.
My specific problem is extending scarce runtime type information
provided by the language with something usable for runtime
reflection. With lots of detail omitted:
module reflect;
Meta[string] metas;
mixin template Reflect(alias object) {
static this()
{
auto m = meta!(object);
metas[m.fullName] ~= m;
}
}
module a;
import reflect;
struct S
{
}
mixin Reflect!S;
The meta-object for S is automatically made available at runtime
through the global metas array. Note that we do not want to force
the user to register the meta-object manually because then it
would not be a "better architecture".
The important (Andrei somehow thinks it is not) requirement is
there must not be circular dependency issues for the users of the
"reflect" module.
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 12:17:49 UTC, Jonathan M Davis wrote: Basic features in the language require static constructors (e.g. static variables frequently do), and some things just can't be done without them. Andrei's std.benchmark proposal actually doesn't work, because in order to do what it does, in needs to mixin in a static constructor and static destructor (to set up the benchmarking and record it at the end, I believe). That completely falls apart due to cyclical imports. It doesn't actually cause any true circular dependencies, but it's treated as such anyway. A _lot_ of us have run into this problem and a number of us consider it to be a huge design problem in the language. Personally, I'd put it towards the top of design mistakes at this point, and I'm very glad to see Walter actually finally being willing to do something about this. In the past when I've brought up similar solutions, he's been completely opposed to them. Yes, we have other major issues that need to be resolved, and those may very well be of a higher priority, but this is still a very import issue, and it's the sort of issue that can probably be implemented with minimal effort. The main thing is getting Walter to agree to it and how it will be done. Once a decision is made, it wouldn't surprise me if someone like Kenji were able to get it done very quickly. - Jonathan M Davis That is understood, but Let me explain how I see things. We are here adding yet a new feature, however small. Nothing stabilize when adding new feature all the time. The annoyance exist, is real, but is not THAT bad, and 'm pretty sure the proposed solution is likely to backfire. Firstly because a module can have several constructors, and more can be added by mixins. Ensuring that constructor A will not create dependancy cycle error may silently break other constructor of the module. Considering the problem static constructor solve, I'd be happy to see some think out of the box. The way things work here may be broken. Many people consider that D runtime should evolve in a way that promote fibers and map them over system threads (this have many benefit : the program adapt autmatically to the provided hardware, yields can be introduced directly into the runtime or some libraries to hide IO latencies, and many other things), and static constructor/variables really get into the way (you would have to run all static constructors each time you start a new fiber or reset one to reuse it). I really wish we can focus on figuring out the uses cases we have for static constructor and start the reflection from theses sues case and not from the feature itself.
Re: Fixing cyclic import static construction problems
On 2012-11-29 16:58, Andrei Alexandrescu wrote: That's workable. I'm hoping, however, to make benchmarks as easy and as trivial to define as unittests. Unit tests have the same problem. One need to create a module containing a main function and importing the modules one wants to benchmark/test. One idea would be to add more support for this in the compiler. Another idea, that might better and is easier to implement, is tool that: 1. Takes a couple of modules on the command line 2. Create a main module which imports the modules on the command line 3. Compile all the modules 4. Run the benchmarks or tests -- /Jacob Carlborg
Re: Fixing cyclic import static construction problems
On 2012-11-29 19:17, deadalnix wrote: That is understood, but Let me explain how I see things. We are here adding yet a new feature, however small. Nothing stabilize when adding new feature all the time. The annoyance exist, is real, but is not THAT bad, and 'm pretty sure the proposed solution is likely to backfire. Firstly because a module can have several constructors, and more can be added by mixins. Ensuring that constructor A will not create dependancy cycle error may silently break other constructor of the module. Considering the problem static constructor solve, I'd be happy to see some think out of the box. The way things work here may be broken. Many people consider that D runtime should evolve in a way that promote fibers and map them over system threads (this have many benefit : the program adapt autmatically to the provided hardware, yields can be introduced directly into the runtime or some libraries to hide IO latencies, and many other things), and static constructor/variables really get into the way (you would have to run all static constructors each time you start a new fiber or reset one to reuse it). I really wish we can focus on figuring out the uses cases we have for static constructor and start the reflection from theses sues case and not from the feature itself. BTW, how does Java handle this? And C# if it has something similar. -- /Jacob Carlborg
Re: Fixing cyclic import static construction problems
Max Samukha wrote: > there must not be circular dependency issues The model shown does not force circular dependencies, because every receiver module only imports one module: `module reflect'. And I can not see, that this module imports any other module. Therefore `module relect' and its `import' statements are not responsible for any circular dependency issue. Where am I wrong? -manfred
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 16:18:10 Paulo Pinto wrote: > On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha wrote: > > On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto > > > > wrote: > >> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei > >> > >> Alexandrescu wrote: > >>> On 11/28/12 9:34 PM, Walter Bright wrote: > For discussion: > >>> [snip] > >>> > >>> I'd say we better finish const, immutable, and shared first. > >>> > >>> Andrei > >> > >> +1 > >> > >> Fully agree. > >> > >> Cyclic imports are a minor nuisance that can be easily > >> solvable with better code architecture. > > > > Show me please how to solve that problem easily with acceptable > > results, would you? > > You just need to have a better architecture. > > In 20 years of software development experience I never found a > case were this wasn't possible. > > Maybe you care to provide an example? Considering that you need static constructors to initialize any static variables at runtime (and in the case of const and immutable variables, you _can't_ work around that by doing the initialization later - it _must_ be in the static constructor), and all it takes for the compiler to declare a circular import is to have two modules import each other - even indirectly - it becomes extremely easy to run into this problem. And if you're dealing with immutable static variables which are initialized at runtime, you can't fix it unless you can contort your modules so that they don't depend on each other, and with a lot of modules, that can become extremely difficult. We had to strip out all static constructors from std.datetime because of this, and in order to fix it, we had to do some nasty voodoo with casting in order to lazily initialize some variables which are supposed to be immutable. _None_ of that sort of thing should be necessary. As it stands, it's basically bad practice to use static constructors, because it's so easy to end up with circular dependencies, and they can involve modules which don't seem even vaguely related because of other stuff that they import and are often a royal pain to debug and fix, if you even can without seriously revamping your design. And for a library like Phobos which needs to avoid breaking backwards compatibility, redesigning things to avoid such circular dependencies isn't necessarily even possible, because those redesigns would break backwards compatibliity. We _need_ a solution for this. Whether now is the best time tackle it is another matter, but the current situation with static constructors is ridiculous. There are features which require them, but you have to avoid them or you're going to run into circular dependency issues very easily. This is a case of some great design decisions in D have leading to a very bad design, and it needs to be fixed. Fortunately, it shouldn't be all that hard to fix with an attribute of some kind. But prior to now, Walter wouldn't even consider it. I do think though that the idea of using only one pragma instead of an attribute/pragma per static constructor is a bad idea because of the silent breakage that it would cause, as has been pointed out in this thread. So, his proposed solution needs some tweaking. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 21:08:58 Jacob Carlborg wrote: > BTW, how does Java handle this? And C# if it has something similar. They just let you blow your foot off. All static variables can be directly initialized at runtime, so it's easy to use variables before they're actually initialized. I don't know how they decide what order to run static constructors in, but AFAIK, it never worries about circular dependencies. We're only running into this problem beacuse we're trying to provide higher safety and better guarantees with regards to when and how variables are initialized. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 21:43:30 UTC, Jonathan M Davis wrote: On Thursday, November 29, 2012 21:08:58 Jacob Carlborg wrote: BTW, how does Java handle this? And C# if it has something similar. They just let you blow your foot off. All static variables can be directly initialized at runtime, so it's easy to use variables before they're actually initialized. I don't know how they decide what order to run static constructors in, but AFAIK, it never worries about circular dependencies. We're only running into this problem beacuse we're trying to provide higher safety and better guarantees with regards to when and how variables are initialized. - Jonathan M Davis Java have static block to pre initialize stuff before it is used. But I'm not sure how they react in case of cyclic dependancies.
Re: Fixing cyclic import static construction problems
On 11/29/2012 09:30 PM, Manfred Nowak wrote: Max Samukha wrote: there must not be circular dependency issues The model shown does not force circular dependencies, because every receiver module only imports one module: `module reflect'. And I can not see, that this module imports any other module. Therefore `module relect' and its `import' statements are not responsible for any circular dependency issue. Where am I wrong? -manfred A precondition for (pseudo-) circular dependency issues of this kind are static constructors. The static constructors are what module reflect imposes in its users in order to operate.
Re: Fixing cyclic import static construction problems
On 11/29/2012 10:45 PM, deadalnix wrote: On Thursday, 29 November 2012 at 21:43:30 UTC, Jonathan M Davis wrote: On Thursday, November 29, 2012 21:08:58 Jacob Carlborg wrote: BTW, how does Java handle this? And C# if it has something similar. They just let you blow your foot off. All static variables can be directly initialized at runtime, so it's easy to use variables before they're actually initialized. I don't know how they decide what order to run static constructors in, but AFAIK, it never worries about circular dependencies. We're only running into this problem beacuse we're trying to provide higher safety and better guarantees with regards to when and how variables are initialized. - Jonathan M Davis Java have static block to pre initialize stuff before it is used. But I'm not sure how they react in case of cyclic dependancies. In Java and C# static constructors are run whenever the class is first referenced. This means that almost any statement in Java or C# may cause arbitrary side-effecting code to run. Circular dependencies are not considered at all. If two static constructors depend on each other's statically initialized parts, at least one of them will see uninitialized data.
Re: Fixing cyclic import static construction problems
On 11/29/2012 01:17 PM, Jonathan M Davis wrote: In the past when I've brought up similar solutions, he's been completely opposed to them. ... It is not a solution, it is a workaround.
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 23:28:07 Timon Gehr wrote: > On 11/29/2012 01:17 PM, Jonathan M Davis wrote: > > In the past when I've brought up similar solutions, he's been completely > > opposed to them. ... > > It is not a solution, it is a workaround. What do you mean? The runtime sees circular dependencies between modules even when there's no actual circular dependency between static constructors. We need to fix that. One way is to just make the runtime not care, which wouldn't be particularly safe. Another is to explicitly tell it that there are no such dependencies. I don't see how that's not a solution. And unless someone can come up with a way for the runtime to somehow determine on its own that there's no actual, circular dependency, I don't see how anything better could be done. Granted, I think that having a single pragma for the whole module like Walter is suggesting (as opposed to marking static constructors individually) is a bad idea because of the silent breakage that it can cause, but the basic idea is solid. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On 11/29/2012 11:58 PM, Manfred Nowak wrote: This is sufficient only for a simple cycle without any branches or a trivial clique like the one shown. Please recall, that in a clique every member is connected to every other member---in this case by an import statement. It is already not sufficient for a simple clique consisting of three modules, because now orderinh would be given for the remaining two modules. If you have: a imports b,c b imports a,c c imports a,b then two of those will need the pragma, that is correct. I don't see an issue with that, because there are two cycles. And it is ambiguous if both of the modules in the example given are marked with that pragma. If: a imports b b imports a and a has the pragma, then the order of construction is a,b. If both have the pragma, then the order is a,b or b,a, yes, it is ambiguous, but it is not an error. One gets picked arbitrarily. In general a topological sorting has to be specified for all strongly connected components of the graph of imports. I believe that the pragma does that. This cannot be done within the module, because this would bind the module to that special strongly connected component in that special set of modules. Which in turn would destroy reusability of the module for other programming tasks. I just don't see the problem.
Re: Fixing cyclic import static construction problems
On 11/30/2012 12:09 AM, Daniel Murphy wrote: I don't think this is sufficient. Imagine a group of modules that really _do_ have a cyclic dependency, and a mixin that adds an independent static this. Ideally you'd be able to mark the mixed-in constructor as independent without tainting the whole module. So just make the pragma apply to declarations, you either mark specific functions (which can then be mixed in) or put `pragma(...):` at the top of your module and you get your behaviour. It is possible for each static constructor to specify independently of the other static constructors which imports must be constructed first. But do we really want to go that far?
Re: Fixing cyclic import static construction problems
On 11/30/2012 9:43 AM, Walter Bright wrote:
It is possible for each static constructor to specify independently of
the other static constructors which imports must be constructed first.
But do we really want to go that far?
One way to do that might be to borrow syntax from classes:
static this() : std.stdio, a, c
{
...
}
and the this static constructor only requires that modules std.stdio,
a, and c be constructed first.
static this() : void
{
...
}
means it has no dependencies on other imports.
static this()
{
...
}
has the current behavior (all imported modules must be constructed first).
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 16:51:29 UTC, Max Samukha wrote:
On Thursday, 29 November 2012 at 15:18:11 UTC, Paulo Pinto
wrote:
On Thursday, 29 November 2012 at 12:04:28 UTC, Max Samukha
wrote:
On Thursday, 29 November 2012 at 11:39:20 UTC, Paulo Pinto
wrote:
On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
Alexandrescu wrote:
On 11/28/12 9:34 PM, Walter Bright wrote:
For discussion:
[snip]
I'd say we better finish const, immutable, and shared first.
Andrei
+1
Fully agree.
Cyclic imports are a minor nuisance that can be easily
solvable with better code architecture.
Show me please how to solve that problem easily with
acceptable results, would you?
You just need to have a better architecture.
In 20 years of software development experience I never found a
case were this wasn't possible.
That's an argument from authority, sorry.
Maybe you care to provide an example?
The general problem is constructing global data structures
based on data introspected at compile-time.
My specific problem is extending scarce runtime type
information provided by the language with something usable for
runtime reflection. With lots of detail omitted:
module reflect;
Meta[string] metas;
mixin template Reflect(alias object) {
static this()
{
auto m = meta!(object);
metas[m.fullName] ~= m;
}
}
module a;
import reflect;
struct S
{
}
mixin Reflect!S;
The meta-object for S is automatically made available at
runtime through the global metas array. Note that we do not
want to force the user to register the meta-object manually
because then it would not be a "better architecture".
For me too much use of meta capabilities is not a better
architecture,
as quite often it leads to write only code.
If you are alreading writing code on the client side for
initialization, like your mixin definition, the client already
needs to make the reflect module of its existence, so why not
call an initialization function that avoids the static
constructors issues altogether?
--
Paulo
Re: Fixing cyclic import static construction problems
On 11/29/12 5:43 PM, Walter Bright wrote: On 11/30/2012 12:09 AM, Daniel Murphy wrote: I don't think this is sufficient. Imagine a group of modules that really _do_ have a cyclic dependency, and a mixin that adds an independent static this. Ideally you'd be able to mark the mixed-in constructor as independent without tainting the whole module. So just make the pragma apply to declarations, you either mark specific functions (which can then be mixed in) or put `pragma(...):` at the top of your module and you get your behaviour. It is possible for each static constructor to specify independently of the other static constructors which imports must be constructed first. But do we really want to go that far? I think we either do it right or leave it as it is. It's not like there's no workaround so if we take a stand here we better have something compelling. Andrei
Re: Fixing cyclic import static construction problems
On Friday, 30 November 2012 at 01:07:57 UTC, Andrei Alexandrescu wrote: I think we either do it right or leave it as it is. It's not like there's no workaround so if we take a stand here we better have something compelling. Andrei Finally some sanity here !
Re: Fixing cyclic import static construction problems
On Thursday, November 29, 2012 20:07:57 Andrei Alexandrescu wrote: > I think we either do it right or leave it as it is. It's not like > there's no workaround so if we take a stand here we better have > something compelling. I think that an attribute per static constructor indicating that it had no circular dependencies would solve the problem just fine (putting it on the module is problematic, because new static constructors which _do_ have actual circular dependencies could be added), but we certainly want to be sure of this before doing it. And if we want to focus on shared or whatever now because it's higher priority, that's fine, but we do need to move forward with outstanding issues like these, and I do think that really need to solve this particular problem rather than considering the workarounds to be okay. A situation where static constructors effectively must be shunned because of the problems that they cause is definitely problematic considering how many features require them (e.g. a const or immutable static variable). And that's what we have right now. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
Walter Bright wrote: > yes, it is ambiguous, but it is not an error. > One gets picked arbitrarily. I did not see that it would not be erroneous. But if it is true, then I do not see a sense in manually adding pragmas: they can be assumed to exist. -manfred
Re: Fixing cyclic import static construction problems
Timon Gehr wrote: > what module reflect imposes Thank you. It seems to be true then, that no issue would be generated if the compiler assumes, that Walters pragma is existent in every module. -manfred
Re: Fixing cyclic import static construction problems
On Friday, November 30, 2012 03:10:57 Manfred Nowak wrote:
> Timon Gehr wrote:
> > what module reflect imposes
>
> Thank you.
>
> It seems to be true then, that no issue would be generated if
> the compiler assumes, that Walters pragma is existent in every
> module.
So, you're suggesting that the runtime should just not worry about circular
dependencies between modules at all? What if you have something like
module a;
import b;
immutable Foo foo;
shared static this()
{
foo = new immutable(Foo)(bar);
}
module b;
import a;
immutable Bar bar;
shared static this()
{
bar = new immutable(Bar)(foo);
}
That's a true circular dependency. It's a big problem if that's allowed. D
rightly disallows it. The problem is that in the vast majority of cases, there
is no such circular dependency, so the circular dependency detection disallows
many valid use cases.
Or do you mean something else by the suggestion that the existence of Walter's
pragma can be assumed? I don't know what else you could mean other than
suggesting that assuming that it's present in all modules which is the same
as throwing away circular dependency detection entirely.
- Jonathan M Davis
Re: Fixing cyclic import static construction problems
On 11/29/2012 5:51 PM, Max Samukha wrote:
On Thursday, 29 November 2012 at 15:18:11 UTC, Paulo Pinto wrote:
Maybe you care to provide an example?
The general problem is constructing global data structures based on data
introspected at compile-time.
My specific problem is extending scarce runtime type information
provided by the language with something usable for runtime reflection.
With lots of detail omitted:
module reflect;
Meta[string] metas;
mixin template Reflect(alias object) {
static this()
{
auto m = meta!(object);
metas[m.fullName] ~= m;
}
}
module a;
import reflect;
struct S
{
}
mixin Reflect!S;
The meta-object for S is automatically made available at runtime through
the global metas array. Note that we do not want to force the user to
register the meta-object manually because then it would not be a "better
architecture".
The important (Andrei somehow thinks it is not) requirement is there
must not be circular dependency issues for the users of the "reflect"
module.
How about running your own set of "constructors" searching the module
info array, searching for specific classes in the module that are added
by a mixin:
--
module register;
RegisterBase[string] registry;
void doRegister(string name, RegisterBase r) { registry[name] = r; }
class RegisterBase
{
abstract void _register();
}
template Register(string name)
{
enum string Register = "
class Register : RegisterBase
{
override void _register() { doRegister(\"" ~ name ~ "\", this);
}
}
";
}
void registerAll()
{
foreach(m; ModuleInfo)
{
TypeInfo_Class[] clss = m.localClasses();
foreach(c; clss)
{
if(c.base is RegisterBase.classinfo)
{
if(auto reg = cast(RegisterBase) c.create())
{
reg._register();
}
}
}
}
}
---
module a;
import register;
mixin(Register!"a");
---
module main;
import std.stdio;
import register;
void main()
{
registerAll();
foreach(a, o; registry)
writeln(a, " ", o);
}
This might also work for the benchmark module.
Re: Fixing cyclic import static construction problems
On 2012-11-29 22:43, Jonathan M Davis wrote: They just let you blow your foot off. All static variables can be directly initialized at runtime, so it's easy to use variables before they're actually initialized. I don't know how they decide what order to run static constructors in, but AFAIK, it never worries about circular dependencies. We're only running into this problem beacuse we're trying to provide higher safety and better guarantees with regards to when and how variables are initialized. I see. -- /Jacob Carlborg
Re: Fixing cyclic import static construction problems
On 11/29/12 23:34, Jonathan M Davis wrote: > On Thursday, November 29, 2012 23:28:07 Timon Gehr wrote: >> On 11/29/2012 01:17 PM, Jonathan M Davis wrote: >>> In the past when I've brought up similar solutions, he's been completely >>> opposed to them. ... >> >> It is not a solution, it is a workaround. > > What do you mean? The runtime sees circular dependencies between modules even > when there's no actual circular dependency between static constructors. We > need to fix that. One way is to just make the runtime not care, which > wouldn't > be particularly safe. Another is to explicitly tell it that there are no such > dependencies. I don't see how that's not a solution. And unless someone can > come up with a way for the runtime to somehow determine on its own that > there's no actual, circular dependency, I don't see how anything better could > be done. It's relatively easy for the /compiler/ to figure it out; it's just that implementing a simple user-provided flag requires the least amount of work. What Walter suggested can be tweaked to be sane (per-ctor flag) and will still be useful if/when the compiler becomes smarter (think lazily initted module fields). For the compiler to check if the value of every imported symbol accessed inside a mod-ctor can be evaluated at compile-time (if you encounter a case where this is not true it means there (potentially) is a true dependency and the ctors should be ordered) would require more work. artur
Re: Fixing cyclic import static construction problems
On Friday, 30 November 2012 at 01:07:57 UTC, Andrei Alexandrescu wrote: On 11/29/12 5:43 PM, Walter Bright wrote: On 11/30/2012 12:09 AM, Daniel Murphy wrote: I don't think this is sufficient. Imagine a group of modules that really _do_ have a cyclic dependency, and a mixin that adds an independent static this. Ideally you'd be able to mark the mixed-in constructor as independent without tainting the whole module. So just make the pragma apply to declarations, you either mark specific functions (which can then be mixed in) or put `pragma(...):` at the top of your module and you get your behaviour. It is possible for each static constructor to specify independently of the other static constructors which imports must be constructed first. But do we really want to go that far? I think we either do it right or leave it as it is. It's not like there's no workaround so if we take a stand here we better have something compelling. Andrei +1 FWIW, I think this proposal sounds like a massive hack. Not a fan.
Re: Fixing cyclic import static construction problems
On 11/30/2012 9:05 PM, Peter Alexander wrote: On Friday, 30 November 2012 at 01:07:57 UTC, Andrei Alexandrescu wrote: I think we either do it right or leave it as it is. It's not like there's no workaround so if we take a stand here we better have something compelling. Andrei +1 FWIW, I think this proposal sounds like a massive hack. Not a fan. Andrei has a point.
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 23:02:17 UTC, Walter Bright
wrote:
On 11/30/2012 9:43 AM, Walter Bright wrote:
It is possible for each static constructor to specify
independently of
the other static constructors which imports must be
constructed first.
But do we really want to go that far?
One way to do that might be to borrow syntax from classes:
static this() : std.stdio, a, c
{
...
}
and the this static constructor only requires that modules
std.stdio, a, and c be constructed first.
static this() : void
{
...
}
means it has no dependencies on other imports.
static this()
{
...
}
has the current behavior (all imported modules must be
constructed first).
Why not simplify?
static this()
{
import std.stdio, a, c; // existing syntax
...
}
static this()
{ // no imports -> no dependencies
...
}
The current behavior should just be dropped.
Re: Fixing cyclic import static construction problems
On 11/29/2012 02:50 PM, Andrei Alexandrescu wrote: On 11/28/12 11:26 PM, bearophile wrote: There are few things left to implement for purity (they are listed in Bugzilla), but what's left to do for const and immutable? Construction flow and copy conversion. ... meaning e.g. being able to effectively .dup or .idup any class, struct, etc.?
Re: Fixing cyclic import static construction problems
On Friday, 30 November 2012 at 14:09:48 UTC, foobar wrote:
Why not simplify?
static this()
{
import std.stdio, a, c; // existing syntax
...
}
static this()
{ // no imports -> no dependencies
...
}
The current behavior should just be dropped.
+2
Simple & Elegant.
Re: Fixing cyclic import static construction problems
On Fri, 30 Nov 2012 14:29:19 -, Tove wrote:
On Friday, 30 November 2012 at 14:09:48 UTC, foobar wrote:
Why not simplify?
static this()
{
import std.stdio, a, c; // existing syntax
...
}
static this()
{ // no imports -> no dependencies
...
}
The current behavior should just be dropped.
+2
Simple & Elegant.
-2
Confusing. What is the scope of the import? How does it interact with
imports above/below the static this?
R
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Fixing cyclic import static construction problems
On 11/30/12 9:09 AM, foobar wrote:
On Thursday, 29 November 2012 at 23:02:17 UTC, Walter Bright wrote:
On 11/30/2012 9:43 AM, Walter Bright wrote:
It is possible for each static constructor to specify independently of
the other static constructors which imports must be constructed first.
But do we really want to go that far?
One way to do that might be to borrow syntax from classes:
static this() : std.stdio, a, c
[snip]
Why not simplify?
[snip]
Why not further simplify?
static this()
{
// JUST AS BEFORE
...
}
There is no need to redundantly specify what modules are used because...
well they are right there in the body of the static constructor.
* no extra syntax
* no change in the symbol visibility rules (why are symbols invisible by
default in static cdtors?)
* no change to the manual
* no breakage of existing code (only code that was broken will be accepted)
* no acceptance of actual circular dependencies go through compilation
This would be purely an improvement to the implementation that would
allow more correct programs to compile. It's a removal of limitation -
the best kind of language change there ever is.
We should have a "bootcamp" area with small compiler and library
projects (such as this after we reach consensus). People who are
interested in helping D, whether or not they've done it before, could
find this area things that are well defined and will definitely be
accepted if properly executed.
Andrei
Re: Fixing cyclic import static construction problems
On Friday, November 30, 2012 10:27:31 Artur Skawina wrote: > On 11/29/12 23:34, Jonathan M Davis wrote: > > On Thursday, November 29, 2012 23:28:07 Timon Gehr wrote: > >> On 11/29/2012 01:17 PM, Jonathan M Davis wrote: > >>> In the past when I've brought up similar solutions, he's been completely > >>> opposed to them. ... > >> > >> It is not a solution, it is a workaround. > > > > What do you mean? The runtime sees circular dependencies between modules > > even when there's no actual circular dependency between static > > constructors. We need to fix that. One way is to just make the runtime > > not care, which wouldn't be particularly safe. Another is to explicitly > > tell it that there are no such dependencies. I don't see how that's not a > > solution. And unless someone can come up with a way for the runtime to > > somehow determine on its own that there's no actual, circular dependency, > > I don't see how anything better could be done. > > It's relatively easy for the /compiler/ to figure it out; it's just that > implementing a simple user-provided flag requires the least amount of work. > What Walter suggested can be tweaked to be sane (per-ctor flag) and will > still be useful if/when the compiler becomes smarter (think lazily initted > module fields). > For the compiler to check if the value of every imported symbol accessed > inside a mod-ctor can be evaluated at compile-time (if you encounter a case > where this is not true it means there (potentially) is a true dependency and > the ctors should be ordered) would require more work. It can't be evaluated at compile time because of .di files. The compiler doesn't necessarily have all of the source to work with - including the static constructors - and if it doesn't have that, it can't do it. If anything figures this out automatically, it has to be the runtime. If we can do that, great, but it means that it'll have actually have to look at individual symbols rather than just at the module level like it's doing now, and right now, we have nothing even close to that. Regardless, while the compiler may be able to provide additional information to the runtime, it's still the runtime that needs to figure this out and not the compiler. - Jonathan M Davis
Re: Fixing cyclic import static construction problems
On Friday, 30 November 2012 at 20:40:23 UTC, Jonathan M Davis wrote: It can't be evaluated at compile time because of .di files. The compiler doesn't necessarily have all of the source to work with - including the static constructors - and if it doesn't have that, it can't do it. If anything figures this out automatically, it has to be the runtime. If we can do that, great, but it means that it'll have actually have to look at individual symbols rather than just at the module level like it's doing now, and right now, we have nothing even close to that. Regardless, while the compiler may be able to provide additional information to the runtime, it's still the runtime that needs to figure this out and not the compiler. I'd bet you can solve most cases anyway.
Re: Fixing cyclic import static construction problems
On 11/30/12 21:40, Jonathan M Davis wrote: > On Friday, November 30, 2012 10:27:31 Artur Skawina wrote: >> On 11/29/12 23:34, Jonathan M Davis wrote: >>> On Thursday, November 29, 2012 23:28:07 Timon Gehr wrote: On 11/29/2012 01:17 PM, Jonathan M Davis wrote: > In the past when I've brought up similar solutions, he's been completely > opposed to them. ... It is not a solution, it is a workaround. >>> >>> What do you mean? The runtime sees circular dependencies between modules >>> even when there's no actual circular dependency between static >>> constructors. We need to fix that. One way is to just make the runtime >>> not care, which wouldn't be particularly safe. Another is to explicitly >>> tell it that there are no such dependencies. I don't see how that's not a >>> solution. And unless someone can come up with a way for the runtime to >>> somehow determine on its own that there's no actual, circular dependency, >>> I don't see how anything better could be done. >> >> It's relatively easy for the /compiler/ to figure it out; it's just that >> implementing a simple user-provided flag requires the least amount of work. >> What Walter suggested can be tweaked to be sane (per-ctor flag) and will >> still be useful if/when the compiler becomes smarter (think lazily initted >> module fields). >> For the compiler to check if the value of every imported symbol accessed >> inside a mod-ctor can be evaluated at compile-time (if you encounter a case >> where this is not true it means there (potentially) is a true dependency and >> the ctors should be ordered) would require more work. > > It can't be evaluated at compile time because of .di files. The compiler It can. The case where all definitions aren't available is of course a potential source of cycles and means that the ctors have to be run in order. There's no way around that, and it's where the attribute (nee pragma) helps, because it can be attached to the declaration, when the programmer thinks he knows better. Andrei's naive suggestion would only handle the trivial toy example case, but fail for much of real code, that needs to access statically known properties of other modules. D has enough features that work for five-liners, but are unusable for real work. The important points are that: 1) a ctor that does not access any symbol from another module is not treated as dependent on that module. 2) accessing just /types/ defined in another module works. No RT dep here. 3) reading known initialized constant data works. That's const/immutable/enum - again, those can never become a RT dep. 4) calling side-effect free code that does not depend on non-local state works. This is why an is-it-ctfeable check works - it will catch even the indirect deps. Yeah, it's conservative, but is has to be. A kind of 'pure-but-w/o-external-refs" thing would help further, but that can be added incrementally and may not even be necessary. > have nothing even close to that. Regardless, while the compiler may be able > to provide additional information to the runtime, it's still the runtime that > needs to figure this out and not the compiler. No. Would, at least, require going from per-module to per-symbol which is a *much* larger change than was proposed here. I don't even want to think about the (runtime) cost. artur
Re: Fixing cyclic import static construction problems
On Saturday, 1 December 2012 at 10:54:37 UTC, Artur Skawina wrote: 3) reading known initialized constant data works. That's const/immutable/enum - again, those can never become a RT dep. const or immutable can be instanciated by another static ctor. 4) calling side-effect free code that does not depend on non-local state works. This is why an is-it-ctfeable check works - it will catch even the indirect deps. Yeah, it's conservative, but is has to be. A kind of 'pure-but-w/o-external-refs" thing would help further, but that can be added incrementally and may not even be necessary. pure is probably enough. CTFEable is more restrictive. have nothing even close to that. Regardless, while the compiler may be able to provide additional information to the runtime, it's still the runtime that needs to figure this out and not the compiler. That is an undemonstrated assertion (and I think it is false). No. Would, at least, require going from per-module to per-symbol which is a *much* larger change than was proposed here. I don't even want to think about the (runtime) cost. This reasonning is based on the assertion above, so is meaningless until the assertion is proven to be true.
Re: Fixing cyclic import static construction problems
On 12/01/12 12:29, deadalnix wrote: > On Saturday, 1 December 2012 at 10:54:37 UTC, Artur Skawina wrote: >> 3) reading known initialized constant data works. That's >> const/immutable/enum - >>again, those can never become a RT dep. > > const or immutable can be instanciated by another static ctor. The key words are "known" and "initialized". IOW const int a = 42; doesn't add a dep, but const int a; does. /Modifying/ initialized const data from a mod ctor is (and should be) illegal. >> 4) calling side-effect free code that does not depend on non-local state >> works. >>This is why an is-it-ctfeable check works - it will catch even the >> indirect deps. >>Yeah, it's conservative, but is has to be. A kind of >> 'pure-but-w/o-external-refs" >>thing would help further, but that can be added incrementally and may not >> even >>be necessary. >> > > pure is probably enough. CTFEable is more restrictive. D's pure isn't enough, because it will allow accesses to external immutable data - which can be modified by a mod ctor. But like i said - this might not be a problem in practice, as it's just about the cross-module non-ctfeable, but truly pure function calls inside mod ctors - is this really something that occurs often enough? >>> have nothing even close to that. Regardless, while the compiler may be able >>> to provide additional information to the runtime, it's still the runtime >>> that >>> needs to figure this out and not the compiler. >> > > That is an undemonstrated assertion (and I think it is false). > >> No. Would, at least, require going from per-module to per-symbol which is a >> *much* >> larger change than was proposed here. I don't even want to think about the >> (runtime) >> cost. >> > > This reasonning is based on the assertion above, so is meaningless until the > assertion is proven to be true. I'm not sure what assertion you think is false, but theoretically it /is/ possible to figure out the deps at runtime, even w/o any hints. Think valgrind. But it's a very bad idea, with a significant runtime cost (not as bad as valgrind since you can figure out some things statically, but determining the order alone would already be too slow for larger projects with many symbols and deps.) artur
Re: Fixing cyclic import static construction problems
On Saturday, 1 December 2012 at 12:08:10 UTC, Artur Skawina wrote: The key words are "known" and "initialized". IOW const int a = 42; doesn't add a dep, but const int a; does. /Modifying/ initialized const data from a mod ctor is (and should be) illegal. That make sense. D's pure isn't enough, because it will allow accesses to external immutable data - which can be modified by a mod ctor. But like i said - this might not be a problem in practice, as it's just about the cross-module non-ctfeable, but truly pure function calls inside mod ctors - is this really something that occurs often enough? Ho yes, this make sense too. CTFEable is still too restrictive but pure isn't enough. I'm not sure what assertion you think is false, but theoretically it /is/ possible to figure out the deps at runtime, even w/o any hints. Think valgrind. But it's a very bad idea, with a significant runtime cost (not as bad as valgrind since you can figure out some things statically, but determining the order alone would already be too slow for larger projects with many symbols and deps.) The assertion is that it must be done at runtime. It is obviously doable at runtime, but complicated. I think that most of it can be done at compile time.
Re: Fixing cyclic import static construction problems
On Thursday, 29 November 2012 at 02:34:11 UTC, Walter Bright
wrote:
For discussion:
Cyclical Imports
Problem:
a.d
module a;
import b;
static this () { ... }
b.d
module b;
import a;
static this() { ... }
-
Static constructors for a module are only run after static
constructors
for all its imports are run. Circular imports, such as the
above, are
detected at run time and the program is aborted.
This can in general be solved by moving the static
constructor(s) into
a third module, c.d, which does not import a or b. But, people
find this
to be unnatural.
Proposed Solution:
Add a pragma,
pragma(cyclic_imports);
This can appear anywhere in a module, and applies globally to
that module.
It means that static constructors from imports that are not
part of the cycle
are run first, and that the static constructor for this module
may be run before
the static constructors of other modules that are part of the
cycle.
If any static constructors in such a module with the pragma have
the @safe attribute, that is a compile time error.
While more complex, would it be possible to declare which
imported modules can be constructed after the current one? That
allows catching any other unintended cyclic imports. For example:
static this {
pragma(cyclic_import, b);
// code that does not depend on
// static constructor(s) in module b
}
Re: Fixing cyclic import static construction problems
Jonathan M Davis wrote: > Or do you mean something else by the suggestion that the > existence of Walter's pragma can be assumed? In http://forum.dlang.org/thread/k96hj2$2lus$1...@digitalmars.com? page=4#post-k98o7j:24qb6:241:40digitalmars.com Walter pointed this out for a clique of imports. All but one members of a clique are to have that pragma anyway---and Walter claims, that there is no error introduced, when the remaining member has that pragma too. -manfred
Re: Fixing cyclic import static construction problems
On Thu, 29 Nov 2012 12:39:19 +0100
"Paulo Pinto" wrote:
> On Thursday, 29 November 2012 at 03:19:55 UTC, Andrei
> Alexandrescu wrote:
> > On 11/28/12 9:34 PM, Walter Bright wrote:
> >> For discussion:
> > [snip]
> >
> > I'd say we better finish const, immutable, and shared first.
> >
> > Andrei
>
> +1
>
> Fully agree.
>
> Cyclic imports are a minor nuisance that can be easily solvable
> with better code architecture.
>
Or just lazy initialization:
Foo f = blah();
-->
private Foo _f;
private bool fInited=false;
@property Foo f()
{
if(!fInited)
{
_f = blah();
fInited = true;
}
return _f;
}
And that boilerplate's trivially wrapped up with a mixin.
