Re: DI Generation Needs your Help!
On 12/19/11, Adam Wilson flybo...@gmail.com wrote: If anyone wishes to test my changes against their code, you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git This seems to work nicely even with the latest release. It's much better than the current .di generation for sure. Any plans on merging this with mainline before it goes stale?
Re: DI Generation Needs your Help!
On 3/14/12, Andrej Mitrovic andrej.mitrov...@gmail.com wrote:
This seems to work nicely even with the latest release. It's much
better than the current .di generation for sure. Any plans on merging
this with mainline before it goes stale?
Although I would really like to be able to keep documentation comments
in the .di files, e.g.:
class Foo {
/** Commented ctor. */
this() { }
}
-
class Foo {
/** Commented ctor. */
this();
}
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 12:30:00 -0700, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: On 12/19/11, Adam Wilson flybo...@gmail.com wrote: If anyone wishes to test my changes against their code, you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git This seems to work nicely even with the latest release. It's much better than the current .di generation for sure. Any plans on merging this with mainline before it goes stale? I am currently maintaining this against DMD HEAD, but until this bug (http://d.puremagic.com/issues/show_bug.cgi?id=7423) gets fixed I can't open a pull request because the patch will break both the Phobos and DRuntime builds badly without it. I would fix the bug myself, but I am getting married in a month and am rather short on time. Earliest I'll likely be able to investigate a fix is May. If anyone else wants to take a shot at a fix before then, I'd happily open up a pull request once they're done.
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 12:35:03 -0700, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
On 3/14/12, Andrej Mitrovic andrej.mitrov...@gmail.com wrote:
This seems to work nicely even with the latest release. It's much
better than the current .di generation for sure. Any plans on merging
this with mainline before it goes stale?
Although I would really like to be able to keep documentation comments
in the .di files, e.g.:
class Foo {
/** Commented ctor. */
this() { }
}
-
class Foo {
/** Commented ctor. */
this();
}
I'll look into it when I get the chance. :-)
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
El 19/12/2011 9:11, Adam Wilson escribió: 1. Function Implementations are removed What about function inlining? would it work from a different module? I think the implementation of small functions should be kept so that client code can inline them. The current DMD does this apparently, it keeps small functions in .di files.
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro alvarodotseg...@gmail.com wrote: El 19/12/2011 9:11, Adam Wilson escribió: 1. Function Implementations are removed What about function inlining? would it work from a different module? I think the implementation of small functions should be kept so that client code can inline them. The current DMD does this apparently, it keeps small functions in .di files. The problem is that in DI generation, at least as near as I can tell, D has now idea how big a function is during DI generation. In my experience it was keeping all functions not just small ones. And frankly DI generation is targeted at library builders who are well aware of the inlining trade-offs. And then comes the question of how do you define small functions to an adequately technical level. Because theoretically you can inline ANYTHING. Yes, there are rules the prevent inlining, but you'll also note that the state of these rules is not guaranteed to be known at generation time. DI generation currently works as such. After the code has been parsed into an AST, the compiler runs a special set of virtual functions called toCBuffer (and it's variants) that is used to print out the AST in source form again. NO semantic analysis of any kind has been performed on the AST yet. And you wouldn't want to generate DI's after semantic analysis as the analysis fundamentally alters the AST such that you would not get the same code back out and some things would be missing from the DI files that you intended to be there. The AST at DI generation is an incredibly naive representation of the SOURCE not the SEMANTICS; which is what you would need to determine the eligibility of inlining. The answer that Walter has always given for objections about how DI files are built is that if you want anything customized about the output, you have to do it yourself. DI generation will probably NEVER be as perfect as everyone wants. But I think this solution represents a best effort to get DI files to a point where the community agrees that they would be most useful in achieving their intended purpose, which is as interface files for compiled libraries. It's not perfect, but it gets you A LOT further than the current one, if you need customization beyond that, well, D lets you do that too. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
Adam Wilson: I think I said this before, but for what it's worth, I think your new approach is spot-on correct.
Re: DI Generation Needs your Help!
On 3/14/12, Adam Wilson flybo...@gmail.com wrote: I would fix the bug myself, but I am getting married in a month and am rather short on time. That's ok I'm in no hurry just curious. Congrats on getting married! :)
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:23:19 -0700, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: On 3/14/12, Adam Wilson flybo...@gmail.com wrote: I would fix the bug myself, but I am getting married in a month and am rather short on time. That's ok I'm in no hurry just curious. Congrats on getting married! :) Thank you! Hehe, well, I have big plans for D, but I need DI files to work better to execute them, so I am a little bit of a hurry, but real life wins for now. ;-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:17:28 -0700, Adam D. Ruppe destructiona...@gmail.com wrote: Adam Wilson: I think I said this before, but for what it's worth, I think your new approach is spot-on correct. Thank you. More importantly though, Walter and Andrei also agree, so in the end, the inlinaholics can just deal. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
El 14/03/2012 22:13, Adam Wilson escribió: On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro alvarodotseg...@gmail.com wrote: The problem is that in DI generation, at least as near as I can tell, D has now idea how big a function is during DI generation. In my experience it was keeping all functions not just small ones. And frankly DI generation is targeted at library builders who are well aware of the inlining trade-offs. And then comes the question of how do you define small functions to an adequately technical level. OK, I rechecked. DMD -H is at least omitting function bodies containing foreach() loops (and maybe other cases). Maybe that led me to assume things. AFAIK some C++ compilers use heuristics (don't really know what that is exactly) to decide what small functions deserved automatic inlining. And well, C++ library builders are aware of that and usually keep the small functions they want inlined in the header file (e.g. in the class declaration). I can see a lot of cases where D libraries would benefit from allowing some of their functions to be inlined. Think of @properties, they're often pretty short. Other than this detail, the improvement in DI generation is very welcome.
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:42:54 -0700, Alvaro alvarodotseg...@gmail.com wrote: El 14/03/2012 22:13, Adam Wilson escribió: On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro alvarodotseg...@gmail.com wrote: The problem is that in DI generation, at least as near as I can tell, D has now idea how big a function is during DI generation. In my experience it was keeping all functions not just small ones. And frankly DI generation is targeted at library builders who are well aware of the inlining trade-offs. And then comes the question of how do you define small functions to an adequately technical level. OK, I rechecked. DMD -H is at least omitting function bodies containing foreach() loops (and maybe other cases). Maybe that led me to assume things. Hmmm, I hadn't seen it do this. But the whole point of DI files is to create an interface to compiled libraries, not be the absolute fastest it can be. It is generally accepted that you are going to take a minuscule (and on modern machines it is truly minuscule) hit every time you cross a library boundary, unless you import the whole source of said library, which is functionally what DI generation does now. AFAIK some C++ compilers use heuristics (don't really know what that is exactly) to decide what small functions deserved automatic inlining. Those heuristics are in the semantic analysis step and I really don't think building the DI file against the constantly changing AST during semantic analysis is a wise idea. DI's get built before that, and as such do no have access to those heuristics. If the compiler is doing something it's based on its own special, and limited analysis prior to the semantic analysis (I think I know which function is doing this, but it doesn't work very well at all). Also, trying to build DI's during semantic analysis would most likely drastically slow down the compiler as the DI's would need to be constantly updated to make the AST changes. Basically, it's an incredible amount of work for a minuscule speed improvement in the final step. There are areas of D that are MUCH more ripe for speed improvements than inlining in DI files ... *cough*GC*cough* And well, C++ library builders are aware of that and usually keep the small functions they want inlined in the header file (e.g. in the class declaration). I can see a lot of cases where D libraries would benefit from allowing some of their functions to be inlined. Think of @properties, they're often pretty short. Well, that's where the do-it-yourself part into play. This is an automated system for generating headers based on a naive AST. It will not be perfect. But C++ doesn't even have that and they have to hand maintain everything, so pretty much anything we do is an improvement. :-) @properties are a valid concern, and also one that can be addressed in the current method of DI generation as they are a valid token that can be checked for and excluded. If the community wishes for @properties to retain their implementations that would an easy thing to do. But be warned, you wont be able to call private functions as those are dropped COMPLETELY from the DI file. Private data will of course be accessible. Other than this detail, the improvement in DI generation is very welcome. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On 2011-12-23 01:22, Adam Wilson wrote: On Thu, 22 Dec 2011 13:02:49 -0800, Jacob Carlborg d...@me.com wrote: Never mind, it seems to be working. Did you check it against my forked code? I want to make sure that there are as few errors as possible before I open up the pull. :-) No, I checked with DMD 1.072. -- /Jacob Carlborg
Re: DI Generation Needs your Help!
On 12/22/11 1:07 AM, Adam Wilson wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Variables in template functions/classes would be stripped of their initializers. 2. Manifest Constants would lose their initializers. DI generation is getting close! Every Phobos DI module I've tried has passed the VisualD parser/lexer (for whatever that is worth). I want to do some build testing with those files, but even complicated modules like std.algorithm and std.regex are passing the lexer/parser. If everything goes well with that testing I'll open up a pull for dmd. Hopefully proper DI generation will make it into 2.058! You can get the latest from my git account here: https://lightben...@github.com/LightBender/dmd.git Great! Did you fix indentation? Also, I think testing with Visual Studio is insufficient. One good test is to extract all unittest code from std modules, and then make sure it compiles and runs when you only import the .di files. Andrei
Re: DI Generation Needs your Help!
On Thu, 22 Dec 2011 08:46:39 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/22/11 1:07 AM, Adam Wilson wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Variables in template functions/classes would be stripped of their initializers. 2. Manifest Constants would lose their initializers. DI generation is getting close! Every Phobos DI module I've tried has passed the VisualD parser/lexer (for whatever that is worth). I want to do some build testing with those files, but even complicated modules like std.algorithm and std.regex are passing the lexer/parser. If everything goes well with that testing I'll open up a pull for dmd. Hopefully proper DI generation will make it into 2.058! You can get the latest from my git account here: https://lightben...@github.com/LightBender/dmd.git Great! Did you fix indentation? Also, I think testing with Visual Studio is insufficient. One good test is to extract all unittest code from std modules, and then make sure it compiles and runs when you only import the .di files. Andrei No I haven't, there is an open pull for something about indentations, so I wasn't sure if I needed to. I will certainly do what I can for indentations, but the generation process is a bit messy so I can't promise it will be perfect. I completely agree, it's just the baseline sanity check I'm using while I hack on it. Unless there is a way to export the unit tests automatically, I don't have the time to hand port all the unit tests for Phobos. I think I'll just pick a few modules with differing levels of complexity and use those unit tests. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Wed, 21 Dec 2011 23:43:56 -0800, Jacob Carlborg d...@me.com wrote:
On 2011-12-22 08:07, Adam Wilson wrote:
The latest DI generation code is now on my Git account and ready for
testing. It fixes the following issues:
1. Variables in template functions/classes would be stripped of their
initializers.
2. Manifest Constants would lose their initializers.
DI generation is getting close! Every Phobos DI module I've tried has
passed the VisualD parser/lexer (for whatever that is worth). I want to
do some build testing with those files, but even complicated modules
like std.algorithm and std.regex are passing the lexer/parser. If
everything goes well with that testing I'll open up a pull for dmd.
Hopefully proper DI generation will make it into 2.058!
You can get the latest from my git account here:
https://lightben...@github.com/LightBender/dmd.git
Can the generator handle this:
id objc_msgSend (ARGS...)(id theReceiver, SEL theSelector, ARGS args)
{
alias extern (C) id function (id, SEL, ARGS) fp;
return (cast(fp)c_objc_msgSend)(theReceiver, theSelector, args);
}
I know it had problems with that, at least in previous versions of DMD.
Well, without the accompanying types that are used in the declaration I'm
not entire sure how I could give you a good answer. However, since it's a
template function, I can say that template functions in DI files are
basically straight implementation dumps of the D file. Can you tell me
where DI generation hiccups in that code? If the hiccup is inside the
function impl than I doubt anything I've changed will fix your error.
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On 2011-12-22 18:33, Adam Wilson wrote:
On Wed, 21 Dec 2011 23:43:56 -0800, Jacob Carlborg d...@me.com wrote:
On 2011-12-22 08:07, Adam Wilson wrote:
The latest DI generation code is now on my Git account and ready for
testing. It fixes the following issues:
1. Variables in template functions/classes would be stripped of their
initializers.
2. Manifest Constants would lose their initializers.
DI generation is getting close! Every Phobos DI module I've tried has
passed the VisualD parser/lexer (for whatever that is worth). I want to
do some build testing with those files, but even complicated modules
like std.algorithm and std.regex are passing the lexer/parser. If
everything goes well with that testing I'll open up a pull for dmd.
Hopefully proper DI generation will make it into 2.058!
You can get the latest from my git account here:
https://lightben...@github.com/LightBender/dmd.git
Can the generator handle this:
id objc_msgSend (ARGS...)(id theReceiver, SEL theSelector, ARGS args)
{
alias extern (C) id function (id, SEL, ARGS) fp;
return (cast(fp)c_objc_msgSend)(theReceiver, theSelector, args);
}
I know it had problems with that, at least in previous versions of DMD.
Well, without the accompanying types that are used in the declaration
I'm not entire sure how I could give you a good answer. However, since
it's a template function, I can say that template functions in DI files
are basically straight implementation dumps of the D file. Can you tell
me where DI generation hiccups in that code? If the hiccup is inside the
function impl than I doubt anything I've changed will fix your error.
Never mind, it seems to be working.
--
/Jacob Carlborg
Re: DI Generation Needs your Help!
On Thu, 22 Dec 2011 13:02:49 -0800, Jacob Carlborg d...@me.com wrote:
On 2011-12-22 18:33, Adam Wilson wrote:
On Wed, 21 Dec 2011 23:43:56 -0800, Jacob Carlborg d...@me.com wrote:
On 2011-12-22 08:07, Adam Wilson wrote:
The latest DI generation code is now on my Git account and ready for
testing. It fixes the following issues:
1. Variables in template functions/classes would be stripped of their
initializers.
2. Manifest Constants would lose their initializers.
DI generation is getting close! Every Phobos DI module I've tried has
passed the VisualD parser/lexer (for whatever that is worth). I want
to
do some build testing with those files, but even complicated modules
like std.algorithm and std.regex are passing the lexer/parser. If
everything goes well with that testing I'll open up a pull for dmd.
Hopefully proper DI generation will make it into 2.058!
You can get the latest from my git account here:
https://lightben...@github.com/LightBender/dmd.git
Can the generator handle this:
id objc_msgSend (ARGS...)(id theReceiver, SEL theSelector, ARGS args)
{
alias extern (C) id function (id, SEL, ARGS) fp;
return (cast(fp)c_objc_msgSend)(theReceiver, theSelector, args);
}
I know it had problems with that, at least in previous versions of DMD.
Well, without the accompanying types that are used in the declaration
I'm not entire sure how I could give you a good answer. However, since
it's a template function, I can say that template functions in DI files
are basically straight implementation dumps of the D file. Can you tell
me where DI generation hiccups in that code? If the hiccup is inside the
function impl than I doubt anything I've changed will fix your error.
Never mind, it seems to be working.
Did you check it against my forked code? I want to make sure that there
are as few errors as possible before I open up the pull. :-)
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Tue, 20 Dec 2011 04:41:11 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/20/11 2:03 AM, Adam Wilson wrote: On Mon, 19 Dec 2011 00:11:25 -0800, Adam Wilson flybo...@gmail.com wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Privates should exist in the DI file to support public templates. 2. Template classes and functions retain their implementations. 3. Immutable types should retain their initializers. Great! At this point I could really use testing; you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git I am trying to get myself setup for building phobos as a test but this is proving to be a lengthy process. Nah, it's much easier than you might think. The posix.mak makefile is very small for what it does, and you need to literally change one line of code to make it generate .di headers. Andrei Is it be add the proper -H options? I'll try it tomorrow, events pretty much overran my day today. If the new DI gen can build valid Phobos DI's I think it is time to consider opening a pull request and get feedback from the maintainers. Any disagreement with that? -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Tue, 20 Dec 2011 03:49:35 -0800, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
Derelict works ok now, good work!
This is fantastic news! Thanks for helping to test these changes. :-)
However, the .di files end up eating newlines.
Before:
double ALLEGRO_USECS_TO_SECS(long x)
{
return x / 1e+06;
}
double ALLEGRO_MSECS_TO_SECS(long x)
{
return x / 1000;
}
double ALLEGRO_BPS_TO_SECS(int x)
{
return 1 / x;
}
After:
double ALLEGRO_USECS_TO_SECS(long x);double ALLEGRO_MSECS_TO_SECS(long
x);double ALLEGRO_BPS_TO_SECS(int x);
I've tried merging
https://github.com/D-Programming-Language/dmd/pull/538 but it doesn't
fix this.
I noticed similar issues before I made changes to DI generation, but
fixing it wasn't as high a priority as getting it working. For now I am
going to continue testing the current changes, however, I would be open to
prettifying it later. I think the simple fix is to add an extra newline
between those functions.
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
Am 19.12.2011 09:11, schrieb Adam Wilson:
As you may all be aware, I've been trying to improve the automated
generation of .di files and I now have something that I feel is
testable. Currently the new code only makes the following changes.
1. Function Implementations are removed
2. Private Function Declarations are removed.
3. Variable Initializers, except for const, are removed.
Everything else is left alone. Templates and mixins are not addressed
with this code and *should* not be modified. That's where I need your
help, the test cases I have written cover some basic scenarios but I
don't have the capability to test these changes with the diverse code
base that the community has created.
drey_ from IRC was kind enough to test build Derelict with the changes
and has discovered a potential issue around private imports. Derelict
uses private imports that contain types which are used in function alias
declarations. As one would expect, this caused many compiler errors.
Currently, I feel that private imports should be stripped from the DI
file as they are intended to be internal to the module. However, I want
to put it to the community to decide, and I would especially appreciate
Mr. Bright's opinion on private imports in DI files.
If anyone wishes to test my changes against their code, you can download
them from my git account here:
https://lightben...@github.com/LightBender/dmd.git
I only ask that you inform me of anything broken and provide a test case
that reproduces the error.
The following is my current test case, it comes from a module in my
project that has been modified to include the various tests I and others
have been able to think of. I know for a fact that it is missing a great
number of features available in D, and I would really appreciate any
tests you might wish to add to this case.
module Horizon.Collections.Basic;
export void TestFunc(int a) {}
private void TestFunc2(int b) {}
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i = 0;
protected int j = 1;
export void ExportedStructFunc() {}
private void PrivateStructFunc() {}
}
export class List : IList
{
export this(uint Capacity)
{
vCapacity = Capacity;
internal = new Object[4];
}
private Object[] internal;
private uint vCapacity = 0;
export @property uint Capacity()
{
return vCapacity;
}
private uint vCount = 0;
export @property uint Count()
{
return vCount;
}
export void Add(Object Value)
{
if(vCount == vCapacity) internal.length *= 2;
internal[++vCount] = Value;
}
export int opApply(int delegate(ref Object) dg) { return 0; }
export IEnumerator GetEnumerator() {return null;}
export @property bool IsFixedSize() {return false;}
export @property bool IsReadOnly() {return false;}
export void Clear() { return; }
export bool Contains(Object Value) {return false;}
export Object opIndex(ulong Index) {return null;}
export void opIndexAssign(Object Value, ulong Index) {}
export int IndexOf(Object Value) {return 0;}
export void Insert(ulong Index, Object Value) {}
export void Remove(Object Value) {}
export void RemoveAt(ulong Index) {}
private int FooTest(int c) { return c*2; }
protected int ProtectedTest(int d) { return d*3; }
}
This is the DI file as exported with the generation changes:
// D import file generated from 'Basic.d'
module Horizon.Collections.Basic;
export void TestFunc(int a);
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i;
protected int j;
export void ExportedStructFunc();
}
export class List : IList
{
export this(uint Capacity);
private Object[] internal;
private uint
Re: DI Generation Needs your Help!
On 12/21/2011 04:58 PM, Benjamin Thaut wrote:
Am 19.12.2011 09:11, schrieb Adam Wilson:
As you may all be aware, I've been trying to improve the automated
generation of .di files and I now have something that I feel is
testable. Currently the new code only makes the following changes.
1. Function Implementations are removed
2. Private Function Declarations are removed.
3. Variable Initializers, except for const, are removed.
Everything else is left alone. Templates and mixins are not addressed
with this code and *should* not be modified. That's where I need your
help, the test cases I have written cover some basic scenarios but I
don't have the capability to test these changes with the diverse code
base that the community has created.
drey_ from IRC was kind enough to test build Derelict with the changes
and has discovered a potential issue around private imports. Derelict
uses private imports that contain types which are used in function alias
declarations. As one would expect, this caused many compiler errors.
Currently, I feel that private imports should be stripped from the DI
file as they are intended to be internal to the module. However, I want
to put it to the community to decide, and I would especially appreciate
Mr. Bright's opinion on private imports in DI files.
If anyone wishes to test my changes against their code, you can download
them from my git account here:
https://lightben...@github.com/LightBender/dmd.git
I only ask that you inform me of anything broken and provide a test case
that reproduces the error.
The following is my current test case, it comes from a module in my
project that has been modified to include the various tests I and others
have been able to think of. I know for a fact that it is missing a great
number of features available in D, and I would really appreciate any
tests you might wish to add to this case.
module Horizon.Collections.Basic;
export void TestFunc(int a) {}
private void TestFunc2(int b) {}
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i = 0;
protected int j = 1;
export void ExportedStructFunc() {}
private void PrivateStructFunc() {}
}
export class List : IList
{
export this(uint Capacity)
{
vCapacity = Capacity;
internal = new Object[4];
}
private Object[] internal;
private uint vCapacity = 0;
export @property uint Capacity()
{
return vCapacity;
}
private uint vCount = 0;
export @property uint Count()
{
return vCount;
}
export void Add(Object Value)
{
if(vCount == vCapacity) internal.length *= 2;
internal[++vCount] = Value;
}
export int opApply(int delegate(ref Object) dg) { return 0; }
export IEnumerator GetEnumerator() {return null;}
export @property bool IsFixedSize() {return false;}
export @property bool IsReadOnly() {return false;}
export void Clear() { return; }
export bool Contains(Object Value) {return false;}
export Object opIndex(ulong Index) {return null;}
export void opIndexAssign(Object Value, ulong Index) {}
export int IndexOf(Object Value) {return 0;}
export void Insert(ulong Index, Object Value) {}
export void Remove(Object Value) {}
export void RemoveAt(ulong Index) {}
private int FooTest(int c) { return c*2; }
protected int ProtectedTest(int d) { return d*3; }
}
This is the DI file as exported with the generation changes:
// D import file generated from 'Basic.d'
module Horizon.Collections.Basic;
export void TestFunc(int a);
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i;
protected int j;
export void ExportedStructFunc();
}
export class List : IList
{
export this(uint Capacity);
Re: DI Generation Needs your Help!
On 12/21/11 2:12 AM, Adam Wilson wrote: On Tue, 20 Dec 2011 04:41:11 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/20/11 2:03 AM, Adam Wilson wrote: On Mon, 19 Dec 2011 00:11:25 -0800, Adam Wilson flybo...@gmail.com wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Privates should exist in the DI file to support public templates. 2. Template classes and functions retain their implementations. 3. Immutable types should retain their initializers. Great! At this point I could really use testing; you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git I am trying to get myself setup for building phobos as a test but this is proving to be a lengthy process. Nah, it's much easier than you might think. The posix.mak makefile is very small for what it does, and you need to literally change one line of code to make it generate .di headers. Andrei Is it be add the proper -H options? I'll try it tomorrow, events pretty much overran my day today. If the new DI gen can build valid Phobos DI's I think it is time to consider opening a pull request and get feedback from the maintainers. Any disagreement with that? I don't think we'll use .di for phobos for the time being. I was suggesting you do so in order to look over the generated .di files and make sure they work. Andrei
Re: DI Generation Needs your Help!
On Wed, 21 Dec 2011 08:41:43 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/21/11 2:12 AM, Adam Wilson wrote: On Tue, 20 Dec 2011 04:41:11 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/20/11 2:03 AM, Adam Wilson wrote: On Mon, 19 Dec 2011 00:11:25 -0800, Adam Wilson flybo...@gmail.com wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Privates should exist in the DI file to support public templates. 2. Template classes and functions retain their implementations. 3. Immutable types should retain their initializers. Great! At this point I could really use testing; you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git I am trying to get myself setup for building phobos as a test but this is proving to be a lengthy process. Nah, it's much easier than you might think. The posix.mak makefile is very small for what it does, and you need to literally change one line of code to make it generate .di headers. Andrei Is it be add the proper -H options? I'll try it tomorrow, events pretty much overran my day today. If the new DI gen can build valid Phobos DI's I think it is time to consider opening a pull request and get feedback from the maintainers. Any disagreement with that? I don't think we'll use .di for phobos for the time being. I was suggesting you do so in order to look over the generated .di files and make sure they work. Andrei Oops! I mean't a pull on dmd for the new DI code. Sorry for the confusion! -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On 12/21/11 10:42 AM, Adam Wilson wrote: Oops! I mean't a pull on dmd for the new DI code. Sorry for the confusion! Yes, absolutely! Andrei
Re: DI Generation Needs your Help!
On Wednesday, 21 December 2011 at 16:42:59 UTC, Adam Wilson wrote: Oops! I mean't a pull on dmd for the new DI code. Sorry for the confusion! Pull requests are always appreciated! :) Remember, it's more like please-review-and-pull-if-of-acceptable-quality request :P
Re: DI Generation Needs your Help!
The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Variables in template functions/classes would be stripped of their initializers. 2. Manifest Constants would lose their initializers. DI generation is getting close! Every Phobos DI module I've tried has passed the VisualD parser/lexer (for whatever that is worth). I want to do some build testing with those files, but even complicated modules like std.algorithm and std.regex are passing the lexer/parser. If everything goes well with that testing I'll open up a pull for dmd. Hopefully proper DI generation will make it into 2.058! You can get the latest from my git account here: https://lightben...@github.com/LightBender/dmd.git -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On 2011-12-22 08:07, Adam Wilson wrote:
The latest DI generation code is now on my Git account and ready for
testing. It fixes the following issues:
1. Variables in template functions/classes would be stripped of their
initializers.
2. Manifest Constants would lose their initializers.
DI generation is getting close! Every Phobos DI module I've tried has
passed the VisualD parser/lexer (for whatever that is worth). I want to
do some build testing with those files, but even complicated modules
like std.algorithm and std.regex are passing the lexer/parser. If
everything goes well with that testing I'll open up a pull for dmd.
Hopefully proper DI generation will make it into 2.058!
You can get the latest from my git account here:
https://lightben...@github.com/LightBender/dmd.git
Can the generator handle this:
id objc_msgSend (ARGS...)(id theReceiver, SEL theSelector, ARGS args)
{
alias extern (C) id function (id, SEL, ARGS) fp;
return (cast(fp)c_objc_msgSend)(theReceiver, theSelector, args);
}
I know it had problems with that, at least in previous versions of DMD.
--
/Jacob Carlborg
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 08:53:21 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/19/11 2:11 AM, Adam Wilson wrote: As you may all be aware, I've been trying to improve the automated generation of .di files and I now have something that I feel is testable. Currently the new code only makes the following changes. 1. Function Implementations are removed 2. Private Function Declarations are removed. 3. Variable Initializers, except for const, are removed. Don't forget immutable. I did it, but it wasn't pretty. I had to pass the immutable state via the HeaderGenState struct. Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. drey_ from IRC was kind enough to test build Derelict with the changes and has discovered a potential issue around private imports. Derelict uses private imports that contain types which are used in function alias declarations. As one would expect, this caused many compiler errors. Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I suspect you'd still need the private imports because template code may use them. Privates are now all in. This is great work. It's almost a textbook example of how one can make a great positive impact on D's development by finding an area of improvement and working on it. Congratulations! You may want to generate DIs for Phobos and take a look at them. Phobos uses a vast array of D's capabilities so it's an effective unittest for DI generation. Thanks, Andrei -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 00:11:25 -0800, Adam Wilson flybo...@gmail.com wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Privates should exist in the DI file to support public templates. 2. Template classes and functions retain their implementations. 3. Immutable types should retain their initializers. At this point I could really use testing; you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git I am trying to get myself setup for building phobos as a test but this is proving to be a lengthy process. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
Derelict works ok now, good work!
However, the .di files end up eating newlines.
Before:
double ALLEGRO_USECS_TO_SECS(long x)
{
return x / 1e+06;
}
double ALLEGRO_MSECS_TO_SECS(long x)
{
return x / 1000;
}
double ALLEGRO_BPS_TO_SECS(int x)
{
return 1 / x;
}
After:
double ALLEGRO_USECS_TO_SECS(long x);double ALLEGRO_MSECS_TO_SECS(long
x);double ALLEGRO_BPS_TO_SECS(int x);
I've tried merging
https://github.com/D-Programming-Language/dmd/pull/538 but it doesn't
fix this.
Re: DI Generation Needs your Help!
On 12/20/11 2:03 AM, Adam Wilson wrote: On Mon, 19 Dec 2011 00:11:25 -0800, Adam Wilson flybo...@gmail.com wrote: The latest DI generation code is now on my Git account and ready for testing. It fixes the following issues: 1. Privates should exist in the DI file to support public templates. 2. Template classes and functions retain their implementations. 3. Immutable types should retain their initializers. Great! At this point I could really use testing; you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git I am trying to get myself setup for building phobos as a test but this is proving to be a lengthy process. Nah, it's much easier than you might think. The posix.mak makefile is very small for what it does, and you need to literally change one line of code to make it generate .di headers. Andrei
DI Generation Needs your Help!
As you may all be aware, I've been trying to improve the automated
generation of .di files and I now have something that I feel is testable.
Currently the new code only makes the following changes.
1. Function Implementations are removed
2. Private Function Declarations are removed.
3. Variable Initializers, except for const, are removed.
Everything else is left alone. Templates and mixins are not addressed with
this code and *should* not be modified. That's where I need your help, the
test cases I have written cover some basic scenarios but I don't have the
capability to test these changes with the diverse code base that the
community has created.
drey_ from IRC was kind enough to test build Derelict with the changes and
has discovered a potential issue around private imports. Derelict uses
private imports that contain types which are used in function alias
declarations. As one would expect, this caused many compiler errors.
Currently, I feel that private imports should be stripped from the DI file
as they are intended to be internal to the module. However, I want to put
it to the community to decide, and I would especially appreciate Mr.
Bright's opinion on private imports in DI files.
If anyone wishes to test my changes against their code, you can download
them from my git account here:
https://lightben...@github.com/LightBender/dmd.git
I only ask that you inform me of anything broken and provide a test case
that reproduces the error.
The following is my current test case, it comes from a module in my
project that has been modified to include the various tests I and others
have been able to think of. I know for a fact that it is missing a great
number of features available in D, and I would really appreciate any tests
you might wish to add to this case.
module Horizon.Collections.Basic;
export void TestFunc(int a) {}
private void TestFunc2(int b) {}
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i = 0;
protected int j = 1;
export void ExportedStructFunc() {}
private void PrivateStructFunc() {}
}
export class List : IList
{
export this(uint Capacity)
{
vCapacity = Capacity;
internal = new Object[4];
}
private Object[] internal;
private uint vCapacity = 0;
export @property uint Capacity()
{
return vCapacity;
}
private uint vCount = 0;
export @property uint Count()
{
return vCount;
}
export void Add(Object Value)
{
if(vCount == vCapacity) internal.length *= 2;
internal[++vCount] = Value;
}
export int opApply(int delegate(ref Object) dg) { return 0; }
export IEnumerator GetEnumerator() {return null;}
export @property bool IsFixedSize() {return false;}
export @property bool IsReadOnly() {return false;}
export void Clear() { return; }
export bool Contains(Object Value) {return false;}
export Object opIndex(ulong Index) {return null;}
export void opIndexAssign(Object Value, ulong Index) {}
export int IndexOf(Object Value) {return 0;}
export void Insert(ulong Index, Object Value) {}
export void Remove(Object Value) {}
export void RemoveAt(ulong Index) {}
private int FooTest(int c) { return c*2; }
protected int ProtectedTest(int d) { return d*3; }
}
This is the DI file as exported with the generation changes:
// D import file generated from 'Basic.d'
module Horizon.Collections.Basic;
export void TestFunc(int a);
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool
Re: DI Generation Needs your Help!
On Monday, 19 December 2011 at 08:11:26 UTC, Adam Wilson wrote: Let me know what you think! Nice work! I think this is going to be more and more relevant now that the situation with shared libraries is getting more interesting in D. I would argue that anything that can't be moved to a different library, like templates, manifest constants etc. must be kept in as-is. D interface generation isn't just useful for hiding source code; some people just want to put as much as possible in a shared library, speed up compilation or hide as much as possible with the restriction of keeping the code working as usual. If people want to hide their source code at all cost, they can make this do a pass to easily identify what needs to be moved to a different module, then do the rest manually (which of course, only needs to be done once).
Re: DI Generation Needs your Help!
Good work Adam!
After class/struct/interface level export we shouldn't need it again.
It already implies that everything (if not stated otherwise via private)
will be exported.
Removing them also would automagicaly solve the inconsistency below.
export class List : IList
{
export void RemoveAt(ulong Index);
protected int ProtectedTest(int d);
}
On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote:
As you may all be aware, I've been trying to improve the automated
generation of .di files and I now have something that I feel is
testable. Currently the new code only makes the following changes.
1. Function Implementations are removed
2. Private Function Declarations are removed.
3. Variable Initializers, except for const, are removed.
Everything else is left alone. Templates and mixins are not addressed
with this code and *should* not be modified. That's where I need your
help, the test cases I have written cover some basic scenarios but I
don't have the capability to test these changes with the diverse code
base that the community has created.
drey_ from IRC was kind enough to test build Derelict with the changes
and has discovered a potential issue around private imports. Derelict
uses private imports that contain types which are used in function alias
declarations. As one would expect, this caused many compiler errors.
Currently, I feel that private imports should be stripped from the DI
file as they are intended to be internal to the module. However, I want
to put it to the community to decide, and I would especially appreciate
Mr. Bright's opinion on private imports in DI files.
If anyone wishes to test my changes against their code, you can download
them from my git account here:
https://lightben...@github.com/LightBender/dmd.git
I only ask that you inform me of anything broken and provide a test case
that reproduces the error.
The following is my current test case, it comes from a module in my
project that has been modified to include the various tests I and others
have been able to think of. I know for a fact that it is missing a great
number of features available in D, and I would really appreciate any
tests you might wish to add to this case.
module Horizon.Collections.Basic;
export void TestFunc(int a) {}
private void TestFunc2(int b) {}
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i = 0;
protected int j = 1;
export void ExportedStructFunc() {}
private void PrivateStructFunc() {}
}
export class List : IList
{
export this(uint Capacity)
{
vCapacity = Capacity;
internal = new Object[4];
}
private Object[] internal;
private uint vCapacity = 0;
export @property uint Capacity()
{
return vCapacity;
}
private uint vCount = 0;
export @property uint Count()
{
return vCount;
}
export void Add(Object Value)
{
if(vCount == vCapacity) internal.length *= 2;
internal[++vCount] = Value;
}
export int opApply(int delegate(ref Object) dg) { return 0; }
export IEnumerator GetEnumerator() {return null;}
export @property bool IsFixedSize() {return false;}
export @property bool IsReadOnly() {return false;}
export void Clear() { return; }
export bool Contains(Object Value) {return false;}
export Object opIndex(ulong Index) {return null;}
export void opIndexAssign(Object Value, ulong Index) {}
export int IndexOf(Object Value) {return 0;}
export void Insert(ulong Index, Object Value) {}
export void Remove(Object Value) {}
export void RemoveAt(ulong Index) {}
private int FooTest(int c) { return c*2; }
protected int ProtectedTest(int d) { return d*3; }
}
This is the DI file as exported with the generation changes:
// D import file generated from 'Basic.d'
module Horizon.Collections.Basic;
export void
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote: Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. I am not exactly sure about your problem with templates and mixins but i'll give it a try regardless :) Since with templates there is no distinction between definition and decleration, exposing them IMO should be solely based on thier module access signatures. private struct A() // hide public struct B() // expose Now if B or some another exposed structure in .di should call A, compiler will take care of it by outputting an error as usual.
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 14:11:31 +0200, so s...@so.so wrote: On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote: Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. I am not exactly sure about your problem with templates and mixins but i'll give it a try regardless :) Since with templates there is no distinction between definition and decleration, exposing them IMO should be solely based on thier module access signatures. private struct A() // hide public struct B() // expose Now if B or some another exposed structure in .di should call A, compiler will take care of it by outputting an error as usual. One last thing! Interfaces shouldn't have any business with export, they should be treated just like templates.
Re: DI Generation Needs your Help!
On Monday, 19 December 2011 at 12:11:32 UTC, so wrote: On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote: Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. I am not exactly sure about your problem with templates and mixins but i'll give it a try regardless :) Since with templates there is no distinction between definition and decleration, exposing them IMO should be solely based on thier module access signatures. private struct A() // hide public struct B() // expose Now if B or some another exposed structure in .di should call A, compiler will take care of it by outputting an error as usual. And if the public template tries to access the private one? Private module members must be treated like any other unless the compiler can prove it removed all references to the private member.
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 14:43:07 +0200, Jakob Ovrum jakobov...@gmail.com wrote: On Monday, 19 December 2011 at 12:11:32 UTC, so wrote: On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote: Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. I am not exactly sure about your problem with templates and mixins but i'll give it a try regardless :) Since with templates there is no distinction between definition and decleration, exposing them IMO should be solely based on thier module access signatures. private struct A() // hide public struct B() // expose Now if B or some another exposed structure in .di should call A, compiler will take care of it by outputting an error as usual. And if the public template tries to access the private one? Private module members must be treated like any other unless the compiler can prove it removed all references to the private member. You are right it was obscure, i'll try to clarify. By B or some another exposed structure i meant things resides completely in .di file. Things like Templates, mixins and if we get them one day inlined functions. In this scenario a original.d file should be interpreted as: original.d public void A()() private void B()() whatitwasmeant.di public void A()() whatitwasmeant.d private void B()() Here B can call A but not the opposite. A simple doesn't see B here. Things that resides completely in .di file have no way to access the things in its .d file unless it wasn't exported. Imagine the final/generated .di file IS the user and the original .d file is the library designer. While i think this is the right way, it probably not that easy to implement.
Re: DI Generation Needs your Help!
On 19.12.2011 09:11, Adam Wilson wrote: Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I'd very much like the dependencies to be shrinked in the di file, but I think you can only remove private imports if you can guarantee that no symbols are referenced by the generated di file (e.g. function signatures, variable types, templates). IIRC removing imports might also have an influence on generated static ctor execution order, because there are some optimizations that remove dependencies if no imports with static ctors are found in a referenced module. Otherwise, having a version of the di file without any implementation sounds like a good idea.
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 14:22:10 +0100, Rainer Schuetze r.sagita...@gmx.de wrote: On 19.12.2011 09:11, Adam Wilson wrote: Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I'd very much like the dependencies to be shrinked in the di file, but I think you can only remove private imports if you can guarantee that no symbols are referenced by the generated di file (e.g. function signatures, variable types, templates). IIRC removing imports might also have an influence on generated static ctor execution order, because there are some optimizations that remove dependencies if no imports with static ctors are found in a referenced module. No it should not, an import will not actually create the module info that contains the import dependencies. So the initialization order will still rely solely on the compiled module. OTOH not declaring private static ctors can lead to unreferenced module infos that would otherwise end up in the final executable. Otherwise, having a version of the di file without any implementation sounds like a good idea.
Re: DI Generation Needs your Help!
On 12/19/11 2:11 AM, Adam Wilson wrote: As you may all be aware, I've been trying to improve the automated generation of .di files and I now have something that I feel is testable. Currently the new code only makes the following changes. 1. Function Implementations are removed 2. Private Function Declarations are removed. 3. Variable Initializers, except for const, are removed. Don't forget immutable. Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. drey_ from IRC was kind enough to test build Derelict with the changes and has discovered a potential issue around private imports. Derelict uses private imports that contain types which are used in function alias declarations. As one would expect, this caused many compiler errors. Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I suspect you'd still need the private imports because template code may use them. This is great work. It's almost a textbook example of how one can make a great positive impact on D's development by finding an area of improvement and working on it. Congratulations! You may want to generate DIs for Phobos and take a look at them. Phobos uses a vast array of D's capabilities so it's an effective unittest for DI generation. Thanks, Andrei
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 03:53:27 -0800, so s...@so.so wrote:
Good work Adam!
After class/struct/interface level export we shouldn't need it again.
It already implies that everything (if not stated otherwise via private)
will be exported.
Well, I tried it that way and it didn't work on Windows. Without the
export in front of the function declaration the function won't get
exported to Windows in a shared library situation. I have verified this
behavior with Walter and he assures me that, under Windows, this is the
correct behavior. On other OS'es public works just fine and D treats
export as public, but I work on Windows so I have to use export. I am
working on an article for the site documenting my experiences with dynamic
libraries and the pitfalls you can run into, including this one. Expect to
see the initial draft after I finish DI generation.
Removing them also would automagicaly solve the inconsistency below.
export class List : IList
{
export void RemoveAt(ulong Index);
protected int ProtectedTest(int d);
}
On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com
wrote:
As you may all be aware, I've been trying to improve the automated
generation of .di files and I now have something that I feel is
testable. Currently the new code only makes the following changes.
1. Function Implementations are removed
2. Private Function Declarations are removed.
3. Variable Initializers, except for const, are removed.
Everything else is left alone. Templates and mixins are not addressed
with this code and *should* not be modified. That's where I need your
help, the test cases I have written cover some basic scenarios but I
don't have the capability to test these changes with the diverse code
base that the community has created.
drey_ from IRC was kind enough to test build Derelict with the changes
and has discovered a potential issue around private imports. Derelict
uses private imports that contain types which are used in function
alias declarations. As one would expect, this caused many compiler
errors. Currently, I feel that private imports should be stripped from
the DI file as they are intended to be internal to the module. However,
I want to put it to the community to decide, and I would especially
appreciate Mr. Bright's opinion on private imports in DI files.
If anyone wishes to test my changes against their code, you can
download them from my git account here:
https://lightben...@github.com/LightBender/dmd.git
I only ask that you inform me of anything broken and provide a test
case that reproduces the error.
The following is my current test case, it comes from a module in my
project that has been modified to include the various tests I and
others have been able to think of. I know for a fact that it is missing
a great number of features available in D, and I would really
appreciate any tests you might wish to add to this case.
module Horizon.Collections.Basic;
export void TestFunc(int a) {}
private void TestFunc2(int b) {}
const gss = __gshared;
export interface IEnumerator
{
export @property Object Current();
export bool MoveNext();
export void Reset();
}
export interface ICollection
{
export @property uint Count();
export int opApply(int delegate(ref Object) dg);
export IEnumerator GetEnumerator();
}
export interface IList : ICollection
{
export @property bool IsFixedSize();
export @property bool IsReadOnly();
export void Add(Object Value);
export void Clear();
export bool Contains(Object Value);
export Object opIndex(ulong Index);
export void opIndexAssign(Object Value, ulong Index);
export int IndexOf(Object Value);
export void Insert(ulong Index, Object Value);
export void Remove(Object Value);
export void RemoveAt(ulong Index);
}
export struct StructTesting
{
private int i = 0;
protected int j = 1;
export void ExportedStructFunc() {}
private void PrivateStructFunc() {}
}
export class List : IList
{
export this(uint Capacity)
{
vCapacity = Capacity;
internal = new Object[4];
}
private Object[] internal;
private uint vCapacity = 0;
export @property uint Capacity()
{
return vCapacity;
}
private uint vCount = 0;
export @property uint Count()
{
return vCount;
}
export void Add(Object Value)
{
if(vCount == vCapacity) internal.length *= 2;
internal[++vCount] = Value;
}
export int opApply(int delegate(ref Object) dg) { return 0; }
export IEnumerator GetEnumerator() {return null;}
export @property bool IsFixedSize() {return false;}
export @property bool IsReadOnly()
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 08:53:21 -0800, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/19/11 2:11 AM, Adam Wilson wrote: As you may all be aware, I've been trying to improve the automated generation of .di files and I now have something that I feel is testable. Currently the new code only makes the following changes. 1. Function Implementations are removed 2. Private Function Declarations are removed. 3. Variable Initializers, except for const, are removed. Don't forget immutable. Added to my list. :-) Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. drey_ from IRC was kind enough to test build Derelict with the changes and has discovered a potential issue around private imports. Derelict uses private imports that contain types which are used in function alias declarations. As one would expect, this caused many compiler errors. Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I suspect you'd still need the private imports because template code may use them. Ok, this is a good reason, I'd still like to hear from Walter on the subject, but I think he'll probably agree. I'll add it to my list. This is great work. It's almost a textbook example of how one can make a great positive impact on D's development by finding an area of improvement and working on it. Congratulations! Thank you! I have to admit that reason I took it on is because it was annoying the daylights out of me on my project, but then I guess that's how most open-source work gets done. :-) You may want to generate DIs for Phobos and take a look at them. Phobos uses a vast array of D's capabilities so it's an effective unittest for DI generation. Now that is a good idea. I may need some help getting the test setup as I've never built anything as big as phobos before, but it would be a fantastic way to nail down the DI behavior and maybe help push Phobos to become a shared library. Thanks, Andrei -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
Btw Adam, a long long time ago (well, few years ago) there was an effort for better DLL support in D: http://dsource.org/projects/ddl/ There was also a presentation here: http://vimeo.com/2264486 and slides: http://replay.waybackmachine.org/20081203030930/http://team0xf.com/conference/DDL.pdf Anyway, it might be a useful resource for what seem to be D's DLL problems.
Re: DI Generation Needs your Help!
On Monday, December 19, 2011 10:53:21 Andrei Alexandrescu wrote: You may want to generate DIs for Phobos and take a look at them. Phobos uses a vast array of D's capabilities so it's an effective unittest for DI generation. It's a great test ground, but I wouldn't want to see Phobos actually be released with .di files instead of d files. It would seriously harm inlining and CTFE. - Jonathan M Davis
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 04:57:54 -0800, so s...@so.so wrote: On Mon, 19 Dec 2011 14:43:07 +0200, Jakob Ovrum jakobov...@gmail.com wrote: On Monday, 19 December 2011 at 12:11:32 UTC, so wrote: On Mon, 19 Dec 2011 10:11:25 +0200, Adam Wilson flybo...@gmail.com wrote: Everything else is left alone. Templates and mixins are not addressed with this code and *should* not be modified. That's where I need your help, the test cases I have written cover some basic scenarios but I don't have the capability to test these changes with the diverse code base that the community has created. I am not exactly sure about your problem with templates and mixins but i'll give it a try regardless :) Since with templates there is no distinction between definition and decleration, exposing them IMO should be solely based on thier module access signatures. private struct A() // hide public struct B() // expose Now if B or some another exposed structure in .di should call A, compiler will take care of it by outputting an error as usual. And if the public template tries to access the private one? Private module members must be treated like any other unless the compiler can prove it removed all references to the private member. You are right it was obscure, i'll try to clarify. By B or some another exposed structure i meant things resides completely in .di file. Things like Templates, mixins and if we get them one day inlined functions. In this scenario a original.d file should be interpreted as: original.d public void A()() private void B()() whatitwasmeant.di public void A()() whatitwasmeant.d private void B()() Here B can call A but not the opposite. A simple doesn't see B here. Things that resides completely in .di file have no way to access the things in its .d file unless it wasn't exported. Imagine the final/generated .di file IS the user and the original .d file is the library designer. While i think this is the right way, it probably not that easy to implement. So the consensus is to keep all private members/imports/etc? -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 06:27:20 -0800, Martin Nowak d...@dawgfoto.de wrote: On Mon, 19 Dec 2011 14:22:10 +0100, Rainer Schuetze r.sagita...@gmx.de wrote: On 19.12.2011 09:11, Adam Wilson wrote: Currently, I feel that private imports should be stripped from the DI file as they are intended to be internal to the module. However, I want to put it to the community to decide, and I would especially appreciate Mr. Bright's opinion on private imports in DI files. I'd very much like the dependencies to be shrinked in the di file, but I think you can only remove private imports if you can guarantee that no symbols are referenced by the generated di file (e.g. function signatures, variable types, templates). IIRC removing imports might also have an influence on generated static ctor execution order, because there are some optimizations that remove dependencies if no imports with static ctors are found in a referenced module. No it should not, an import will not actually create the module info that contains the import dependencies. So the initialization order will still rely solely on the compiled module. OTOH not declaring private static ctors can lead to unreferenced module infos that would otherwise end up in the final executable. Otherwise, having a version of the di file without any implementation sounds like a good idea. At this point it sounds like I need to keep all private members and imports. No problem, that's easy enough. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 09:36:27 -0800, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: Btw Adam, a long long time ago (well, few years ago) there was an effort for better DLL support in D: http://dsource.org/projects/ddl/ There was also a presentation here: http://vimeo.com/2264486 and slides: http://replay.waybackmachine.org/20081203030930/http://team0xf.com/conference/DDL.pdf Anyway, it might be a useful resource for what seem to be D's DLL problems. Those are awesome links. Thanks Andrej! -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 22:24:18 +0200, Adam Wilson flybo...@gmail.com wrote: So the consensus is to keep all private members/imports/etc? Ideally to get most out of the auto generation of di files (by definition of module system) we need to strip ALL private stuff. Only this way we can say the automation is on par with handcrafting. But if it does not look doable somehow, we can always handcraft them (which i'll generally prefer),
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 23:04:27 +0200, so s...@so.so wrote: On Mon, 19 Dec 2011 22:24:18 +0200, Adam Wilson flybo...@gmail.com wrote: So the consensus is to keep all private members/imports/etc? Ideally to get most out of the auto generation of di files (by definition of module system) we need to strip ALL private stuff. Only this way we can say the automation is on par with handcrafting. But if it does not look doable somehow, we can always handcraft them (which i'll generally prefer), ALL except imports :)
Re: DI Generation Needs your Help!
On Monday, 19 December 2011 at 21:04:28 UTC, so wrote:
On Mon, 19 Dec 2011 22:24:18 +0200, Adam Wilson
flybo...@gmail.com wrote:
So the consensus is to keep all private members/imports/etc?
Ideally to get most out of the auto generation of di files (by
definition of module system) we need to strip ALL private stuff.
Only this way we can say the automation is on par with
handcrafting.
But if it does not look doable somehow, we can always handcraft
them (which i'll generally prefer),
Private members can still be used from public templates and
whatnot. You can't take them out unless you can statically proved
all references within the same .di to that private member was
removed in the stripping process.
foo.d
--
private i = 42;
// Is a public template, has to be left in
void bar()()
{
/* Uses 'i' */
}
void baz()
{
/* Also uses 'i' */
}
--
foo.di
--
void bar()()
{
/* Still uses 'i', where did it go!? */
}
void baz(); // Fine, no longer need 'i'
--
Also, on a side note, function parameters can use types from
private imports too. Be it function, data or import, it has to be
left in even if private.
Re: DI Generation Needs your Help!
On Mon, 19 Dec 2011 15:53:02 -0800, Jakob Ovrum jakobov...@gmail.com
wrote:
On Monday, 19 December 2011 at 21:04:28 UTC, so wrote:
On Mon, 19 Dec 2011 22:24:18 +0200, Adam Wilson flybo...@gmail.com
wrote:
So the consensus is to keep all private members/imports/etc?
Ideally to get most out of the auto generation of di files (by
definition of module system) we need to strip ALL private stuff.
Only this way we can say the automation is on par with handcrafting.
But if it does not look doable somehow, we can always handcraft them
(which i'll generally prefer),
Private members can still be used from public templates and whatnot. You
can't take them out unless you can statically proved all references
within the same .di to that private member was removed in the stripping
process.
It sounds at this point we will just be leaving everything private in the
DI files, sans implementations.
foo.d
--
private i = 42;
// Is a public template, has to be left in
void bar()()
{
/* Uses 'i' */
}
void baz()
{
/* Also uses 'i' */
}
--
foo.di
--
void bar()()
{
/* Still uses 'i', where did it go!? */
}
void baz(); // Fine, no longer need 'i'
--
Also, on a side note, function parameters can use types from private
imports too. Be it function, data or import, it has to be left in even
if private.
This was discovered by drey_ on IRC and will be fixed by leaving all
privates intact. I'll upload a patch to my git account this evening.
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
