shorthand template and static conditionals?

[Jeremie Pelletier]

Could it be a valid option to have a shorthand template syntax if the template 
body has only one statement? Something like:

template Tuple(A...) alias Tuple = A;

Another suggestion would be to have the ?: syntax supported for static 
statements, as we're currently forced to use static ifs. It really adds a lot 
of code to templates for even simple conditions.

Compile time float binary representation

[Jeremie Pelletier]

Is there a way to convert a float (or double/real) to an integral number 
without changing its binary representation at compile time?

I need to extract the sign, exponent and mantissa yet I cant use bit shifting.
"Error: 'R' is not of integral type, it is a real" is the error I get.

The usual *cast(uint*)&value wont work either at compile time.

Any suggestions?

Re: Compile time float binary representation

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Is there a way to convert a float (or double/real) to an integral number 
> > without changing its binary representation at compile time?
> > 
> > I need to extract the sign, exponent and mantissa yet I cant use bit 
> > shifting.
> > "Error: 'R' is not of integral type, it is a real" is the error I get.
> > 
> > The usual *cast(uint*)&value wont work either at compile time.
> > 
> > Any suggestions?
> 
> I got this to work, would it be applicable to your need?
> 
> void main(string[] args)
> {
>  union A
>  {
>  double x;
>  int y;
>  }
>  enum A a = A(0.3);
>  writeln(a.x);
>  writeln(a.y);
> }
> 
> 
> Andrei

Good suggestion, however it doesnt work the way I need it:

---
template FloatParts(real R) {
union A {
real theReal;
struct {
ushort hi;
ulong lo;
}
}

A a = cast(A)R; // Error: cannot cast real to immutable(A)
enum A a = A(R); // members cannot be used at runtime
}
---

I need to gather .sign, .exp and .mantissa among others (possibly a .radix too) 
and use them at compile time (generic metaprogramming, or else i would just 
fallback to runtime).

Unless there are arithmetic methods to get these, I dont see how it can be done 
without changes in the compiler first.

To Walter: is this something possible for DMD2.32?

Re: Compile time float binary representation

[Jeremie Pelletier]

Jeremie Pelletier Wrote:

> Andrei Alexandrescu Wrote:
> 
> > Jeremie Pelletier wrote:
> > > Is there a way to convert a float (or double/real) to an integral number 
> > > without changing its binary representation at compile time?
> > > 
> > > I need to extract the sign, exponent and mantissa yet I cant use bit 
> > > shifting.
> > > "Error: 'R' is not of integral type, it is a real" is the error I get.
> > > 
> > > The usual *cast(uint*)&value wont work either at compile time.
> > > 
> > > Any suggestions?
> > 
> > I got this to work, would it be applicable to your need?
> > 
> > void main(string[] args)
> > {
> >  union A
> >  {
> >  double x;
> >  int y;
> >  }
> >  enum A a = A(0.3);
> >  writeln(a.x);
> >  writeln(a.y);
> > }
> > 
> > 
> > Andrei
> 
> Good suggestion, however it doesnt work the way I need it:
> 
> ---
> template FloatParts(real R) {
> union A {
> real theReal;
> struct {
> ushort hi;
> ulong lo;
> }
> }
> 
> A a = cast(A)R; // Error: cannot cast real to immutable(A)
> enum A a = A(R); // members cannot be used at runtime
> }
> ---
> 
> I need to gather .sign, .exp and .mantissa among others (possibly a .radix 
> too) and use them at compile time (generic metaprogramming, or else i would 
> just fallback to runtime).
> 
> Unless there are arithmetic methods to get these, I dont see how it can be 
> done without changes in the compiler first.
> 
> To Walter: is this something possible for DMD2.32?

After some toying around, I managed to get some results:

---
import std.metastrings;
immutable union A {
double x;
int y;
}
pragma(msg, ToString!(A(5.2).y));
---

but it crashed dmd (2.031) instead of printing 3435973837 to stdout.

Re: Compile time float binary representation

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Jeremie Pelletier Wrote:
> > 
> >> Andrei Alexandrescu Wrote:
> >>
> >>> Jeremie Pelletier wrote:
> >>>> Is there a way to convert a float (or double/real) to an integral number 
> >>>> without changing its binary representation at compile time?
> >>>>
> >>>> I need to extract the sign, exponent and mantissa yet I cant use bit 
> >>>> shifting.
> >>>> "Error: 'R' is not of integral type, it is a real" is the error I get.
> >>>>
> >>>> The usual *cast(uint*)&value wont work either at compile time.
> >>>>
> >>>> Any suggestions?
> >>> I got this to work, would it be applicable to your need?
> >>>
> >>> void main(string[] args)
> >>> {
> >>>  union A
> >>>  {
> >>>  double x;
> >>>  int y;
> >>>  }
> >>>  enum A a = A(0.3);
> >>>  writeln(a.x);
> >>>  writeln(a.y);
> >>> }
> >>>
> >>>
> >>> Andrei
> >> Good suggestion, however it doesnt work the way I need it:
> >>
> >> ---
> >> template FloatParts(real R) {
> >> union A {
> >> real theReal;
> >> struct {
> >> ushort hi;
> >> ulong lo;
> >> }
> >> }
> >> 
> >> A a = cast(A)R; // Error: cannot cast real to immutable(A)
> >> enum A a = A(R); // members cannot be used at runtime
> >> }
> >> ---
> >>
> >> I need to gather .sign, .exp and .mantissa among others (possibly a .radix 
> >> too) and use them at compile time (generic metaprogramming, or else i 
> >> would just fallback to runtime).
> >>
> >> Unless there are arithmetic methods to get these, I dont see how it can be 
> >> done without changes in the compiler first.
> >>
> >> To Walter: is this something possible for DMD2.32?
> > 
> > After some toying around, I managed to get some results:
> > 
> > ---
> > import std.metastrings;
> > immutable union A {
> > double x;
> > int y;
> > }
> > pragma(msg, ToString!(A(5.2).y));
> > ---
> > 
> > but it crashed dmd (2.031) instead of printing 3435973837 to stdout.
> > 
> 
> Yah, I crashed it a couple of times with similar code now too. Could you 
> please do the bugzilla honors.
> 
> (Also you may want to check std.numeric.CustomFloat. It doesn't help 
> with this particular problem, but it allows defining floating point 
> types with specific mantissa and exponent sizes.)
> 
> 
> Andrei

Thanks again for the quick reply Andrei, however, as useful as the CustomFloat 
template may be, I need a reverse algorithm usable entirely at compile time. If 
you know a way to extract this information without resorting to bitwise logic, 
please let me know, for the moment I'm adding a static assert(0) to my template 
until dmd supports compile time binary casts.

Also I created http://d.puremagic.com/issues/show_bug.cgi?id=3220.

Re: reddit.com: first Chapter of TDPL available for free

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> KennyTM~ wrote:
> > Andrei Alexandrescu wrote:
> >> http://www.reddit.com/r/programming/comments/975ng/diving_into_the_d_programming_language_tdpl/
> >>  
> >>
> >>
> >> (Don't tell anyone, but I plan to rewrite it.)
> >>
> >> Andrei
> > 
> > What's a "so-called IEEE 754 double-extended 79-bit format"? :p
> 
> Heck, I didn't know until I googled for it!
> 
> BTW folks, please please vote the reddit link up. It's a simple and 
> effective way to increase exposure to D and grow the community.
> 
> http://www.reddit.com/r/programming/comments/975ng/diving_into_the_d_programming_language_tdpl/
> 
> 
> Andrei

And I always thought it was double-extended 80-bit. The IEEE standard actually 
specifies >= 79 bits and Intel implements it as 80bits.

Well you learn something new everyday!

Re: Triggers

[Jeremie Pelletier]

Sjoerd van Leent Wrote:

> OK, to make matters worse.
> 
> As I was reading so many things about properties, keywords and a bunch of 
> other interesting things, perhaps triggers.
> 
> Triggers are, in my opinion, functions that act as delegates, and are called 
> before or after one or more other functions. Maybe we can do something with 
> this? It would introduce some aspect programming to D.
> 
> file 1:
> 
> class Foo
> {
> int mybar;
> int myother;
> 
> void bar(int i)
> {
> mybar = i;
> }
> 
> 
> void other(int i)
> {
> myother = i;
> }
> }
> 
> 
> file 2:
> 
> trigger(before) : "*.void * (int *)"
> {
> writefln("calling ", this.currentFunctionName);
> }
> 
> trigger(after) : "*.void * (int *)"
> {
> writefln("called ", this.currentFunctionName);
> }
> 
> 
> This might be something for D3 though...

You can already do something like that, although very hackish: turn on trace 
profiling and override the prolog/epilog code from druntime (I think these are 
called _trace_pro_n and _trace_epi_n, just grep the source for their syntax).

Just make *very* sure to use only naked declarations in these and any routine 
they may call, or else you will just call back into _trace_pro_n until you get 
a stack overflow.

Re: Naming things in Phobos - std.algorithm and writefln

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Michel Fortin wrote:
> > Alternatively, "writefln" could be an exception to the rules, but then 
> > the exception would need a better rationale than "it shouldn't look like 
> > Java". I mean, if Phobos makes unjustified exceptions to its naming 
> > conventions here and there for no good other reason than "it looks 
> > good", it breaks the concistency and makes function names less 
> > predictable and less readable.
> 
> I agree that Phobos' names could use a good overhaul. That would make it 
> easier for growing it too.
> 
> Certain names could be kept short and intuitive although they don't fit 
> common conventions.
> 
> 
> Andrei

You could also use aliases to make everyone happy, thats what I do in my local 
phobos source, its just a bitch to upgrade to the newest dmd while keeping my 
own changes ;)

One of the most annoying names I've had in phobos was the std.utf.encode/decode 
functions. When I would come across these in some code it wasnt descriptive 
enough as to what was being done. So I rewrote the std.utf module to use names 
such as toUTF8, toUTF16 and toUnicode, and made a generic toUTF template to 
call the proper one. Then aliased encode and decode to their corresponding 
toUTF calls to keep compatibility with the rest of phobos, works like a charm.

I can mail you my version of std.utf if you want Andrei.

Global operator overloading?

[Jeremie Pelletier]

Are there any plans to have global operator overloading in D?

It would be terribly useful with ranges, since std.array already define range 
primitives for arrays, I tried to implement array primitives to ranges the same 
way, ie:

---
bool opEquals(T, U)(T t, U, u) if(isInputRange!T && isInputRange!U) {
return equal(t, u);
}
---

so I wouldnt have to change every a==b to equal(a, b) when using ranges where 
built-in arrays were previously used.

Of course if theres another way to do it (other than manually adding opEquals 
and others to every range declaration in phobos), feel free to suggest it.

Re: Global operator overloading?

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Are there any plans to have global operator overloading in D?
> > 
> > It would be terribly useful with ranges, since std.array already define 
> > range primitives for arrays, I tried to implement array primitives to 
> > ranges the same way, ie:
> > 
> > ---
> > bool opEquals(T, U)(T t, U, u) if(isInputRange!T && isInputRange!U) {
> > return equal(t, u);
> > }
> > ---
> > 
> > so I wouldnt have to change every a==b to equal(a, b) when using ranges 
> > where built-in arrays were previously used.
> > 
> > Of course if theres another way to do it (other than manually adding 
> > opEquals and others to every range declaration in phobos), feel free to 
> > suggest it.
> 
> I guess the compiler could rewrite a == b into a.opEquals(b) and then 
> let normal rules find opEquals(a, b). The latter could sit in object.d.
> 
> Andrei

Yeah, global operators overloads would be at the very end of the lookup chain, 
and built-in types ignored too. So only custom aggregates would be able to 
match these.

Anywho, I've fixed my issue by adding a template mixin to most ranges in my 
local phobos, but I would much prefer a more generic solution.

Re: Templates - Numeric Types only

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jarrett Billingsley wrote:
> > On Thu, Aug 6, 2009 at 9:53 PM, Andrei
> > Alexandrescu wrote:
> >> Jarrett Billingsley wrote:
> >>> On Thu, Aug 6, 2009 at 9:19 PM, Andrei
> >>> Alexandrescu wrote:
>  Kevin wrote:
> > Hi,
> > simple Question:
> > Is it possible to allow just numeric types in templates?
> > For example to create a Vector Class like this:
> >
> > class Vector(T:"numeric" = float, int size = 4)
> >
> > Thanks,
> > Kevin
>  import std.traits;
> 
>  class Vector(T, int size = 4) if (isNumeric!T)
>  {
>    ...
>  }
> >>> Unfortunately,
> >>>
> >>> struct Vec(T) if(isNumeric!T) {}
> >>> struct Vec(T) {}
> >>> Vec!(int) x; // error
> >>>
> >>> foo.d(14): Error: template instance Vec!(int) matches more than one
> >>> template declaration, Vec(T) if (isNumeric!(T)) and Vec(T)
> >>>
> >>> *sigh*
> >>>
> >>> Wouldn't it be nice.
> >> struct Vec(T) if(isNumeric!T) {}
> >> struct Vec(T) if(!isNumeric!T) {}
> >>
> >> is actually superior because it's more modular; the semantics of the latter
> >> does not depend on the presence of the former.
> > 
> > But the condition of the latter now depends on the condition of the
> > former.  And on the condition of any other 'specializations':
> > 
> > struct Vec(T) if(isNumeric!T) {}
> > struct Vec(T) if(isSomeString!T) {}
> > struct Vec(T) if(!isNumeric!T) {} // uhoh
> > Vec!(string) x; // matches two
> > 
> > Now the fallback has to be declared as
> > 
> > struct Vec(T) if(!isNumeric!T && !isSomeString!T) {}
> > 
> > and it gets worse the more you add.
> 
> I know, and I explained why that's arguably better.
> 
> Andrei

I for myself favor lots of small templates to keep function declarations 
shorter, for example:

template isStringOrNumeric(T) {
enum isStringOrNumeric = isSomeString!T || isNumeric!T;
}
struct Vec(T) if(isSomeString!T) {}
struct Vec(T) if(isNumeric!T) {}
struct Vec(T) if(!isStringOrNumeric!T) {}

It also allows for the templates to have static asserts of their own to make 
bugs easier to isolate, and they can be moved to another file which makes for 
more generic programming. Chances are you'll be using the template again if you 
need it here.

I usually try to keep my template if statements down to one or two short 
expressions at most, otherwise it quickly becomes more confusing than useful.

Re: delete and references?

[Jeremie Pelletier]

Jarrett Billingsley Wrote:

> On Fri, Aug 7, 2009 at 1:20 PM, Aaron Watters wrote:
> > Hi guys.  D looks cool.  A couple of things confuse me.
> > The first is: what happens if I do a
> >
> > delete cl;
> >
> > after storing a reference to cl somewhere?  Can I use the stored
> > reference?  Is it invalid?  Please elucidate, thanks.
> 
> If you delete something and attempt to use another reference to the
> deleted object, the behavior's undefined.  Basically the same thing as
> accessing a dangling pointer in C or C++.  'delete' is really just
> there for when you *know* that there are no other references and you
> want to ensure that it gets cleaned up immediately.

If you're interested in how this behaves in the runtime, "delete cl;" tells the 
memory manager to reclaim the memory at 'cl'. It does so by putting its block 
of data back onto its internal freelist. At that point the first 4 bytes are 
used in a linked list. If you still have references to that memory and then 
modify it, you break the memory manager internal state, which is gonna be hard 
to track back. After the memory has been reallocated, your old reference and 
the new one are now fighting for the same memory block. In either cases, your 
old reference will cause trouble that will be very hard to isolate.

A good rule of thumb in D is that if you're unsure whether references are still 
alive, don't delete the object, the GC will automatically reclaim the memory 
once it detects there are no more references to it. It is also faster to go 
that way, lots of delete calls are more expensive than one GC sweep, and the 
sweep is going to happen sooner or later anyways.

However, delete is most useful when you need large blocks of data for short 
periods of time.

Re: delete and references?

[Jeremie Pelletier]

Jarrett Billingsley Wrote:

> On Sat, Aug 8, 2009 at 9:42 AM, Andrei
> Alexandrescu wrote:
> > Jarrett Billingsley wrote:
> >>
> >> On Sat, Aug 8, 2009 at 9:17 AM, Andrei
> >> Alexandrescu wrote:
> >>>
> >>> FWIW, I am trying to convince Walter to not reclaim memory in delete, but
> >>> instead only call destructors. D continues C++'s mistake of conflating
> >>> lifetime termination with memory reclamation.
> >>
> >> Why?  Instead of dangling pointers, you'd end up with pointers to
> >> finalized objects, which would probably lead to harder-to-catch bugs
> >> (since then you wouldn't even get a segfault).
> >
> > Getting a segfault == lucky
> 
> And the alternative - using a "dead" object - is any better?
> 
> >> And what if you used it to try to delete a temporary array?  Yes, you
> >> could use malloc/free for the temp array, but.. it just seems silly to
> >> have to dip into C functions to do that.
> >
> > Why? It's not a frequent need.
> 
> That's enough of a justification to change delete's behavior?
> 
> I just don't know why it should be changed.  It's a blunt tool for
> blunt uses.  When you delete something, you're telling the GC "I don't
> need this anymore, *trust me*", so *not* freeing memory associated
> with it seems silly.

I just had an idea for new semantics for delete. With proper use it could speed 
up the GC a lot, and with misuse it could make it worse.

When deleting memory, that block goes on a special linked list with an internal 
entry wrapper as to not alter the memory. It is not finalized, nor is its GC 
state changed (has pointers, no scan, etc). Once the GC is asked to do a sweep, 
that linked list is checked first for unused memory, and free blocks are put 
back on the freelist. If enough can be reclaimed there for the allocation 
request which triggered the sweep, the entire set of allocations doesn't have 
to be scanned.

Well thats the idea in a nutshell, it popped in my mind as I read Andrei's 
comment about delete doing only a finalize. Since we don't have a moving GC 
with multiple heaps (short lived heap moving up to long lived heap, and only 
the short lived one is scanned often), we could optimize how the sweep is being 
done.

That would however require new ABI routines to use with do-it-now deletes, such 
as scoped allocations as their usage may rely on the fact that the finalizer is 
executed when the scope exits.

Please give me some input on that idea.

Re: delete and references?

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Michel Fortin wrote:
> > On 2009-08-08 09:17:28 -0400, Andrei Alexandrescu 
> >  said:
> > 
> >> Great description.
> >>
> >> FWIW, I am trying to convince Walter to not reclaim memory in delete, 
> >> but instead only call destructors. D continues C++'s mistake of 
> >> conflating lifetime termination with memory reclamation.
> > 
> > I don't see how this changes anything. Instead of accessing a 
> > deallocated object, you'll access a finaized but not yet deallocated 
> > object. In both cases, it's a bug.
> 
> The former case has you using memory that was possibly allocated to an 
> object of different type. This makes for intractable errors.
> 
> > Wouldn't it be better to have a system to track unique pointers? If you 
> > knew that no other pointer points to a given object or memory block, you 
> > can finalize and deallocate it safely. In fact, the current semantics of 
> > a scope object assume that the programmer will not leave any dangling 
> > pointers at the end of the scope, so it's already assuming uniqueness, 
> > just not enforcing it.
> 
> Unique pops in an out as a topic for future directions. It is quite 
> difficult to make it at the same time correct, inconspicuous, and 
> useful. We have firmly decided to do away with unique for D2.
> 
> 
> Andrei

I agree, scope arguments can easily be casted away and live beyond the scope. 
It's impossible to know if a pointer is unique without scanning the entire heap 
and pausing all threads to scan their stacks. And adding reference counts to 
allocations would break the need to have a GC in the first place.

The best thing to do is to not use delete when uncertain about the uniqueness 
of a pointer.

Re: delete and references?

[Jeremie Pelletier]

bearophile Wrote:

> Jeremie Pelletier:
> >It's impossible to know if a pointer is unique without scanning the entire 
> >heap and pausing all threads to scan their stacks.<
> 
> Are you saying this after reading Bartoz blog posts?
> 
> Bye,
> bearophile

No I haven't, do you have a link?

unittext extension proposal

[Jeremie Pelletier]

I just had an idea to help keep track of unittests, right now we're turning on 
printf's at the beginning of a test to know which one fails, and adding printfs 
everywhere quickly becomes redundant. Also if the test succeeds and execution 
fails at some other point, the last printf is then misleading.

---
module sample;
unittest("myTest") {}
---

With a few runtime changes, it could be made to print "unittest: 
sample.myTest..." before calling the test routine, and "PASS\n" as the routine 
returns. It could have its own compiler switch to turn it on or off globally, 
and support for a custom output handler should the user want to redirect output 
to a GUI or something.

Right now I made the following template to do that:
---
private template UnittestTrace(string test) {
debug(TEST)
immutable UnittestTrace =
"printf(\"unittest: std.text.String." ~ test ~ "... 
\");"
"scope(success) printf(\"PASS\n\");";
else
immutable UnittestTrace = "";
}
unittest {
mixin(UnittestTrace!"myTest");
}
---

However being a dreamer and all, I would like to see support for such a feature 
pushed to the D runtime.

J

Re: Memory allocation problem

[Jeremie Pelletier]

bearophile Wrote:

> In a small program on Windows XP I have to allocate a large chunk of RAM, 
> about 1847 MB of RAM. This PC has 2 GB RAM. So I use std.c.stdio.malloc(), 
> with DMD v1.042 (or v2.031). But it's not able to allocate it, and produces 
> at runtime:
> Error: Access Violation
> 
> An equal program written in C and compiled with GCC, that allocates the same 
> amount of memory using malloc, is able to run (with just a little hard disk 
> trashing at the beginning).
> Do you know why DMD doesn't allow me to allocate more RAM, and can this be 
> fixed?
> 
> Bye,
> bearophile

i just did the test on my 6Gb ram laptop, it failed, the exception comes from 
the error handling crashing on OutOfMemory exceptions (I think).

Your C program probably compiled in 64bit, which has MUCH more room for virtual 
memory.

DMD only compiles 32bit binaries so far, and the virtual memory it can map for 
a single process is 4Gb, 2 of which are reserved for shared system memory.

What that means is, when you allocate memory, even if you DO have 2Gb available 
in ram or pagefile, you may not have enough contiguous free pages of virtual 
memory to map the allocation, 64bit applications rarely hit that wall anymore.

You could always try and stream the memory and allocate the minimum required, 
and work on it one slice at a time.

Re: YAPP - reminder

[Jeremie Pelletier]

aarti_pl Wrote:

> Hello!
> 
> This is just another reminder about ongoing voting about properties:
> 
> http://www.igsoft.net/dpolls/index.php
> 
> Current results:
> 
> * about 68% of responders want to have special syntax for properties
> * from people wanting new syntax most people want C# syntax (26 votes) 
> and then almost ex-aequo syntax with "property" keyword (25 votes)
> * seems that only one person from 88 voters like opGet_ syntax
> * quite a big group of responders (~32%) wants just fix problems in 
> existing property syntax
> * 60% of people (but only 75 voters) want to remove possibility to omit 
> parentheses from function call.
> 
> 
> 
> Comments:
> 
> I think that these results are already quite representative, but poll 
> should be open till tomorrow. So there is still chance to vote.
> 
> 
> There appeared other proposals from time where poll was created. If you 
> like something other than options in poll, then I think you should vote 
> for "I want other syntax than above" option.
> 
> 
> Personally I don't like so much C# syntax because of magic "value" 
> variable, which refers to property value used in setter. There is 
> another place in D where such a magic values appears: it is in variadic 
> parameters functions. IMHO such a magic parameters could stay if there 
> would be general syntax for getting compile time/runtime parameters of 
> functions. In such a case there would be no more "magic" in language but 
> rather clear rules how to read function parameter values using reflections.
> 
> Currently my choice would be something like proposal in DIP6 
> (attributes). Using '@' at the beginning of attributes could be also 
> used in another place: in imports it could be used to escape keywords, 
> so that following would be possible:
> --
> import s...@traits; //then we could remove underscores from __traits
> import d...@for.masses;
> --
> It's most general way of annotating source code with special compiler 
> understandable meanings. Looking at result of poll & also seeing many NG 
> posts I believe that most of D community prefers generality over 
> one-time hackish solutions. And that's something to think about it... :-)
> 
> 
> It seems that almost no one likes "ugly" functions with underscores and 
> messing operator names with property names. I am not so surprised about 
> underscores at all: it was common practice in C++, but it's not so 
> common in other modern languages. But then why in D we have so many of 
> them although no one really likes it (__traits, foreach_reverse, 
> _argptr, _arguments)?
> 
> 
> And finally: why polls are not integral part of digitalmars web page? It 
> took me only one hour to set up this poll...
> 
> 
> BR
> Marcin Kuszczak
> (aarti_pl)

Oops, I got the only one "keep things as they are now" vote haha.
I meant to vote for the one with resolved +=.

Took you an hour to set it up? Did you have to code it from scratch?

Compile time code paths

[Jeremie Pelletier]

If a function has both an asm and D implementations inside its body, and the D 
version can be executed at compile time, but the asm one is much faster at 
runtime. Is it possible to have the compiler use the D code path at compile 
time (ie to fill in enums and whatnot), and have the asm version available at 
runtime.

Re: T[new]

[Jeremie Pelletier]

bearophile Wrote:

> grauzone:
> 
> > I see two things a dotnet implementation of D could have over native D:
> > - better garbage collector (the D one barely does its job...)
> 
> The dotnet GC is probably better than the current D GC, but I think it's 
> designed for mostly movable objects. Currently most (or all) D objects are 
> pinned (they can't be moved around in memory), so I don't know if the dotnet 
> GC will do much good.
> D needs a GC designed for its peculiar characteristics. And I believe D will 
> also need some extra semantics to allow the creation of such efficient 
> half-movable half-pinned GC (I have explained such ideas one time in the 
> past).
> 
> Bye,
> bearophile

I just finished reading about Bartosz's early entry about the shared qualifier 
and how it could lead to per-thead memory heaps (as only shared allocations 
should use the shared heap). This means the GC can maintain different heaps per 
thread, and perform collection locally without pausing the world, and the 
entire heap can just be trashed when the thread exits.

I think it could be much more efficient than a moving GC (as maintaining 
different shared heaps and moving memory between them is a slow process). I am 
really tempted to modify my custom GC to do just that heh.

Re: reddit.com: first Chapter of TDPL available for free

[Jeremie Pelletier]

Daniel Keep Wrote:

> 
> 
> Andrei Alexandrescu wrote:
> > Michel Fortin wrote:
> >> On 2009-08-09 11:10:48 -0400, Andrei Alexandrescu
> >>  said:
> >>
>  It's also arguable that all functions in std.string should take
>  const(char)[]. Or, you know, const(T)[], since D supports encodings
>  other than UTF-8, despite what std.string leads you to believe.
> >>>
> >>> Yah, I think they should all be parameterized so they can work with
> >>> various character widths and even encodings.
> >>
> >> But shouldn't they work with *ranges* in general, a string being only
> >> a specific case?
> > 
> > That's true as well! In my dreams, me and the famous actress... oh wait,
> > wrong dream. In my dreams, I eliminate std.string and put all of its
> > algorithms, properly generalized, in std.algorithm, to work on more than
> > just arrays, and more than just characters.
> > 
> > Andrei
> 
> How do you define 'tolower' on non-characters?

You don't, character-specific functions can test for the element type to be a 
character type, be it an array or a range.

If you want to use tolower with a non-character array, you have to cast it to a 
string type.

Re: unittext extension proposal

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Leandro Lucarella wrote:
> > Sergey Gromov, el 10 de agosto a las 16:32 me escribiste:
> >> Sat, 08 Aug 2009 17:32:30 -0400, Jeremie Pelletier wrote:
> >>
> >>> I just had an idea to help keep track of unittests, right now we're 
> >>> turning on printf's at the beginning of a test to know which one fails, 
> >>> and adding printfs everywhere quickly becomes redundant. Also if the test 
> >>> succeeds and execution fails at some other point, the last printf is then 
> >>> misleading.
> >>>
> >>> ---
> >>> module sample;
> >>> unittest("myTest") {}
> >>> ---
> >> Named unittests is a rather often requested feature.  Others also wanted
> >> __UNITTEST__ to expand into a name of the current unittest.  Also a
> >> 'weak assert' was requested which tests and prints a message but delays
> >> exit until the end of the current unit test.
> > 
> > It would be nice if unittest could be extended to have import statements.
> 
> I use:
> 
> version(unittest) import std.stdio;
> 
> 
> Andrei

Thats the thing, I dont need every unittest compile to fill stdout with 
progress status. It's only needed when you have 300 unittests in a single 
module and you cant tell which one is failing.

I for myself use an additional debug(TEST) check alongside with 
version(unittest) for my test tracing features.

Re: Compile time code paths

[Jeremie Pelletier]

David Gileadi Wrote:

> Daniel Keep wrote:
> > Jeremie Pelletier wrote:
> >> If a function has both an asm and D implementations inside its body, and 
> >> the D version can be executed at compile time, but the asm one is much 
> >> faster at runtime. Is it possible to have the compiler use the D code path 
> >> at compile time (ie to fill in enums and whatnot), and have the asm 
> >> version available at runtime.
> > 
> > Not that I know of.  There's no way to switch based on run time/compile
> > time.  This was going to be solved, at least in part, using static
> > arguments, but that got dropped.
> > 
> > As it stands, you just have to use a suffix or prefix or something to
> > distinguish CTFE methods from runtime methods.
> 
> Is this a case for version(CompileTime){}?

No because it would also compile this block into the binary. 

The easy way is of course to have different symbols for compile-time and 
run-time. But this doesn't go well with generic programming where the function 
needing such a check can be deep in the compile-time call stack.

For example:
int foo() { return bar + 1; }
int bar() { return foobar * 2; }
int foobar() {
static if(isCompileTime) return ...; /// asm cannot execute at compile time, 
needed to keep foo and bar able to do CTFE
else asm { ...; } /// asm optimized for runtime
}

Re: Unit test practices in Phobos

[Jeremie Pelletier]

Lars T. Kyllingstad Wrote:

> I just filed a bug report (3240) that describes a case where IFTI is 
> used in Phobos, and where this causes errors when the function is used 
> with a different type than the one used in the unittest. (The well known 
> "IFTI doesn't work with implicit conversions" problem.) I have a strong 
> suspicion that there are many other cases like this waiting to be 
> discovered.
> 
> I have encountered such errors in my own code many times, and lately 
> I've been trying to get into the habit of writing unittests for all (or 
> at least more than one) types. Not full-fledged functionality tests, 
> mind you -- something like this is usually sufficient:
> 
>T foo(T)(T x) if (isFloatingPoint!T) { return x + 1.0; }
> 
>unittest
>{
>// Test different types
>alias foo!float foo_float;
>alias foo!double foo_double;
>alias foo!real foo_real;
> 
>// Test functionality
>assert (foo(2.0) == 3.0);
>}
> 
> For the cases where any type is allowed (or a lot of them, at least) 
> even this can become a time-consuming task. In these cases it should at 
> least be possible to make a representative selection of types to check.
> 
> I just wanted to recommend this as "good practice" to all, but 
> especially to the Phobos authors. In my experience this catches a lot of 
> bugs which are otherwise hard to spot.
> 
> -Lars

I just go with type tuples:

T foo(T)(T x) if(isFloatingPoint!T) { return x + 1.0; }
unittest {
foreach(T; allFloatingPointTuple) assert(foo!T(1.0) == 2.0);
}

Re: T[new]

[Jeremie Pelletier]

Lars T. Kyllingstad Wrote:

> Walter Bright wrote:
> > Lars T. Kyllingstad wrote:
> >> I've always wondered: Why are strings of type immutable(char)[], and 
> >> not immutable(char[])?
> > 
> > So:
> > 
> >string a = "hello";
> >a = "foo";
> > 
> > works.
> 
> 
> Ah, of course. :) Thanks.
> 
> -Lars

The problem with immutable(char)[] was that any string can be resized, even 
slices.

Walter: what will the string types be aliased to now: still immutable(char)[] 
or immutable(char)[new]?

I think it would be best to have them use the array [new] type. Functions which 
do not modify the string length can mark the string as an in parameter, and 
immutable(char)[] should be castable to const(immutable(char)[new]) in such 
cases to still allow slices to be passed.

Re: Memory allocation problem

[Jeremie Pelletier]

bearophile Wrote:

> Robert Fraser:
> > I agree it's a bug, and probably a rather major one. However in a real 
> > use case, any program that needs 1800+ MB arrays should be 64-bit only.
> 
> In that program there's essentially just that large array.
> What is the size of biggest array you suggest to use in a D/C program on a 32 
> bit OS running on a 2 GB RAM PC?
> 
> Bye,
> bearophile

I don't think there's any ideal value for a max allocation size, I can't even 
think of an use for such large arrays.

There is always a way to split the allocation in smaller ones which will be 
easy to map in the available virtual memory space.

If its a single stream, loading it all in memory at once is overkill, it would 
be more efficient to create a specialized range to load something like 0x1000 
bytes at once (aligned to the same boundary) and operate on this slice.

Re: Unit test practices in Phobos

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Lars T. Kyllingstad Wrote:
> > 
> >> I just filed a bug report (3240) that describes a case where IFTI is 
> >> used in Phobos, and where this causes errors when the function is used 
> >> with a different type than the one used in the unittest. (The well known 
> >> "IFTI doesn't work with implicit conversions" problem.) I have a strong 
> >> suspicion that there are many other cases like this waiting to be 
> >> discovered.
> >>
> >> I have encountered such errors in my own code many times, and lately 
> >> I've been trying to get into the habit of writing unittests for all (or 
> >> at least more than one) types. Not full-fledged functionality tests, 
> >> mind you -- something like this is usually sufficient:
> >>
> >>T foo(T)(T x) if (isFloatingPoint!T) { return x + 1.0; }
> >>
> >>unittest
> >>{
> >>// Test different types
> >>alias foo!float foo_float;
> >>alias foo!double foo_double;
> >>alias foo!real foo_real;
> >>
> >>// Test functionality
> >>assert (foo(2.0) == 3.0);
> >>}
> >>
> >> For the cases where any type is allowed (or a lot of them, at least) 
> >> even this can become a time-consuming task. In these cases it should at 
> >> least be possible to make a representative selection of types to check.
> >>
> >> I just wanted to recommend this as "good practice" to all, but 
> >> especially to the Phobos authors. In my experience this catches a lot of 
> >> bugs which are otherwise hard to spot.
> >>
> >> -Lars
> > 
> > I just go with type tuples:
> > 
> > T foo(T)(T x) if(isFloatingPoint!T) { return x + 1.0; }
> > unittest {
> > foreach(T; allFloatingPointTuple) assert(foo!T(1.0) == 2.0);
> > }
> 
> Yah, same here. I have unit tests in Phobos that have nested loops 
> testing against so many types, the release build takes forever. Some 
> edge case for the optimizer. I must disable them in release builds.
> 
> 
> Andrei

Don't you disable unittests in release builds?

Re: reddit.com: first Chapter of TDPL available for free

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Steven Schveighoffer wrote:
> > On Mon, 03 Aug 2009 18:00:59 -0400, Andrei Alexandrescu 
> >  wrote:
> > 
> >> http://www.reddit.com/r/programming/comments/975ng/diving_into_the_d_programming_language_tdpl/
> >>  
> >>
> >>
> >> (Don't tell anyone, but I plan to rewrite it.)
> >>
> >> Andrei
> > 
> > Wow, my head's spinning :)
> > 
> > That's a lot of data/concepts in one chapter.  Have you considered how 
> > this chapter will be for a newbie programmer?
> 
> Like K&R, TDPL is intended for people who already know how to program in 
> another language. Knowledge of a specific language is not recommended or 
> required. The preface will make that clear.
> 
> Some reviewers are still concerned that I discuss topics a bit too 
> advanced in the first chapter. I was suggested to adapt The Case for D 
> instead of this first chapter.
> 
> > *disclaimer: I'm not an expert on teaching or writing books.
> 
> Well you are an expert on reading books and that's what matters here.
> 
> 
> Andrei

I agree that your preview, while being really insightful into D, is gonna make 
the head of beginners explode. For example, you dive rather headfirst into 
variables, using immutable and auto before even talking about variable types.

I for one believe the market for such a book is mostly beginner to intermediate 
programmers. I only needed the language reference on the digitalmars website to 
learn about the syntax and semantics of D, and reading the code in the runtime 
and phobos gave more than enough examples to get comfortable using the language.

Re: Unit test practices in Phobos

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Andrei Alexandrescu Wrote:
> > 
> >> Jeremie Pelletier wrote:
> >>> Lars T. Kyllingstad Wrote:
> >>>
> >>>> I just filed a bug report (3240) that describes a case where IFTI is 
> >>>> used in Phobos, and where this causes errors when the function is used 
> >>>> with a different type than the one used in the unittest. (The well known 
> >>>> "IFTI doesn't work with implicit conversions" problem.) I have a strong 
> >>>> suspicion that there are many other cases like this waiting to be 
> >>>> discovered.
> >>>>
> >>>> I have encountered such errors in my own code many times, and lately 
> >>>> I've been trying to get into the habit of writing unittests for all (or 
> >>>> at least more than one) types. Not full-fledged functionality tests, 
> >>>> mind you -- something like this is usually sufficient:
> >>>>
> >>>>T foo(T)(T x) if (isFloatingPoint!T) { return x + 1.0; }
> >>>>
> >>>>unittest
> >>>>{
> >>>>// Test different types
> >>>>alias foo!float foo_float;
> >>>>alias foo!double foo_double;
> >>>>alias foo!real foo_real;
> >>>>
> >>>>// Test functionality
> >>>>assert (foo(2.0) == 3.0);
> >>>>}
> >>>>
> >>>> For the cases where any type is allowed (or a lot of them, at least) 
> >>>> even this can become a time-consuming task. In these cases it should at 
> >>>> least be possible to make a representative selection of types to check.
> >>>>
> >>>> I just wanted to recommend this as "good practice" to all, but 
> >>>> especially to the Phobos authors. In my experience this catches a lot of 
> >>>> bugs which are otherwise hard to spot.
> >>>>
> >>>> -Lars
> >>> I just go with type tuples:
> >>>
> >>> T foo(T)(T x) if(isFloatingPoint!T) { return x + 1.0; }
> >>> unittest {
> >>> foreach(T; allFloatingPointTuple) assert(foo!T(1.0) == 2.0);
> >>> }
> >> Yah, same here. I have unit tests in Phobos that have nested loops 
> >> testing against so many types, the release build takes forever. Some 
> >> edge case for the optimizer. I must disable them in release builds.
> >>
> >>
> >> Andrei
> > 
> > Don't you disable unittests in release builds?
> 
> Phobos has the builds: debug, release, unittest/debug, and 
> unittest/release. Client apps use the release version. I like being able 
>   to unittest the release version to make sure that the optimizer etc. 
> don't do shenanigans.
> 
> Andrei

Oh yeah I just did a test compile with -O -release -inline -unittest, took 
about 30 seconds to compile.

Still this is lightning fast when compared to C. Even using make with 40 jobs 
on my quad core a lot of programs take minutes to compile, DMD does it all in 
one big swoop, no jobs needed, no makefile needed.

Re: GPU/CPU roadmaps

[Jeremie Pelletier]

davidl Wrote:

> ÔÚ Tue, 11 Aug 2009 08:54:07 +0800£¬bearophile   
> дµÀ:
> 
> > D2/D3 may become a good language to create video games, this is a new  
> > interesting document that shows some of the things D2 users may want to  
> > use it for:
> > http://graphics.cs.williams.edu/archive/SweeneyHPG2009/TimHPG2009.pdf
> >
> > I don't know how D2 can adapt itself to help in such regards.
> >
> > Bye,
> > bearophile
> 
> At the first sight, I thought that guy were insane to abandon the merit of  
> current GPU computing. Then, when I went through the slice of productivity  
> is vital. I think I'm completely convinced by this exact point.
> GPU programming and blending into games is pretty specialized skill for  
> programmers. Maybe 1 of 1000 programmers has this skill. And maybe 1 of 2  
> has the sufficient knowledge to handle it well.
> I would expect an easier development model.
> 
> -- 
> ʹÓà Opera ¸ïÃüÐԵĵç×ÓÓʼþ¿Í»§³ÌÐò: http://www.opera.com/mail/

Sweeney makes some strong points in his presentation, it's true that the line 
between CPU and GPU is growing thinner and thinner,  and GPU programming is not 
the easiest thing to learn. I had an easier time learning ASM than I did 
learning the GPU pipeline in OpenGL heh. However it's just like everything 
else, its much harder to learn it than to use it once you get the hang of it.

The D language definitely has what it takes to fully exploit the concepts of 
this presentation, most of what he mentions about the language and compiler are 
either already covered by D or are being talked about.

I wouldn't be surprised to see major games releases built in D in a few years.

Re: GPU/CPU roadmaps

[Jeremie Pelletier]

language_fan Wrote:

> Tue, 11 Aug 2009 00:37:46 -0400, Jeremie Pelletier thusly wrote:
> 
> > I wouldn't be surprised to see major games releases built in D in a few
> > years.
> 
> It's quite optimistic to think that the toolchain will be usable and 
> robust enough on the large scale in a few years. For instance adding the 
> keywords immutable, const, and pure will not magically turn D into the 
> state of the art research language in the category of pure functional 
> languages.

Yeah it's optimistic, but not unrealistic. Most of the concepts talked about in 
the presentation are covered on Bartosz's blog, which I've been reading with 
much interest in the past days, and will most likely end up in D sooner or 
later.

>From what I read of the presentation, functional programming is only a quarter 
>of his framework concept (dealing with physics and the likes), there is still 
>STM for game state, good old sequencial programming to talk to the hardware, 
>and then highly parallel vector processing.

For the moment libraries like OpenCL could be used to provide the vector 
processing, and STM only requires a software implementation, it doesn't have to 
be part of the language, because we don't want it everywhere (that 30% 
performance overhead is scary).

As for the masses of libraries already available in C++, they can all be ported 
to D, with less code, and performing more efficiently. 3rd party middleware 
usually have a C API so its not a problem either.

Thats the beauty of abstractions, if we abstract OpenCL now for vector 
processing, we can just replace this module in the future to use intel's 
larrabee and all the code using the interface wouldn't see the difference.

So yeah, I'm optimistic, because I know its something we can start designing 
right away, and thinking about things like that is exactly why I love 
programming so much.

Re: Compile time code paths

[Jeremie Pelletier]

Don Wrote:

> Jeremie Pelletier wrote:
> > David Gileadi Wrote:
> > 
> >> Daniel Keep wrote:
> >>> Jeremie Pelletier wrote:
> >>>> If a function has both an asm and D implementations inside its body, and 
> >>>> the D version can be executed at compile time, but the asm one is much 
> >>>> faster at runtime. Is it possible to have the compiler use the D code 
> >>>> path at compile time (ie to fill in enums and whatnot), and have the asm 
> >>>> version available at runtime.
> >>> Not that I know of.  There's no way to switch based on run time/compile
> >>> time.  This was going to be solved, at least in part, using static
> >>> arguments, but that got dropped.
> >>>
> >>> As it stands, you just have to use a suffix or prefix or something to
> >>> distinguish CTFE methods from runtime methods.
> >> Is this a case for version(CompileTime){}?
> > 
> > No because it would also compile this block into the binary. 
> 
> I think something like this might work:
> 
> static if (__traits(CompileTime)) {
> ... // ctfe code
> } else {
> asm {
>   ...
> }
> }
> 
> Not sure what the exact syntax should be.

Good idea, I agree with the use of __traits here. The syntax looks great to me 
as it is.

Re: Explicitly saying ref or out when invoking a function

[Jeremie Pelletier]

Ary Borenszweig Wrote:

> In C# when you define a function that takes an out or ref parameter, 
> when invoking that function you must also specify ref or out. For example:
> 
> void fun(ref uint x, double y);
> 
> uint a = 1;
> double b = 2;
> fun(ref a, b);
> 
> When I first started using C# it really annoyed me that I had to put 
> that keyword there just to get my program compiled. "I know what I'm 
> doing", I thought. But later, when reading the code, I found it very 
> helpful to know that my "a" could be changed when invoking "fun". As 
> always, code is read much more times than written, and I think this 
> little tips help better understand the code.
> 
> What do you think?

I think it's overkill, and it still doesn't say if the ref is const or mutable. 
Much like in C/C++ when you pass a pointer you dont know if its a const or 
mutable parameter unless you look at the function declaration.

It's especially bad since if you modify the function prototype and change ref, 
you have all your calls to update too.

shared class constructor

[Jeremie Pelletier]

Just came across this while porting my code to enforce the use of shared.

---
module test;
shared class Foo {
   this() {}
}
---
Error: cannot implicitly convert expression (this) of type shared(Foo) to 
test.Foo

Of course I could just use new shared(Foo) but that would allow non-shared 
instances of Foo to be created as well, which I do not want.

Shouldn't "this" resolve to the fully qualified class?

Re: I don't think this is a bug but...

[Jeremie Pelletier]

Jesse Phillips Wrote:

> Benjamin Shropshire Wrote:
> 
> > The same expression twice gets different results
> > 
> > code:
> > 
> > import std.stdio;
> > 
> > bool Fn(float i){ return true; }
> > const bool b = Fn(cast(int)0);
> > 
> > static if(b) bool Fn(int i){ return false; }
> > const bool c = Fn(cast(int)0);
> > 
> > void main()
> > {
> >writef("%s\n", b);
> >writef("%s\n", c);
> > }
> > 
> > output:
> > 
> > true
> > false
> > 
> 
> The value of 'b' is assigned during compile time, but since it is indirectly 
> called the behavior seems odd. I suppose it is something to be aware of, but 
> it is behaving correctly.

Exactly, when 'b' is declared, Fn(int) isn't declared yet, as it depends on b 
being true, int is implicitly convertible to float so it matches the first 
declaration. When 'c' is declared both functions are available and the second 
declaration is chosen.

Re: I don't think this is a bug but...

[Jeremie Pelletier]

BCS Wrote:

> Reply to Jesse,
> 
> > Benjamin Shropshire Wrote:
> > 
> >> The same expression twice gets different results
> >> 
> >> code:
> >> 
> >> import std.stdio;
> >> 
> >> bool Fn(float i){ return true; }
> >> const bool b = Fn(cast(int)0);
> >> static if(b) bool Fn(int i){ return false; }
> >> const bool c = Fn(cast(int)0);
> >> void main()
> >> {
> >> writef("%s\n", b);
> >> writef("%s\n", c);
> >> }
> >> output:
> >> 
> >> true
> >> false
> > The value of 'b' is assigned during compile time, but since it is
> > indirectly called the behavior seems odd. I suppose it is something to
> > be aware of, but it is behaving correctly.
> > 
> 
> The only other option (beside saying this is correct) would be to make it 
> illegal to add a new function to an overload set after the set is used. That 
> could get very tricky to implement.
> 
> 

I don't think this is a good idea, since overload sets can be hijacked there is 
nothing that prevents a module from using a local overload set, and another 
then using it alongside its local hijacks.

Having such a "feature" would only limit the programmer in what they can do.

Re: The future of D 1.x

[Jeremie Pelletier]

Dominik Wrote:

> Is there any or there will be a "big switch" to 2.x once it is considered 
> final/stable? 

I already made the switch to D2 almost a year ago, and definitely wont be going 
back to D1. I didn't need to make many changes to my custom runtime either, the 
ABI between the two versions is for the most part the same.

I must say I was a bit confused with the added const and immutable qualifiers 
at first, but now I couldn't live without them. The shared qualifier is making 
me drool too, I can't wait until it is implemented properly. 

D1 was enough to make me an instant fan of the language and switch the planning 
of a few personal projects from C++ to D. D2 was what really made me dig 
headfirst nearly fulltime in the language.

I don't think D1 will vanish after D2 hits final, because it's an easier 
language to deal with, but D2 will most likely attract more programmers in the 
long run, especially if it continues getting better at concurrent programming.

Re: The future of D 1.x

[Jeremie Pelletier]

Dominik Wrote:

> 
> "Lutger"  wrote in message 
> news:h68ngs$1ef...@digitalmars.com...
> > Though they overlap, phobos and tango do have different scopes and 
> > somewhat
> > different programming styles.
> 
> I understand taht completeley. I was more making a general statement that 
> one of the reasons I am not considering D2 because of Tango being D1 only.
> 
> > Would you consider using D2 if Tango was available for it?
> 
> I would quite possibly consider it then since I believe fully in the 
> competence of Tango people - their work has proven me that they know their 
> stuf and I use Tango exclusively (I've started first with D1/Phobos). Tango 
> not being available for D2 yet sends mixed signals - it is either that D2 is 
> not considered stable yet for any effort of making a D2 version yet, or D2 
> is considered not good from the perspective of D1 users.
> 
> I have dabbled a bit with D2 and it seems like a different language to me, 
> but if Tango would be available for D2 - that would signal me that people 
> that I trust and are more competent than me to judge a language are 
> confident in D2 enough, so there would be much more reason for me to go 
> through with it.

I would assume the main reason why Tango hasn't been ported to D2 yet is that 
it's not a production language yet, a lot of new releases breaks the code of 
the previous ones as new syntax and semantics are added. As soon as it hits 
final I'm sure Tango will branch out a D2 version, with lots of new modules to 
cover the new features made possible by D2.

Right now D2 is more for people who wants to play with the new language 
features and contribute to the language.

I do believe D1 has a good chance of success with programmers coming from 
scripting backgrounds, but D2 is where the language will really shine against 
C++ and other established system languages.

Re: It's awfully quiet

[Jeremie Pelletier]

Paul D. Anderson Wrote:

> It's awfully quiet on the newsgroup today. Does that mean Walter is busy 
> getting the new release ready?
> 
> Or does it only mean I'm not getting new posts :( 
> 
> Suspiciously,
> 
> Paul

It IS awfully quiet this week. I hope a new release is soon, I can't wait for 
the shared qualifier to be working properly on classes and structs. The new 
T[new] arrays are sexy too!

Re: It's awfully quiet

[Jeremie Pelletier]

Jason House Wrote:

> Jeremie Pelletier Wrote:
> 
> > Paul D. Anderson Wrote:
> > 
> > > It's awfully quiet on the newsgroup today. Does that mean Walter is busy 
> > > getting the new release ready?
> > > 
> > > Or does it only mean I'm not getting new posts :( 
> > > 
> > > Suspiciously,
> > > 
> > > Paul
> > 
> > It IS awfully quiet this week. I hope a new release is soon, I can't wait 
> > for the shared qualifier to be working properly on classes and structs. The 
> > new T[new] arrays are sexy too!
> 
> I hope I'm not spoiling anything, but here are other things I know is going 
> on:
> 1. An updated gdb patch was submitted. It was tested against the upcoming 
> 2.032 release (and maybe the D1 equivalent?). Walter has made some tweaks to 
> how debug info is written.
> 2. Bartosz is rewriting thread related stuff. He has an upcoming blog on 
> reference counting which is part of his implementation. As part of this, I 
> know Bartosz got a few shared related bugs fixed. I'm unsure if they're in 
> bugzilla or not... 

Yeah I have been reading Bartosz's blog a lot lately, I especially liked his 
entries on thin-locks and the use of shared qualifiers, which is why I can't 
wait to toy with his concepts once D fully supports the primitives to implement 
them.

The recent thread about CPUs/GPUs also is of big interest, I really like 
Sweeney's concept of having four different processing models to form an 
all-around framework. I will definitely be working on an implementation soon.

Re: OT - Which Linux?

[Jeremie Pelletier]

Jason House Wrote:

> Paul D. Anderson Wrote:
> 
> > I'm going to add Linux to my PC to get a dual-boot configuration. (I'm 
> > tired of slw start ups and want to tap into the great tools available.) 
> > The tutorial I'm looking at suggests Ubuntu. Is there a significant 
> > difference in Linux implementations? Is Ubuntu one of the better ones? Does 
> > it make a difference for running D2?
> > 
> > Thanks in advance for your hellp.
> > 
> > Paul
> 
> 
> Ubuntu is the most popular Linux distro at the moment. It's easy to install, 
> keep up to date, and find help. I highly recommend it for newcomers to Linux. 
> I have not tried the other distro's recommended in this thread but do like 
> Ubuntu better than others I have tried. 
> 

Not only to newcomers, I've been using linux for at least 6 years (mostly 
redhat and slackware, also built a few linux from scratchs) and am now using 
Ubuntu on my laptop alongside win7. Not because its an easy linux distribution 
to work with, because its a convenient one to work with. I know I can do 
anything from the shell, but having a popup that says "it seems like you are 
missing this to execute that, would you like me to install and configure it for 
you while you drink your coffee?" is just plain neat.

Heck even windows isnt as neat as Ubuntu.

Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

I just noticed that when a method has a ref parameter for a struct, it doesn't 
get inlined:

union Matrix4x4 {
struct { float _11, _12, ...}
float[4][4] m;
float[16] v;

Matrix4x4 opMul(const ref Matrix4x4 m) const { ... }
void opMulAssign(const ref Matrix4x4 m) { this = opMul(m); }

void Translate(float x, float y, float z) {
const m = GetTranslation(x, y, z);
opMulAssign(m);
}

static Matrix4x4 GetTranslation(float x, float y, float z) { ... }
}

// RVO and inlining works like a charm here
Matrix4x4 m1 = Matrix4x4.GetTranslation(10, 0, 0);

// No inlining whatsoever here
m1.Translate(0, 1,  0);

I know in C++ when passing by reference (const Matrix4x4& m) the code gets 
inlned.

It's even worse when using the in (I know it means const scope) storage class 
for these members:

Matrix4x4 opMul(in Matrix4x4 m) const { ... }
void opMulAssign(in Matrix4x4 m) { this = opMul(m); }

void Translate(float x, float y, float z) {
opMulAssign(GetTranslation(x, y, z));
}

Not only does it not get inlined, but the whole 64bytes of the struct is copied 
into every stack frame using a bunch of MOVS instructions.

Isn't there a way to implement RVO to work on parameters (PVO?) too if the 
storage is const? The compiler could even detect that GetTranslation() is going 
to write to the stack frame of opMulAssign and inline the entire sequence. Even 
in debug compiles you could skip the whole data copy thing.

For now I pass all my references to any struct larger than 8bytes in my code by 
explicitely dereferencing (in Matrix4x4* m) to get the inline to work, but that 
kind of kills the newer syntax of D for such things. I think this should be 
left for compatibility with C and get the D storage classes working properly to 
prevent having to use pointers all over D too.

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

Bill Baxter Wrote:

> On Wed, Aug 26, 2009 at 11:01 AM, Jeremie Pelletier wrote:
> > I just noticed that when a method has a ref parameter for a struct, it 
> > doesn't get inlined:
> 
> Here's the bug you want to vote up:
> 
> http://d.puremagic.com/issues/show_bug.cgi?id=2008
> 
> It is indeed a sad situation.
> 
> --bb

Just voted, it should be marked as urgent, I can't even begin to imagine the 
amount of code that will need to be converted back to D syntax once it is fixed.

Thanks.

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

Jeremie Pelletier Wrote:

> Bill Baxter Wrote:
> 
> > On Wed, Aug 26, 2009 at 11:01 AM, Jeremie Pelletier 
> > wrote:
> > > I just noticed that when a method has a ref parameter for a struct, it 
> > > doesn't get inlined:
> > 
> > Here's the bug you want to vote up:
> > 
> > http://d.puremagic.com/issues/show_bug.cgi?id=2008
> > 
> > It is indeed a sad situation.
> > 
> > --bb
> 
> Just voted, it should be marked as urgent, I can't even begin to imagine the 
> amount of code that will need to be converted back to D syntax once it is 
> fixed.
> 
> Thanks.

Just to stress out how important this is; I have a small loop calculating a 3x2 
rotation matrix used with either Direct2D or Cairo every 10ms which transform 
the text "Hello World" in a window. CPU usage drops by 7-10% when using C style 
pointers instead of D storage classes. I haven't even optimized with SSE and 
whatnot yet, just removing the stack frame initialization overhead.

It would be way worse with the out storage class since the memory also has to 
be zero filled, which is overkill to me since its much better to just throw an 
error "out variable x used before initialization".

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

bearophile Wrote:

> Jeremie Pelletier:
> 
> > Just to stress out how important this is; I have a small loop calculating a 
> > 3x2 rotation matrix used with either Direct2D or Cairo every 10ms which 
> > transform the text "Hello World" in a window. CPU usage drops by 7-10% when 
> > using C style pointers instead of D storage classes. I haven't even 
> > optimized with SSE and whatnot yet, just removing the stack frame 
> > initialization overhead.<
> 
> Are you in a situation where you use the LDC compiler?
> 
> Bye,
> bearophile

DMD 2.031 here on windows 7 x64, I'm writing a Direct2D backend for my display 
package. I don't mind not having 64-bit support for now, but I definitely care 
about being on the bleeding edge of the D2 language. I also use the same 
compiler on Ubuntu.

I use a customized runtime forked from D1 in '06 and almost entirely rewritten 
kepping up to date with every new D release. I dropped support for D1 last 
year. Last I heard LDC only supported D1 under unix, haven't checked it in a 
while.

Does dmd have SSE intrinsics?

[Jeremie Pelletier]

While writing SSE assembly by hand in D is fun and works well, I'm wondering if 
the compiler has intrinsics for its instruction set, much like xmmintrin.h in C.

The reason is that the compiler can usually reorder the intrinsics to optimize 
performance.

I could always use C code to implement my SSE routines but then I'd lose the 
ability to inline them in D.

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

Walter Bright Wrote:

> Jeremie Pelletier wrote:
> > Isn't there a way to implement RVO to work on parameters (PVO?) too
> > if the storage is const?
> 
> No, and it doesn't work for C++ either. Consider:
> 
> struct S { int a; }
> 
> void foo(const ref S s)
> {
>  assert(s.a == 3);
>  bar();
>  assert(s.a == 3); // OOPS!
> }
> 
> S g;
> 
> void bar()
> {
>  g.a = 4;
> }
> 
> void main()
> {
>  foo(g);
> }

Yes but that would be a logic error from the programmer, as it would be 
possible to do the very same thing had foo been declared as void foo(in S* s).

Isn't it possible to make 'const ref S' or 'in S' generate the same machine 
code as 'in S*'? To me it would seem the semantics of the two are the same, 
with 'const S*' being useful syntax for C compatibility while 'in S' and 'const 
ref S' are both D syntax.

I don't use ref and out whatsoever in D because of their overhead and inability 
to inline, which is sad because the first time I read about D I pictured having 
every single parameter in my programs having at least one of in, ref or out and 
finally saying bye bye to pointers.

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

downs Wrote:

> Walter Bright wrote:
> > Jeremie Pelletier wrote:
> >> Isn't it possible to make 'const ref S' or 'in S' generate the same
> >> machine code as 'in S*'? To me it would seem the semantics of the two
> >> are the same, with 'const S*' being useful syntax for C compatibility
> >> while 'in S' and 'const ref S' are both D syntax.
> > 
> > The thing about const is it only specifies a read only view of an
> > object, it does *not* specify that the referenced object will not
> > change. That is why pass by value cannot be "optimized" to be pass by
> > reference.
> > 
> 
> To elaborate on this, consider this case:
> 
> import std.stdio;
> 
> struct X { int v; }
> 
> void test(X x, void delegate() dg) { writefln(x.v); dg(); writefln(x.v); }
> 
> void main() {
>   X ecks;
>   test(ecks, { ecks.v = 17; });
> }

Ok, I understand why it cant be done for 'in S' but I don't see why 'const ref 
S' cannot have the same semantics as 'in S*', unless 'ref' doesn't mean that 
the struct is implicitly dereferenced.

Here is some code to illustrate my point of view:

struct S { int i; }
S s;

void Stuff() { s.i++; }

void Foo(in S* s) {
writefln(s.i);
Stuff();
writefln(s.i);
}
void Bar(const ref S s) {
writefln(s.i);
Stuff();
writefln(s.i);
}

int main() {
// Those both do the exact same thing
Foo(&s);
Bar(s);
}

If they are meant to have different semantics, then when is a good time to use 
ref? It would seem to me 'in S*' and 'S*' carry both behaviors you want in a 
referenced parameter: const and mutable. In any case only the reference is 
passed by value, not the struct itself.

If the method calls another method which modifies the const view on the 
reference, then it should be a logic error from the programmer (good old 
shooting yourself in the foot) without the compiler getting in the way. Making 
fool-proof language semantics is a good idea, but IMO it shouldn't impact 
performance, or else any bit of code looking for time critical performance will 
never use the syntax that makes D shine, and a lot of confusion will spread 
around as both types of syntax are used. It also makes it confusing to 
interface with IDL.

Alls I'm suggesting is that 'const ref S' and 'ref S' generate the same machine 
code as 'in S*' and 'S*', which would prevent us from using different syntax to 
get the performance boost, when in the end the intended behavior is the same.

Re: Reference value of structs not optimized or inlined?

[Jeremie Pelletier]

Walter Bright Wrote:

> Ary Borenszweig wrote:
> > Walter Bright escribió:
> >> Ary Borenszweig wrote:
> >>> Walter Bright escribió:
>  There are a lot of D specific optimization opportunities that are 
>  left undone for now.
> >>> Why?
> >>
> >> Which of the thousand things people want done in D should be done first?
> > 
> > Those that you feel like doing first.
> > 
> > Ok, you win. :-)
> > 
> > (I use the same reasoning most of the time when coding Descent)
> 
>  But I'm not kidding about the thousand things. There are more than 
> that in Bugzilla.

The core feature set should be more important for now, since they drive D2 
closer to a stable specification, and I know how adding new features can break 
optimizations and whatnot.

I would rather see T[new] arrays and working shared qualifiers myself. However, 
proper ref semantics would also allow us to switch back from C to D syntax for 
these.

Maybe I should get familiar with dmd2's source and see if I can contribute to 
those issues once I get more free time, but even in its current development 
state, I have more fun coding in D than I ever had in C or C++.

Re: OT: What's your favorite codeline?

[Jeremie Pelletier]

Manfred_Nowak Wrote:

> incognito wrote:
> 
> > foreach(Problem p; World.problems) find_answer_to(p);
> > 
> > It just always seems to get stuck.
> 
> ... because it doesn't have `World.plan' to its disposal.

You are lacking a world context, cultural semantics, a problem solver, and a 
very very large pool of random. Maybe an abstract country factory.

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Walter Bright Wrote:

> http://www.reddit.com/r/programming/comments/9fk6g/how_nested_functions_work_part_1/

I really like the way nested functions and closures are done in D. Especially 
because they are the same thing as delegate and closures.

But speaking of closures, I did notice something that could be optimized:

void foo {
int a;
void foo2() { a++;}
bar(&foo2);
}
void bar(in void delegate() dg) { dg(); }

Here foo() is using _d_allocmemory to get storage for the stack frame. I 
understand it is for cases where the closure is executed long after the owner 
method has returned, but if the delegate has a scope storage class it could use 
the thread's stack instead.

Using nested functions alone does use the stack for storage, unless you need to 
use a local delegate to fix forward references within nested functions 
resulting in _d_allocmemory allocating the stack frame, as in the following 
example:

void myFoo() {
int a;
scope void delegate() bar = void;
void foo() { bar(); }
void foo2() { a++; if(a != 10) foo(); }
bar = &foo2;
foo();
}

Closures in D is an amazing feature and I really miss it when I need to use 
another language (I can't find an employer who wants D code, but I keep my 
fingers crossed!), but here is yet another feature I try to avoid in time 
critical code right now because of the expensive call into the memory manager 
when I can write longer code that executes on stack storage.

Anywho, thanks for the link Walter, I'm not really familiar with C# and it's 
nice to know D beats its syntax, I would hate to have to assign delegates for 
every nested function!

Re: const and immutable objects

[Jeremie Pelletier]

Graham St Jack Wrote:

> I have been trying (again) to start using const and immutable objects in 
> a project, and found I kept getting stuck with the need to change the 
> value of object references, which isn't allowed. I don't quite follow the 
> arguments that mean that this has to be the case, but I'm prepared to 
> accept them.
> 
> After some experimenting with Andrei's Rebindable template, I still 
> couldn't get past the problems. For a start, the get method is private so 
> I couldn't test for null.
> 
> In the end I cooked up the following, which is a very simple workaround 
> that just lets me assign to a const or immutable object reference. It 
> works just fine, and can be used in templates like containers without 
> forcing the container to care about what it is containing.
> 
> Comments?
> 
> //
> // Assign dest to src, using brute-force methods as necessary if T
> // is an immutable or const object.
> //
> // The intent is to avoid needing routine nasty tricks in
> // application code, especially in template code which should be
> // spared the complication of handling this very annoying
> // restriction on objects. Use of this template function doesn't
> // let you change the object itself, just the reference to it - so
> // all should be well.
> //
> void forcefulAssign(T)(ref T dest, T src) {
> static if (!is(T X == const(U), U) &&  !is(T X == invariant(U), U)) {
> // T is not const or immutable - just assign normally
> dest = src;
> }
> else static if (is(T : Object)) {
> // T is a const or immutable object - use brute force
> U * ptr = cast(U *) &dest;
> *ptr = cast(U) src;
> }
> else {
> // T is some other const or immutable type - not assignable
> static assert(false, "cannot assign to " ~ T.stringof);
> }
> }
> 
> // assign the default initializer to something
> void forcefulInitialize(T)(ref T dest) {
> static if (!is(T X == const(U), U) &&  !is(T X == invariant(U), U)) {
> // T is not const or immutable - just initialise normally
> dest = T.init;
> }
> else static if (is(T : Object)) {
> // T is a const or immutable object - use brute force
> U * ptr = cast(U *) &dest;
> *ptr = U.init;
> }
> else {
> // T is some other const or immutable type - not assignable
> static assert(false, "cannot assign to " ~ T.stringof);
> }
> }
> 

I agree that D lacks a mechanism to separate the object from it's reference. 
Maybe syntax like the following could be used to apply the storage class to the 
object value, and not the reference value:

class Foo;
void bar(in Foo& foo) {}

It's quite ugly and C-like, but that's the first thing that came to mind. The 
reference value is unique to the current method and shouldn't share the same 
storage qualifiers as it's referenced memory.

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Jarrett Billingsley Wrote:

> On Sun, Aug 30, 2009 at 9:00 PM, Jeremie Pelletier wrote:
> > Walter Bright Wrote:
> >
> >> http://www.reddit.com/r/programming/comments/9fk6g/how_nested_functions_work_part_1/
> >
> > I really like the way nested functions and closures are done in D. 
> > Especially because they are the same thing as delegate and closures.
> >
> > But speaking of closures, I did notice something that could be optimized:
> >
> > void foo {
> >    int a;
> >    void foo2() { a++;}
> >    bar(&foo2);
> > }
> > void bar(in void delegate() dg) { dg(); }
> >
> > Here foo() is using _d_allocmemory to get storage for the stack frame. I 
> > understand it is for cases where the closure is executed long after the 
> > owner method has returned, but if the delegate has a scope storage class it 
> > could use the thread's stack instead.
> 
> It already is optimized if you use "void bar(scope void delegate()
> dg)". If it doesn't optimize it when you use 'in', it's probably a
> bug. :)

I need to try that out, its most likely a bug since 'in' means 'const scope' 
now. 

Check my second code bit, the delegate is declared as scope and it doesn't 
optimize, maybe the compiler only checks if the closure is being dereferenced 
and if so switches to memory storage instead of stack storage without checking 
the storage class of the closure's delegate. And even without 'scope' in this 
case the compiler could detect that the delegate doesn't leave the function 
scope and optimize away.

Better yet, allow forward references for nested functions. The need to assign 
closures to delegates to get a nested function is so C# ;)

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Walter Bright Wrote:

> That's very nice, could you post that on reddit?

Just did, I even changed "D" to "the D programming language" because I know you 
like it for search engine referencing :)

http://www.reddit.com/r/programming/comments/9fk6g/how_nested_functions_work_part_1/c0cm3bk

Re: const and immutable objects

[Jeremie Pelletier]

Graham St Jack Wrote:

> On Sun, 30 Aug 2009 21:28:18 -0400, Jeremie Pelletier wrote:
> 
> 
> > I agree that D lacks a mechanism to separate the object from it's
> > reference. Maybe syntax like the following could be used to apply the
> > storage class to the object value, and not the reference value:
> > 
> > class Foo;
> > void bar(in Foo& foo) {}
> > 
> > It's quite ugly and C-like, but that's the first thing that came to
> > mind. The reference value is unique to the current method and shouldn't
> > share the same storage qualifiers as it's referenced memory.
> 
> I think the time for pining over this particular syntax feature of D is 
> over - as nice as it would be to be able to fix the problem in the 
> language, it would be too disruptive right now.

I think it would be better to fix it in D2  instead of patching a language 
design flaw with a template. It wouldn't be the first time D2 has changes that 
breaks existing code anyways (and the upcoming T[new] will definitely break a 
lot of code, so does shared when it works, etc), so it would be less disruptive 
to do it now than never do it.

Re: [OT] Haxe "if" context-sensitive? (Q for Grammar Experts)

[Jeremie Pelletier]

Nick Sabalausky Wrote:

> "Nick Sabalausky"  wrote in message 
> news:h7fl1o$f8...@digitalmars.com...
> > I've been trying to make a grammar for the Haxe langauge, and I've been 
> > having a hell of a time emulating it's expression-based if/if-else (as 
> > opposed to statement-based as in D). I'm sure a big part of it is my 
> > inexperience with writing grammars, but I've also been starting to wonder 
> > if it's impossible to do context-free. For those unfamiliar, here's how 
> > the relevant parts in Haxe work:
> >
> > --
> 
> > --
> >
> > So, anyone know if a grammar that handles this would need to be 
> > context-sensitive? Or am I just *really* bad at this? ;)
> >
> 
> I guess the key I'm looking for is this:
> 
> // With "if" being an expr, not a stmt:
> foo = if(blah) bar = 1;
> foo = if(blah) bar = 1; else bar = 2;
> 
> If I make assignment bind tighter than the "if expressions", then I can't 
> support "foo = if...", I can only support "foo = (if...)"
> 
> But if I make the "if expressions" bind tighter than assignment, then I 
> can't seem to solve the dangling-else conflict without introducing other 
> ambiguities.
> 
> Anyway, I'm not really looking to get an exact solution, just wondering if 
> there's something about it that causes it to be impossible for a 
> context-free grammar.
> 

I don't know what your parser implementation looks like, but assuming it uses 
Statement and Expression base classes, and your assignment statements are in 
the form "LValueExpression = Expression;", this is how I would structure it:

class IfStatement : Statement {
Expression _expr;
Expression _elseExpr
...
}
class IfExpression : Expression {
IfStatement _stmt;
...
}

Where ... contains your semantics analysis and either machine code or byte code 
generation.

Your parser first detects an assignment statement, has the lvalue expression 
node, and now begins to parse the assigned expression, which it detects to be 
an IfExpression. That expression generates its own underlying IfStatement which 
spans the ; else tokens too, resulting in both your if and else being part of 
your IfExpression, the later being used in your AssignStatement.

I could toy more with the idea, but this is definitely where I would start 
experimenting.

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Walter Bright Wrote:

> language_fan wrote:
> > This seems more like an advertisement of D than practical new information 
> > for compiler construction. Nesting functions is the basic feature of 
> > functional languages. Moreover even procedural Algol based languages such 
> > as Pascal have always supported them, too.
> 
> But not C, C++, Java, etc., so quite a lot of programmers have little 
> experience with them, and even less understanding.
> 
> > This information is also 
> > taught in basic university level compiler courses.
> 
> I bet only a tiny fraction of programmers have taken university compiler 
> classes. (Also, they were not covered in compiler classes I took.) 
> Consider also that the Java JVM doesn't support static links, and last I 
> heard the .NET IL didn't, either.
> 
> 
> > Now that I checked what wikipedia had to say to the matter, it also 
> > seemed to mention D. Apparently 'c-like syntax' plus 'advanced feature 
> > ' always equals 'innovation'.
> 
> 
> Nested functions aren't innovative; they just are apparently lacking in 
> many other popular languages, and seem to confuse a lot of people. If 
> you google it, you'll find there's a lot of programmer confusion about 
> them. Hence an article as to how they work is in order.
> 
> In part 2, I'll cover innovative things D does with nested functions.

I agree with Walter here. While D is not the first place I see closures and 
nested functions, it is the first language I come across that blends delegates, 
closures and nested functions in a simple, elegant and intuitive manner.

PHP for one does not support closures at all, and delegates are done using the 
ugly array(object => 'method') syntax. JavaScript works like C#; you have to 
assign a closure to a delegate value to get a nested function. C/C++ also don't 
support closures, and delegates must be manually handled (function ptr + void* 
param) with very little support from many existing APIs (win32 is especially 
bad for that as it only uses static callbacks).

Its no surprise that I was literally drooling when I first read about how D 
does it, I had just spent a few weeks doing jQuery in JavaScript for a client 
back then and fell in love with closures, D had it all with better syntax :)

In D you can even send a function pointer to a delegate value!

Re: const and immutable objects

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Graham St Jack Wrote:
> > 
> >> On Sun, 30 Aug 2009 21:28:18 -0400, Jeremie Pelletier wrote:
> >> 
> >> 
> >>> I agree that D lacks a mechanism to separate the object from it's
> >>>  reference. Maybe syntax like the following could be used to
> >>> apply the storage class to the object value, and not the
> >>> reference value:
> >>> 
> >>> class Foo; void bar(in Foo& foo) {}
> >>> 
> >>> It's quite ugly and C-like, but that's the first thing that came
> >>> to mind. The reference value is unique to the current method and
> >>> shouldn't share the same storage qualifiers as it's referenced
> >>> memory.
> >> I think the time for pining over this particular syntax feature of
> >> D is over - as nice as it would be to be able to fix the problem in
> >> the language, it would be too disruptive right now.
> > 
> > I think it would be better to fix it in D2  instead of patching a
> > language design flaw with a template. It wouldn't be the first time
> > D2 has changes that breaks existing code anyways (and the upcoming
> > T[new] will definitely break a lot of code, so does shared when it
> > works, etc), so it would be less disruptive to do it now than never
> > do it.
> 
> I agree with the sentiment. The issue is, however, that the change adds 
> a fair amount of complexity to the const system for an arguably 
> not-often-used need.
> 
> Andrei

Correct me if I'm wrong, but I don't see how complex that could be, since the 
compiler already needs to make the difference between the reference value and 
the object being referenced. The only thing that would need change is how that 
view is reflected in the syntax.

How about a .ref property?

class A {}
void foo(in A a) {

}

Re: const and immutable objects

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jeremie Pelletier wrote:
> > Graham St Jack Wrote:
> > 
> >> On Sun, 30 Aug 2009 21:28:18 -0400, Jeremie Pelletier wrote:
> >> 
> >> 
> >>> I agree that D lacks a mechanism to separate the object from it's
> >>>  reference. Maybe syntax like the following could be used to
> >>> apply the storage class to the object value, and not the
> >>> reference value:
> >>> 
> >>> class Foo; void bar(in Foo& foo) {}
> >>> 
> >>> It's quite ugly and C-like, but that's the first thing that came
> >>> to mind. The reference value is unique to the current method and
> >>> shouldn't share the same storage qualifiers as it's referenced
> >>> memory.
> >> I think the time for pining over this particular syntax feature of
> >> D is over - as nice as it would be to be able to fix the problem in
> >> the language, it would be too disruptive right now.
> > 
> > I think it would be better to fix it in D2  instead of patching a
> > language design flaw with a template. It wouldn't be the first time
> > D2 has changes that breaks existing code anyways (and the upcoming
> > T[new] will definitely break a lot of code, so does shared when it
> > works, etc), so it would be less disruptive to do it now than never
> > do it.
> 
> I agree with the sentiment. The issue is, however, that the change adds 
> a fair amount of complexity to the const system for an arguably 
> not-often-used need.
> 
> Andrei

Correct me if I'm wrong, but I don't see how complex that could be. I assume 
the compiler already makes the difference between the reference value and the 
referenced object, all that needs to be done is to reflect that view in the 
syntax.

Maybe a .ref property?

class A {}
void foo(in A a) {
a = new A(); // fail, cannot modify const
a.ref = new A(); // ok, only reference value is modified
}

If you want to make the reference immutable, you can use the .ref on the class 
identifier:

const(A.ref)[] AList; // slice of const references to A

It may not be perfect, but I do firmly believe that the D syntax needs a 
mechanism to separate references from objects, even if rarely used, I can think 
of at least 15-20 places in my code I had to use workarounds for such cases.

I don't think A* should be used since that is a pointer to an object reference, 
it would break too much code and be incredibly hard to track back. A& looks 
nice because that's how references are already handled in C++, and we'd  get 
const(A)& the same way we get const(int)*.

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Walter Bright Wrote:

> language_fan wrote:
> > For what it's worth, they also seem somewhat buggy in D. Hopefully 
> > writing the articles motivates fixing those issues :) For example one 
> > can't call symbols before their declaration on non-global scope.
> 
> That's actually not a bug. In function scopes, you can't reference a 
> variable before use, either.

You have to agree however that this is counter-intuitive, since functions in D 
can be used before they are declared, yet nested functions cant.

While it follows the convention used by local variables, this isnt the same, 
local variables need storage, the nested functions merely need a pointer to the 
stack frame.

Also, using delegates to dereference a nested function so it can be used as a 
prototype (which to me feels more like a hack than a feature) forces the entire 
stack frame to be allocated in _d_allocmemory.

Nested functions should be parsed in the same pass as their parent method, 
fixing the need to use delegates which also needlessly use 8bytes in the stack 
frame.

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> Jarrett Billingsley wrote:
> > On Mon, Aug 31, 2009 at 1:14 PM, Walter
> > Bright wrote:
> >> Thanks!
> >>
> > 
> > I don't mean to pull a Stewart, but please be sure to quote a bit of
> > the post you're responding to. If you don't, it's virtually impossible
> > to tell who you're talking to when viewing the mailing lists.
> 
> Same happened to me, but I've been brought back to order by others who 
> pointed out that turning "threaded" on in your newsreader clarifies what 
> the parent message is.
> 
> Andrei

Am I the only one here using web-news?

Re: Grand Central Dispatch & Blocks in Apple's C

[Jeremie Pelletier]

Michel Fortin Wrote:
> > Plus, they have OpenCL.  I've seen about a half dozen OpenCL examples
> > specifically for non-Mac machines, but none bother to tell you how to
> > actually GET an OpenCL implementation, or if they're even available.  *sigh*
> 
> OpenCL is Apple's baby, no wonder they're a little ahead in their 
> implementation. Just be pacient, I'm sure others will appear soon.

The specification is overseen by the Khronos group, Apple was a major 
contributor, and they had the original idea. But its not their work alone :)

ATI already has a beta implemention of OpenCL, nVidia has one it releases to 
developers but I am too lazy to fill in the long form with just "i want to toy 
with it" answers.

Hopefully these both reach release quality at the same time D handles 'shared' 
correctly :)

Re: Apple Blocks added to C++?

[Jeremie Pelletier]

Michel Fortin Wrote:

> On 2009-09-02 00:27:46 -0400, S.  said:
> 
> > Been awhile since I posted.
> > 
> > I was wondering what other people thought about this addition to C++ by 
> > Apple.   Heh.
> > 
> > http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/10
> 
> I don't find the syntax that ugly. And I'd say the feature is at its 
> best when used with Grand Central Dispatch. As page 13 of this same 
> review says, it enables developer to transform synchronous operations 
> such as this:
> 
>   NSDictionary *stats = [myDoc analyze];
>   [myModel setDict:stats];
>   [myStatsView setNeedsDisplay:YES];
>   [stats release];
> 
> into asynchrnous with minimal effort:
> 
>   dispatch_async(dispatch_get_global_queue(0, 0), ^{
> NSDictionary *stats = [myDoc analyze];
> dispatch_async(dispatch_get_main_queue(), ^{
>   [myModel setDict:stats];
>   [myStatsView setNeedsDisplay:YES];
>   [stats release];
> });
>   });
> 
> Without it, you'd have to create a bunch of different functions, 
> probably accompanied by a context struct, to make this work, 
> complexifying the conde and making it harder to follow. It's just 
> syntaxic sugar when you think about it, but it's that syntaxic sugar 
> that makes it practical to express operations in a way they can run 
> asynchrnously.
> 
> -- 
> Michel Fortin
> michel.for...@michelf.com
> http://michelf.com/
> 

How hard is it to implement asynchronous work queues anyways? Not hard at all.

Apple's blocks and global queue are not portable anyways. I would *much* rather 
use D delegates (which can be static functions, methods, closures or nested 
functions) and a process-local work queue.

There's a huge difference between complex code and long code, especially in 
compiled languages. Those 8 lines in your code are not convenient in that you 
don't get what they do on first sight. Declaring your things separately and 
then registering the closures to the queues would result in the exact same 
machine code being generated, but also be much easier to read.

The only place where I expect to see code unreadable like this is in scripting 
where the parser has an impact on the performance.

I also don't get why you need two closures on two different queues. My take at 
this in D would look like:

DispatchAsync({
   writefln("Look, async!");
});

No global queue to dispatch on the main queue, that's just redundant and slow, 
custom dispatcher that is portable, and D closures, simple and elegant :)

Re: How Nested Functions Work, part 1

[Jeremie Pelletier]

Nick Sabalausky Wrote:

> "Jeremie Pelletier"  wrote in message 
> news:h7h5i1$5m...@digitalmars.com...
> >
> > While D is not the first place I see closures and nested functions, it is 
> > the first language I come across that blends delegates, closures and 
> > nested functions in a simple, elegant and intuitive manner.
> >
> 
> I almost agree. But not quite. D certainly handles much of it better than a 
> lot of languages, and the syntax of the *actual* nested functions is nice. 
> But there are still issues that bug the hell out of me.
> 
> For instance, it's fairly common for me, in D, to write a function that 
> takes a delegate and then call it with a delegate literal (anon func). But 
> out of all the times I've done that, I don't think there's been a single 
> time I haven't had to go look up the syntax. And here's why:
> 
> bool delegate(int a)
> delegate bool(int a)
> 
> Umm, yea. If you don't already know what I'm getting at with that, then...
> Quick! For each of these, valid or invalid?:
> 
> // D 
> void repeat(int n, void delegate(int i) dg)
> {
> foreach(int i; 0..n)
> dg(i);
> }
> repeat(5, delegate void(int i) { writefln("hi"); });
> //
> 
> And now the secondary question: Which, if any, of those instances of 
> "delegate" should be changed to "function"? (I know it's not a real serious 
> problem, but the thing is, 99.9% of the time, I don't care, and more 
> importantly, I don't *want* to care whether or not there's a scope being 
> passed around with the...whatever it is.)
> 
> So...If you could correctly, and confidently, identify the correct answers 
> without hesitation, especially on the first part, well...then you've got a 
> far better mind than I do.

The only valid syntax is up there, and you could use a simple closure instead 
of a delegate literal. As for function pointers, you can send a function 
pointer to a delegate value, the context only gets set to null and the method 
calls with a null "this", but calls nonetheless.

I almost only use delegates in D, since it allows functions pointers, 
delegates, closures and nested functions to be passed. Function pointers only 
allow function pointers, but are useful for C compatibility.

I almost never use delegate literals since closures can be used instead, even 
with a return value (works like 'auto' return values).

void foo(bool delegate() dg) {
dg();
}
foo({return true;});

Re: Nullable or Optional? Or something else?

[Jeremie Pelletier]

Andrei Alexandrescu Wrote:

> I plan to add a Nullable struct to Phobos (akin to C#'s Nullable, 
> Boost's Optional).
> 
> Apparently a good design is to define Optional!T with a minimum of 
> member functions (ideally none) and have it use the "alias this" feature 
> to masquerade as a T. That way Optional!T looks and feels much like a T, 
> except that it supports a function
> 
> bool isNull(T)(Optional!T value);
> 
> Am I on the right track? If so, what is the name you'd prefer for this 
> artifact?
> 
> 
> Andrei


I just recently converted tons of COM headers in win32 to D (gotta love 
extern(C++)) and I really like how they hint the compiler of what parameters 
are used for. They have all sorts of macros like __in, __inout, __out, 
__in_opt, __inout_opt, __out_opt.

Why can't these be used in D too and implicitly add the appropriate contracts 
to the function:

void foo(in_opt int* a) { ... }

can be the same as

void foo(in int* a)
in { assert(a); }
body { ... }

I know I'm trying to push a lot of library stuff to the language spec, but it 
would just be so much more convenient that way. in_opt would be semantically 
the same as in, with the added contract.

Re: dmd 1.047 and 2.032 releases

[Jeremie Pelletier]

Brad Roberts Wrote:

> Walter Bright wrote:
> > This will probably be the last OSX 10.5 release, the next should be 10.6.
> > 
> > http://www.digitalmars.com/d/1.0/changelog.html
> > http://ftp.digitalmars.com/dmd.1.047.zip
> > 
> > 
> > http://www.digitalmars.com/d/2.0/changelog.html
> > http://ftp.digitalmars.com/dmd.2.032.zip
> > 
> > Many thanks to the numerous people who contributed to this update.
> 
> Nice.. looks like 5 of 27 regression bugs have been fixed:
> 
>   2560 -- 2.x
>   2665 -- 2.x
>   3168 -- 1.x
>   3169 -- 2.x
>   3196 -- 1.x and 2.x
> 
> http://d.puremagic.com/issues/buglist.cgi?product=D&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=regression
> 
> 68 ever filed
> 22 left after this round
> 
> Coincidently meaning 68% of the regression bugs have been fixed.  32% left.
> 
> Good stuff,
> Brad

Awesome, I'm gonna install and update my code today :)

This may be early, but what can we expect for 2.0.33? New features such as 
T[new] and working shared aggregates?

Re: The Linker is not a Magical Program

[Jeremie Pelletier]

Walter Bright Wrote:

> A rant I wrote a few months ago:
> 
> http://programmer.97things.oreilly.com/wiki/index.php/The_Linker_Is_not_a_Magical_Program

Great read, reminded me of my first weeks in C/C++ land, back when I couldn't 
tell the compiler from the linker :) 

These days only the OMF format on Windows still seems magic because I can't 
find any decent documentation on its format. I had the hardest of times putting 
together a CodeView reader, finally got one working after consulting at least 
10 different documents, looking at shitloads of codeview data in hex editors, 
and plenty of trial and error magic, and I still only support version 4.10, but 
I guess that makes me a magician!

Now if I only can find enough documentation about OMF to write a COFF to OMF 
converter.. there are still a bunch of libraries I use in C which I'd like to 
compile to static libraries in VC++ and link in DMD. I tried compiling with DMC 
to generate static OMF libraries directly, but they always fail on missing 
files like  and other misc issues.

I also searched the web high and low for a coff2omf binary and only found one 
which generate broken libraries, dmd won't see any symbols in the converted 
files and the linker goes crazy on unresolved references. I tried link /CONVERT 
only to find out there is no /CONVERT option.

So yeah, the linker is not a magical program, but it does operate in a magical 
world.

Re: The Linker is not a Magical Program

[Jeremie Pelletier]

Walter Bright Wrote:

> Jeremie Pelletier wrote:
> > Now if I only can find enough documentation about OMF to write a COFF
> > to OMF converter..
> 
> 
> http://www.azillionmonkeys.com/qed/Omfg.pdf

Sweet, you're the man Walter!

Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Well I've decided to get a look at the dmd2 source and see what I can 
contribute to it, only to notice it is very VC++ unfriendly. After a few hours 
of work, I finally got it to compile, and it works great, for the most part.

VC++ has no real support for long double; it supports the syntax as distinct 
types but they are implemented as double precision, not extended precision. 
There are no stdint.h and fenv.h headers, I found isnan() in float.h but I had 
to leave out backend/strtold.c for now, there was no unresolved references from 
this. alloca() was also found in malloc.h, not stdlib.h. The joys of not using 
a C99 compliant compiler heh. I also couldn't get the compiler to compile *.c 
as C++, even with the /TP switch (Compile as C++), I had to rename every file 
to *.cpp.

Other than that, it was standard compilation, I disabled a few warnings to not 
get my output flooded, defined MARS, __I86__ and _DH, generated the different 
table files, and compiled, all from within the IDE.

Jumping back in C++ reminded me why I loved D so much, I will definitely start 
working on, or help developing it if already planned, a D port of dmd once the 
spec gets stable.

In any ways, I am wondering where I can submit my changes for review, since I 
doubt there is any public access to the repository on dsource, and I would very 
much like to contribute to the D programming language.

Re: The Linker is not a Magical Program

[Jeremie Pelletier]

Christopher Wright Wrote:

> Walter Bright wrote:
> > A rant I wrote a few months ago:
> > 
> > http://programmer.97things.oreilly.com/wiki/index.php/The_Linker_Is_not_a_Magical_Program
> >  
> 
> For reference, GNU ld has to be compiled with -linvisible-pink-unicorn.

How can you know its pink if its also invisible?

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Jason House Wrote:

> Don Wrote:
> 
> > Interestingly since DMD has extern(C++), it ought to be possible to 
> > convert a single module to D, and still link it in...
> 
> extern(C++) support in D is nearly a joke. There are some big areas for 
> improvement, such as support for non-virtual functions, or allowing use of 
> new, ... It's unusable to the point that I can't connect D code to existing 
> C++ or even write efficient wrappers. I've resorted to simply using extern(C) 
> code in C++ and then wrappers in D to use those functions...

>From what I know it was mostly intended for interfaces, and it works wonders 
>when binding to COM code in Windows. A lot of the new features of recent 
>windows versions are only available through COM, extern(C++) makes it dead 
>easy to write their bindings.

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Jarrett Billingsley Wrote:

> On Fri, Sep 4, 2009 at 10:22 AM, Jeremie Pelletier wrote:
> > Jason House Wrote:
> >
> >> Don Wrote:
> >>
> >> > Interestingly since DMD has extern(C++), it ought to be possible to
> >> > convert a single module to D, and still link it in...
> >>
> >> extern(C++) support in D is nearly a joke. There are some big areas for 
> >> improvement, such as support for non-virtual functions, or allowing use of 
> >> new, ... It's unusable to the point that I can't connect D code to 
> >> existing C++ or even write efficient wrappers. I've resorted to simply 
> >> using extern(C) code in C++ and then wrappers in D to use those 
> >> functions...
> >
> > From what I know it was mostly intended for interfaces, and it works 
> > wonders when binding to COM code in Windows. A lot of the new features of 
> > recent windows versions are only available through COM, extern(C++) makes 
> > it dead easy to write their bindings.
> 
> You've been able to bind to COM interfaces in D for years. extern(C++)
> hasn't changed that, as far as I know.

True, but it has not always been that simple, now I can take the IDL file, run 
it through a few regular expressions and I got my D binding :)

extern(C++):
interface ID2D1Factory : IUnknown { ... }

I don't remember what the syntax in D1 looks like for COM interfacing, but I 
remember having some issues writing bindings.

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Don Wrote:

> Jeremie Pelletier wrote:
> > Well I've decided to get a look at the dmd2 source and see what I can
> > contribute to it, only to notice it is very VC++ unfriendly. After a
> > few hours of work, I finally got it to compile, and it works great,
> > for the most part.
> > 
> > VC++ has no real support for long double; it supports the syntax as
> > distinct types but they are implemented as double precision, not
> > extended precision. There are no stdint.h and fenv.h headers, I found
> > isnan() in float.h but I had to leave out backend/strtold.c for now,
> > there was no unresolved references from this. alloca() was also found
> > in malloc.h, not stdlib.h. The joys of not using a C99 compliant
> > compiler heh. I also couldn't get the compiler to compile *.c as C++,
> > even with the /TP switch (Compile as C++), I had to rename every file
> > to *.cpp.
> > 
> > Other than that, it was standard compilation, I disabled a few
> > warnings to not get my output flooded, defined MARS, __I86__ and _DH,
> > generated the different table files, and compiled, all from within
> > the IDE.
> 
> What are the advantages of VC++? Just the browsing?

It has one of the most powerful IDEs I have seen so far, and its debugging 
support is amazing. I also think its a good idea to make the dmd source 
compilable on a few different compilers, it will allow a wider range of 
developers to contribute to it in the long run.

> > Jumping back in C++ reminded me why I loved D so much, I will
> > definitely start working on, or help developing it if already
> > planned, a D port of dmd once the spec gets stable.
> 
> Oh yeah. Looking at the source, I keep thinking, "this would be SO much 
> easier in D!".
> Interestingly since DMD has extern(C++), it ought to be possible to 
> convert a single module to D, and still link it in...
> 
> > In any ways, I am wondering where I can submit my changes for review,
> > since I doubt there is any public access to the repository on
> > dsource, and I would very much like to contribute to the D
> > programming language.
> 
> It'd be fantastic if you could help.
> I've just been submitting my patches to Bugzilla. Walter sends me his 
> source from time to time. If you get active enough with patches we'll 
> make a better arrangement. It's only yesterday that Walter made a commit 
> to the repository for the first time, so at least he has it set up now .
> 
> I suggest that to start off, you look at the .stringof bugs. .stringof 
> has a reputation as one of the most buggy features in the compiler, but 
> I think it's only because of neglect rather than being particularly 
> complicated.

Sure, I'll give it a look, I need to get familiar with the code structure 
anyways :)

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Walter Bright Wrote:

> Jeremie Pelletier wrote:
> > Well I've decided to get a look at the dmd2 source and see what I can
> > contribute to it, only to notice it is very VC++ unfriendly. After a
> > few hours of work, I finally got it to compile, and it works great,
> > for the most part.
> 
> Can you send me the diffs?
> 

Sure, as soon as I figure out why my dmd produces different object code than 
yours. I ran a simple hello world executable and the "hello world" string is 
not properly aligned in the resulting executable, causing garbage to be 
appended by printf. The bug is somewhere between the parsing, which works fine, 
and the object generation (the call to obj_bytes() has the proper data pointer, 
but incorrect nbytes count).

> > 
> > VC++ has no real support for long double; it supports the syntax as
> > distinct types but they are implemented as double precision, not
> > extended precision.
> 
> There are two routes to go here, one is to just make a D that uses only 
> double precision for real precision, the other is to write a simple 
> emulator for 80 bit reals so the compiler can still generate 80 bit real 
> code despite the shortcomings in VC++.
> 
> 
> > In any ways, I am wondering where I can submit my changes for review,
> > since I doubt there is any public access to the repository on
> > dsource, and I would very much like to contribute to the D
> > programming language.
> 
> Bugzilla is a good place to start.

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Jeremie Pelletier Wrote:

> Walter Bright Wrote:
> 
> > Jeremie Pelletier wrote:
> > > Well I've decided to get a look at the dmd2 source and see what I can
> > > contribute to it, only to notice it is very VC++ unfriendly. After a
> > > few hours of work, I finally got it to compile, and it works great,
> > > for the most part.
> > 
> > Can you send me the diffs?
> > 
> 
> Sure, as soon as I figure out why my dmd produces different object code than 
> yours. I ran a simple hello world executable and the "hello world" string is 
> not properly aligned in the resulting executable, causing garbage to be 
> appended by printf. The bug is somewhere between the parsing, which works 
> fine, and the object generation (the call to obj_bytes() has the proper data 
> pointer, but incorrect nbytes count).

Finally found the bug, which was within a change I made in obj_bytes() to get 
it to compile under VC++, but didn't take in account the goto statement :x At 
least this little debugging session got me familiar with how dmd and dmc works.

Anywho, the patch file is about 1000 lines long, where should I mail it?

I also made a simple D script to rename all c files to cpp:

---
module conv;
import core.stdc.stdlib, core.stdc.stdio, std.file;

int main(string[] args) {
if(args.length != 2) goto Error;
bool isDir = void;
try isDir = isdir(args[1]);
catch(FileException) {}
if(!isDir) goto Error;

char[256] newpath = void;
bool delegate(DirEntry*) dg = void;
bool rename(DirEntry* de) {
if(de.name[$-2 .. $] == ".c") {
newpath[0 .. de.name.length] = de.name[];
newpath[de.name.length .. de.name.length + 2] = "pp";
.rename(de.name, newpath[0 .. de.name.length + 2]);
}
else if(de.isdir() && de.name[$-2 ..  $] != "tk") 
listdir(de.name, dg);
return true;
}
dg = &rename;
listdir(args[1], dg);
return 0;

Error:
printf("Usage: %.*s ", args[0]);
return 1;
}
---

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

bearophile Wrote:

> Jeremie Pelletier:
> > I also made a simple D script to rename all c files to cpp:
> 
> Generally I suggest to use a scripting language to do a similar task (or a 
> simper linux shell script, probably one or two lines), but I agree that doing 
> this I can't see possible faults/troubles of the same D code. So in the end 
> writing it in D is a good experiment.

D can be used as a scripting language too, and it's cross platform, plus it was 
a good test for my newly compiled dmd.

> You are probably an expert programmer, yet in your script I see some things 
> that in my opinion are better done in other ways:
> - Don't init things to void unless your performance experiment tells you to, 
> and do it with care. This is explained in D documentation too. The speed gain 
> is very limited and the risk of giving false pointers to the conservative GC 
> is present, because currently it's not very precise.

I am well aware of what =void implies, I had this habit ever since my first 
steps into D. The only times I don't use =void on variable declarations is when 
i want them to be initialized to their default values.

> - Gotos aren't evil, but it's better to use them only as performance 
> optimization or for tricky flows of the code. Generally in D there are better 
> ways to solve such problems. For example you can just throw an exception if 
> args.length!=2, and the uncaught exception will show an error message to the 
> user and the code will stop.

I generally don't use them a lot except to move error handling to the footer of 
a function or replace for(;;) loops. The dmd and druntime sources (I think 
phobos too) are full of such gotos anyways!

> - try ... catch(FileException) {} looks not nice.

Agreed, but it does what I want it to; redirect the exception to the Error 
label.

> - if(de.name[$-2 .. $] == ".c") is a bit unsafe, because unlike Python D 
> slices aren't saturating (and in the meantime I have asked to the Python 
> developers, they have told me that this design is desired in Python, it's not 
> a design mistake left for backward compatibility as someone here told me. And 
> it's a very handy thing. The only advantage of the way D slices are done is 
> that D ones are a bit faster. But their unsafety unnerves me and forces me to 
> write uglier and longer code). Generally it's better to use string functions, 
> that are safer. Python has endswith() and startwith() string methods that in 
> D (as functions) are even more important.

I don't see how unsafe that is in this context. I know it can be unsafe 
sometimes to send a slice to be saved somewhere, and later modify the original 
array. I only use slices to pass data around. As soon as a slice leaves the 
scope it is in (ie, assigning it to a property) or the original array is shared 
between threads, I use copies.

Besides, with the upcoming T[new] arrays, we're gonna see a major change in how 
arrays are used in D which should solve a lot of the current issues with 
slices. And once the shared qualifier is fully operational, its gonna solve 
even more issues.

> - In D the main doesn't need to return an int in an explicit way. If your 
> program stops because you have thrown and exception, I think the return value 
> is not zero.

Oh yeah, forgot about that one. Old habits again :)

> - Generally don't use this: printf("Usage: %.*s ", args[0]); printf() 
> is good mostly to speed up output if you have to print tons of things, or in 
> debugging write* functions, or for a simple compatibility with Tango. 
> Generally use writeln or writefln.

printf isn't that fast, in fact it is quite slow, anything that has to parse 
the format string on every call is gonna be slow. In this case, writefln caused 
an unresolved toUTF8 reference which I was too lazy to look into.

> - Unless performance is critical, it's generally better to write script-like 
> D programs in a high-level style, because in such kind of programs the 
> purpose is to write the code quickly, to have readable code, and most of all 
> to have the most reliable code as possible. In such small programs saving 
> microseconds is usually not positive.

I also agree here, however I'm not all that familiar with higher level 
concepts, even the PHP code I do at work looks a lot more like systems 
programming than web scripting, but it runs fast.

> - Note for D developers, listdir() desiged like that isn't good. Python too 
> used to have something similar (os.path.walk()), but it's better to invent 
> its design, and give an interable that instead yields the nodes of the tree 
> (os.walk() from Python 2.3).
>
> Bye,
> bearophile

Thanks for the comments though, I may argue a lot but I will remember what you 
said :)

J

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

div0 Wrote:
> Jeremie Pelletier wrote:
> 
> 
> Sweet. If dmd can compile w/ VS outa the box, I'll start poking around
> in it. Hey you can knock mirco-soft all you like, but VS is the nuts.

Yeah, I side with you here. I may not be a fan of the business model Microsoft 
uses, but their IDE is oh so sweet. I wish D had such a powerful and 
full-featured IDE. I use Descent on Ubuntu and Poseidon on Windows; both have 
their pros and cons yet I don't feel at home like I do in VS in either of them. 
Poseidon is simple, light and fast, but doesn't have that many features, 
although the customizable syntax highlighter, project manager and very simple 
build process are awesome. Descent on the other hand has a very nice syntax 
analyzer and tons of features, but requires eclipse which is too heavy for my 
tastes (thanks to java), has no syntax customization so new keywords are marked 
as errors, and the build process is a pain to setup, even with ant.

Anywho, I've sent my patch for VC++ compatibility to Walter, hopefully the 
changes will get into the next dmd release.

> - --
> My enormous talent is exceeded only by my outrageous laziness.
> http://www.ssTk.co.uk

I've seen that quote in a comment on slashdot a few weeks ago!

Re: D naming style?

[Jeremie Pelletier]

bearophile Wrote:

> Jarrett Billingsley:
> > I'm consistently confused by D (and Java and C#) code that doesn't
> > indicate member names, especially when they're named really common
> > things that often conflict with local variables, like "len" or "src"
> > or such. Unless the type is small enough to fit on a single screen,
> > it's easy to forget the member names.
> 
> I usually follow Python usage, and use this.fieldName. Is this bad practice 
> in D?
> 
> Bye,
> bearophile

I haven't coded in python, but in php members are accessed through 
$this->fieldName, which is just as ugly. I used to name my member variables 
like any other variable in D (without accessing them with this.) until I 
confused myself looking back at D code I wrote a year earlier and not being 
able to tell which variables were function local and which were from the class, 
which had a nasty superclass stack. I don't like using 'this' to access every 
member variable since I view it more as a way to access a member shadowed by a 
local symbol than as a convention, so I declare my member variables with a '_' 
prefix.

Re: D naming style?

[Jeremie Pelletier]

Michel Fortin Wrote:

> On 2009-09-04 21:07:01 -0400, Jarrett Billingsley 
>  said:
> 
> > On Fri, Sep 4, 2009 at 8:42 PM, Ali Cehreli wrote:
> >> Is there a common(-ish) naming style for D?
> >> 
> >> - camel case or underscores within words of names?
> >> 
> >> - type names begin with capital?
> >> 
> >> - underscore before or after member names?
> >> 
> >> - enum values lowercase?
> >> 
> >> - constant names?
> >> 
> >> - etc.? :)
> >> 
> >> Do you have a document that you would like to share?
> > 
> > There is a "D style guide" in the D spec that briefly mentions naming
> > conventions, but I'm not sure how many people use it / are aware of
> > its existence.
> 
> Here it is: .
> 
> I've also written a guide on how to name things, but, as far as I know, 
> nobody is following it at the moment. Hopefully it can be improved and 
> serve as the basis for naming things in Phobos (although I'm beginning 
> to be skeptical about that). See 
> .
> 
> -- 
> Michel Fortin
> michel.for...@michelf.com
> http://michelf.com/
> 

I just read your guide, it follows a lot of the logic I already use myself. But 
I think you need more explanations to justify your rules; I believe it is way 
easier for someone to remember something if they know and understand why they 
should remember it.

Also, about your bit on wchar vs wchar_t. wchar in D is fixed to 16bit while 
wchar_t is 16bit on Windows but 32bit on linux, some people care about 
differences like that, or learn them the hard way like I did: from segmentation 
faults!

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

Rainer Deyke Wrote:

> Jeremie Pelletier wrote:
> > bearophile Wrote:
> >> - if(de.name[$-2 .. $] == ".c") is a bit unsafe, (...)
> > I don't see how unsafe that is in this context.
> 
> Potential buffer underrun.
> http://www.digitalmars.com/d/1.0/arrays.html: "A program may not rely on
> array bounds checking happening"
> 
> 
> -- 
> Rainer Deyke - rain...@eldwood.com

Oh yeah, didn't think about that one here. I usually test for the string length 
before slicing it.

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

div0 Wrote:
> Jeremie Pelletier wrote:
> > div0 Wrote:
> >> Jeremie Pelletier wrote:
> >> 
> >>
> >> Sweet. If dmd can compile w/ VS outa the box, I'll start poking around
> >> in it. Hey you can knock mirco-soft all you like, but VS is the nuts.
> > 
> > Yeah, I side with you here. I may not be a fan of the business model 
> > Microsoft uses, but their IDE is oh so sweet. I wish D had such a powerful 
> > and full-featured IDE. I use Descent on Ubuntu and Poseidon on Windows; 
> > both have their pros and cons yet I don't feel at home like I do in VS in 
> > either of them. Poseidon is simple, light and fast, but doesn't have that 
> > many features, although the customizable syntax highlighter, project 
> > manager and very simple build process are awesome. Descent on the other 
> > hand has a very nice syntax analyzer and tons of features, but requires 
> > eclipse which is too heavy for my tastes (thanks to java), has no syntax 
> > customization so new keywords are marked as errors, and the build process 
> > is a pain to setup, even with ant.
> > 
> > Anywho, I've sent my patch for VC++ compatibility to Walter, hopefully the 
> > changes will get into the next dmd release.
> 
> Nice.
> How about the project files? They going to be in as well or at least
> dl-able from somewhere?

I also mailed them to Walter.

> >> - --
> >> My enormous talent is exceeded only by my outrageous laziness.
> >> http://www.ssTk.co.uk
> > 
> > I've seen that quote in a comment on slashdot a few weeks ago!
> 
> It's not a quote. It's original to myself; I've only ever used it in
> this NG from a couple of years back. Though it wouldn't surprise me if
> somebody else came up with it as well. Seems like an obvious one...
> 
> I staring reading slashdot back when it was called Chips and technology,
> but the signal to noise ratio dropped too low for my liking. It started
> going downhill the day they added user comments and accelerated once
> that open source company got involved. I had a div0 account there, but I
> don't recall ever making a comment. Haven't been back there in years.
> 
> Nick Sabalausky asked if he could appropriate it a while back,
> maybe he posts on slashdot.
> 

Well it's a great quote, made me laugh out loud the first time I saw it :)

Re: D 2.00 official spec

[Jeremie Pelletier]

Justin Johansson Wrote:

> Having trolled all over DM site in search for an official D 2.0 spec similar 
> to that as exists for D 1.0 (spec_DMD_1.00.pdf), I seem to be out of luck.
> 
> Does such a document exist and would someone please point me to it?
> 
> Thanks for all help.
> 

D2 is still in development, therefore there is no spec yet.

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Justin Johansson Wrote:
> I really really hope that my current excitement with D continues and that 
> another 30-60 days down the track I don't end up becoming disillusioned with 
> D as I did with Scala.
> 
> -- Justin Johansson 

I've been using D for over 2 years now, and my excitement about it still 
continues to grow :)

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

negerns Wrote:

> Jeremie,
> 
> I tried compiling the sources using VS2008 SP1. I did run into some of 
> what you mentioned but got stuck midway. Could you elaborate on how you 
> dealt with the missing header

#ifndef _MSC_VER
#include 
#endif

You don't need this header on VS.

Re: Compiled dmd2.032 in VC++ 2009!

[Jeremie Pelletier]

negerns Wrote:

> I ran into the __pascal calling convention. VS2008 no longer support
> 
> The __pascal calling convention is no longer supported in VS2008. How 
> did you resolve this?

I assumed these were remainders of older code, and removed the __pascal from 
both the prototypes and declarations.

You're gonna hit a few other issues, mostly unresolved symbols. In backend/cc.h 
there's a prototype for cpp_prettyident but no declaration, I hacked it away:

inline char *cpp_prettyident (Symbol *s) { return s->Sident; }

In cg87.cpp the compiler will whine about not having the M_*_L symbols, just 
define them as aliases of the * symbols (#define M_PI_L PI). VS has no long 
double support other than knowing its a different type.

In cgobj.cpp the compiler also whines about a few char arrays being out of 
bounds. I exploded the strings into array literals: "SMCLV" -> 
{'S','M','C','L','V'}. There are a few of them in the file.

In the same file, the compiler string has a hex character in it and VS is too 
dumb to notice it should be 8 bytes, fix it by breaking the string in two 
parts: "\0\xDB" "Digital Mars C/C++".

Again in the same file, in the function obj_bytes(), VS whines about ((char 
*)p)++; not being a l-value, just add a local char* variable to use as the 
pointer, be careful about the goto here, you have to replace p with your 
variable everywhere in the function, not just the first half.

In backend/cod3.cpp, the FLUSH() macro should be the same as the one for ELFOBJ.

That's about it, these were the few tricky issues I had, there are plenty more 
but they are trivial to solve.

Re: Modern Windows GUI visual styles

[Jeremie Pelletier]

Valery Wrote:

> Recently I spent a few hours to find a way to enable a new styles of Windows 
> XP controls from the code (without manual created external manifest files, 
> resource files, ...). The only solution I found in DFL library and it looks 
> quite complicated.
> 
> Maybe should create that function at the level of the compiler or linker?

As far as I know, the only way to get the newer common controls library is to 
use an assembly manifest, be it external or compiled as a resource. I haven't 
looked into DFL but I'm pretty sure it simply generates the proper manifest 
resource and link it in the executable.

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Justin Johansson Wrote:
> 5. Newcomers to the language will find it's type system concepts overwhelming 
> - co-variance and contra-variance etc.  (don't know how D2 will address this 
> better though). Yes these issues are important for OO libraries but feel 
> there must be a more practical way out of the language complexity.  
> Personally I always kept away from the hairy and scary bits of C++; you don't 
> need 'em in a practical language.
> 
> I've heard Scala's argument that all the complexity is hidden in the 
> libraries so need to worry about it.  Unfortunately I don't believe her.  I 
> learn a lot about a language by studying the library code and expect it to be 
> as easy to read and understand as mainline code.

I couldn't agree more, I learned how to use D by studying its runtime library 
over the past few years. To me it is especially useful to study a runtime 
library when it is used to implement features of the language, so you get a 
clear grasp of what using these features imply. I lost count on how many neat 
tricks I learned reading Andrei's metaprogramming code.

shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

I decided to play once again with shared and see what 2.032 is capable of. 
Turns out a lot of the previous issues I was having last time are gone, 
however, there are still a few things left which prevent me from rewriting my 
code.

The first issue that jumped to my face straight away was how 'shared const' 
methods are not callable from 'shared' objects.

shared class Foo {
void bar() const;
}
auto foo = new Foo; // foo is of type shared(Foo)
foo.bar; //  Error: function Foo.bar () shared const is not callable using 
argument types () shared

Considering how 'const' methods can be called from mutable objects, this looks 
like either a bug or a really awkward feature to me. Sending a shared(Foo) to a 
method expecting a shared(const(Foo)) also triggers a similar error from the 
compiler.

The other issue may be an intended feature, but it doesn't sound practical to 
me. Marking a method as shared assumes all used properties in the method's 
scope are also shared. Here is an example to illustrate my point:

class SimpleReader {
   this(LocalFile file) { _stream = new FileInputStream(file); }
   ...
private:
synchronized void read(ubyte[] buf, long offset) {
_stream.seek(offset);
_stream.read(buf);
}
FileInputStream _stream;
}

The FileInputStream here is a generic blocking binary stream which is not 
thread-safe by design. The reader is a composite class where every instance has 
its own unique stream instance and use it to implement asynchronous reads over 
the file format it abstracts, which in my case is a specialized read-only 
archive using a lot of random accesses from different threads.

This is where the issue shows its ugly head. The 'synchronized' keyword tags 
the read method as shared, which in itself is quite neat, what is annoying 
however is that it also changes the type of _stream in the method's scope to 
shared(FileInputStream) and therefore triggers compiler errors because 
_stream.seek and _stream.read are not shared:

Error: function FileInputStream.read (ubyte[]) is not callable using argument 
types (ubyte[]) shared

While it may be an attempt to keep shared usage safe, it isn't very practical. 
The stream object here is not shared because it is not thread-safe. While it 
may be used by different threads, it is unique to the reader's context and its 
accesses are synchronized by the reader, the stream should therefore be 
completely oblivious to the fact it is being used by different threads.

Maybe this could be the time to implement an unique qualifier; this is a 
context where having _stream be of type unique(FileInputStream) would solve the 
problem and allow further compiler optimizations. I don't know if it can be 
done with templates, and without any overhead whatsoever. I know I would much 
rather see unique(Foo) than Unique!Foo, and it would allow the use of 'is(foo : 
unique)'.

Furthermore, tagging a method with shared does not make it thread-safe, it may 
however use synchronized within its scope to protect its shared or unique data. 
This may be confusing when calling shared methods vs calling synchronized 
methods; one may think the shared one is not thread-safe and optionally 
synchronize the call, resulting in another monitor being used for nothing, or 
no monitor being used at all:

class Foo {
shared void bar() {
// Do stuff with local or immutable data
synchronized(this) { /* do stuff with shared data */ }
}
shared void bar2() {
// Do stuff on shared data
}
}

Someone seeing only the prototype of Foo.bar may assume the method is not 
thread-safe and call it as 'synchronized(foo) foo.bar()'. Just like they could 
see the prototype of bar2 and assume it is thread-safe, calling it as 
'foo.bar2()'.

What could be a good design against this sort of misleading behavior?

Phew, that's about enough issues and questions for now :)

Re: shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

Jason House Wrote:

> I'm glad to see I'm not the only one trying to use shared. I tried to use it 
> with 2.031 and rapidly hit bug after bug... I submitted several bug reports 
> for basic functionality, and none of it appeared in the changelog.
> 
> http://d.puremagic.com/issues/show_bug.cgi?id=3089

shared methods no longer trigger errors in 2.032, this issue should be marked 
as fixed, it must be a side effect from fixing another issue.

> http://d.puremagic.com/issues/show_bug.cgi?id=3090

I just made a quick template this seems to work in 2.032:

immutable template isShared(T) {
static if(is(T U : shared U))
bool isShared = true;
else
bool isShared = false;
}

> http://d.puremagic.com/issues/show_bug.cgi?id=3091

This one still isn't solved, I too found it annoying that you cant use 'new 
shared Foo()'. You can however declare Foo as 'shared class Foo', this works 
just like 'immutable class' or 'const class' by marking all properties and 
members with the qualifier. As a side note, I would like the same behavior for 
'static class'.

However using 'shared class' is not always the wanted behavior, you may only 
want a subset of the members and properties to be shared.

We also miss a unique qualifier to allow unshared objects to be used in shared 
contexts without the need for shared methods.

It's also awkward to use, const members may be called from either const or 
mutable objects. Shared members must be called from shared objects, so why 
allow a class to have shared and unshared members, if all instances are going 
to be shared anyways? It makes it much harder to draw the line between shared 
and unshared; you may have only a few objects actually shared, but you are 
required to make all the other objects they may use shared, even if they are 
synchronized or unique. This makes all the other contexts these objects are 
used in as shared, and soon your entire program is shared.

Re: shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

Graham St Jack Wrote:

> I'm also having the same problems.
> 
> As Jeremie said, as soon as you start introducing shared methods (via 
> synchronized for example), you rapidly get into trouble that can only be 
> overcome by excessive casting.
> 
> It may be possible to contain the problem by refactoring multi-threaded 
> code so that the shared objects are very small and simple, but even then 
> the casting required is too much. This approach might be ok if you could 
> define classes as being shared or immutable, and ALL instance of them 
> were then implicitly shared or immutable. Also, immutable objects should 
> be implicitly shareable.

I agree, this is also one of my main concerns with shared in its current state. 
It's an amazing and powerful concept and has the potential to make multi-thread 
code much easier and safer to write. But all the required casting is killing 
the safety, and makes it harder to use than it would be not having shared at 
all. The lack of an unique qualifier certainly doesn't help either.

Unique data could only be used for aggregate properties, const/immutable data 
would also be implicitly unique. This qualifier alone would simplify shared 
quite a lot, allowing the use of unshared objects in shared contexts safely.

The compiler should make the distinction between shared code and shared data 
and allow both shared and unshared instances to use shared methods, just like 
both const and mutable instances may call const methods. An error should also 
be triggered when calling a shared method of a shared object without 
synchronization, and maybe have a __sync keyword to override this. If a 
synchronized method is called from a non-shared object, no synchronization 
takes place.

Allow me to illustrate my point with some code:

class Foo {
int bar() shared { return a; }
__sync bar2() { synchronized(this) return a; }
synchronized void foo() { a = 1; }
int a;
}
auto foo1 = new shared(Foo)();
auto foo2 = new Foo;

foo1.foo(); // ok, synchronized call
synchronized(foo1) foo1.foo(); // warning: recursive synchronization
foo2.foo(); // ok, unsynchronized call
synchronized(foo2) foo2.foo(); // ok synchronized call

foo1.bar(); // error, unsynchronized call to bar() shared
synchronized(foo1) foo1.bar(); // ok, synchronized call
foo2.bar(); // ok, unsynchronized call
synchronized(foo1) foo1.bar(); // ok, synchronized call

foo1.bar2(); // ok, method handles synchronized
synchronized(foo1) foo1.bar2(); // warning, recursive synchronization
foo2.bar2(); // ok, method handles synchronized, even on unshared object
synchronized(foo2) foo2.bar2(); // warning, recursive synchronization, even on 
unshared object

That's about it, I believe this behavior would allow quite a number of 
multi-threaded techniques to be easily implemented and documented. It would 
only be the most natural thing since its quite similar to how const works.

Re: shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

Robert Jacques Wrote:

> On Sun, 13 Sep 2009 15:04:57 -0400, Jeremie Pelletier   
> wrote:
> [snip]
> > Unique data could only be used for aggregate properties, const/immutable  
> > data would also be implicitly unique. This qualifier alone would  
> > simplify shared quite a lot, allowing the use of unshared objects in  
> > shared contexts safely.
> 
> Neither const nor immutable data can be considered unique. First, any  
> const data may be being mutated by another routine, so it can't be safely  
> accessed without synchronization. Second, unique data is mutable while  
> const/immutable data is not. Third, most implementations of unique allow  
> for deterministic memory reclamation, which isn't possible if the unique  
> data might actually be const/immutable.

Good points, I can only agree with you here. However I still believe immutable 
data should be able to be used in shared contexts without being 'shared' or 
protected by a monitor.

> > The compiler should make the distinction between shared code and shared  
> > data and allow both shared and unshared instances to use shared methods,  
> > just like both const and mutable instances may call const methods. An  
> > error should also be triggered when calling a shared method of a shared  
> > object without synchronization, and maybe have a __sync keyword to  
> > override this. If a synchronized method is called from a non-shared  
> > object, no synchronization takes place.
> 
> I think you have the wrong paradigm in mind. Shared and non-shared aren't  
> mutable and const. They're mutable and immutable. From a technical  
> perspective, synchronization of shared methods are handled by the callee,  
> so there is no way not to call them and non-shared objects don't have a  
> monitor that can be synchronized. Now you can have the compiler use the  
> same code to generate two different object types (vtables, object layouts,  
> etc) with have the same interface, but that doesn't sound like what you're  
> suggesting.

I know that shared/unshared is not const/mutable. What I meant is that right 
now in D if a method is 'shared' it cannot be called from a non-shared object, 
which makes unshared instance of the class unusable without plenty of dirty 
casts. Take the following objects:

class Foo { void foo() const; }
class Bar { void bar() shared; }

Foo foo; foo.foo(); // ok, mutable object can call const method
Bar bar; bar.bar(); // error, unshared object may not call shared method

I had only presented the concept, your idea of using two virtual tables for 
shared/unshared instances is also what I had in mind for the implementation, 
and it would give exactly the behavior I had in mind. 

> > Allow me to illustrate my point with some code:
> >
> > class Foo {
> > int bar() shared { return a; }
> > __sync bar2() { synchronized(this) return a; }
> > synchronized void foo() { a = 1; }
> > int a;
> > }
> > auto foo1 = new shared(Foo)();
> > auto foo2 = new Foo;
> >
> > foo1.foo(); // ok, synchronized call
> > synchronized(foo1) foo1.foo(); // warning: recursive synchronization
> 
> Why a warning? Monitors are designed to handle recursive synchronization.

Its a performance issue that can easily be avoided, but still generates valid 
code.
 
> > foo2.foo(); // ok, unsynchronized call
> > synchronized(foo2) foo2.foo(); // ok synchronized call
> >
> > foo1.bar(); // error, unsynchronized call to bar() shared
> > synchronized(foo1) foo1.bar(); // ok, synchronized call
> > foo2.bar(); // ok, unsynchronized call
> > synchronized(foo1) foo1.bar(); // ok, synchronized call
> >
> > foo1.bar2(); // ok, method handles synchronized
> > synchronized(foo1) foo1.bar2(); // warning, recursive synchronization
> > foo2.bar2(); // ok, method handles synchronized, even on unshared object
> > synchronized(foo2) foo2.bar2(); // warning, recursive synchronization,  
> > even on unshared object
> >
> > That's about it, I believe this behavior would allow quite a number of  
> > multi-threaded techniques to be easily implemented and documented. It  
> > would only be the most natural thing since its quite similar to how  
> > const works.
> 
> The major benefit of const isn't method declaration, but object use: i.e.  
> only having to declare func(const T var) and not func(immutable T var) and  
> func(T var). Currently, there's no planned type to fill this role though  
> there have been some proposals.

I disagree, I think const methods are just as useful as const objects, since 
they are the only methods that can be called on such objects. They do 

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

bearophile Wrote:

> Justin Johansson:
> 
> >would you mind saying what salient things there are about D that presumably 
> >attracts to the language.  It just helps to know why others are here as one 
> >ventures into new territory.<
> 
> That's not an easy question. This is a personal answer, other people will 
> like other sides of D. I like D1/D2 for:
> - I don't think of it as a propetary language, like C#.
> - Sometimes I want the freedom to use memory as I like, with structs, and 
> even unions. If you have large datasets you find that using more than 20 
> bytes for a number as in Python doesn't help. Values also reduce indirection, 
> this speeds up things. This allows a more efficient usage of memory, and this 
> helps increase cache locality, that increases performance. Unfortunately 
> GC-managed D pointers can't be tagged, so I have to use memory from the C 
> heap for them. And sometimes you need pointers. That's why I'd like D to have 
> more/better ways to increase safety when using pointers (like using memory 
> regions when not in release mode, etc).

I haven't had to use the C heap whatsoever so far in D, could you give me an 
example of where you need it? In fact, the *only* place I use the C heap is in 
my garbage collector's internals, for pool structs and mark ranges. I use 
pointers to GC memory all the time too, there are plenty of algorithms, 
especially in loops, that can run faster with pointer arithmetic than slices 
and it's still the fastest way to pass struct references around.

> - I like this newsgroups, I can talk to people, and they sometimes answer my 
> numerous questions. I am learning a lot. Sometimes I receive no answers, but 
> it's acceptable. For its nature this newsgroup attracts some strange people 
> too.
> - I often use Python, it's the language I like more, but for some purposes 
> it's too much slow. And I am tired of writing vectorized code in NumPy and 
> the like. Cython reference count makes me sick and ShedSkin while nice is a 
> failed experiment. D feels like freedom, while sometimes using Python feels 
> like programming with mittens for me.
> - There are some things that I'd like to see in a language, and D being in 
> development still and being not controlled by an ivory tower committee give 
> me the illusion to see some of my ideas realized. So far I haven't influenced 
> a lot the development of D. On the other hand if everyone can influence a lot 
> the language the result may be a patchwork. So some dynamic compromise has to 
> be found every day.

I also like this community driven model, but this forum has more people 
submitting ideas than people able to implement them on time, I'm pretty sure 
the TODO list is rather huge at this time :) I for one much prefer D 
development the way it is now than the "working group" model used by the W3C or 
Khronos for example.

The public bugzilla is really nice too, once you get used to it, one of the 
issues I submitted got fixed in 2.032, I've also sent maybe 3 or 4 patches to 
the compiler source in other issues so far too, hopefully they'll be used in 
2.033!

> - D looks a lot like C, yet in D I can write code several times faster than 
> C. Sometimes 5-10 times faster. This is an enormous difference.

Indeed, and sometimes it's way faster than that. I've ported many C headers to 
D and I'm always amazed at how many things I can throw out, just the DirectX 
headers were at least 50% smaller in D and MUCH easier to read. Such simplicity 
is also reflected in the compiler by having quite a lot less tokens and parse 
nodes to create and analyze.

I must admit however that I sometimes miss the C preprocessor, or at least wish 
mixins had a syntax closer to that used by the C preprocessor. But it's a good 
idea to keep D without a preprocessor, its much better for everything to have a 
scope.

> - I am quite sensitive to syntax noise, boilerplate code. I like elegance, 
> clarity in semantics, I hate corner cases, yet I want a language that's 
> efficient, readable, and the less bug-prone as possible. C++ looks too much 
> complex for me. D1 is simple enough for me, and D2 is getting a bit too much 
> complex. I may appreciate the idea of a D 1.5 language that fixes some holes 
> and limits of D1 while keeping language simple enough (struct constructors, 
> and few other things. Such things don't significantly increase the difficulty 
> in using the language).

C++ isn't anymore complex than D2, its syntax just isn't as elegant. Other than 
multiple inheritance which is partially solved through object composition, I 
can't think of many features C++ has over D2. I can name quite a few features 
D2 has over C++ :)

What I like about D is that while its elegant, it still allows for dirty work 
to be done when you need it. You don't need to code your application core in C 
and your application behavior in a scripting language on top of the C core. D 
allows you to write it all in one language with the sam

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Tom S Wrote:

> Jeremie Pelletier wrote:
> > I haven't had to use the C heap whatsoever so far in D, could you give me 
> > an example of where you need it? In fact, the *only* place I use the C heap 
> > is in my garbage collector's internals, for pool structs and mark ranges. I 
> > use pointers to GC memory all the time too, there are plenty of algorithms, 
> > especially in loops, that can run faster with pointer arithmetic than 
> > slices and it's still the fastest way to pass struct references around.
> 
> I use the C heap a lot when I need slabs of memory that the GC should 
> not look into for performance reasons. This includes images/textures, 
> mesh data and some data structures that I release manually - again, for 
> efficiency reasons.

The garbage collector in D already mark allocations which contains pointers and 
scans these only. If you want to know if a type contains pointers, check the 
'flags' property of the typeinfo or classinfo, test for bit0 and bit1 
respectively. This is what the GC uses at runtime when allocating memory to 
know if it should tag the allocation as containing possible pointers.

I myself allocate all my meshes and textures directly on the GC and I'm pretty 
sure its faster than C's malloc and much safer.

> >> - I like how D doesn't totally ignore safety as C does, in D sometimes the 
> >> default way is the safer one, and the unsafe way is used only where you 
> >> ask for it.  I'd like to see more safeties added to D, like optional 
> >> run-time and compile-time integral overflow tests, some pointer safety, 
> >> better template error messages (template constraints help some in such 
> >> regard), stack traces, less compiler bugs, safer casts (in C# you need 
> >> explicit casts to convert double => float), a safer printf, some optional 
> >> annotations inspired by Plint (a lint program) to give more semantics to 
> >> the compiler, that can be used to both speed up code and avoid bugs. 
> >> There's lot that can be done in this regard. And release-mode performance 
> >> can be usually kept unchanged.
> > 
> > Stack traces is a feature for the runtime, I made one for mine, which shows 
> > a dialog window with the stack trace, current registers values, loaded 
> > modules and whatnot. Took me almost a week to piece together my CodeView 
> > reader, I still need a DWARF reader for linux. I'm gonna try and implement 
> > it in druntime and submit the patch to bugzilla.
> 
> Tango's runtime already does stack tracing on Windows and *NIX, however 
> its CV parser is subject to some licensing issues :( Perhaps you could 
> release yours under some liberal license so it can be plugged there? :)
> 

Sure, I wouldn't mind at all, I'm not into licenses myself so I might just 
release it to public domain. I'll try and get a standalone package ready and 
post it somewhere, I just don't know where yet :x

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Tom S Wrote:

> Jeremie Pelletier wrote:
> > Tom S Wrote:
> > 
> >> Jeremie Pelletier wrote:
> >>> I haven't had to use the C heap whatsoever so far in D, could you give me 
> >>> an example of where you need it? In fact, the *only* place I use the C 
> >>> heap is in my garbage collector's internals, for pool structs and mark 
> >>> ranges. I use pointers to GC memory all the time too, there are plenty of 
> >>> algorithms, especially in loops, that can run faster with pointer 
> >>> arithmetic than slices and it's still the fastest way to pass struct 
> >>> references around.
> >> I use the C heap a lot when I need slabs of memory that the GC should 
> >> not look into for performance reasons. This includes images/textures, 
> >> mesh data and some data structures that I release manually - again, for 
> >> efficiency reasons.
> > 
> > The garbage collector in D already mark allocations which contains pointers 
> > and scans these only. If you want to know if a type contains pointers, 
> > check the 'flags' property of the typeinfo or classinfo, test for bit0 and 
> > bit1 respectively. This is what the GC uses at runtime when allocating 
> > memory to know if it should tag the allocation as containing possible 
> > pointers.
> 
> Yea I know, but I want structures with pointers manually managed as well.

Then just inform the GC to not scan the allocations you want, or better yet 
have a static ctor modify the flag of the typeinfo you don't want scanned.

> > I myself allocate all my meshes and textures directly on the GC and I'm 
> > pretty sure its faster than C's malloc and much safer.
> 
> Hm, why would it be faster with the GC than malloc? I'm pretty sure it's 
> the opposite :P Plus, I could use a specialized malloc implementation, 
> like TLSF.

The D GC is already specialized, and given its being used quite a lot in D, 
there are good chances its already sitting in the CPU cache, its heap already 
having the available memory block waiting on a freelist, or if the alloc is 
more than 0x1000 bytes, the pages available in a pool. You'd need to use malloc 
quite a lot to get the same optimal performance, and mixing the two would 
affect the performance of both.

> >>>> - I like how D doesn't totally ignore safety as C does, in D sometimes 
> >>>> the default way is the safer one, and the unsafe way is used only where 
> >>>> you ask for it.  I'd like to see more safeties added to D, like optional 
> >>>> run-time and compile-time integral overflow tests, some pointer safety, 
> >>>> better template error messages (template constraints help some in such 
> >>>> regard), stack traces, less compiler bugs, safer casts (in C# you need 
> >>>> explicit casts to convert double => float), a safer printf, some 
> >>>> optional annotations inspired by Plint (a lint program) to give more 
> >>>> semantics to the compiler, that can be used to both speed up code and 
> >>>> avoid bugs. There's lot that can be done in this regard. And 
> >>>> release-mode performance can be usually kept unchanged.
> >>> Stack traces is a feature for the runtime, I made one for mine, which 
> >>> shows a dialog window with the stack trace, current registers values, 
> >>> loaded modules and whatnot. Took me almost a week to piece together my 
> >>> CodeView reader, I still need a DWARF reader for linux. I'm gonna try and 
> >>> implement it in druntime and submit the patch to bugzilla.
> >> Tango's runtime already does stack tracing on Windows and *NIX, however 
> >> its CV parser is subject to some licensing issues :( Perhaps you could 
> >> release yours under some liberal license so it can be plugged there? :)
> >>
> > 
> > Sure, I wouldn't mind at all, I'm not into licenses myself so I might just 
> > release it to public domain. I'll try and get a standalone package ready 
> > and post it somewhere, I just don't know where yet :x
> 
> Sweet :D As for a place, there are plenty of options, e.g. 
> http://dsource.org/projects/scrapple/ or a separate dsource project.

I thought of that, but I don't feel like opening a project for just a few 
random code snippets or standalone classes. I think I'll just post it in this 
forum and let interested people grab it for now.

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Tom S Wrote:

> Jeremie Pelletier wrote:
> > Tom S Wrote:
> > 
> >> Jeremie Pelletier wrote:
> >>> I myself allocate all my meshes and textures directly on the GC and I'm 
> >>> pretty sure its faster than C's malloc and much safer.
> >> Hm, why would it be faster with the GC than malloc? I'm pretty sure it's 
> >> the opposite :P Plus, I could use a specialized malloc implementation, 
> >> like TLSF.
> > 
> > The D GC is already specialized, and given its being used quite a lot in D, 
> > there are good chances its already sitting in the CPU cache, its heap 
> > already having the available memory block waiting on a freelist, or if the 
> > alloc is more than 0x1000 bytes, the pages available in a pool. You'd need 
> > to use malloc quite a lot to get the same optimal performance, and mixing 
> > the two would affect the performance of both.
> 
> It might be specialized for _something_, but it definitely isn't 
> real-time systems. I'd say with my use cases there's a very poor chance 
> the GC is sitting in the CPU cache since most of the time my memory is 
> preallocated and managed by specialized structures and/or malloc. I've 
> found that using the GC only for the hard-to-manually-manage objects 
> works best. The rest is handled by malloc and the GC has a very shallow 
> vision of the world thus its collection runs are very fast. Of course 
> there's a drawback that both the GC and malloc will have some pages 
> cached, wasting memory, but I don't let the GC touch too much so it 
> should be minimal. YMMV of course - all depends on the memory allocation 
> patterns of the application.

I understand your points for using a separate memory manager, and I agree with 
you that having less active allocations make for faster sweeps, no matter how 
little of them are scanned for pointers. However I just had an idea on how to 
implement generational collection on a non-moving GC which should solve your 
issues (and well, mines too) with the collector not being fast enough. I need 
to do some hacking on my custom GC first, but I believe it could give yet 
another performance boost. I'll add my memory manager to my list of code 
modules to make public :)

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Nick B Wrote:

> Jeremie Pelletier wrote:
> > Tom S Wrote:
> > 
> >> Jeremie Pelletier wrote:
> >>> Tom S Wrote:
> >>>
> >>>> Jeremie Pelletier wrote:
> >>>>> I myself allocate all my meshes and textures directly on the GC and I'm 
> >>>>> pretty sure its faster than C's malloc and much safer.
> >>>> Hm, why would it be faster with the GC than malloc? I'm pretty sure it's 
> >>>> the opposite :P Plus, I could use a specialized malloc implementation, 
> >>>> like TLSF.
> >>> The D GC is already specialized, and given its being used quite a lot in 
> >>> D, there are good chances its already sitting in the CPU cache, its heap 
> >>> already having the available memory block waiting on a freelist, or if 
> >>> the alloc is more than 0x1000 bytes, the pages available in a pool. You'd 
> >>> need to use malloc quite a lot to get the same optimal performance, and 
> >>> mixing the two would affect the performance of both.
> >> It might be specialized for _something_, but it definitely isn't 
> >> real-time systems. I'd say with my use cases there's a very poor chance 
> >> the GC is sitting in the CPU cache since most of the time my memory is 
> >> preallocated and managed by specialized structures and/or malloc. I've 
> >> found that using the GC only for the hard-to-manually-manage objects 
> >> works best. The rest is handled by malloc and the GC has a very shallow 
> >> vision of the world thus its collection runs are very fast. Of course 
> >> there's a drawback that both the GC and malloc will have some pages 
> >> cached, wasting memory, but I don't let the GC touch too much so it 
> >> should be minimal. YMMV of course - all depends on the memory allocation 
> >> patterns of the application.
> > 
> > I understand your points for using a separate memory manager, and I agree 
> > with you that having less active allocations make for faster sweeps, no 
> > matter how little of them are scanned for pointers. However I just had an 
> > idea on how to implement generational collection on a non-moving GC which 
> > should solve your issues (and well, mines too) with the collector not being 
> > fast enough. I need to do some hacking on my custom GC first, but I believe 
> > it could give yet another performance boost. I'll add my memory manager to 
> > my list of code modules to make public :)
> Jeremie
> 
> If the code is really usefull, why not offer it to the Tango team, for 
> formal inclusion  in the next release ?
> 
> Nick B

Because I dropped support for D1 long ago. If either the Tango or Phobos team 
like my code once I publish it, they are free to adapt it for their runtime. 

I rewrote the GC from scratch and optimized over the past 2 years to support my 
custom D runtime. It cannot be used as-is with neither phobos or tango without 
either changing the public interface of the GC or rewriting every runtime 
routine calling into the GC. I would only release it to public domain as an 
example of how to implement a tracing generational non-moving GC. I still need 
to implement the generational part, but I got the general algorithm down on 
paper today so I should have it working sometime this week.

I'm not a big fan of code licenses and therefore like to write most of my code 
myself, if only to learn how it works. I rarely mind people asking for my code 
either, so long as I get credited for it :)

Re: shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

Robert Jacques Wrote:

> On Sun, 13 Sep 2009 18:08:57 -0400, Jeremie Pelletier   
> wrote:
> 
> > Robert Jacques Wrote:
> >
> >> On Sun, 13 Sep 2009 15:04:57 -0400, Jeremie Pelletier  
> >> 
> >> wrote:
> >> [snip]
> >> > Unique data could only be used for aggregate properties,  
> >> const/immutable
> >> > data would also be implicitly unique. This qualifier alone would
> >> > simplify shared quite a lot, allowing the use of unshared objects in
> >> > shared contexts safely.
> >>
> >> Neither const nor immutable data can be considered unique. First, any
> >> const data may be being mutated by another routine, so it can't be  
> >> safely
> >> accessed without synchronization. Second, unique data is mutable while
> >> const/immutable data is not. Third, most implementations of unique allow
> >> for deterministic memory reclamation, which isn't possible if the unique
> >> data might actually be const/immutable.
> >
> > Good points, I can only agree with you here. However I still believe  
> > immutable data should be able to be used in shared contexts without  
> > being 'shared' or protected by a monitor.
> 
> One of the purposes behind immutable was lock-free access. As far as I  
> know you can use immutable data in shared contexts today without any other  
> modifiers. A quick test seems to indicate this works today, but if you've  
> got a test case where it doesn't, I'd recommend filing it as a bug.

Oh yeah, I'm confusing it with 'shared' methods not able to call 'const shared' 
methods, which is a pain in the ass :(

> >> > The compiler should make the distinction between shared code and  
> >> shared
> >> > data and allow both shared and unshared instances to use shared  
> >> methods,
> >> > just like both const and mutable instances may call const methods. An
> >> > error should also be triggered when calling a shared method of a  
> >> shared
> >> > object without synchronization, and maybe have a __sync keyword to
> >> > override this. If a synchronized method is called from a non-shared
> >> > object, no synchronization takes place.
> >>
> >> I think you have the wrong paradigm in mind. Shared and non-shared  
> >> aren't
> >> mutable and const. They're mutable and immutable. From a technical
> >> perspective, synchronization of shared methods are handled by the  
> >> callee,
> >> so there is no way not to call them and non-shared objects don't have a
> >> monitor that can be synchronized. Now you can have the compiler use the
> >> same code to generate two different object types (vtables, object  
> >> layouts,
> >> etc) with have the same interface, but that doesn't sound like what  
> >> you're
> >> suggesting.
> >
> > I know that shared/unshared is not const/mutable. What I meant is that  
> > right now in D if a method is 'shared' it cannot be called from a  
> > non-shared object, which makes unshared instance of the class unusable  
> > without plenty of dirty casts. Take the following objects:
> >
> > class Foo { void foo() const; }
> > class Bar { void bar() shared; }
> >
> > Foo foo; foo.foo(); // ok, mutable object can call const method
> > Bar bar; bar.bar(); // error, unshared object may not call shared method
> >
> > I had only presented the concept, your idea of using two virtual tables  
> > for shared/unshared instances is also what I had in mind for the  
> > implementation, and it would give exactly the behavior I had in mind.
> 
> Bartosz took the concept one step further: when declared as shared, all  
> methods are implicitly wrapped in synchronize blocks. He then added a  
> keyword for more manual, lock-free style programming. But this syntactic  
> sugar isn't implemented yet.

The current D keywords (synchronized and shared) are already designed for that, 
since synchronized implies shared. I don't want implicit synchronization, I'd 
much rather have a shared class marking all its members/properties as shared 
and letting me explicitely decide where the synchronization takes place.

> >> > Allow me to illustrate my point with some code:
> >> >
> >> > class Foo {
> >> > int bar() shared { return a; }
> >> > __sync bar2() { synchronized(this) return a; }
> >> > synchronized void foo() { a = 1; }
> >> > int a;

Re: Template Metaprogramming Made Easy (Huh?)

[Jeremie Pelletier]

Lutger Wrote:

> language_fan wrote:
> 
> > Mon, 14 Sep 2009 07:33:59 -0400, bearophile thusly wrote:
> > 
> >> But lot of people will judge D against more modern languages like C#,
> >> Scala or Java) and not against C.
> > 
> > Programmers often belong to three kinds of groups. First come the fans of
> > traditionally weakly typed compiled languages (basic, c, c++). They have
> > tried some "dynamic" or "academic" languages but did not like them. They
> > fancy efficiency and close to metal feel. They think compilation to
> > native code is the best way to produce programs, and think types should
> > reflect the feature set of their cpu. They believe the syntax C uses was
> > defined by their God.
> > 
> > The second group started with interpreted languages built by amateurs
> > (php, ruby, python, some game scripting language etc). They do not
> > understand the meaning the types or compilation. They prefer writing
> > short programs that usually seem to work. They hate formal specifications
> > and proofs about program properties. They are usually writing simple web
> > applications or some basic shareware utilies no one uses. They also hate
> > trailing semicolons.
> > 
> > The members of the last group have studied computer science and
> > languages, in particular. They have found a pet academic language,
> > typically a pure one, but paradigms may differ. In fact this is the group
> > which uses something other than the hybrid object-oriented/procedural
> > model. They appreciate a strong, orthogonal core language that scales
> > cleanly. They are not scared of esoteric non-C-like syntax. They use
> > languages that are not ready to take a step to the "real world" during
> > the 70 next years.
> > 
> 
> That's a fancy way of saying that anyone who has not studied CS is a moron 
> and therefore cannot understand what is good about languages, thus they lose 
> any argument automatically. Am I right?
> 

I dunno if that's what OP meant, but studying CS does not make you a reference 
in programming languages. I didn't even complete my first year of CS because I 
wasn't learning as fast as I wanted. School teaches you theory anyways, a job 
will teach you how to apply it in the real world. Anyone who can read and has 
the slightest interest in programming can learn the theory by themselves.

As for the different classes of programmers, I think the OP pushed more the 
extremes than the general cases. I came across a series of articles by Eric 
Lippert a few weeks ago talking about the matter:

http://blogs.msdn.com/ericlippert/archive/tags/Cargo+Cult+Programming/default.aspx

Re: Compile-time AAs

[Jeremie Pelletier]

Don Wrote:

> bearophile wrote:
> > Don has recently said that adding dynamic arrays at compile-time looks 
> > easy. I'd also like to have compile-time associative arrays. So you can 
> > fill them inside a CT function (the compiler may try to do the same for AAs 
> > created inside a static this(){}), and save some run time.
> > Even if such compile-time AAs have to be immutable at run-time they can be 
> > useful anyway.
> > A smarter implementation of such CT AAs may even use a perfet hashing, to 
> > make them quite fast.
> 
> Indeed. I think perfect hashing is one of the primary appeals of a 
> compile-time AA.
> 
> BTW, you can use AAs inside CTFE already. There's probably missing 
> functionality, though -- create a Bugzilla test case for anything you 
> find. The primary thing which is missing is that you can't use an AA 
> literal to populate a runtime AA (this is a backend issue, not a CTFE 
> limitation). You can get the AA keys and values as arrays, though, so 
> you could populate the AA yourself.

Just out of curiosity, how are compile time AAs implemented? The D runtime 
already handles the creation and lookups of these arrays, so let's suppose I 
changed my runtime to have a completely different AA implementation, what would 
happen when you mix both compile time and runtime AAs?

Re: Type unions in D

[Jeremie Pelletier]

Justin Johansson Wrote:

> What's the best way of emulating a system of quantified type unions in D  
> (D1)?
> 
> By type unions (sometimes called algebraic sums in the literature), I don't 
> mean the C/C++ "union" facility.
> 
> Suppose there is some class X, I'd like to have a first class type for 
> representing the following quantified types (using RegExp notation):
> X?  zero or one instances of an X  (call this type XQ)
> X a single instance of X
> X* zero or more instances of X  (call this type XS)
> X+ at least one X (call this type XP)
> 
> To unify the whole thing, now let X, XQ, XS and XP all be derived from a 
> common base type T.
> 
> Note that Scala has the Option, Some and None classes which kind of handles 
> (but not exactly) the XQ case.
> 
> Brendan Eich, JavaScript inventor, apparently has suggested type unions for 
> that language.  Some FP languages also have type systems that support type 
> unions.
> 
> http://wiki.ecmascript.org/doku.php?id=meetings:minutes_oct_13_2005
> 
> "Type Unions
> Brendan thinks we should have some ability for type unions in the language.
> 
> var x : Number | Null
> 
> It would be good to have something like this because otherwise developers end 
> up enforcing their invariants on their own, with the potential for buggy 
> code."
> 
> As always, thanks for all replies.
> 
> Justin Johansson
> 

What you want sounds a lot like a variant type. Check std.variant in phobos, it 
has a template to let you define custom variants through type tuples.