Nitpick on forum interface: searches bring up wrong post in unthreaded view.

[Chad J]

I just searched for something in the property discussion.  I wanted to 
link a friend to one of deadalnix's posts, so I put "I'm so happy to 
read this " into the search box and punched it.


What it brought up was one of David Nadlinger's posts.  I want to say 
nothing bad about David Nadlinger's fine post; it just wasn't what I was 
searching for.  Now, deadalnix's post was on the page, so it wasn't a 
complete miss, but I expected it to be the first one and it wasn't.  If 
I sent that link to my friend then he would probably read the wrong part.


There's more frustration in this though: partially because the result 
was unthreaded.  It's unthreaded, so I can't just click on the post I 
/actually/ want and get a new URL in my browser that I can copy-pasta. 
When I click on "View this thread in threaded view mode" it DOES NOT go 
to deadalnix's post as my search initially intended: it still selects 
David's post.  Now I have to scan through all of the 7625342 subtopics 
in the property discussion /by hand/ and find the one I'm interested in. 
 I eventually found it and sent the link.  Very time consuming. 
Thankfully it is fast.


Please fix?

I'm sorry if this is already a known bug or something.  I don't actually 
remember what the forum front-end is called or where I would go to 
properly find/add issues about it.

Re: @property - take it behind the woodshed and shoot it?

[Chad J]

On 01/24/2013 12:57 PM, Andrei Alexandrescu wrote:

On 1/24/13 8:41 AM, deadalnix wrote:
[snip]

Waiting for the shitstorm . . .


Nothing like that at least from me but I can plainly say this proposal
has merit but will never get implemented.

Andrei


Wait, so Walter would be willing to ice @property, but deprecating it in 
favor of this meritorious proposal will never happen?


I, for one, would love to see good ideas go into the language that I 
like to use.  I'd be willing to s/@property/@getter|@setter/ all of my 
code for this.


I wonder how hard it would be to write a tool that does the conversion 
and gets the disambiguation right in 90% of the cases...

Re: @property - take it behind the woodshed and shoot it?

[Chad J]

On 01/24/2013 03:45 PM, Jonathan M Davis wrote:

On Thursday, January 24, 2013 00:34:42 Walter Bright wrote:

This has turned into a monster. We've taken 2 or 3 wrong turns somewhere.

Perhaps we should revert to a simple set of rules.

1. Empty parens are optional. If there is an ambiguity with the return value
taking (), the () go on the return value.

2. the:
f = g
rewrite to:
f(g)
only happens if f is a function that only has overloads for () and (one
argument). No variadics.

3. Parens are required for calling delegates or function pointers.

4. No more @property.


Another issue to consider is enhancements such as

http://d.puremagic.com/issues/show_bug.cgi?id=8006

As a property function is supposed to be an abstraction for a variable, and
you really should be able to swap out property functions and public variables
without breaking code, things like

foo.prop += 5;

and

++foo.prop;

should work. Unfortunately, D's current property implementation is poor enough
that that code doesn't currently, but that can be fixed in a backwards-
compatible-manner.



I think what you want is semantic property rewriting.
Please see this:
http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Property#Semanticrewritingofproperties


But implementing something like that becomes more problematic without explicit
properties (potentially even dangerous, depending on what ends up happening
with those sorts of expressions with non-@property functions which are called
without parens). Maybe it could be done, but it seems to me that it's the sort
of thing that should be controlled. We don't want lots of voodoo happening
with functions called without parens, as we risk getting some nasty bugs when
using functions that weren't really meant to be properties. But we _do_ want
some of that voodoo with functions that are designed to be properties.

So, I really think that we need to have explicit properties, even if we allow
paran-less function calls in some other cases.

But honestly, I wish that we'd just gone with something closer to what C# did
with properties from the get-go.

- Jonathan M Davis

Re: @property - take it behind the woodshed and shoot it?

[Chad J]

On 01/24/2013 07:34 PM, Jesse Phillips wrote:

On Thursday, 24 January 2013 at 08:35:01 UTC, Walter Bright wrote:

1. Empty parens are optional. If there is an ambiguity with the return
value taking (), the () go on the return value.


As mentioned, this seems dangerous. I'd suggest requiring when there is
ambiguity. It still has generic problems, but it will never be silent.


4. No more @property.


I'm on the side that would miss optional parens if they died. In would
be nice if @property actually made a function behave like a field.

foo += bar;

It seems there are lots of complications around this? So yes kill
@property.



This has nothing to do with @property.  It has to do with defining what 
+= /does/ to a property.


I think that what you want is semantic property rewriting.

See this:
http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Property#Semanticrewritingofproperties


On another note,

To properly correct this situation we will break code. So before you get
started be sure to try out the new feature preview release approach :)

Re: @property - take it behind the woodshed and shoot it?

[Chad J]

On 01/24/2013 03:34 AM, Walter Bright wrote:

This has turned into a monster. We've taken 2 or 3 wrong turns somewhere.

Perhaps we should revert to a simple set of rules.

1. Empty parens are optional. If there is an ambiguity with the return
value taking (), the () go on the return value.

2. the:
f = g
rewrite to:
f(g)
only happens if f is a function that only has overloads for () and (one
argument). No variadics.

3. Parens are required for calling delegates or function pointers.

4. No more @property.


I somehow feel like someone read my article from a couple years ago 
(http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Property) and you 
implemented the less important part and consciously ignored the more 
important part.


The section titled "semantic rewriting of properties" is the important 
one.  Do that.  It removes a source of bugs and gives intuitive behavior 
to our property-functions (ex: prop++; works in general).


@property is considerably less meaningful: take it or leave it, I 
certainly won't care.  It's a bikeshed full of personal opinions.


If we had godly hindsight, then we would have made fields 
non-addressable by default so that people get safe design behavior by 
default: non-addressable fields can be turned into properties (assuming 
property rewriting exists) without breaking API, which creates closure 
for the whole concept.  There would, of course, have to be a way to make 
it possible to explicitly mark variables as addressable in cases where 
you need to make pointers to them.  That non-addressability semantic 
might have helped reference-counting too.  The mistake's made though; 
we'll probably just have to take that one in the gut.

Re: Ready for review: new std.uni

[Chad J]

On 01/13/2013 05:23 AM, Jonathan M Davis wrote:

On Sunday, January 13, 2013 04:58:16 Chad J wrote:

On 01/13/2013 02:28 AM, Jonathan M Davis wrote:

I really should ask Andrei why he made length require O(log n) instead
O(1)...

- Jonathan M Davis


Are there cases where it can't be O(1)?


Most definitely. Take a doubly linked list for example. Either length can be
O(1) and splicing is then O(n), or length is O(n) and splicing is then O(1).
That's because if you want to keep track of the length, you have to count the
number of elements being spliced in. For instance, it's a relatively common
mistake in C++ to use std::List's size function and compare it with 0 to see
whether the list is empty, because size is O(n). The correct thing to do is to
call empty and check its result.

You _could_ make std::list's size function be O(1), but that would mean that
splicing becomes O(n), which C++98 definitely did not do (though I hear that
C++11 made the interesting choice of changing how std::list works so that size
is O(1) and splicing is O(n); I don't know if that's good or not).

std.container.slist and std.container.dlist don't define length precisely
because it can't be O(1) given their design.

- Jonathan M Davis


Thanks!

That's a good example.  It made sense once I realized that you could be 
splicing in arbitrary sections of another list, not just an entire other 
list.


If I were to used cached lengths and splice in another list with a 
cached length, then I would just add the lengths of the two lists and do 
an O(1) splice.  I can see how this wouldn't cover all cases though.

Re: Ready for review: new std.uni

[Chad J]

On 01/14/2013 12:33 AM, David Nadlinger wrote:

On Monday, 14 January 2013 at 04:42:33 UTC, Andrei Alexandrescu wrote:

I'll candidly mention that David's request was unclear to me. It's
been winding enough that I missed the key sentence. Was it that we
switch dmd to using llvm as backend?


If you want, yes - but not in the form of an actionable proposal yet. I
was trying to argue that the benefits of using an existing solution like
GCC or LLVM are large enough (resp. the costs of using a custom backend
high enough) that we should seriously consider doing so. Especially
because it looks as if the amount of work needed to keep the DMD backend
and thus the reference D compiler competitive is going to increase
further as other backends are gaining things like auto-vectorization to
light up modern CPUs and ARM is gaining in importance.

David


"gaining" he says ;)

D just missed out on the leading edge of smartphone games.  That is a 
HUGE market packed with small developers that can easily adopt new 
tooling.  We got the invite and we stood them up.  IMO, sticking to an 
x86-centric toolset cost D one of its perfect opportunities for being a 
killer tool.  That makes me kinda sad.


Sorry for the downer.  I bring it up in the hope that we can learn from 
it.  I like to think that we'll see more opportunities in the future.

Re: Ready for review: new std.uni

[Chad J]

On 01/13/2013 02:28 AM, Jonathan M Davis wrote:


I really should ask Andrei why he made length require O(log n) instead O(1)...

- Jonathan M Davis


Are there cases where it can't be O(1)?

My current intuition is that length can always be cached.  If a 
container has a long length calculation, then it can be shortened by 
wrapping it in a proxy container that does nothing but expose the same 
operations as jackets around the original container.  These jackets 
forward all data in and out with no changes, except for one: they count 
the number of elements entering and leaving the underlying container and 
store this count in a length variable.

Re: [RFC] ColorD

[Chad J]

On 10/23/2012 04:42 PM, Jens Mueller wrote:

Jens Mueller wrote:

Chad J wrote:


That's a reasonable suggestion.  The only thing that can't be solved
is the trailing ) enclosing the text to be formatted.  That needs a
% before it to prevent ambiguity with parentheses in the text
itself.  So I could make your example:


writefln("The %c(red,white)(widgetometer%) is a device

formeasuring"); // for writing red on white

I was also considering the possibility of separating layout and
style by allowing some kind of style specification before the
printing, with a limited formatting spec for using the styles:

stdout.addTermTextStyle("id=myStyle, fg=red, bg=white, dark, underline");

writefln("The %c(myStyle)(widgetometer%) is a device for measuring");


Ideally, we get some users and ask them what they find easy to read. Or
look how other languages solve this.
Because I myself don't color my terminals that much I find it hard to
imagine.
I searched for some ruby libraries. I find ruby syntax often very easy.
We could use UFCS, e.g.
"some string".red
or
"some string".foreground(Color.red)

What do you think?
I find this way easier than format strings.


Just read the other post. This has the same problem.

Jens


Also I'm intentionally shooting for something very concise.  If 
verbosity conflicts, then it loses.  I say verbosity because I find that 
things get /less/ readable in situations like these if the syntax/naming 
is lengthy.  It causes alignment issues, text wrapping, noise, etc.


The thing I liked about the styling notion is that it allows things to 
be spelled out more, but places this noise outside of the string being 
formatted.


More thoughts:

// Do style parsing at compile-time if desired.
const myStyle = parseTermTextStyle("id=myStyle, fg=red, bg=white, dark, 
underline"


// At runtime, stdout is told to use this.
stdout.addTermTextStyle(myStyle);

// I'm thinking it might look better by dropping a pair of parens,
//   but using a . to make it clear where the formatter type ends
//   and the style id begins.
writefln("The %C.myStyle(widgetometer%) is a device for measuring");

// Overloaded for convenience.
stdout.addTermTextStyle("id=rw, fg=red, bg=white, dark, underline");

// The user can choose how verbose they want their formatter to look.
// This is a very concise one.
writefln("The %C.rw(widgetometer%) is a device for measuring");

// Other note: %C is necessary.  %c can't be used because that is
//   already used for formatting single characters.

Re: [RFC] ColorD

[Chad J]

On 10/23/2012 03:56 PM, Philippe Sigaud wrote:

writefln("The %c(red,white)(widgetometer%) is a device formeasuring"); //
for writing red on white


Would something like the following be possible?

// col is a string-accepting function that returns a correctly formatted string
// red and white are from a general Color enum
alias col!(Color.red, Color.white) rw;

writeln("The ", rw("widgetometer"), " is a device for measuring...");


Nope.  Windows requires function calls to do color formatting.  It does 
not use escape sequences.


That said, it is always possible to scan all text about to be sent off 
to Windows and look for escape sequences, then reinterpret them as 
WinAPI coloring calls.  The difficulty then is getting the "rw" 
construct above to know if it should emit escape sequences or not: the 
text it creates might eventually be bound for a file, a network socket, 
or some buffer in memory, but not a terminal.  If it's heading for a 
terminal it has to know which one because they might use different 
escape sequences.  So there always has to be some way of contextualizing 
the color formatting to its destination so that it can select the right 
form of output, including no formatting if its inappropriate for the 
destination.


Also, it doesn't nest.  It should be possible to push/pop terminal 
attributes in mid-string.

Re: [RFC] ColorD

[Chad J]

On 10/23/2012 03:51 AM, Jens Mueller wrote:

Chad J wrote:

On 10/22/2012 03:47 AM, Jens Mueller wrote:

Chad J wrote:

There is no weakness to this.  The only shred of a counterargument I
can think of is that it makes the format strings more difficult to
learn. Other than that, it is possible to detect the destination of
the formatter, so color codes will never end up in places where they
shouldn't.  A conservative approach to this should handle most
desires and never interfere with all the people with no interest in
color.

On the upshot are the things I've mentioned:
- A format specifier is potentially more discoverable.
- A format specifier is more concise.  This keeps your lines from
wrapping.  They are probably too long already.


Do you consider this
writecf(Color.red, "something %s", "here")
concise as well?



The case is too easy.  You're formatting an entire line.


Am I? I think it's not a line. But I see your point.
You mean something like
writec(Color.red, "red")
writec(Color.blue, "blue")
writec(Color.green, "green")
is too verbose.
You want something like
writef("%CFred(red%)%CFblue(blue%)%CFgreeng(reen%)");
Right?


- To cement the previous point: nesting requires a few extra
characters with a format specifier, rather than a couple extra
/lines/ for extra function calls.


Don't understand this point. Can you give an example?



Option A:

 auto save = getConsoleState();
 scope (exit) setConsoleState(save);
 setConsoleColors(Fg.red, Bg.blue);
 writeln("Red text on blue background.");

Option B:

 writefln("%CFredBblu(Red text on blue background.%)");


I see. Though I find the last line difficult to decipher (because there
are no spaces).


- Calls to stateful console functions allow people to write bugs
like saving console state and then forgetting to restore it (or
throwing an exception and neglecting to restore from within a scope
guard).  Format specifiers do not have this problem.


The same holds for
writecf(Color.red, "something %s", "here")



See my above example.  In that case the formatter no longer requires
even using the scope feature because there are no resources to clean
up.  The library handles that mess.

Also statefulness is a pain to deal with.  Stack-like operation with
push/pop or bracketing constructs is usually much less troublesome
for this sort of thing.


It'll be nice then if you can built something using format specifiers on
top of a basic library.


- etc (I'm sure I'm forgetting one or two.)

These are the reasons why my ideal language has color formatting
built into its I/O routines.


Just wanted to point out that instead of that you can add writec*
functions. I think the only thing is that these are less discoverable
but they also work without formatting, e.g.
writec(Color.red, "my text");



The thing I found very difficult with other color formatting APIs
was formatting individual words or characters.  Entire lines are
easy-peasy stuff in any API.  Solving the entire-lines case won't
impress me. ;)


I see your point now. But we should keep it simple.


Here's my more typical use case:

writefln("The %CFred(widgetometer%) is a device for measuring");
writefln("widget effectiveness.  It is possible to ");
writefln("reconcile transcendental properties by hitting");
writefln("%CFblu(B%) for blue coefficients, %CFgrn(G%) for green");
writefln("coefficients, or %CFyel(Y%) for yellow coefficients.");
writefln("Here is a correspondence table:");
writefln("  %CFnue( %) | %CFblu( B %) %CFgrn( G %) %CFyel( Y %) ");
writefln("  %CFnue(-%)-+-%CFnue(---%)-%CFnue(---%)-%CFnue(---%)-");
writefln("  %CFblu(B%) | %CFblu(200%) %CFwht(330%) %CFwht(303%) ");
writefln("  %CFgrn(G%) | %CFwht(110%) %CFgrn(020%) %CFwht(033%) ");
writefln("  %CFyel(Y%) | %CFwht(101%) %CFwht(011%) %CFyel(002%) ");

I realized that I wanted a "nue" color that has no effect but allows
me to align things effectively ;)

Anyhow, please try to write the above example using any other style.
Interleaved function calls are particularly "fun"


I'm convinced. But I find that it difficult to read. Though that's a
problem I usually have with format strings. Can these format strings be
made easier to read. I mean

writefln("The %CF(red)(widgetometer) is a device for measuring");
or
writefln("The %c(red,white)(widgetometer) is a device for measuring"); // for 
writing red on white

is already easier to my eyes.
I'd be happy to see it built on top.

Jens


That's a reasonable suggestion.  The only thing that can't be solved is 
the trailing ) enclosing the text to be formatted.  That needs a % 
before it to prevent ambiguity with parentheses in the text 

Re: [RFC] ColorD

[Chad J]

On 10/22/2012 03:47 AM, Jens Mueller wrote:

Chad J wrote:

There is no weakness to this.  The only shred of a counterargument I
can think of is that it makes the format strings more difficult to
learn. Other than that, it is possible to detect the destination of
the formatter, so color codes will never end up in places where they
shouldn't.  A conservative approach to this should handle most
desires and never interfere with all the people with no interest in
color.

On the upshot are the things I've mentioned:
- A format specifier is potentially more discoverable.
- A format specifier is more concise.  This keeps your lines from
wrapping.  They are probably too long already.


Do you consider this
writecf(Color.red, "something %s", "here")
concise as well?



The case is too easy.  You're formatting an entire line.


- To cement the previous point: nesting requires a few extra
characters with a format specifier, rather than a couple extra
/lines/ for extra function calls.


Don't understand this point. Can you give an example?



Option A:

auto save = getConsoleState();
scope (exit) setConsoleState(save);
setConsoleColors(Fg.red, Bg.blue);
writeln("Red text on blue background.");

Option B:

writefln("%CFredBblu(Red text on blue background.%)");


- Calls to stateful console functions allow people to write bugs
like saving console state and then forgetting to restore it (or
throwing an exception and neglecting to restore from within a scope
guard).  Format specifiers do not have this problem.


The same holds for
writecf(Color.red, "something %s", "here")



See my above example.  In that case the formatter no longer requires 
even using the scope feature because there are no resources to clean up. 
 The library handles that mess.


Also statefulness is a pain to deal with.  Stack-like operation with 
push/pop or bracketing constructs is usually much less troublesome for 
this sort of thing.



- etc (I'm sure I'm forgetting one or two.)

These are the reasons why my ideal language has color formatting
built into its I/O routines.


Just wanted to point out that instead of that you can add writec*
functions. I think the only thing is that these are less discoverable
but they also work without formatting, e.g.
writec(Color.red, "my text");



The thing I found very difficult with other color formatting APIs was 
formatting individual words or characters.  Entire lines are easy-peasy 
stuff in any API.  Solving the entire-lines case won't impress me. ;)


Here's my more typical use case:

writefln("The %CFred(widgetometer%) is a device for measuring");
writefln("widget effectiveness.  It is possible to ");
writefln("reconcile transcendental properties by hitting");
writefln("%CFblu(B%) for blue coefficients, %CFgrn(G%) for green");
writefln("coefficients, or %CFyel(Y%) for yellow coefficients.");
writefln("Here is a correspondence table:");
writefln("  %CFnue( %) | %CFblu( B %) %CFgrn( G %) %CFyel( Y %) ");
writefln("  %CFnue(-%)-+-%CFnue(---%)-%CFnue(---%)-%CFnue(---%)-");
writefln("  %CFblu(B%) | %CFblu(200%) %CFwht(330%) %CFwht(303%) ");
writefln("  %CFgrn(G%) | %CFwht(110%) %CFgrn(020%) %CFwht(033%) ");
writefln("  %CFyel(Y%) | %CFwht(101%) %CFwht(011%) %CFyel(002%) ");

I realized that I wanted a "nue" color that has no effect but allows me 
to align things effectively ;)


Anyhow, please try to write the above example using any other style. 
Interleaved function calls are particularly "fun" 

Re: [RFC] ColorD

[Chad J]

On 10/21/2012 06:55 PM, Jens Mueller wrote:

Chad J wrote:

On 10/21/2012 06:11 PM, Jens Mueller wrote:

Chad J wrote:

On 10/21/2012 05:01 PM, Jens Mueller wrote:

It seems to have a hard ncurses/termcap/etc dependency.


Yes. I think you cannot make it portable without. Please proof me wrong
and I'll fix this.



Well, traditionally it's done with automake/autoconf.  You'd end up
with preprocessor defines that tell you whether the lib has been
statically linked or not.  This isn't available here because Phobos
doesn't use these as a build system and I hope it never does.


I mean to detect if your terminal is ANSI compatible without adding
another dependency.
It's easy to provide different version(...) to support different modes.
One could do something like:
1. Check at run time if ncurses etc. are available.
   * If they are use them.
   * Otherwise fall back to ANSI codes or throw an Exception.

What do you think?



I completely agree.

The difficulty I encountered is actually /doing/ the runtime detection. 
 Does Phobos have a way to dynamically link .so files yet?


If yes, then we could search the obvious places ("/lib/libncurses.so.5" 
matches on my machine right now) and link to any files found.


Since color is probably never necessary for program correctness, I think 
it is acceptable to ignore color formatting and produce plain text when 
detection fails.  It would make sense to make this configurable though: 
the Terminal object could have a .throwOnDetectionFailure flag that can 
be set if people want to be a bit more hardcore about it.



I'll admit when I started trying to work on doing this thing, I
never got anything onto the screen.  What stopped me was that I
couldn't figure out how to detect ncurses/termcap/etc.  I was going
to shoot for Phobos inclusion and making Phobos always link with
ncurses seems like a bad idea.


Dependence on Phobos is bad. If you can detect whether a terminal is
ANSI compatible then this mode should be default. But I don't know how
to detect this.



Wrong direction on the dependency.  I wouldn't expect Terminal
coloring/detection to rely on Phobos.  I'd expect it to be one of
the lower-level modules built into Phobos.


I mean it's bad to have Phobos depend on ncurses.
Though one can go with loading at run time.



Yes.


Ultimately I expect it to work with writeln or writefln to make it
discoverable and easy to work with.


One could try this. At least for Linux. You just have to add the
appropriate escape sequences. But this won't work on Windows.



I remember having a plan for this.  See below.


Back then I did design a format spec for introducing colors into
format strings:
www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html


I doubt that the Phobos maintainers will accept this. This is very
invasive.


Hmmm, depends what is meant by invasive.

I feel it's the only way to have discoverable and concise syntax.
I'd be pretty disappointed if they didn't, regardless of who submits
the pull request.

I remember it being possible in Phobos to determine the destination
of the format operation.  If the destination is a string in memory,
then no color formatting would be applied.  If the destination is a
Linux terminal of some kind, then some ncurses terminal info would
be looked up (possible a cached lookup) and escape sequences
generated based on that.  If the destination is a Windows terminal,
then these approaches can be considered:
(1) Split the formatted text up on the color format boundaries.
Send the slices into the stream one by one, calling the necessary
WinAPI color formatting functions inbetween.  I think this might not
have been possible with Phobos' architecture.
(2) Insert ANSI escape sequences into the text.  The I/O code for
Windows would then have to intercept these and convert them into the
appropriate WinAPI calls.  I think this was possible, and even
distinguishable from the case of writing to a string in memory.

If the invasiveness worry comes from the possibility of dumping
escape sequences into non-terminal destinations, then I hope the
above wall of text can alleviate that concern.


Checking whether something is a terminal can be done using isatty on the
file handle. I think this will work.
But it is invasive because you want to add it to the formatting spec. Is
this the usual way it is done? I don't know how it is done in Python
or other languages.



Is it relevant?  I posit that having format syntax is simply better than 
not.  This is on an objective basis.


There is no weakness to this.  The only shred of a counterargument I can 
think of is that it makes the format strings more difficult to learn. 
Other than that, it is possible to detect the destination of the 
formatter, so color codes will never end up in places where they 
shouldn't.  A conservative approach to this should handle most desires 
and never in

Re: Why does std.string.splitLines return an array?

[Chad J]

On 10/21/2012 06:35 PM, Jonathan M Davis wrote:

On Sun, 2012-10-21 at 18:00 -0400, Chad J wrote:

std.string.splitLines returns an array, which is pretty grody.  Why not
return a lazily-evaluated range struct so that we can avoid allocations
on this simple but common operation?


If you want a lazy range, then use std.algorithm.splitter. std.string
operates on and returns strings, not general ranges.

- Jonathan M Davis



std.algorithm.splitter is simply not acceptable for this.  It doesn't 
have this kind of logic:


bool matchLineEnd( string text, size_t pos )
{
if ( pos+1 < text.length
  && text[pos] == '\r'
  && text[pos+1] == '\n' )
return true;
else if ( pos < text.length
  && (text[pos] == '\r' || text[pos] == '\n') )
return true;
else
return false;
}

I've never used std.algorithm.splitter for line splitting, despite 
trying.  It's always more effective to write your own.


I'm with bearophile on this one:
http://d.puremagic.com/issues/show_bug.cgi?id=4764

I think his suggestions about naming also just make *sense*.  I'm not 
sure how practical some of those naming changes would be if there is a 
lot of wild D2 code that uses the current weirdly-named stuff that 
emphasizes eager evaluation and extraneous allocations.  I'm not sure 
how necessary it is to even /have/ functions that return arrays when 
there are lazy versions: the result of a lazy function can always be fed 
to std.array.array(range).  Heh, even parentheses nesting is nicely 
handled by UFCS now.

Re: [RFC] ColorD

[Chad J]

Additionally note that the format syntax handles Walter's concerns here:
http://forum.dlang.org/post/k61t63$pi4$1...@digitalmars.com

The color format syntax uses a pair of matched parentheses, and thus 
makes it impossible to leave the console in a different state than when 
the formatting call was entered.


Lower level stuff like terminal functions dedicated specifically to 
saving/restoring state would make sense and be used underneath the 
formatter to accomplish what it does.

Re: [RFC] ColorD

[Chad J]

On 10/21/2012 06:11 PM, Jens Mueller wrote:

Chad J wrote:

On 10/21/2012 05:01 PM, Jens Mueller wrote:

It seems to have a hard ncurses/termcap/etc dependency.


Yes. I think you cannot make it portable without. Please proof me wrong
and I'll fix this.



Well, traditionally it's done with automake/autoconf.  You'd end up with 
preprocessor defines that tell you whether the lib has been statically 
linked or not.  This isn't available here because Phobos doesn't use 
these as a build system and I hope it never does.



I'll admit when I started trying to work on doing this thing, I
never got anything onto the screen.  What stopped me was that I
couldn't figure out how to detect ncurses/termcap/etc.  I was going
to shoot for Phobos inclusion and making Phobos always link with
ncurses seems like a bad idea.


Dependence on Phobos is bad. If you can detect whether a terminal is
ANSI compatible then this mode should be default. But I don't know how
to detect this.



Wrong direction on the dependency.  I wouldn't expect Terminal 
coloring/detection to rely on Phobos.  I'd expect it to be one of the 
lower-level modules built into Phobos.



Ultimately I expect it to work with writeln or writefln to make it
discoverable and easy to work with.


One could try this. At least for Linux. You just have to add the
appropriate escape sequences. But this won't work on Windows.



I remember having a plan for this.  See below.


Back then I did design a format spec for introducing colors into
format strings:
www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html


I doubt that the Phobos maintainers will accept this. This is very
invasive.


Hmmm, depends what is meant by invasive.

I feel it's the only way to have discoverable and concise syntax.  I'd 
be pretty disappointed if they didn't, regardless of who submits the 
pull request.


I remember it being possible in Phobos to determine the destination of 
the format operation.  If the destination is a string in memory, then no 
color formatting would be applied.  If the destination is a Linux 
terminal of some kind, then some ncurses terminal info would be looked 
up (possible a cached lookup) and escape sequences generated based on 
that.  If the destination is a Windows terminal, then these approaches 
can be considered:
(1) Split the formatted text up on the color format boundaries.  Send 
the slices into the stream one by one, calling the necessary WinAPI 
color formatting functions inbetween.  I think this might not have been 
possible with Phobos' architecture.
(2) Insert ANSI escape sequences into the text.  The I/O code for 
Windows would then have to intercept these and convert them into the 
appropriate WinAPI calls.  I think this was possible, and even 
distinguishable from the case of writing to a string in memory.


If the invasiveness worry comes from the possibility of dumping escape 
sequences into non-terminal destinations, then I hope the above wall of 
text can alleviate that concern.



I added writecf, writec, etc. with additional arguments.
writec(Color.red, "some text")
or
writecf(Color.red, "%s", "some text")
This is fine I think. But better options may be worth investigating.

Jens


I really think this should be in Phobos.  If it doesn't go into Phobos, 
then people will write crappy terminal apps with no color.  If it does 
go into Phobos, then the better devs will see the opportunity and use 
it.  Something 3rd party is much less discoverable and won't have nearly 
as much impact.  The use case is almost all CLI apps, so it's not like 
an uncommon corner-case or something.


I run a Gentoo system where things are configured to use color output 
wherever possible.  The portage devs went through all of the necessary 
contortions to get Python to output colored text, somehow.  I feel the 
end result is indispensable.  Color is an extremely useful tool for 
making sure that the user doesn't overlook important bits while scanning 
text.  Outside of Gentoo, I find this most notable in grep: uncolored 
grep output is just awful, but the coloring makes it possible to easily 
identify why the regular expression behaved the way it did.


I look forward to a better CLI ecosystem where highly reliable D 
programs are written quickly and write beautiful colored output ;)

Why does std.string.splitLines return an array?

[Chad J]

std.string.splitLines returns an array, which is pretty grody.  Why not 
return a lazily-evaluated range struct so that we can avoid allocations 
on this simple but common operation?

Re: [RFC] ColorD

[Chad J]

On 10/21/2012 05:01 PM, Jens Mueller wrote:

Robik wrote:

Hello,

I would like to introduce ColorD, small library that allows to
simply manipulate console output colors, both on Windows and Posix
operating systems. It also supports font styles such as underline
and strikethrough(Posix feature only).


Simple example:

import std.stdio, colord;
void main()
{
 setConsoleColors(Fg.red, Bg.blue);
 writeln("Red text on blue background.");
 resetConsoleColors(); // Bring back initial state
}


Feedback welcome.

GitHub: https://github.com/robik/ColorD


Interesting looks solid to me.
Some nit-picks:
* Coloring on Posix depends a ANSI terminal. Can you check that a
   terminal is ANSI compatible?
* There are some magic numbers in the code. These may be difficult to
   figure out for maintaining.
* enum Color should maybe be same on all systems.
   This is a rather small issue. But I may transfer the Color to another
   system. So if it is possible there should only be one enum Color for
   all systems.
* Call it terminal or similar. Because other terminal related stuff can
   be added and IMHO it's a better name.

I have written a similar library. Not finished. Let's join forces.
https://github.com/jkm/terminal

Johannes Pfau has written a progress bar. I will add this.

Jens


Hey that looks cool.

It seems to have a hard ncurses/termcap/etc dependency.

I'll admit when I started trying to work on doing this thing, I never 
got anything onto the screen.  What stopped me was that I couldn't 
figure out how to detect ncurses/termcap/etc.  I was going to shoot for 
Phobos inclusion and making Phobos always link with ncurses seems like a 
bad idea.


Ultimately I expect it to work with writeln or writefln to make it 
discoverable and easy to work with.


Back then I did design a format spec for introducing colors into format 
strings:

www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html

Re: DIP20: Volatile read/write intrinsics

[Chad J]

On 10/11/2012 10:18 PM, Alex Rønne Petersen wrote:


Oops. It should be clearer now.



It is!

Re: DIP20: Volatile read/write intrinsics

[Chad J]

On 10/11/2012 10:35 AM, Alex Rønne Petersen wrote:

On 11-10-2012 14:42, Chad J wrote:

On 10/11/2012 01:40 AM, Alex Rønne Petersen wrote:


I suppose a simple D_Volatile version identifier will do, like we have
D_SIMD for core.simd.__simd.



Cool.


OK, updated.



I do have a critique for your description:


Compilers that support these intrinsics must define the D_Volatile version 
identifier. Compilers that do not may support them, but programmers should not 
rely on it.


I read this as meaning:
Compilers that support volatile intrinsics must define D_Volatile.
Compilers can support volatile intrinsics without defining D_Volatile.

And I see a contradiction ;)

Re: DIP20: Volatile read/write intrinsics

[Chad J]

On 10/11/2012 01:40 AM, Alex Rønne Petersen wrote:


I suppose a simple D_Volatile version identifier will do, like we have
D_SIMD for core.simd.__simd.



Cool.

Re: DIP20: Volatile read/write intrinsics

[Chad J]

On 10/10/2012 02:31 PM, Alex Rønne Petersen wrote:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP20

This supersedes DIP17.

The primary reason to use intrinsics instead of something built into the
language is that the latter is too complex for current (and likely also
future) compiler implementations.

Destroy!



Since this can affect semantics but might not always be available, is 
there any chance you could add a feature detection intrinsic for this? 
Maybe a compile-time "haveVolatileOps" identifier would work.


Volatility might be absent if:
- The backend target can't forward the volatility guarantees, ex: a D 
compiler that emits JavaScript code.

- The compiler has not implemented volatile operations yet.

Thoughts on tuple indexing syntax...

[Chad J]

From what I read on the "DIP19: Remove comma operator..." thread it 
sounded like one of the difficult challenges for tuples is ambiguities 
created by indexing tuple elements that might also be random access 
ranges (arrays).  It gets especially nasty for single-element tuples.


Would it help if we used something /besides/ the normal index notation 
for this?


My immediate thoughts were something like:
(int,float) a;
a[[0]] = 1;
a[[1]] = 1.0;

But the idea of introducing the [[ and ]] tokens is probably not going 
to cut it.


Next I thought of this:
(int[]) a;
a.[0] = new int[10];
a[0] = 1;
assert(a.[0][0] == 1);

Thus a trailing dot in paths with brackets would indicate tuple 
dereferencing.


Another thought was to use a trailing @ sign for this:
(int[]) a;
a@[0] = new int[10];
a[0] = 1;
assert(a@[0][0] == 1);

In fact, I wonder if the extra brackets are necessary at all:

(int[]) a;
a.0 = new int[10];
a[0] = 1;
assert(a.0[0] == 1);

(int[]) a;
a@0 = new int[10];
a[0] = 1;
assert(a@0[0] == 1);

Assuming I haven't missed some bad interactions with other syntax, then 
these notations should make tuple element access unambiguous.


Does this help?

Re: References in D

[Chad J]

On 10/07/2012 02:22 AM, David Piepgrass wrote:

void main()
{
void* x = a(b());
c();
while(goobledegook)
{
x = p();
d(x);
}
e(x); /+ Crash! x is null. +/
}

Where did x's null value come from? Not a. Not p; the while loop
happened to be never executed. To say "b" would be closer, but still
imprecise. Actually it was created in the q() function that was called
by u() that was called by b() which then created a class that held the
null value and was passed to a() that then dereferenced the class and
returned the value stored in the class that happened to be null. nulls
create very non-local bugs, and that's why they frustrate me to no end
sometimes.


Since this thread's attracted lots of commotion I thought I'd just drop
by and +1 for non-nullable types, and +1 for your arguments.

I keep wondering, though, if it is 'enough' to solve the null problem,
or if it would be possible to devise a more general mechanism for
solving other problems too, like say, the fact that certain integers
have to always be positive, or if you want to go more general, that a
certain relationship must hold between two structures...



I agree.


Not having used D's invariants so far (well, I haven't used D itself for
a real project actually)... what's stopping D's invariant mechanism from
handling all this?

http://dlang.org/class.html#invariants (as is typical of D
documentation, this says nothing about invariants on structs, but the
page about structs says that they support invariants with an X.)

I mean, problems are detected at runtime this way, and slightly too
late, but still, it would be better than most popular languages that
can't do anything about nulls at all. Since D's devs don't even seem to
have enough time to implement D as described in TDPL (published more
than two years ago), I wouldn't expect to see this feature in the D
language in the near future.


Invariants might work... create a proxy struct and then have assignment 
always check the invariant.  I don't like the idea that they get removed 
during release mode.  I'd like to be able to control which checks I pay 
for.  I think this might just mean that the important thing is 
overloading assignment to do checks, which could be done in principle, 
and this could work without invariants and thus give the desired control.


I just haven't had much luck creating proxy types in the past.  As of 
some months ago, D just wasn't there yet.  :(


In another post in this thread I mentioned something similar:



It would be cool to have templates like this:

51.  bool isPrime(int val)
52.  {
53.  ...
54.  }

101. Watch!(int,&isPrime) primeNumber = primeGenerator();
102. primeNumber = 8;
103. doStuff();
104. checkPrime(primeNumber); /+ Crash! +/
Error: checkPrime(primeNumber): primeNumber is not prime.
primeNumber: isPrime returned false after assignment at
  (foobar.d, line 102)

~or~

101. Constrain!(int,&isPrime) primeNumber = primeGenerator();
102. primeNumber = 8; /+ Crash! +/
103. doStuff();
104. checkPrime(primeNumber);
foobar.d, line 102: isPrime returned false after assignment.

For convenience one could define this:
alias Constrain!(int,&isPrime) PrimeInt;


Maybe these could be turned on/off in release mode on a case-by-case 
basis.  I would probably leave the checks and tracking in always, with 
the one exception of code that has been shown (with profiling) to need 
the extra performance.

Re: References in D

[Chad J]

On 10/06/2012 04:18 AM, "Franciszek Czekała" " wrote:

B) The description of a() says the return value cannot be null. Then a()
should check its return value before returning or make otherwise sure it
is not null. If it returns null it is a bug. One of the infinite number
of possible bugs that can happen. Again it is not the problem of the
language. The problem of divergence of specification and code is a human
problem that cannot be solved formally. Insistance on formal tools is a
misunderstanding that leads to design bloat and eventually failure (Ada).

D competes directly with C++ as Ada did before. Ada drowned under the
weight of its "safety" and so will D if it goes the same route. The only
thing needed now are mature compilers and good systems API integration.
If anything I would rather consider removing features from the language
than adding them.




I have another thing to bring up: why the hating on Ada?

Because, if you're hating on Ada because it requires a bunch of extra 
boilerplate and verbosity that most programmers would find unnecessary, 
then I will happily join you in hating on Ada (and Pascal and the like). 
 Keep in mind that one of D's current idiomatic objectives seems to be 
the elimination of as much boilerplate as possible.


I firmly believe that safety and brevity are NOT exclusive to each 
other.  Moreover, they even synergize well if the language and tools are 
designed right: verbose code will be less readable and therefore more 
prone to error.

Re: References in D

[Chad J]

On 10/06/2012 04:18 AM, "Franciszek Czekała" " wrote:

On Saturday, 6 October 2012 at 04:10:28 UTC, Chad J wrote:


I find this to be very suboptimal at the least.

This prevents null values from traveling "up" the stack, but still
allows them to move "down" (as return values) and allows them to
survive multiple unrelated function calls.

It catches null values once they've already ended up in a place they
shouldn't be. Too late.

Nulls can also be placed into variables within structs or classes that
then get passed around. Checking for those can require some complex
traversal: impractical for casual one-off checks at the start of a
function in some cases.

void main()
{
void* x = a(b());
c();
while(goobledegook)
{
x = p();
d(x);
}
e(x); /+ Crash! x is null. +/
}

Where did x's null value come from? Not a. Not p; the while loop
happened to be never executed. To say "b" would be closer, but still
imprecise. Actually it was created in the q() function that was called
by u() that was called by b() which then created a class that held the
null value and was passed to a() that then dereferenced the class and
returned the value stored in the class that happened to be null. nulls
create very non-local bugs, and that's why they frustrate me to no end
sometimes.

What I really want to know is where errant null values come FROM.

I also want to know this /at compile time/, because debugging run-time
errors is time consuming and debugging compile-time errors is not.

The above example could yield the unchecked null assignment at compile
time if all of the types involved were typed as non-nullable, except
for the very bare minimum that needs to be nullable. If something is
explicitly nullable, then its enclosing function/class is responsible
for handling null conditions before passing it into non-nullable space.
If a function/class with nullable state tries to pass a null value
into non-nullable space, then it is a bug. This contains the
non-locality of null values as much as is reasonably possible.

Additionally, it might be nice to have a runtime nullable type that
uses its object file's debugging information to remember which
file/function/line that its null value originated from (if it happens
to be null at all). This would make for some even better diagnostics
when code that HAS to deal with null values eventually breaks and
needs to dump a stack trace on some poor unsuspecting sap (that isn't
me) or ideally sends an email to the responsible staff (which is me).



returned the value stored in the class that happened to be null.


Happened? "I was driving carefully and then it happened I drove into the
tree, officer." Every function should define its interface, its contract
with the outside world. If a() function returns a pointer it is a part
of the contract whether it can be null. Two possibilities:

A) The contract says it can be null. Then it is your duty to check for
null. Period. Learn to read the signs before you start driving. You
assinged the value without checking, it is your fault, not a()'s, not
the language's.



I am unconvinced by the driving analogy.  When driving, most of the 
important bits become muscle memory (acceleration, braking, turn 
signals, etc) and the rest falls under the category of "be aware".  The 
factor in our advantage is that awareness in driving usually only 
requires you to focus on one thing at a time: "turn your head before 
changing lanes or turning", "look at the sides of the road", "check your 
rear view", etc.


Programming involves the management of complex logical relationships. 
It is more akin to mathematics.  I could continue, but I'll stop here 
and leave it at "I'm unconvinced".


Even if I grant the premise, I'll expand on what Timon wrote:
We'd have a lot less accidents if well-designed robots drove our 
vehicles for us (with manual overrides, of course).



B) The description of a() says the return value cannot be null. Then a()
should check its return value before returning or make otherwise sure it
is not null. If it returns null it is a bug. One of the infinite number
of possible bugs that can happen. Again it is not the problem of the
language. The problem of divergence of specification and code is a human
problem that cannot be solved formally. Insistance on formal tools is a
misunderstanding that leads to design bloat and eventually failure (Ada).



As I understand it, you would have written my code snippet this way:

void main()
{
MyType j = b();
assert( j !is null );
assert( j.qux !is null );
assert( j.qux.yarly !is null ); /+ Crash! yarly is null. +/

void* x = a(j);
assert( x !is null );

c();
while(goobledegook)
{
x = p();
assert(x !is null);

d(x);

// Note: be sure to put this one in!
//   The previous dev f

Re: DIP19: Remove comma operator from D and provision better syntactic support for tuples

[Chad J]

On 09/23/2012 04:40 PM, Andrei Alexandrescu wrote:

I discussed this with Walter, and we concluded that we could deprecate
the comma operator if it helps tuples. So I started with this:

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP19

Unfortunately, I started much cockier than I ended. The analysis in
there fails to construct a case even half strong that deprecating the
comma operator could significantly help tuples. Well it essentially
concludes that tuples are mostly fine as they are, and attempts to
embellish them syntactically are marred with unexpected problems.
Nevertheless, I sure have missed aspects all over, so contributions are
appreciated.


Thanks,

Andrei


Might some of the ambiguity be reduced by defining special tokens for 
dereferencing tuple elements?


(int[]) a = ([1,2,3]);
assert(a[0]   == 1);
assert(a[[0]] == [1,2,3]);

I suspect that this would allow us to indulge in the attractive notion 
of single-element tuples.


Would the addition of such a token be considered heresy?

Re: DIP19: Remove comma operator from D and provision better syntactic support for tuples

[Chad J]

On 09/24/2012 11:55 AM, Caligo wrote:

If tuples are ever introduced, I hope parentheses will not be used.

I would prefer something like this:

tuple<2,1,8>


Not using parentheses: a possibly valid idea!

Using angle brackets: never going to happen.

People HATE angle brackets.  There is extremely good justification for 
this hatred, because C++ already stepped on that landmine and suffered 
dearly for it.


If you were to use angle brackets, then I would try to do this:
tupleb,c> d
Which way should this be parsed?
It might be a value tuple of type 
or it might be a declaration: d is of type 

It is very easy to create cases with angle brackets that are 
syntactically ambiguous and make it impossible to parse code as a 
context-free-grammar.  It is harder, but probably possible, to create 
cases where such a syntax is also completely ambiguous semantically too.

Re: References in D

[Chad J]

On 10/03/2012 01:31 PM, "Franciszek Czekała" " wrote:

On Wednesday, 3 October 2012 at 16:33:15 UTC, Simen Kjaeraas wrote:

On 2012-10-03, 18:12, wrote:




They make sure you never pass null to a function that doesn't expect
null - I'd say that's a nice advantage.


No, it is meaningless. If you have a class which is supposed to hold a
prime number and you pass it to a function are you going to check each
time that the value is indeed prime? That would kill the efficiency of
your program guaranteed. So you would be happy to know that the
reference is non-null but you would take it for granted the value is
indeed prime? Does it make any sense?
I maintain that this non-null "advantage" does not warrant to make the
language more complicated even by a tiny bit. It is dwarfed by normal
considerations related to program correctness.



It would be cool to have templates like this:

51.  bool isPrime(int val)
52.  {
53. ...
54.  }

101. Watch!(int,&isPrime) primeNumber = primeGenerator();
102. primeNumber = 8;
103. doStuff();
104. checkPrime(primeNumber); /+ Crash! +/
Error: checkPrime(primeNumber): primeNumber is not prime.
primeNumber: isPrime returned false after assignment at
  (foobar.d, line 102)

~or~

101. Constrain!(int,&isPrime) primeNumber = primeGenerator();
102. primeNumber = 8; /+ Crash! +/
103. doStuff();
104. checkPrime(primeNumber);
foobar.d, line 102: isPrime returned false after assignment.

For convenience one could define this:
alias Constrain!(int,&isPrime) PrimeInt;

I think this would be sufficient for the cases that are considerable 
less common than errant null references.  I do think these capabilities 
should exist.  Assumptions suck: allowing invalid data to propogate in 
non-local ways is BAD.



With default null references:
A)either null is an expected non-value for the type (like in the chess
example), checking for it is part of normal processing then
B) or null is not a valid value, then there is no need to check for it.
If you get a null reference it is a bug. It is like getting a 15 for
your prime number. You do not put checks like that in your code. You
test your prime generation routine not the consumers. If your function
gets a null reference when it should not, some other part of your
program is buggy. You do not process bugs in your code - you remove them
from it.



Please explain how a program printing
Segmentation fault
tells me where the null came from?

If I'm lucky, I even get a stack trace, which is still not good enough 
in the cases that are actually non-trivial to solve.


My problem with using nullable values /everywhere/ is that they make it 
very difficult to determine exactly what code is /producing/ the null. 
If there's no stack trace information then it isn't even possible to 
know where the consumer is in a lot of cases.



However with D, dereferencing an uninitialized reference is well defined
- null is not random data: you get a well-defined exception and you know
you are dealing with unitialized data. This is easy to fix. You just go
up the stack and check where the reference comes from. Much easier
probably than finding out why your prime numbers turn out to be
divisible by 3. How about introducing some syntax that will rule this out?



You... you... you /JUST/ go up the stack.  Nope.

I've been betrayed by this approach many times in the past.  Can you 
tell? ;)


Copy-pasta from my previous post:

void main()
{
void* x = a(b());
c();
while(goobledegook)
{
x = p();
d(x);
}
e(x); /+ Crash! x is null. +/
}

Where did x's null value come from?  Not a. Not p; the while loop 
happened to be never executed.  To say "b" would be closer, but still 
imprecise.  Actually it was created in the q() function that was called 
by u() that was called by b() which then created a class that held the 
null value and was passed to a() that then dereferenced the class and 
returned the value stored in the class that happened to be null.  nulls 
create very non-local bugs, and that's why they frustrate me to no end 
sometimes.


(end of copy-pasta)


To quote (loosely) Mr. Walter Bright from another discussion: how many
current bugs in dmd are related to default null references?


I don't know, but when I was trying to debug a certain dmd bug that was 
blocking my progress.  It took a day of figuring out what was creating 
the damn null values.  That could have been much faster.  The bug was 
not caused by null values, but they severely complicated debugging.  So 
no, I don't want to do that work.  I want the compiler to do it for me. 
 (I gave up on fixing that bug because it required rewriting 
non-trivial portions of dmd.)


--

If you have not experienced these frustrations, then you are very 
fortunate.  Please spread your fortune and be more understanding of the 
problems faced by others.

Re: References in D

[Chad J]

On 10/05/2012 08:31 AM, Regan Heath wrote:

On Fri, 05 Oct 2012 05:19:13 +0100, Alex Burton
 wrote:


On Saturday, 15 September 2012 at 17:51:39 UTC, Jonathan M Davis wrote:

On Saturday, September 15, 2012 19:35:44 Alex Rønne Petersen wrote:

Out of curiosity: Why? How often does your code actually accept null as
a valid state of a class reference?


I have no idea. I know that it's a non-negligible amount of the time,
though
it's certainly true that they normally have values. But null is how you
indicate that a reference has no value. The same goes for arrays and
pointers.
Sometimes it's useful to have null and sometimes it's useful to know
that a
value can't be null. I confess though that I find it very surprising
how much
some people push for non-nullable references, since I've never really
found
null to be a problem. Sure, once in a while, you get a null
pointer/reference
and something blows up, but that's very rare in my experience, so I
can't help
but think that people who hit issues with null pointers on a regular
basis are
doing something wrong.

- Jonathan M Davis


In my experience this sort of attutide is not workable in projects
with more than one developer.


Almost all my work is on projects with multiple developers in C/C++ and
making extensive use of null.


It all works OK if everyone knows the 'rules' about when to check for
null and when not to.


As every good C/C++ developer does. The rule is simple, always check for
nulls on input passed to "public" functions/methods. What you do with
internal protected and private functions and methods is up to you (I use
assert).


Telling team members that find bugs caused by your null references
that they are doing it wrong and next time should check for null is a
poor substitute for having the language define the rules.


Having language defined rules is a nice added /bonus/ it doesn't let you
off the hook when it comes to being "null safe" in your code.


A defensive attitude of checking for null everywhere like I have seen
in many C++ projects makes the code ugly.


That's a matter of opinion. I like to see null checks at the top of a
function or method, it makes it far more likely to be safe and it means
I can ignore the possibility of null from then on - making the code much
cleaner.

R



I find this to be very suboptimal at the least.

This prevents null values from traveling "up" the stack, but still 
allows them to move "down" (as return values) and allows them to survive 
multiple unrelated function calls.


It catches null values once they've already ended up in a place they 
shouldn't be.  Too late.


Nulls can also be placed into variables within structs or classes that 
then get passed around.  Checking for those can require some complex 
traversal: impractical for casual one-off checks at the start of a 
function in some cases.


void main()
{
void* x = a(b());
c();
while(goobledegook)
{
x = p();
d(x);
}
e(x); /+ Crash! x is null. +/
}

Where did x's null value come from?  Not a. Not p; the while loop 
happened to be never executed.  To say "b" would be closer, but still 
imprecise.  Actually it was created in the q() function that was called 
by u() that was called by b() which then created a class that held the 
null value and was passed to a() that then dereferenced the class and 
returned the value stored in the class that happened to be null.  nulls 
create very non-local bugs, and that's why they frustrate me to no end 
sometimes.


What I really want to know is where errant null values come FROM.

I also want to know this /at compile time/, because debugging run-time 
errors is time consuming and debugging compile-time errors is not.


The above example could yield the unchecked null assignment at compile 
time if all of the types involved were typed as non-nullable, except for 
the very bare minimum that needs to be nullable.  If something is 
explicitly nullable, then its enclosing function/class is responsible 
for handling null conditions before passing it into non-nullable space. 
 If a function/class with nullable state tries to pass a null value 
into non-nullable space, then it is a bug.  This contains the 
non-locality of null values as much as is reasonably possible.


Additionally, it might be nice to have a runtime nullable type that uses 
its object file's debugging information to remember which 
file/function/line that its null value originated from (if it happens to 
be null at all).  This would make for some even better diagnostics when 
code that HAS to deal with null values eventually breaks and needs to 
dump a stack trace on some poor unsuspecting sap (that isn't me) or 
ideally sends an email to the responsible staff (which is me).

Re: Will the D GC be awesome?

[Chad J]

On 10/03/2012 05:26 PM, DypthroposTheImposter wrote:

D is pretty cool, perhaps someday I can use it instead of C++ and have
cool shit like fast build times, modules, no more bloody headers, sane
templates, CTFE, UFCS etc

But can the D GC ever be made:

1. precise
2. able to scale to large-ish data set(2gig+)


Others have already addressed these 2 points better than my knowledge.


3. No long stalls(anything over a couple millisecond(<3))


One of the cool things about D's memory management is that it has a lot 
of features that help you /avoid heap allocation/:


- Stack allocation.  Just about anything in D can be stack allocated. 
As long as you know that your memory won't need to survive past its 
scope, you should be able to put it in the stack.


- Array slices.  This can be used to reduce copying.  Many other 
languages seem to implement string slicing as a copy operation, which 
introduces allocations.


- Ranges allow you to traverse arbitrary containers /without/ first 
converting them into an array, thus avoiding copying allocations. 
(Exception: if you need random access on a range that doesn't support 
random access, then you may have to do this.)


- Preallocation with conventional malloc.  Any time you find yourself 
frequently allocating and freeing the same struct/object/whatever, then 
you may want to preallocate it and avoid the allocation/deallocation 
overhead.


Using these things is probably a much better strategy for real-time 
software than leaning on a GC.  There will probably be times when a GC 
is still all-too-convenient or perhaps even necessary (if you've ever 
looked at MMO code).  As long as you keep the GC stuff down to only what 
is strictly necessary, then it will probably do well.


Try to do array slicing in Java or C#.  You probably won't be able to do 
it.  You'll get string /copies/ and this will incur heap allocations in 
the GC heap.  Those languages /need/ good garbage collection to be 
performant because they abuse the poor GC heavily.


The one thing I find missing (as of a couple months ago, anyways) is 
reference counting.  For soft-real-time apps it would be nice if 
transitively-atomic types (ex: strings) could be reference counted. 
This would allow a lot of copy-on-write standard library functions to be 
called without incurring the wrath of the GC.  D is already so powerful 
at memory management in other regards that I won't be worrying about 
this unless it gets in my way some day.





Q. Curious, would it be compacting?

If not then I'm stuck not using it much--

Which leaves me with structs, and lets just say D struct are not
impressive--


* Oh and on a totally unrelated note, D needs Multiple return values.
Lua has it, it's awesome. D doesn't want to be left out does it?

* OpCmp returning an int is fugly I r sad

* why is haskell so much shorter syntax, can D get that nice syntax
plss



I'll stick with the comments of others for these points too.


STAB!



PAWNCH!

Re: Which D features to emphasize for academic review article

[Chad J]

On 08/10/2012 06:01 PM, Walter Bright wrote:

On 8/10/2012 1:38 AM, F i L wrote:

Walter Bright wrote:

3. Floating point values are default initialized to NaN.


This isn't a good feature, IMO. C# handles this much more conveniently
with just
as much optimization/debugging benefit (arguably more so, because it
catches NaN
issues at compile-time). In C#:

class Foo
{
float x; // defaults to 0.0f

void bar()
{
float y; // doesn't default
y ++; // ERROR: use of unassigned local

float z = 0.0f;
z ++; // OKAY
}
}

This is the same behavior for any local variable,


It catches only a subset of these at compile time. I can craft any
number of ways of getting it to miss diagnosing it. Consider this one:

float z;
if (condition1)
z = 5;
... lotsa code ...
if (condition2)
z++;

To diagnose this correctly, the static analyzer would have to determine
that condition1 produces the same result as condition2, or not. This is
impossible to prove. So the static analyzer either gives up and lets it
pass, or issues an incorrect diagnostic. So our intrepid programmer is
forced to write:

float z = 0;
if (condition1)
z = 5;
... lotsa code ...
if (condition2)
z++;

Now, as it may turn out, for your algorithm the value "0" is an
out-of-range, incorrect value. Not a problem as it is a dead assignment,
right?

But then the maintenance programmer comes along and changes condition1
so it is not always the same as condition2, and now the z++ sees the
invalid "0" value sometimes, and a silent bug is introduced.

This bug will not remain undetected with the default NaN initialization.



To address the concern of static analysis being too hard: I wish we 
could have it but limit the amount of static analysis that's done. 
Something like this: the compiler will test branches of if-else 
statements and switch-case statements, but it will not drop into 
function calls with ref parameters nor will it accept initialization in 
looping constructs (foreach, for, while, etc).  A compiler is an 
incorrect implementation if it implements /too much/ static analysis.


The example code you give can be implemented with such limited static 
analysis:


void lotsaCode() {
... lotsa code ...
}

float z;
if ( condition1 )
{
z = 5;
lotsaCode();
z++;
}
else
{
lotsaCode();
}

I will, in advance, concede that this does not prevent people from just 
writing "float z = 0;".  In my dream-world the compiler recognizes a set 
of common mistake-inducing patterns like the one you mentioned and then 
prints helpful error messages suggesting alternative design patterns. 
That way, bugs are prevented and users become better programmers.

Re: @property

[Chad J]

On 08/04/2012 06:50 PM, Adam D. Ruppe wrote:

On Saturday, 4 August 2012 at 22:19:17 UTC, Chad J wrote:

There was a very long discussion on this before that lead me to write
this:


I'll have to read it... my thing right now works, as far as I can tell,
perfectly on free function properties.

But member properties are different and I haven't gotten them to work
without massive breakage yet. (I forgot all about them until just an
hour ago.) For some reason, rewriting the members breaks random things
like opCall too.

The good is the free function change doesn't actually break anything...
the bad is member properties are the more interesting ones and if they
don't work, this isn't cool

But it doesn't look like I'll be done with this today after all :( I
still have a lot of stuff I was supposed to be doing today!


Best of luck, and thank you for thinking about this and getting the ball 
rolling!

Re: @property

[Chad J]

There was a very long discussion on this before that lead me to write this:

http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Property

I should draw attention to the tables near the bottom.  It is really too 
bad that variables in D allow the taking of addresses by default.  If 
things were non-addressable by default, then they could always be 
promoted to properly-implemented properties.  Addressabilitiy could be 
added explicitly by authors that are confident that they will never need 
to turn their public fields into properties.  Too late for all of that 
now.


I tried to get DMD to do property rewriting in a vaguely general sense. 
 I think it's still very worthwhile to attempt it.


It didn't work for me because this was in the days of SVN and I tend to 
get my free time in 1-3 hour chunks.  By the time I had more free time 
I'd spend it all merging/updating my code to the latest revision.  I 
also spent a lot of my time learning how DMD's semantic analysis works. 
 It's pretty tangled (sorry, Walter).  So yeah, I didn't get to spend 
much time actually working on the property rewrite due to confusion and 
backtracking.  I realized that while I really wanted the feature, I 
wasn't having fun, nor did I have any free time to program at all. 
(Implementing the recursive logic for such a rewrite is really easy! 
Dealing with DMDs architectural corner cases... (was) not so easy.)


It'd probably be good to be minimal about it.  We might not be able to 
do property rewriting in a %100 awesome way without breaking backwards 
compatibility.  It'll still be worth it to get the property rewriting 
that doesn't break backwards compatibility.  Example:


struct Bar
{
private Foo m_a;

@property Foo a()
{
return m_a;
}
}

struct Foo
{
private int qux;

Foo opBinary(string op : "+")(Foo other)
{
Foo result;
result.qux = this.qux + other.qux
return result;
}
}

void main()
{
Bar a;
Foo b;

// Error: attempt to write to a non-writable property a.a
b = a.a + b;
}

To make the property rewrite bulletproof, we have to assume that (a.f + 
b) will modify a.a because a.a.opBinary!("+") is non-const.  We should 
then compile-time-error because the property a.a does not have a setter.
However, in current code, this would not give a compile-time error. 
Instead it would just work, because the caller and callee implicitly 
assume that the opBinary call is const, even if it's not marked as such. 
 There is probably a significant amount of code that would no longer 
compile because of this.  The answer would be to mark all such functions 
as const, but I'm not sure that Walter & Co would be fine with that.


It's OK though.  If Walter & Co don't want to break various calls to 
non-const functions at the end of expressions containing properties, 
then we can let that slide and only focus on the more likely cases like 
using += on a property field that returns a value type like an int 
(array.length += 1 is the classic case, with its own special-cased 
solution and everything).  It'd be extremely useful and possibly remove 
a lot of potential sources of bugs/headaches.


Anyhow, that's a lot of my background info on this subject.  I do hope 
someone else tackles this.  They will probably be more successful than 
I.  If not, I might still try it someday if the pull request might get 
accepted.

Re: std.d.lexer requirements

[Chad J]

On 08/02/2012 03:09 AM, Bernard Helyer wrote:

http://i.imgur.com/oSXTc.png

Posted without comment.


Hell yeah Alexander Brandon.

Re: D language and .NET platform

[Chad J]

On 07/29/2012 01:11 PM, Timon Gehr wrote:

On 07/29/2012 07:00 PM, Timon Gehr wrote:

On 07/29/2012 06:32 PM, Alex Rønne Petersen wrote:

On 29-07-2012 17:36, bearophile wrote:

Alex Rønne Petersen:


.NET is too limited to represent the language,


Can you tell us why?

Bye,
bearophile


Array slices. The .NET type system has no way to represent them because
it's designed for precise GC, and array slices allow interior pointers
in the heap (as opposed to the stack when passing a field of an object
by reference to a function, or whatever).



I think all of CTFE-D should map to .NET without much hassle.



This could get a little hairy:

struct S{
int x,y;
}

void main(){
S s;
auto p = &s.y;
// ...
}

It would have to be translated to something like (pseudo-code)

class S{
int x;
int y;

S copy(){...}
}

class SyPointer : Pointer {
private S instance;
override int deref(){ return instance.y; }
override void derefAssign(int y){ instance.y = y; }
}

Out of the window goes native value type support.


I still always thought it funny that we didn't reach for it anyways.  It 
seems like the VMs can weaken performance significantly, but they should 
always be able to implement D's semantics.  It's a matter of finding 
effective hacks.

Re: Impressed

[Chad J]

On 07/26/2012 08:54 PM, Jonathan M Davis wrote:

On Friday, July 27, 2012 02:46:20 Adam D. Ruppe wrote:

On Friday, 27 July 2012 at 00:25:49 UTC, Jonathan M Davis wrote:

scope on local variables is going to be deprecated, because
it's unsafe. scope
on function parameters and scope statements are here to stay.


There's also scope(exit), scope(success), and scope(failure),
which aren't going anywhere.


Yeah. Those are scope statements.

It's just scope on local variable declarations which is going away.

- Jonathan M Davis


Stuart & other readers:

I just asked about this in the other thread, and Jonathan mentioned that 
std.typecons.scoped can be used to accomplish the same thing.  So the 
functionality isn't being removed; it's just being moved from the 
language and into the library.


I think this is worth mentioning so that it doesn't look like we are 
depriving folks of things that they had come to expect from D.

Re: Just where has this language gone wrong?

[Chad J]

On 07/19/2012 10:21 AM, Petr Janda wrote:
...
I think the other points have been adequately covered.
...
> auto r = [5, 3, 5, 6, 8].sort.uniq.map!(x => x.to!string);
...


I'm sorry I don't mean to be a criticizer, but it seems to me that D is
trying to be a dynamic-like compiled language way too hard.



Everything in that snippet is statically (and probably strongly too?) 
typed at compile-time.


What's going on is a lot of type inference and syntax sugar that allows 
for very terse code.  You'll still get immediate compiler errors if you 
screw up the types and you try to treat "r" like it isn't a range of 
strings.  There won't be any bugs introduced in your programs due to 
breaches of type safety.  This is the beauty of it: the code is terse, 
does what it says it does, potentially very optimized, and also heavily 
verified at compile-time.  The anonymous function is even passed at 
compile-time for easy inlining (not sure how aggressively dmd handles 
that right now though).  It's hard to get that combo of (development 
speed)|(correctness)|(execution speed) in other places, in my 
not-so-humble opinion. ;)

Re: Rust updates

[Chad J]

On 07/11/2012 02:00 PM, David Piepgrass wrote:


I am particularly a fan of structural typing. I don't know if Rust uses
it but Opa and other functional languages often do. You see, there's a
problem that pops up in .NET all the time, and probably the same problem
exists in D.

Any time two libraries want to use the same concept, but the concept is
not in the standard library, they need to define it. For instance if
there is no "Point" type in the standard library, but two unrelated
libraries need points, they will both define their own (amazingly,
Points are poorly thought out in .NET and tightly bound to GUI
libraries, so people define their own in some cases):

// JoesLibrary
struct Point!T { T x, y; /* followed by some manipulation functions */ }

// FunkyLibrary
struct Point!T { T x, y; /* followed by other manipulation functions */ }

Sadly, the two point types are not compatible with each other. A client
that wants to use both libraries now has an interoperability problem
when he wants to pass data between the.

Even a client that uses only one of the library, let's call it
"JoesLibrary" has to import Point from "JoesLibrary", even if its
functionality is not quite what the client wants. It would be much nicer
if the client could define his own Point struct that seamlessly
interoperates with Joes'. In D this is currently impractical, but I
would enjoy designing a way to make it work (before you point out that
"what if x and y are in a different order in the two structs" and "it
could be T X,Y in one and T x,y in the other", yes, I know, It's on my
list of problems to cleverly solve)

A similar problem exists with interfaces, where two unrelated libraries
expose two similar classes with some common functions, but you can't
cast them to a common type in D. This is a solved problem in Go
(http://www.airs.com/blog/archives/277) and it's actually pretty easy
for a compiler to magically cast a class to an interface that the class
did not declare--if the underlying language is designed for that, anyway.

In fact, in .NET at least, the same problem exists even if the libraries
DO know about each other and are even written by the same person and use
identical interfaces. The problem is, if I write two libraries A and B,
and I want them to be interoperable, then I need to factor out the
common structs and interfaces to a microscopic third library, I. But
from the client's perspective, if a client only knows about A or B, he
will think it's stupid that I require him to use and deploy two
DLLs/so's (A and I, or B and I) instead of one. In D I guess it's not
the same, though.



I'm pretty sure this interoperability thing is solved in D, at least at 
compile time.


Example:

import std.traits;

/// Returns true if T is a Point type.
template isPoint(T)
{
enum bool isPoint = is(typeof(
{
T point;

// (x,y) should be numeric.
static assert ( isNumeric!(typeof(point.x)) );
static assert ( isNumeric!(typeof(point.y)) );

auto x = point.x; // x is readable
point.x = x;  // x is writable

auto y = point.y; // y is readable
point.y = y;  // y is writable
}));
}

struct MyPointType
{
float x;
float y;
}

struct AnotherPointType
{
int x;
int y;
}

struct NotAPoint
{
char[] x;
char[] y;
}

// TODO: are x and y allowed to have differing types?
static assert(isPoint!MyPointType);
static assert(isPoint!AnotherPointType);
static assert(!isPoint!NotAPoint);

auto myPointCalculation(P1,P2)( P1 p1, P2 p2 ) if ( isPoint!P1 && 
isPoint!P2 )

{
p1.x += p2.x;
p1.y += p2.y;
return p1;
}

import std.stdio;
void main()
{
MyPointType p1;
AnotherPointType p2;
p1.x = 3.5;
p1.y = 5.0;
p2.x = 2;
p2.y = 2;
writefln("(before) p1 == (%s, %s)",p1.x,p1.y);
p1 = myPointCalculation(p1,p2);
writefln("(after)  p1 == (%s, %s)",p1.x,p1.y);
}



At the command line:
chad@Hugin ~/dprojects/dtesting/points $ dmd points.d
chad@Hugin ~/dprojects/dtesting/points $ ./points
(before) p1 == (3.5, 5)
(after)  p1 == (5.5, 7)

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 07:04 PM, Roman D. Boiko wrote:

On Saturday, 7 July 2012 at 22:45:01 UTC, Chad J wrote:

I see what you mean.

And I think that I expressed myself badly. Let me rephrase.

When the memory hierarchy is deep, every level would require at least
one bit position. Or even every base class would require a separate bit.
(I think that the former + several bits to distinguish among
hierarchies.) Otherwise it would not be easy to check by a bit-mask.
Even if the above is incorrect (and that is likely since I didn't try to
encode that compactly for the real grammar), I think that in general
that information would only be possible to store in a fairly large
integral. Especially if we try to generate parser from grammar, and thus
can't do fine-tuning to pack the information tightly. This overhead
would be paid per each AST node __instance__. But that is not necessary,
since we could store information in a lookup table only once per node
__type__.


Yep. Makes sense.

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 04:26 PM, David Piepgrass wrote:

auto captures = syntaxNode.matchNodes(
TOK_WHILE_NODE,
OP_ENTER_NODE,
OP_CAPTURE(0),
OP_BEGIN,
TOK_EXPRESSION,
OP_END,
OP_CAPTURE(1),
OP_BEGIN,
TOK_STATEMENT,
OP_END,
OP_LEAVE_NODE);


I'm glad to hear you like the tree-parsing approach, Chad, although the
particular syntax here looks pretty unfriendly :O -- does this represent
something that you are working on right now?



Yes and yes.

I didn't choose this because it because it's pretty.

I chose it because:
(1) It's easy to implement.
(2) Both the implementation and syntax can be altered easily.

I do not have time to write a language for the tree pattern recognition 
and substitution that is needed to do this in an aesthetically pleasing 
way.  I've tried to sketch what it might look like before, and even then 
it is hard to make it nice, much less begin implementing the thing.  I'd 
love to have such a language, but resource constraints exist.


I also think that this approach would allow me to find out what my usage 
patterns look like before I commit to a more complicated 
architecture/tool.  I really think the design of this regex/language/DSL 
thing should be dominated by its usage.  This is a tricky 
chicken-and-egg thing because it's not currently used.  The hacky syntax 
you see is the bootstrapping.


Point (2) is important because, since we don't have existing usage 
patterns, this thing is going to change.  It's going to change /a lot/. 
 I want it to be easy to change.  I think a complete DSL will be harder 
to change quickly.


I also like how it doesn't require a lot of CTFE trickery or pushing DMD 
too far.  D has really cool features, but I find that when I use things 
like CTFE aggressively then I lose productivity because I end up 
spending a lot of time finding compiler bugs.  This leads to my current 
strategy: use the simpler features that work for sure, and only use the 
more advanced stuff when I really need to.  I think my syntax fits this 
strategy and thus contributes to point (1).


That said, it is good that even mostly-working CTFE exists and that a 
powerful template and metaprogramming system exists, because I don't 
think a compiler like this would be very practical to program otherwise. 
 It would be doable in other languages, but could easily suffer from 
performance pessimizations due to being forced to compute everything at 
runtime.


If anyone has an approach that shares the above strengths and looks 
nicer or is more powerful, I'd love to see it.




5. It's risky 'cause I've never heard of anyone taking this approach
before. Bring on the danger!



The danger is the fun part! 




I wanted to make such a front-end so that I could easily make a C
backend. I believe such a compiler would be able to do that with great
ease.

Needing to use D in places where it isn't available is a real
pain-point for me right now, and I'll probably find ways to spend time
on it eventually.


Yeah, with a tree-transforming parser, I imagine the same thing, except
my current fetish is to convert a certain subset of D to multiple other
languages automatically. Then I could write libraries that can easily be
used by an astonishingly large audience. I certainly would like to see D
targetting Android, but that's best done directly from D to ARM.



That does sound very cool.  Possibly difficult though, due to having to 
cater to the lowest-common-denominator in all of your API designs.  No 
templated functions or ranges in your API, that's for sure.  I'm sure 
there are some things where this is very doable though; it probably 
depends on what kind of libraries you are writing.


As for D targeting Android, my intent is really to target X where X is 
any CPU/OS combo you can think of.  I want to be able to get D, the 
language, not necessarily phobos or other niceties, to work on any 
platform, and to do so without much work.  Cross-compiling to a new 
platform that has never been cross-compiled before should require zero 
coding.  Perhaps it might be possible to have a text file with some 
key-value configuration that tells it certain common features are 
available on the target, thus allowing you to have more features with 
almost no effort involved.


Still, I'll always take a crippled but existent D compiler that targets 
Android over a perfect but non-existent D compiler that targets Android.


I think that the D-directly-to-ARM is the current approach for 
cross-compiling.  I critique it for its underwhelming lack of results.



Anyway, the devil's in the detail. Originally I wanted to do a parser
generator and a "completely general AST" in C# and couldn't seem to work
out the details, but D is more flexible and is likely better suited to
the task.


I can easily see how this is the case.  I don't think I'd be interested 
in doing a project like this in any other language.  I imagined trying 
to do something like this in C or Java or C# and it just doesn't seem 
practical.  For insta

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 04:26 PM, Timon Gehr wrote:

On 07/07/2012 08:55 PM, Chad J wrote:


The specifics will easily change.


I'd suggest:

AstOp!`
Lower
while ( boolExpr )
{
statements;
}

Into
loopAgain:
if ( !boolExpr ) {
statements;
} else goto exitLoop
goto loopAgain
exitLoop:

`.run(syntaxNode);



I wish.

Can you make it happen?

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 06:18 PM, Roman D. Boiko wrote:

On Saturday, 7 July 2012 at 22:03:20 UTC, Chad J wrote:

enum : SyntaxElement
{
AST_EXPRESSION = 0x0001___,
AST_UNARY_EXPR = 0x_0001__ |


This would cause wasting space (probably a lot). Definitely it would not
be easy to implement in a parser generator, when various language
properties are not known beforehand for fine-grained tuning.


This approach of course has shameful nesting limitations, but I feel
like these determinations could be fairly well optimized even for the
general case. For example: another approach that I might be more
inclined to take is to give each token/symbol a low-valued index into
a small inheritance table.


Depending on implementation, that might introduce the multiplier
overhead of table access per each comparison (and there would be many in
case of searching for nodes of specific type).


I would expect the regex engine to call the isA function as one of
it's operations. Thus placing an AST_EXPRESSION into your expression
would also match an AST_NEGATE_EXPR too.


But actually it is not so difficult to implement in a very similar way
to what you described. I was thinking about a lookup table, but
different from a traditional inheritance table. It would be indexed by
AST node type (integral enum value), and store various classification
information as bits. Maybe this is what you meant and I misunderstood
you... Example is here:
https://github.com/roman-d-boiko/dct/blob/May2012drafts/fe/core.d
(sorry, it doesn't show how to do classification, and has a different
context, but I hope you get the idea). The advantage over storing
hierarchical information directly in each token is obviously memory usage.


I see what you mean.

I'm not sure that I buy that language properties are known before-hand. 
 The front-end knows what the language grammar looks like and it knows 
what kind of things it can find in an AST.


Thus you can make the regex DSL do this transformation and let the DFAs 
handle everything:


expr -> (expr | unary_expr | negate_expr | binary_compliment | incr_expr 
| decr_expr | binary_expr | assign_expr | add_assign_expr | nary_expr | ...)


Because what we really want it to do is match any of those expression 
kinds when we place the expression symbol in our regex.


I think the important point here though is that inheritance can be 
reduced to bit-twiddling optimizations in this case.

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 03:13 PM, Roman D. Boiko wrote:

On Saturday, 7 July 2012 at 18:55:57 UTC, Chad J wrote:


In this vision I do not use classes and inheritance for my AST.
Instead I use structs that contain some kind of nodeType member that
would be one of the tokens/symbols in the grammar, like TOK_WHILE_NODE
in the above code. Dynamic dispatch is instead performed by (very
fast) DFAs recognizing parts of the AST.


Exactly. This idea first came to me in April after I implemented the
first top-down recursive descent custom parser for a D subset. I tried
Visitor pattern before that and wasn't happy. There are some subtle
difficulties which I believe will be possible to overcome, most
important being the need to introduce a mechanism for hierarchical
classification (like a pow expression being an assign expression at the
same time).



From some of my earlier scribblings:

enum : SyntaxElement
{
  AST_EXPRESSION  = 0x0001___,
AST_UNARY_EXPR= 0x_0001__ | AST_EXPRESSION,
  AST_NEGATE_EXPR = 0x__0001_ |  AST_UNARY_EXPR,
  AST_COMPLIMENT_EXPR = 0x__0002_ |  AST_UNARY_EXPR,
  AST_POST_ADD_EXPR   = 0x__0003_ |  AST_UNARY_EXPR,
  AST_POST_SUB_EXPR   = 0x__0004_ |  AST_UNARY_EXPR,
  AST_PRE_ADD_EXPR= 0x__0005_ |  AST_UNARY_EXPR,
  AST_PRE_SUB_EXPR= 0x__0006_ |  AST_UNARY_EXPR,
AST_BINARY_EXPR   = 0x_0002__ | AST_EXPRESSION,
  AST_AND_EXPR= 0x__0001_ |  AST_BINARY_EXPR,
  AST_OR_EXPR = 0x__0002_ |  AST_BINARY_EXPR,
  AST_XOR_EXPR= 0x__0003_ |  AST_BINARY_EXPR,
  AST_AND_AND_EXPR= 0x__0004_ |  AST_BINARY_EXPR,
  AST_OR_OR_EXPR  = 0x__0005_ |  AST_BINARY_EXPR,
  AST_ADD_EXPR= 0x__0006_ |  AST_BINARY_EXPR,
AST_TRINARY_EXPR  = 0x_0003__ | AST_EXPRESSION,
AST_NARY_EXPR = 0x_0004__ | AST_EXPRESSION,
  AST_STATEMENT   = 0x0002___,
}


bool isA( SyntaxElement leafier, SyntaxElement rootier )
{
SyntaxElement mask = 0;

if ( rootier & 0x___ )
{
if ( rootier & 0x___ )
mask = 0x___;
else
mask = 0x___;
}
else
{
if ( rootier & 0x___ )
mask = 0x___;
else
mask = 0x___;
}

return (leafier & mask) == rootier;
}

unittest
{
assert(  isA( AST_EXPRESSION,  AST_EXPRESSION) );
assert(  isA( AST_NEGATE_EXPR, AST_NEGATE_EXPR) );
assert(  isA( AST_NEGATE_EXPR, AST_EXPRESSION) );
assert(  isA( AST_NEGATE_EXPR, AST_UNARY_EXPR) );
assert( !isA( AST_EXPRESSION,  AST_STATEMENT) );
assert( !isA( AST_NEGATE_EXPR, AST_BINARY_EXPR) );
assert( !isA( AST_NEGATE_EXPR, AST_STATEMENT) );
assert( !isA( AST_NEGATE_EXPR, AST_COMPLIMENT_EXPR) );
assert(false);
}

This approach of course has shameful nesting limitations, but I feel 
like these determinations could be fairly well optimized even for the 
general case.  For example: another approach that I might be more 
inclined to take is to give each token/symbol a low-valued index into a 
small inheritance table.


I would expect the regex engine to call the isA function as one of it's 
operations.  Thus placing an AST_EXPRESSION into your expression would 
also match an AST_NEGATE_EXPR too.

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 02:23 PM, David Piepgrass wrote:

Note that PEG does not impose to use packrat parsing, even though it
was developed to use it. I think it's a historical 'accident' that put
the two together: Bryan Ford thesis used the two together.


Interesting. After trying to use ANTLR-C# several years back, I got
disillusioned because nobody was interested in fixing the bugs in it
(ANTLR's author is a Java guy first and foremost) and the source code of
the required libraries didn't have source code or a license (wtf.)

So, for awhile I was thinking about how I might make my own parser
generator that was "better" than ANTLR. I liked the syntax of PEG
descriptions, but I was concerned about the performance hit of packrat
and, besides, I already liked the syntax and flexibility of ANTLR. So my
idea was to make something that was LL(k) and mixed the syntax of ANTLR
and PEG while using more sane (IMO) semantics than ANTLR did at the time
(I've no idea if ANTLR 3 still uses the same semantics today...) All of
this is 'water under the bridge' now, but I hand-wrote a lexer to help
me plan out how my parser-generator would produce code. The output code
was to be both more efficient and significantly more readable than
ANTLR's output. I didn't get around to writing the parser-generator
itself but I'll have a look back at my handmade lexer for inspiration.


However, as I found a few hours ago, Packrat parsing (typically used to
handle PEG) has serious disadvantages: it complicates debugging
because of
frequent backtracking, it has problems with error recovery, and
typically
disallows to add actions with side effects (because of possibility of
backtracking). These are important enough to reconsider my plans of
using
Pegged. I will try to analyze whether the issues are so fundamental
that I
(or somebody else) will have to create an ANTLR-like parser instead, or
whether it is possible to introduce changes into Pegged that would
fix these
problems.


I don't like the sound of this either. Even if PEGs were fast,
difficulty in debugging, error handling, etc. would give me pause. I
insist on well-rounded tools. For example, even though LALR(1) may be
the fastest type of parser (is it?), I prefer not to use it due to its
inflexibility (it just doesn't like some reasonable grammars), and the
fact that the generated code is totally unreadable and hard to debug
(mind you, when I learned LALR in school I found that it is possible to
visualize how it works in a pretty intuitive way--but debuggers won't do
that for you.)

While PEGs are clearly far more flexible than LALR and probably more
flexible than LL(k), I am a big fan of old-fashioned recursive descent
because it's very flexible (easy to insert actions during parsing, and
it's possible to use custom parsing code in certain places, if
necessary*) and the parser generator's output is potentially very
straightforward to understand and debug. In my mind, the main reason you
want to use a parser generator instead of hand-coding is convenience,
e.g. (1) to compress the grammar down so you can see it clearly, (2)
have the PG compute the first-sets and follow-sets for you, (3) get
reasonably automatic error handling.

* (If the language you want to parse is well-designed, you'll probably
not need much custom parsing. But it's a nice thing to offer in a
general-purpose parser generator.)

I'm not totally sure yet how to support good error messages, efficiency
and straightforward output at the same time, but by the power of D I'm
sure I could think of something...

I would like to submit another approach to parsing that I dare say is my
favorite, even though I have hardly used it at all yet. ANTLR offers
something called "tree parsing" that is extremely cool. It parses trees
instead of linear token streams, and produces other trees as output. I
don't have a good sense of how tree parsing works, but I think that some
kind of tree-based parser generator could become the basis for a very
flexible and easy-to-understand D front-end. If a PG operates on trees
instead of linear token streams, I have a sneaky suspicion that it could
revolutionize how a compiler front-end works.

Why? because right now parsers operate just once, on the user's input,
and from there you manipulate the AST with "ordinary" code. But if you
have a tree parser, you can routinely manipulate and transform parts of
the tree with a sequence of independent parsers and grammars. Thus,
parsers would replace a lot of things for which you would otherwise use
a visitor pattern, or something. I think I'll try to sketch out this
idea in more detail later.


I was thinking the same thing.

My intent is to create a kind of regular-expression-of-nodes with 
push/pop operators to recognize ascent and descent on the tree.  Such a 
regular expression would allow one to capture subtrees out of 
generalized patterns and then place them into new trees that then become 
the input for the next pattern or set of patterns.  I think t

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 01:01 PM, Roman D. Boiko wrote:

On Saturday, 7 July 2012 at 16:56:06 UTC, Chad J wrote:

Yeah, it's good to hear this notion reinforced. I had this suspicion
that the packrat parser is not necessarily the best/fastest solution,
mostly because of the large allocation that has to happen before you
get O(n) performance. Thus I figured that pegged might eventually use
different parsing strategies underneath it all, possibly with a lot of
special-casing and clever hand-tuned and profiled optimizations. At
least that's what makes sense to me.


At the very least, we could use DFA instead of backtracking where
possible. This is the approach implemented in ANTLR, but I wanted to
introduce them before I knew about existence of the latter, simply
because this would likely produce the fastest parsers possible.


These were my thoughts exactly, although somewhat unsubstantiated in my 
case ;)

Re: Let's stop parser Hell

[Chad J]

On 07/07/2012 12:37 PM, Roman D. Boiko wrote:

On Saturday, 7 July 2012 at 16:27:00 UTC, Philippe Sigaud wrote:


Note that PEG does not impose to use packrat parsing, even though it
was developed to use it. I think it's a historical 'accident' that put
the two together: Bryan Ford thesis used the two together.

Note that many PEG parsers do not rely on packrat (Pegged does not).
There are a bunch of articles on Bryan Ford's website by a guy
writting a PEG parser for Java, and who found that storing the last
rules was enought to get a slight speed improvement, buth that doing
anymore sotrage was detrimental to the parser's overall efficiency.


That's great! Anyway I want to understand the advantages and limitations
of both Pegged and ANTLR, and probably study some more techniques. Such
research consumes a lot of time but can be done incrementally along with
development.


Yeah, it's good to hear this notion reinforced.  I had this suspicion 
that the packrat parser is not necessarily the best/fastest solution, 
mostly because of the large allocation that has to happen before you get 
O(n) performance.  Thus I figured that pegged might eventually use 
different parsing strategies underneath it all, possibly with a lot of 
special-casing and clever hand-tuned and profiled optimizations.  At 
least that's what makes sense to me.

Re: Let's stop parser Hell

[Chad J]

On 07/05/2012 08:28 AM, Roman D. Boiko wrote:

Pegged may eventually become standard, if it will be performanceoptimized and a 
bit more customizable


Interesting, I thought you were hand-writing this stuff.

I'm a fan of pegged and made some pull requests, one of which was aimed 
at making it more customizable by allowing the user to define what type 
gets used as a parse tree node, thus allowing one to potentially use 
their parse tree as an AST (and maybe do a couple other things too). 
It's a WIP, but the proof of concept is done: DMD can handle the extra 
templating at compile time, so it works.


What kind of things did you want in terms of customizability?

Re: How do I submit documentation...

[Chad J]

On 06/24/2012 03:27 PM, Brad Anderson wrote:




We've actually made it even easier than this.  Click the "Improve this
page" button on the top-right of any dlang.org  page.
  You could make use of the autotester to add unit tests but it's
probably easier if you have a complicated edit to just do it locally.


Where's the autotester?

Re: How do I submit documentation...

[Chad J]

On 06/24/2012 03:27 PM, Brad Anderson wrote:

On Sun, Jun 24, 2012 at 1:35 AM, Tobias Pankrath mailto:tob...@pankrath.net>> wrote:

This is in stark contrast to wikis and the like where there's a
button to /just edit the page dammit/.  I understand that this
would be way too unmoderated for D's central projects,


Github can do this. Just use the "Edit this file"-Button for minor
changes.



We've actually made it even easier than this.  Click the "Improve this
page" button on the top-right of any dlang.org  page.
  You could make use of the autotester to add unit tests but it's
probably easier if you have a complicated edit to just do it locally.

Regards,
Brad Anderson


Yes!  That is cool.

Re: How do I submit documentation...

[Chad J]

On 06/24/2012 01:33 PM, Jesse Phillips wrote:

On Sunday, 24 June 2012 at 01:30:14 UTC, Chad J wrote:


No way it is that easy. My situation called for this:

src/web
src/phobos
src/druntime


I was and am not in Linux to look at my setup so I may have gotten the
names wrong but I know I don't have them start with src/ (hmmm, maybe it
has changed). The name for phobos didn't matter since that is where you
build from, and the make file did mention d-programming-language.org as
a directory so I just assumed. web I'm pretty sure is created by
building the docs, and I completely forgot to mention druntime.


dmd
linux/bin64/dmd -> ../../dmd/src/dmd
linux/bin64/dmd.conf (Copy this from a released zip file.)
linux/lib64/libphobos2.a ->
../../src/phobos/generated/linux/debug/32/libphobos2.a
linux/lib64/libdruntime.a ->
../../src/druntime/lib/libdruntime.a

dmd.conf may need editing.


This was all hidden under the "you'll have to convince your
environment to use the one you build though" I created a bash file to
call DMD providing the proper libraries. I've had issues with it though,
but hopefully I've convinced it to stop grabbing the system libraries I
have installed.

This really isn't a problem with the repositories though. The setup is
pretty simple, but ld will be looking in your system libraries first and
the repositories cannot mess with your environment.


Oh, but I think it is.

If you have all of the repositories tracked by a single repository that 
cements the directory structure of a D distribution, then you can 
streamline the build process.  The makefile in the ./dmd/src should be 
able to place the compiler executable into .//bin/dmd. 
Also, druntime and phobos makefiles should place their library files 
into .//lib/*.*.  dmd.conf and any other release 
requirements should also be included in this meta-repository.  I should 
be able to sit at the toplevel of this meta-repository (./ that is) and 
type "make all" and end up with all of the contents that end up in a zip 
file in all of the places they end up in the zip file, plus website 
things.  It should be everything needed to make a D release, minus the 
secret keys to actually write to the digitalmars.com website.  With a 
setup like this, it would require /zero/ time to convince things to use 
the correct paths (aside from maybe editing your PATH variable once or 
running the installer that is hopefully also included in the meta-repo).

Re: How do I submit documentation...

[Chad J]

On 06/23/2012 08:39 PM, Chad J wrote:

...


As I reread this a day later I realize it is unreasonably harsh.  I am 
sorry for that.  I was frustrated.


I still have stuff I want to contribute eventually.  Please Walter, 
Andrei, and Co., help me help you.

Re: How do I submit documentation...

[Chad J]

On 06/24/2012 03:35 AM, Tobias Pankrath wrote:

This is in stark contrast to wikis and the like where there's a button
to /just edit the page dammit/. I understand that this would be way
too unmoderated for D's central projects,


Github can do this. Just use the "Edit this file"-Button for minor changes.




Hey, I missed that.  That looks very handy.  Thanks!

Re: How do I submit documentation...

[Chad J]

On 06/23/2012 08:59 PM, Jesse Phillips wrote:

On Sunday, 24 June 2012 at 00:40:08 UTC, Chad J wrote:


So how about it, all-encompassing D git repo and buildscripts please?


The problem is, the documentation/website is built from the source code.
The source code doesn't rely on the website, though you may need the in
development dmd to build Phobos. dmd does not need Phobos. So while,
better, you'd still just end up with a single repository with three
submodules which would need downloaded. Or two repositories, on for the
Phobos/dmd dependency then another for the website/phobos/dmd dependency.

Directory structure.

cwd: /path/to/some/place

d-programming-language.org/
phobos
dmd

dmd actually doesn't matter, you'll have to convince your environment to
use the one you build though.


No way it is that easy.  My situation called for this:

src/web
src/phobos
src/druntime
dmd
linux/bin64/dmd -> ../../dmd/src/dmd
linux/bin64/dmd.conf  (Copy this from a released zip file.)
linux/lib64/libphobos2.a ->
  ../../src/phobos/generated/linux/debug/32/libphobos2.a
linux/lib64/libdruntime.a ->
  ../../src/druntime/lib/libdruntime.a

dmd.conf may need editing.

I am also left wondering if that is incomplete.  I probably forgot 
something.


(N, you made me look!)

I don't like how much harder it is to find versions of DMD that compile 
versions of phobos.  Because they are in different repositories, there 
is no easy way to figure out which commits are in sync with which 
commits in the other repo.  I remember some amount of time recompiling 
while searching for commit versions that line up.

Re: Phobos: Arbitrary delimiter variants of (std.string) stripLeft and stripRight

[Chad J]

On 06/23/2012 08:25 PM, Jonathan M Davis wrote:

On Saturday, June 23, 2012 20:04:11 Chad J wrote:

I'm not exactly sure what the arbitrary delimiter variants of stripLeft
and stripRight are, but they should be referenced in the documentation
of those functions.  And if they don't exist, why don't we have them?


There are no such variants. They could be added, but they don't currently
exist.


I was looking around for trimLeft and trimRight, but no luck.


trim and strip are the same thing. Having both would be bad design. It's
understandable if you were looking for trim first, since a library could go
with either name, and you probably have seen others using trim, but we chose
strip, and we're not going to have both.



I've seen them used to differentiate between whitespace and 
any-delimiter variants in other places, and the vocabulary seems to be 
used entirely inconsistently.  I just kind of run under the assumption 
that trim|strip mean remove /something/ from the right|left side of a 
given string.  The /something/ may or may not be whitespace or some 
delimiter of your choosing depending on what language you are in.


So yeah, I'm all for using something besides "trim". ;)


chomp
seems to be the arbitrary delimiter variant of trimRight, but then there
is no chompLeft.


chompPrefix is chompLeft. And no, _neither_ is the delimiter variant of
stripLeft or stripRight. stripRight with an arbitrary delimiter would remove
all instances of that delimiter from the right-hand side of the array (and
that delimiter would have to be a single character), whereas chomp takes a
substring and removes _exactly_ that substring. The same would go for
stripLeft and chompPrefix.



Ah.  Looks like a doc rewrite may be in order (and add unittests/examples!):

Returns s sans the trailing delimiter, if any. If no delimiter is given, 
then any trailing '\r', '\n', "\r\n", std.uni.lineSep, or 
std.uni.paraSeps are removed.


My parse of this: "any trailing ...s are removed."

I took the plural to mean that more than one newline/separator may be 
removed.


It should parse like this: "one trailing ... is removed."


If we had chompLeft I'd expect chompRight to exist and
chomp to remove both the left and right parts.

There are also no examples/unittests in the documentation on the website
for stripLeft, stripRight, chomp, or a bunch of others.


They could be added.

- Jonathan M Davis


I would say "should be added" myself, given how bread-and-butter this 
stuff is for me (at least at work).  I had to write all of those myself 
when I started using this Synergy/DE|DBL language at work, and it would 
be just silly to have to do that in a powerful language like D.


I'd add all of this stuff myself except that 
http://forum.dlang.org/thread/js5nl8$h94$1...@digitalmars.com

How do I submit documentation...

[Chad J]

...without wanting to slit my wrists?

I see that https://github.com/D-Programming-Language still has a bunch 
of the core D projects huddled together, but no single repository that 
binds it all together.


This is immensely frustrating if I want to do any contributions to DMD, 
Phobos, or the documentation.  Right now I'd like to contribute to 
documentation mostly.  I haven't done so because there is an annoying 
barrier to entry: I better be able to download and compile everything so 
that I can write unittests so that I can write documentation.  I really 
want to be able to download /one single repository/ and then type "make 
all" and have it /just work/.  But no, I have to download each of these 
things individually and figure out where each of them goes in some kind 
of canonical directory structure that everyone except me knows.  Oh, the 
me from a few months ago knew where to put things.  The me from now will 
need to be told again -- or do a grindy search over old emails.


If I'm going to make a pull request for every few words I edit or every 
example/unittest I want to add, then the {pull->edit->make docs->test 
with browser->push} cycle has to be very fast.  Right now, it would 
probably be a few hours before I even have things setup well enough that 
I can even /start/ editing.


I know it takes a while to mess with this stuff because I've done it 
some months ago.  It was thanks to bug #5278 
(http://d.puremagic.com/issues/show_bug.cgi?id=5278) which makes it so I 
can't use DMD without applying my own patch and building from source. 
I've done it a couple other times when I wanted to mess with DMD/phobos 
internals too.  Every time it takes hours of completely unproductive 
wasted time to build from source.


This is in stark contrast to wikis and the like where there's a button 
to /just edit the page dammit/.  I understand that this would be way too 
unmoderated for D's central projects, but it would be very nice if we 
had something closer to that.  At this point though I'd settle for just 
being able to build dmd/phobos/docs easily because it seems like a 
reasonable thing to ask for.


So how about it, all-encompassing D git repo and buildscripts please?

Phobos: Arbitrary delimiter variants of (std.string) stripLeft and stripRight

[Chad J]

I'm not exactly sure what the arbitrary delimiter variants of stripLeft 
and stripRight are, but they should be referenced in the documentation 
of those functions.  And if they don't exist, why don't we have them?


I was looking around for trimLeft and trimRight, but no luck.  chomp 
seems to be the arbitrary delimiter variant of trimRight, but then there 
is no chompLeft.  If we had chompLeft I'd expect chompRight to exist and 
chomp to remove both the left and right parts.


There are also no examples/unittests in the documentation on the website 
for stripLeft, stripRight, chomp, or a bunch of others.


Link for your convenience:
http://dlang.org/phobos/std_string.html#stripLeft

Re: Nimrod language

[Chad J]

On 05/28/2012 04:21 AM, Araq wrote:

On Saturday, 26 May 2012 at 11:49:47 UTC, Chad J wrote:

On 05/24/2012 07:21 PM, Araq wrote:

On Thursday, 24 May 2012 at 22:56:52 UTC, Kevin Cox wrote:

On May 24, 2012 6:53 PM, "Froglegs"  wrote:


Nimrod is full of constructs that have inlining semantics and as such
declaration order matters quite a bit. The D compiler has/had bugs with
this feature for a reason. ;-)


OK, now I'm curious. Why?



Because it's hard to implement? ;-)


t() # the compiler needs to *expand* t here; parsing t's
# header and put it into the symbol table is not enough

template t() = ...



I'm reading this to mean that t might contain other definitions, so 
building a symbol table to completion would involve leapfrogging 
template/macro expansion with symbol table invalidation/insertions.  At 
some level though, I expect the compiler to do this for me.




And Nimrod supports symbol table inspection:

when not defined(p):
proc p = echo "a"
else:
proc p = echo "b"
p() # what should that do?

These things can be solved, but it's lots of work and I have no idea how
the result would affect compile times (probably negligible).




p() should print "a": p wasn't defined to begin with.  It wouldn't be 
intuitive to me if the when statement is evaluated more than once, not 
unless it's found within a macro or something that iterates.



I'm considering to weaken the requirement but I don't mind this feature:
Having the order reflect the call graph has its advantages too. Many
consider the resulting order *backwards*, but at least there is *an*
order.



Wouldn't it be good enough to define the order arbitrarily when
building your symbol table? What kind of information is this allowing
the programmer to convey to the compiler?



I was talking about readability here. You can read an ordinary
(smallish) Nimrod program from *bottom to top*. (And a forward
declaration often screams "warning: mutually recursive procs ahead").
Granted, from *top to bottom* would be nicer, but I have never seen that
happen in practice in C# (which allows for arbitrary order); but
admittedly that's a very weak point.



But... I like my arbitrary ordering.  I don't necessarily define 
functions according to their call graph, but instead with functionality 
groupings and mnemonics.



I'm already skeptical because I have no intuition for how this allows
me to better optimize my code ;)



Btw, I've looked at Nimrod a while ago (year+) and found it very
elegant. I love good metaprogramming. I think my only complaints were
the bus-factor and the apparent lack of array slices (the kind that
doesn't cause copying). Still, very promising.


See my other answer about the slicing. The bus factor got better as
there are now 3 core developers. :-)



Congrats!

Re: Nimrod language

[Chad J]

On 05/24/2012 07:21 PM, Araq wrote:

On Thursday, 24 May 2012 at 22:56:52 UTC, Kevin Cox wrote:

On May 24, 2012 6:53 PM, "Froglegs"  wrote:


Like the design, syntax is way better than D

But half of what makes a language are the compilers/debuggers/tool


I like many ideas of the language but there are some show-stoppers for
me.
For example the fact that you have to define things in order. I shouldn't
have to deal with that in this day and age.


Nimrod is full of constructs that have inlining semantics and as such
declaration order matters quite a bit. The D compiler has/had bugs with
this feature for a reason. ;-)


OK, now I'm curious.  Why?


I'm considering to weaken the requirement but I don't mind this feature:
Having the order reflect the call graph has its advantages too. Many
consider the resulting order *backwards*, but at least there is *an* order.



Wouldn't it be good enough to define the order arbitrarily when building 
your symbol table?  What kind of information is this allowing the 
programmer to convey to the compiler?


I'm already skeptical because I have no intuition for how this allows me 
to better optimize my code ;)


Btw, I've looked at Nimrod a while ago (year+) and found it very 
elegant.  I love good metaprogramming.  I think my only complaints were 
the bus-factor and the apparent lack of array slices (the kind that 
doesn't cause copying).  Still, very promising.

Re: Getting the const-correctness of Object sorted once and for all

[Chad J]

On 05/15/2012 08:18 PM, Chris Cain wrote:

On Tuesday, 15 May 2012 at 23:36:38 UTC, Chad J wrote:

The idea /was/ to store the @instance variable with the object
specifically to avoid complex indirections. Still have to trust
programmers though :/


But you /can't/ store the @instance variable with the object. As per the
language reference, immutables may be stored in ROM. If the object is
immutable (and const objects might be immutable), then the @instance
variable would be stored in ROM, in which case it physically couldn't
change no matter how hard you tried (or it would vomit run-time errors
or some undefined behavior). The only "workable" solution would be an
immutable pointer to the mutable variable.

Link: http://dlang.org/const3.html



I guess this matters for toHash... so now we not only want 
pure/const/nothrow methods, but there is this implication that we are 
desiring to mutate immutable things.  If we really wanted to overcome 
this, we could have things that behave as if immutable, but carry 
mutable state internally.  These would not be eligible for placement in 
ROM.


The other thing that I'm wondering about is if there are use-cases 
/besides/ hashing.  Are there any?  If not, it might be worth 
special-casing somehow.  It's a difficult route to travel though, 
because it's very difficult to know that there won't be any roadblocks 
besides caching in the future.  Still, I suspect this is not well 
researched.

Re: Getting the const-correctness of Object sorted once and for all

[Chad J]

On 05/15/2012 06:41 PM, Chris Cain wrote:

On Tuesday, 15 May 2012 at 21:18:11 UTC, Chad J wrote:

On 05/15/2012 03:32 PM, Chris Cain wrote:

On Tuesday, 15 May 2012 at 18:07:12 UTC, Chad J wrote:

An idea I thought of is to introduce a method local declaration that
allows a method to access instance-specific-state that isn't
accessible to the rest of the class:


This is an interesting idea (as it seems to really try to keep
the state changes internal to the function, which can be seen as
how D handles purity)... however, it breaks the point of purity due to
this:

pure nothrow hash_t toHash() const {
@instance hash_t bad = 0;
++bad;
return hashfn(field) + bad;
}

Now it violates everyone's definition of purity and we can no
longer make our sweet optimizations and reasoning about the code.
Sure, we could "trust the programmers to not do this" ... but
that's another debate entirely.



Yes. I intend to "trust the programmers to not do this".

Otherwise we need to find some way to ensure that a function that
alters external state will always return the same value as long as the
rest of the program doesn't change the state it looks at.


It still wouldn't work though. Your @instance variables couldn't
be stored with the object (and, thus, would have to be a pointer
to mutable memory). So it'd require some backend work to make
sure that's even feasible (it is, you could have an immutable
pointer to a mutable pointer to the int, but let's face it:
that's further spitting in the face of the way D's type system
has been designed).

So, you'd have invisible indirections, additional complexity, and
you'd have to trust the programmers to be responsible. In other
words, C++ + the slowness of indirections.



The idea /was/ to store the @instance variable with the object 
specifically to avoid complex indirections.  Still have to trust 
programmers though :/


Otherwise it's better to just memoize things using external lookups and 
whatnot.  That solution does have complex indirections, but it doesn't 
require trusting the programmer and it exists already.



On Tuesday, 15 May 2012 at 22:33:56 UTC, Era Scarecrow wrote:

Perhaps an alternate workaround... a thought coming to mind, is that
the constructor for a const object lets you set it once in the
constructor (but not touch it again after) So Maybe...?


This is a good approach, but I think a lot of people want something
that's lazy ... if they don't need a hash, they don't want it to be
calculated for them.


FWIW, my idea: the thing we really need to do is to R&D some
design patterns & idioms for D. There's solutions for using const
and caching and such, but formalizing them and working out the
kinks to make sure it's optimal for everyone's circumstances
would be helpful. And people will have to accept that C++'s
particular idioms can't be used with D (and vice versa ... some
things you can do easily in D is infeasible or incorrect/invalid
with C++).

Book idea? :) I'd do it, but I'm just a student, so I haven't
seen even a decent subset of all possible software engineering
problems.

Re: Getting the const-correctness of Object sorted once and for all

[Chad J]

On 05/15/2012 03:32 PM, Chris Cain wrote:

On Tuesday, 15 May 2012 at 18:07:12 UTC, Chad J wrote:

An idea I thought of is to introduce a method local declaration that
allows a method to access instance-specific-state that isn't
accessible to the rest of the class:


This is an interesting idea (as it seems to really try to keep
the state changes internal to the function, which can be seen as
how D handles purity)... however, it breaks the point of purity due to
this:

pure nothrow hash_t toHash() const {
@instance hash_t bad = 0;
++bad;
return hashfn(field) + bad;
}

Now it violates everyone's definition of purity and we can no
longer make our sweet optimizations and reasoning about the code.
Sure, we could "trust the programmers to not do this" ... but
that's another debate entirely.



Yes.  I intend to "trust the programmers to not do this".

Otherwise we need to find some way to ensure that a function that alters 
external state will always return the same value as long as the rest of 
the program doesn't change the state it looks at.





To communicate intents to invalidate the cache:

class Foo
{
private toStringCacheValid = false;

public void methodThatInvalidatesCache()
{
...
toStringCacheValid = false;
}

public pure nothrow string toString() const
{
// strCache gets stored in an instance of Foo
// strCache is only accessable in this method body.
@instance string strCache = null;

if ( !toStringCacheValid )
{
// Observable change in strCache!
// ... because isCaching reveals it
// to everyone.
strCache = someComplicatedCalculation();
toStringCacheValid = true;
return strCache;
}
else
return strCache;
}
}


... and setting toStringCacheValid to true in toString violates
const, so this is absolutely not allowed. Sorry.



Yep, ya got me.



Maybe there's a solution, but I doubt the solution is something
the programmer can/should do completely transparently.

Re: Getting the const-correctness of Object sorted once and for all

[Chad J]

On 05/15/2012 02:49 PM, Christophe wrote:

Chad J , dans le message (digitalmars.D:167461), a écrit :

An idea I thought of is to introduce a method local declaration that
allows a method to access instance-specific-state that isn't accessible
to the rest of the class:


It is not good for caching results, since the cache often has to be
erased when the object is modified.


I did outline a way to invalidate caches with this.

Re: Getting the const-correctness of Object sorted once and for all

[Chad J]

On 05/13/2012 12:39 PM, Stewart Gordon wrote:

http://d.puremagic.com/issues/show_bug.cgi?id=1824

This has gone on for too long.

Object.toString, .toHash, .opCmp and .opEquals should all be const.
(It's also been stated somewhere that they should be pure and nothrow,
or something like that, but I forget where.)

This makes it a nightmare to use const objects in data structures, among
other things, at best forcing the use of ugly workarounds. There are
probably other, more serious effects of this that can't easily be worked
around.

It seems that the main obstacle is rewriting the relevant methods in
std.stream. The current implementation doesn't make sense anyway -
reading the entire contents of a file is certainly not the way to
generate a hash or string representation of the stream. I'm thinking the
hash should probably be the stream handle, and the string representation
could perhaps be the full pathname of the file. Of course, what it
should be for non-file streams is another matter. (This would be a
change at the API level, but when the API's as fundamentally flawed as
this)

Are there any other bits of druntime/Phobos that need to be sorted out
before these methods can be declared const/pure/nothrow once and for all?

Stewart.


I haven't read all of the replies here, but the gist I'm getting is that 
we have two contradictory interests:
(1.)  .toString(), .toHash(), .opCmp(), .opEquals(), should be 
const/pure/nothrow because their operations are inherently 
const/pure/nothrow and it would be both unintuitive and less reusable if 
they weren't.
(2.)  Marking these as const/pure/nothrow prevents caching and 
optimizations that are important for real-world code.


When I see a dichotomy like this forming, I have to think that we're 
missing something.  There is definitely a better way!  I, for one, 
wouldn't give up until it's found.




So I'll toss out an idea:

I think the const we want is a kind of "interface const" rather than an 
"implementation const".  Interface const means that calling the method 
will not cause any /observable/ state-changes in the referred object. 
Implementation const is stricter: it means that calling the method will 
not cause ANY state-changes in the referred object at all.


I am going to be fairly strict about my notion of observable.  Changes 
to private members are observable:


class Foo
{
private string strCache = null;

// The isCaching method makes strCache "observable".
public bool isCaching()
{
if ( strCache is null )
return false;
else
return true;
}

public string toString()
{
if ( strCache is null )
{
// Observable change in strCache!
// ... because isCaching reveals it
//   to everyone.
strCache = someComplicatedCalculation();
return strCache;
}
else
return strCache;
}
}


An idea I thought of is to introduce a method local declaration that 
allows a method to access instance-specific-state that isn't accessible 
to the rest of the class:


class Foo
{
// The isCaching method is no longer possible.

public pure nothrow string toString() const
{
// strCache gets stored in an instance of Foo
// strCache is only accessable in this method body.
@instance string strCache = null;

if ( strCache is null )
{
// Observable change in strCache!
// ... because isCaching reveals it
//   to everyone.
strCache = someComplicatedCalculation();
return strCache;
}
else
return strCache;
}
}

Now it is not possible (or at least, not very easy at all) for the 
statefulness of strCache to leak into the rest of the class (or 
program).  It is not "observable".  If the implementor does their 
caching wrong, then the statefulness might be observable from the 
toString method, but nowhere else (except for methods that call 
toString).  It's not a perfect situation, but it's /a lot/ better.  We 
may be required to trust the implementor a little bit and assume that 
they know how to make sure strCache's statefulness isn't observable (ex: 
two calls to toString() should return the same results).


To communicate intents to invalidate the cache:

class Foo
{
private toStringCacheValid = false;

public void methodThatInvalidatesCache()
{
...
toStringCacheValid = false;
}

public pure nothrow string toString() const
{
// strCache gets stored in an instance of 

Re: Wasn't someone trying to work on a C backend for DMD?

[Chad J]

On 04/29/2012 01:53 AM, Daniel Murphy wrote:

"Chad J"  wrote in message
news:jniju1$1pf7$1...@digitalmars.com...


Bummer, no commits in the last 3 months or so.



It was never a serious project unfortunately, just an experiment.


At any rate, I almost wonder if I can compile the output C code on the
OpenVMS system at work.  I have a project in C I work on from time to
time, but it's an uncomfortable (to say the least) environment to work
with.  There is stuff like this:
http://h30499.www3.hp.com/t5/Languages-and-Scripting/How-do-I-get-C-program-tracebacks-to-print-SOURCE-line-numbers/td-p/5570015
If I had DMD with C-backend at my disposal, then maybe I could instrument
it with a calling convention that gives me exception handling and proper
debug information.  Then I'd also get arrays and various other really nice
D features, even with just the basics.  I wouldn't bother getting the GC
to work, but it'd make me consider implementing some kind of reference
counting.  Even without ref counting I'd have way more at that point than
what I have currently with straight C code.

And if I /could/ get reference counting, and if the backend were solid
enough, then the thing could probably be used for writing games.  I'd have
the ability to output extremely portable C code to pretty much any target
except the web, and there's the JS backend for that.  Very enticing!



It is a lng way from being able to do that.  Your best bet is probably
to try and configure gdc or ldc to output c or compiled code for your
platform.  Completing MicroD would require porting most of druntime as well.


I wonder what it would take to get this sort of thing merged into
mainline.


A huge amount of work.




I was afraid you might say that.

Thanks for the input!

Re: Wasn't someone trying to work on a C backend for DMD?

[Chad J]

On 04/29/2012 01:02 AM, bls wrote:

On Saturday, 28 April 2012 at 23:40:39 UTC, Chad J wrote:

I've googled for it a bit and searched over newsgroup messages but I
can't find the post that mentioned this. I think it might have been
amidst the discussion of the javascript backend.

I want to download it and try it out.



http://www.dsource.org/projects/tdc



Interesting!  I'd forgotten about that (or possibly not heard of it?). 
5 years old though.  *sigh*, I wish stuff like this had taken off much 
earlier.

Re: Wasn't someone trying to work on a C backend for DMD?

[Chad J]

On 04/28/2012 11:36 PM, Nick Sabalausky wrote:

"Chad J"  wrote in message
news:jnhv5n$qe3$1...@digitalmars.com...

I've googled for it a bit and searched over newsgroup messages but I can't
find the post that mentioned this.  I think it might have been amidst the
discussion of the javascript backend.

I want to download it and try it out.


MicroD: https://github.com/yebblies/dmd/tree/microd

D->JS thread mentioning it in a few places:
http://forum.dlang.org/thread/yfmgvgprfpiquakiy...@forum.dlang.org?page=1




Hey, thanks!

Bummer, no commits in the last 3 months or so.

At any rate, I almost wonder if I can compile the output C code on the 
OpenVMS system at work.  I have a project in C I work on from time to 
time, but it's an uncomfortable (to say the least) environment to work 
with.  There is stuff like this:

http://h30499.www3.hp.com/t5/Languages-and-Scripting/How-do-I-get-C-program-tracebacks-to-print-SOURCE-line-numbers/td-p/5570015
If I had DMD with C-backend at my disposal, then maybe I could 
instrument it with a calling convention that gives me exception handling 
and proper debug information.  Then I'd also get arrays and various 
other really nice D features, even with just the basics.  I wouldn't 
bother getting the GC to work, but it'd make me consider implementing 
some kind of reference counting.  Even without ref counting I'd have way 
more at that point than what I have currently with straight C code.


And if I /could/ get reference counting, and if the backend were solid 
enough, then the thing could probably be used for writing games.  I'd 
have the ability to output extremely portable C code to pretty much any 
target except the web, and there's the JS backend for that.  Very enticing!


I wonder what it would take to get this sort of thing merged into mainline.

Wasn't someone trying to work on a C backend for DMD?

[Chad J]

I've googled for it a bit and searched over newsgroup messages but I 
can't find the post that mentioned this.  I think it might have been 
amidst the discussion of the javascript backend.


I want to download it and try it out.

Re: [off-topic] Sony releases PS Vita SDK

[Chad J]

On 04/22/2012 03:43 PM, Victor Vicente de Carvalho wrote:


Another option that crossed my mind was to do something like a
kickstarter funding to pay someone to do that fulltime. What you guys
think?



I think "FUCK YES".  I would dump money on that. ;)

Re: Orphan ranges

[Chad J]

On 04/15/2012 12:34 PM, Andrei Alexandrescu wrote:

I'm making good progress on an allocator design. If things come together
as I hope, it'll kick some serious ass.

I'm currently looking at four allocation models:

* straight GC, safe (no deallocation)
* GC + the ability to free memory
* malloc/free, unsafe, buyer beware (STL-level safety)
* reference counted (based on either malloc or GC+free)
* region (scoped)

I need to kink out a few details, most important being - is safety part
of the allocator or an extricable property that's a different policy?
For now I have a related question about orphan ranges.

...
Thanks,

Andrei


How about
* User-defined allocator
??

At the very least, I think we should be able to choose from the above 
strategies for a given container.  If this isn't done, then there will 
always be people wanting to re-implement the containers just to 
accomplish silly stuff like allocating a little differently.


As for safety:
I would probably want to be able to tweak the ways my containers are 
safe/unsafe.  I'd want safety by default, with the ability to choose my 
vulnerabilities.  "Safety" is a fairly vague notion, since there are a 
number of things that could cause memory corruption or undefined 
behavior.  Allowing me to decide which risk factors to take allows me to 
decide which causes of corruption/undefined behavior I want to expose 
myself to.  Once there's a sufficiently large panel of twiddley knobs 
and toggle switches, then it might make sense to create "policies" like 
"safe" vs "blazing fast".


As for orphan ranges:
I've always written under the assumption that an array like 'x' would 
keep 'b' around.  It would be nice if large amounts of memory wasted 
from orphans could be reused in some way, but not at the expense of a 
lot of speed.  If large waste from orphans sticks around, a warning in 
the docs would be nice.


Other note:
For the longest time I've thought that D should do reference counting 
for types that can be proven to have no reference cycles.  I don't 
understand why this is difficult, but even if it is it might be worth 
it.  I don't actually need it personally, but I constantly hear 
complaints about "oh, D requires garbage collection because you can't 
use most of Phobos without it".  What they really want, I bet, is 
deterministic memory management for realtime systems that does not cause 
thread contention/world stops.  Add reference counting and this will 
suddenly be doable for what I'd image would be a lot of important stuff 
in Phobos (think CoW string functions--probably reference counter 
friendly).  At this point the "reference counted" option gets subsumed 
into the "straight GC" / "GC+free" options, because the 
language/compiler would decide at compile-time which strategy to use for 
a range.


I see D's current coverage of memory management techniques looking like 
this:


- GC garbage collection: allocation for objects with complicated 
lifespans, with the caveat of complex allocation/deallocation code 
(check, not %100 featureful)


- malloc/free for mediocre-speed allocation of objects whose lifespans 
can't be easily calculated but are known by the programmer (check)


- custom allocators for when everything else fails (check, in most cases)

- memory reuse with slices/immutability which gives us an awesome 
zero-overhead solution in a number of situations (check)


- reference counting for almost-deterministic allocation of objects with 
easily calculated lifespans (**D does not cover this case!**)


- stack for fast allocation of short-lived objects (check)

Re: Precise GC

[Chad J]

Hey, that sounds awesome.  I think I geeked out a bit.

Would this make it any easier to reference count types that can be 
statically proven to have no cyclical references?

Re: Discussion on Go and D

[Chad J]

On 04/07/2012 05:42 PM, Andrei Alexandrescu wrote:

On 4/7/12 4:26 PM, Chad J wrote:

I keep reading that 'delete' is going to go away. Is this even
future-proof, or is code written in this fashion going to suffer a
reckoning later on?


Biggest confusions in the history of humankind:

1. Pyramids have been built by aliens;

2. Flying with a device heavier than air is impossible;

3. The ability to dispose of memory will disappear along with the delete
keyword.


Andrei


Oh.  Whoops.

Thanks!

Re: Discussion on Go and D

[Chad J]

On 04/06/2012 02:01 PM, Walter Bright wrote:

On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

I hope there is something wrong with my reasoning, and that you could
give me
some hints to avoid the memory bloat and the application stalls.


A couple of things you can try (they are workarounds, not solutions):

1. Actively delete memory you no longer need, rather than relying on the
gc to catch it. Yes, this is as unsafe as using C's free().



I keep reading that 'delete' is going to go away.  Is this even 
future-proof, or is code written in this fashion going to suffer a 
reckoning later on?


As an aside: I think this point generalizes to "avoid using the GC in 
cases where it is easy to do so".  D is very good at this due to having 
expressive structs, scope keyword, array slices, and a number of other 
features that synergize to track ownership/lifetime of memory without GC 
help.



2. Null out pointers & references when you are done with them. This
helps reduce unwanted pinning of unused gc memory.

3. Minimize use of global memory, as that is a major source of source of
roots.

Re: Regarding implementing a stable sort for Phobos

[Chad J]

On 03/13/2012 02:31 AM, Xinok wrote:

I've been playing with sorting algorithms a lot in recent months, so I
want to implement a *working* stable sort for Phobos which is broken at
the moment. I have a working library and I'm still adding to it. It's
much more complex than a simple merge sort, being over 300 lines of code
at the moment.

...


Hey, I'd love to see more sorting algorithms in phobos.  Being stuck 
with one seems kind of... wrong.


If the range is a linked list, shouldn't it be possible to do a merge 
sort with optimally in-place and use no extra memory whatsoever?  I know 
it can be done in-place, but I've never benchmarked it.  I wonder if 
it's worth considering, and how it would compare against array-based 
merge sort with allocations and such.


Although it's probably out of your scope right now, I'd like to see 
insertion sort some day.  I would use it for things like broadphase 
sorting in collision detection (that is, you sort everything by say, x 
coordinates first, and then you can walk through the whole simulation 
from left-to-right and have very few things to check collisions for at 
each point).  Since the ordering of the objects in the simulation is 
unlikely to change much between frames, it will be almost entirely 
sorted each time.  I have to imagine insertion sort would be awesome at 
that; nearly O(n).  Maybe if it hits more than log(n) nonlocal 
insertions it would bail out into a merge sort or something.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/13/2012 01:34 AM, H. S. Teoh wrote:

On Tue, Mar 13, 2012 at 03:17:42PM +1300, James Miller wrote:

On 13 March 2012 15:17, H. S. Teoh  wrote:

We could start off with said module just doing colors for now, and
then gradually add more stuff to it later.


We could end up at a D-flavoured ncurses library!

[...]

That would be a dream come true for me.

I have to admit that I find the ncurses API quite ugly, and not very
well-designed. It's the curse (pun intended) of inheriting a decades-old
API that was designed back in the days when people didn't know very much
about good API design.


T



Yes.

Maybe someday...

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/13/2012 01:41 AM, James Miller wrote:

On 13 March 2012 18:24, Chad J  wrote:

I'm not sure I agree with resetting to a default color.  What if I want to
write to the stream without altering the terminal's graphics settings?


Actually, I meant more to make sure that any output is reset to the
terminal's default. I'm pretty sure there is a way to do this. The
point is that not undoing mode changes is bad form.

Otherwise, I can live with the colourings being nested, but I would
suggest a change in syntax, I understand that yours is mostly just for
show, but using parenthesis will be annoying, I'd probably use braces
('{' and '}') instead, since they are less common.

writefln('%Cred(\(this is in color\))');
  vs
writefln('%Cred{(this is in color)}');

Neither are /that/ pretty, but at least the second one requires less
escaping in the common case.

--
James Miller


Oh, I see what you mean.

This is why the second paren always had a % before it:

writefln('%Cred((this is in color)%)');

Is this OK?  I know that escaping is still involved, but the text itself 
does not need escaping: only the special closing element does.


I like this constraint because it means that the only character you ever 
have to escape in your normal text is %, which you write by using %% 
instead.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/13/2012 12:15 AM, James Miller wrote:

On 13 March 2012 16:58, Chad J  wrote:

On 03/12/2012 10:37 PM, James Miller wrote:



I think the problem with putting it into formatting is that it is
inherently not output. IOW formatting should go anywhere, but colored
output is terminal-only.

Also, there are differences between terminals and all sorts of crap
that just make this harder to do "simply". However, there's no reason
why there cant be an easy way to colorize output in std.terminal (or
whatever), that are basically just modified writef(ln) calls
(colorWritef?) that only output to stdout (and maybe stderr). I think
this would be a good way around it, because then everything that is
terminal-specific is kept in one place, and you don't get mistakes
like outputting color to a file because you did the wrong sequence, or
forgot to check that stdout is a terminal and all that.

--
James Miller



I do want to be able to format things besides color with the color
formatting function.  Maybe I can pick out the color format specifiers first
and then pass the rest to format.  It'd be a shame to reimplement format.

At that point it would be cool if thrown exceptions and the like could do
color formatting, and also know when to strip it out when writing to log
files and such.  I don't know how difficult or practical it would be, but I
think that stack traces with color highlights would be awesome.  It's pretty
in-your-face user experience-wise too; might be good PR for D.

So then, now for the fun part.  What to name this function?

zoosmellPooplord(ln)("%Cred(Text.%)");


I wasn't suggesting to actually re-implement format, actually my idea
was similar to yours in that it would wrap over writef(ln). But doing
it this way, rather than extending writef, means that you can check
for output conditions first. It also allows people to realise that the
function they are reading /isn't/ just a bog-standard writefln, and
should be parsed (mentally) differently.



Why?  This would be an addition of features for writef(ln), not an 
alteration.  Existing writef(ln) usage would remain exactly the same.



However, I think that any more than just sequential output should
probably have its own set of functions, so a clearLine function,
clearScreen function, setColor functions and all that. This also
suggests that a color-writef should reset to a "default" color at the
end of output, which makes sense.



I totally agree with separating all of that stuff out into a separate 
block of code/functionality/API.  I actually see those all as being 
quite different in usage from the very simple sequential case.


I'm not sure I agree with resetting to a default color.  What if I want 
to write to the stream without altering the terminal's graphics settings?


(IMO those terminal graphics attributes are user data, and user data is 
always sacred.)



One thing I would change is to not use syntax quite so similar to
writef though, and also (since it matches better with terminal
operation anyway) not have it wrap, just set the color for the
remaining output, otherwise parsing will be more difficult, since you
have to check for yet more escape sequences, and using it would be
more error-prone. (For an example, vim has some weird escaping rules
for its regex-implementation that catch me out all the time)

--
James Miller


Ah, maybe it's preference.  I'd probably do both nesting and not-nesting 
variants.  I always /hated/ being /forced/ to use the modal 
(non-nesting) idea of changing THE color.  Instead, I see it very easily 
as a stack: push and pop the colors.  Otherwise I start writing 
"original format and then %Cred this text is red and then back to the 
%C. .. ... " FFF nooo!  what do I do to get back to the original 
formatting if I don't know what it is?!?!  Maybe I'm drainbamaged, but 
stuff like that will drive me nuts and leave me in mental lockout for a 
number of minutes.


Care to share some examples of the things that frustrate you?

I would prefer that any unclosed/unmatched nesting formatters would be 
automatically closed at the end of the format string.  If only nesting 
formatters are used, it would be least surprising to me if the formatter 
just didn't change the terminal's original settings at all:


// Terminal is currently set to bold-green.
writefln("%Cred(But this text will be red.");
writefln("And this text will be bold-green.");

If I used a non-nesting variant at toplevel and change the current mode, 
then I would expect /that/ to alter the terminal's original settings and 
leave it in a different state than from before the formatted write:


// Terminal is currently set to bold-green.
writefln("%Cred(But this text will be red.");
writefln("And this text will be bold-green.");
writefln("Still bold-green, until %Cblu hey, it's blue now.");
// Terminal is currently set to blue.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/12/2012 11:58 PM, Chad J wrote:

On 03/12/2012 10:37 PM, James Miller wrote:


I think the problem with putting it into formatting is that it is
inherently not output. IOW formatting should go anywhere, but colored
output is terminal-only.

Also, there are differences between terminals and all sorts of crap
that just make this harder to do "simply". However, there's no reason
why there cant be an easy way to colorize output in std.terminal (or
whatever), that are basically just modified writef(ln) calls
(colorWritef?) that only output to stdout (and maybe stderr). I think
this would be a good way around it, because then everything that is
terminal-specific is kept in one place, and you don't get mistakes
like outputting color to a file because you did the wrong sequence, or
forgot to check that stdout is a terminal and all that.

--
James Miller


I do want to be able to format things besides color with the color
formatting function. Maybe I can pick out the color format specifiers
first and then pass the rest to format. It'd be a shame to reimplement
format.

At that point it would be cool if thrown exceptions and the like could
do color formatting, and also know when to strip it out when writing to
log files and such. I don't know how difficult or practical it would be,
but I think that stack traces with color highlights would be awesome.
It's pretty in-your-face user experience-wise too; might be good PR for D.

So then, now for the fun part. What to name this function?

zoosmellPooplord(ln)("%Cred(Text.%)");


Actually, wait a sec.

So I don't do it in format.

But I can create a color-format function that takes a terminal spec does 
some color formats and optionally some std.format formats as well.  It 
would output a string with the correct escape sequences (hey, you never 
know when you might want to inspect the escape sequences it produces).


So we define
string std.terminal.colorFormat(TerminalSpec spec, string fmtstr);
and std.format remains untouched.
And then...

Why not have writef(ln) use it?  writef(ln) functions know what they are 
attached to right?


They can just strip the formatting out for non-terminal destinations.

I would much prefer this.  I don't like the idea of having different 
writefln and termfln when they both do the same damn thing sans some 
simple coloring functionality.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/12/2012 10:37 PM, James Miller wrote:


I think the problem with putting it into formatting is that it is
inherently not output. IOW formatting should go anywhere, but colored
output is terminal-only.

Also, there are differences between terminals and all sorts of crap
that just make this harder to do "simply". However, there's no reason
why there cant be an easy way to colorize output in std.terminal (or
whatever), that are basically just modified writef(ln) calls
(colorWritef?) that only output to stdout (and maybe stderr). I think
this would be a good way around it, because then everything that is
terminal-specific is kept in one place, and you don't get mistakes
like outputting color to a file because you did the wrong sequence, or
forgot to check that stdout is a terminal and all that.

--
James Miller


I do want to be able to format things besides color with the color 
formatting function.  Maybe I can pick out the color format specifiers 
first and then pass the rest to format.  It'd be a shame to reimplement 
format.


At that point it would be cool if thrown exceptions and the like could 
do color formatting, and also know when to strip it out when writing to 
log files and such.  I don't know how difficult or practical it would 
be, but I think that stack traces with color highlights would be 
awesome.  It's pretty in-your-face user experience-wise too; might be 
good PR for D.


So then, now for the fun part.  What to name this function?

zoosmellPooplord(ln)("%Cred(Text.%)");

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/12/2012 07:51 PM, Damian Ziemba wrote:

On Monday, 12 March 2012 at 03:32:26 UTC, Chad J wrote:

On 03/11/2012 11:27 PM, Damian Ziemba wrote:

On Monday, 12 March 2012 at 02:52:15 UTC, Andrei Alexandrescu wrote:

On 3/11/12 9:16 PM, Chad J wrote:

I remember doing colored terminal output in Python. It was pretty
nifty,
and allows for some slick CLI design. I think D can do better by
putting
it in the standard library.

...

So, would this sort of thing make it in?


I don't know, seems interesting but I wonder how portable that could
be. Probably I'd define a more general means a la %q to mean "send a
control sequence" and then would define control sequences as functions
or constants.


...


Andrei


It could work.
In my small framework I use version blocks and I use ansi escape
sequences for posix and SetConsoleTextAttribute for windoze.

Ofcourse there would be a need to create unified enumeration with colors
as they differ on those platforms too.



Good catch.



public enum Font
{
Normal = 0,
Underline = 0x8000,
Reverse = 0x4000,
}

public enum Color
{
Default = 0x,
Blue = 0x0001,
Green = 0x0002,
Aqua = 0x0003,
Red = 0x0004,
Purple= 0x0005,
Yellow= 0x0006,
Gray = 0x0008,
LightBlue = 0x0009,
LightGreen = 0x000A,
LightAqua = 0x000B,
LightRed = 0x000C,
LightPurple= 0x000D,
}

Those are colors and font-attributes that I found to match both Windows
and Posix



If you can show me you're work and it's licensed in a way that I can
use it in Phobos, then I'll happily try to make use of it. It'd be
much appreciated.


Hey Chad.

Sorry for delay, been a bit busy.
Here it is: https://gist.github.com/2025473
I hope it can help you somehow.
(There is place for improvments I know but I used it mostly for
debbuging stuff ;))

And yea, I think like others that it should have its own module like
std.terminal/std.console or maybe somekind of spot in std.stdio.

Best Regards,
Damian Ziemba


Hey, np about the delay.  I'm not going to get to working on this stuff 
that soon anyways.


Thanks for the code!

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/12/2012 11:02 AM, H. S. Teoh wrote:

On Mon, Mar 12, 2012 at 10:51:08AM +0100, Jacob Carlborg wrote:

On 2012-03-12 03:16, Chad J wrote:

I remember doing colored terminal output in Python. It was pretty
nifty, and allows for some slick CLI design. I think D can do better
by putting it in the standard library.

I was thinking something along the lines of this:
http://www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html

I figure it would probably be easy to get some of the basics down.
More advanced stuff would probably involve messing with terminfo or
.  Windows' (terribly bad) command prompt can have some of
these capabilities implemented too, but in a roundabout way because
Windows defines API functions that need to be called to affect
terminal graphics and coloring. I figure that would require the help
of the I/O routines if I were to work on that.

If there's interest, I might take a stab at it.

So, would this sort of thing make it in?


I think it would nice to have, but not in std.format. std.terminal or
similar would be better.

[...]

+1.

It's better not to pollute std.format with stuff that, strictly
speaking, isn't related to formatting per se, but is tied to a
particular output medium.  This is proven by the fact that the
translation of color escapes only makes sense w.r.t. a particular
terminal, so you'll get garbage if you call std.format on the string,
save it to file, say, then load it later and output it to a possibly
different terminal type.

So what you *really* want is to translate these escape sequences *at
output time*, not at string formatting time. If we do it that way, we
can have the nicer behaviour that these escapes will work on any
terminal and can be freely moved around from terminal to terminal,
because they are only interpreted at the instant when they are actually
being output to that terminal.


T




I never felt it would go into std.terminal because I have a fairly 
narrow purpose in mind.  If I wanted to do anything a bit fancier, I 
would be all about making std.terminal.  But currently I feel that this 
would be orthogonal to such a module.  This is very lightweight on the 
learning side, while std.terminal would include code to handle multiple 
terminals and interactivity: it would be much heavier and a bigger 
mental investment from the user.



Anyhow, here are my objectives:

What I really want is to have an /easy/ way to do color formatting.  I 
want to be able to write this:


writefln("%Cred(Red text:%) Normal text.");

The syntax should be similar to format strings so that memorization is 
easier due to the extra association, and also so that we don't escapify 
yet another special character: '%' and '\' are already required to be 
written as '%%' and '\\' in various contexts and it would be annoying to 
have yet another one.


I suppose that could be handled output side, but I don't know how I'd 
avoid stuff like this:


writefln("%%Cred(Red text:%%) %d complete.",percentComplete);

It also made a lot of sense to me that I would find documentation on 
coloring in the place I'd find documentation on formatting.  Maybe this 
isn't what others feel.  Nonetheless, I'm inclined to start from 
intuition and refine if there are objections.


There are probably going to be multiple ways to push things to the 
terminal (maybe in multiple source files too: stdio vs stream?) and it 
should be possible to colorize all of them in a uniform manner. 
Documentation should be easy to find; discovery should be intuitive.



If someone has a good vision that accomplishes these things, then please 
do share.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/12/2012 09:36 PM, Christian Manning wrote:

On Monday, 12 March 2012 at 09:51:08 UTC, Jacob Carlborg wrote:


I think it would nice to have, but not in std.format. std.terminal or
similar would be better.


It would be great if an std.terminal contained general stuff for
manipulating/querying a terminal portably, as well as colour output, eg.
get terminal size, move cursor around, erase line... just things to help
with building UIs, progress bars, etc. that are easy to use.


Although this would be cool, it is out of the scope of what I am willing 
to spend time on.


My intent is to make color output easy.  Handling interactive stuff is a 
completely different beast from the usage standpoint.


I might do it someday if I feel like writing a game using terminal 
graphics, but even then there is no guarantee it would be phobos worthy.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/11/2012 11:27 PM, Damian Ziemba wrote:

On Monday, 12 March 2012 at 02:52:15 UTC, Andrei Alexandrescu wrote:

On 3/11/12 9:16 PM, Chad J wrote:

I remember doing colored terminal output in Python. It was pretty nifty,
and allows for some slick CLI design. I think D can do better by putting
it in the standard library.

...

So, would this sort of thing make it in?


I don't know, seems interesting but I wonder how portable that could
be. Probably I'd define a more general means a la %q to mean "send a
control sequence" and then would define control sequences as functions
or constants.


...


Andrei


It could work.
In my small framework I use version blocks and I use ansi escape
sequences for posix and SetConsoleTextAttribute for windoze.

Ofcourse there would be a need to create unified enumeration with colors
as they differ on those platforms too.



Good catch.



public enum Font
{
Normal = 0,
Underline = 0x8000,
Reverse = 0x4000,
}

public enum Color
{
Default = 0x,
Blue = 0x0001,
Green = 0x0002,
Aqua = 0x0003,
Red = 0x0004,
Purple= 0x0005,
Yellow= 0x0006,
Gray = 0x0008,
LightBlue = 0x0009,
LightGreen = 0x000A,
LightAqua = 0x000B,
LightRed = 0x000C,
LightPurple= 0x000D,
}

Those are colors and font-attributes that I found to match both Windows
and Posix



If you can show me you're work and it's licensed in a way that I can use 
it in Phobos, then I'll happily try to make use of it.  It'd be much 
appreciated.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/11/2012 11:05 PM, Adam D. Ruppe wrote:

One concern I have with this is format() creates a
string, which isn't necessarily output; color is
a thing of output.

The unix implementation will use the ansi escape
sequences, surely, which isn't correct almost
anywhere else.



I was kind of intending to /not/ do that, for exactly the reasons you 
mention.  ASCII escape sequences should work anyways.  I don't think 
anyone will panic if I waste a byte or two for every 3+ on fairly rare 
coloring/gfx operations.  And then there's always terminfo in the long 
picture.



Of course, you could choose to not use these
special specifiers, and put a note in the
documentation explaining what it is, so
not a big deal... but I just think it is
somewhat wrong to put special control
sequences in the middle of a regular string
that might be used anywhere.


The primary intent of this is to be used with writefln or any other 
routines that will be writing to the terminal.


That this can emit escape sequences to other targets is a consequence of 
std.format being abstracted from the I/O routines.  I think it is fine. 
 I doubt people will put color formatting into strings that they know 
will not end up on a terminal.  And if they don't realize that 
limitation, then perhaps I should change the way it's documented so that 
it's /really obvious/ that the resulting strings will only colorize on 
process's attached terminal, and produce gobbledygook otherwise.

Re: How about colors and terminal graphics in std.format?

[Chad J]

On 03/11/2012 10:52 PM, Andrei Alexandrescu wrote:

On 3/11/12 9:16 PM, Chad J wrote:

I remember doing colored terminal output in Python. It was pretty nifty,
and allows for some slick CLI design. I think D can do better by putting
it in the standard library.

I was thinking something along the lines of this:
http://www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html

...

If there's interest, I might take a stab at it.

So, would this sort of thing make it in?


I don't know, seems interesting but I wonder how portable that could be.
Probably I'd define a more general means a la %q to mean "send a control
sequence" and then would define control sequences as functions or
constants.



My intent is to make it easy to use.  If it's not easy to use, I 
wouldn't use it.  If someone wants low-level, they can help themselves 
to curses.  It should be possible to do this reasonably portably.  I 
know Python was able to do this well on both my Linux install and my 
Windows install.


If other OSes have known issues, it can be solved by using either better 
feature detection or simply version()'ing out the code that inserts 
control sequences on those OSes.


My plan would be something like this:

version(Windows)
{
// There is only one possible way to do this:
// Call WinAPI functions.
}
else version(Posix)
{
Look for the terminfo database.
Check $TERMINFO,
then ~/.terminfo,
then /usr/share/terminfo,
then /usr/lib/terminfo,
then /etc/terminfo,
then give up if not found.
if ( terminfo found )
{
query terminfo for terminal capabilities and sequences
emit sequences as appropriate
}
else
{
Don't output control sequences.
}

// Using  might be easier,
//   but I don't know how portable it is. :/
}
else
{
// Don't output control sequences.
}



Oh, on an unrelated note, Phobos' documentation make target is quite
broken:
blahblah/dmd.git/src/phobos $ make -fposix.mak html
make: *** No rule to make target `../web/phobos-prerelease/index.html',
needed by `html'. Stop.

I examined the makefile and concocted this line of bash that constructs
my desired html file:
dmd -m32 -d -c -o- -version=StdDdoc -I../druntime/import std/format.d
std.ddoc -Dfstd_format.html
and copied std.ddoc from a release version of dmd (it's in src/phobos).


Since recently the Phobos doc build is meant to be driven from the site
build. I'll fix the standalone thing because it's useful too, just I
don't know when.



Andrei


Alright, thanks!

Btw, did we ever get a git repo that includes the release files for D 
and tracks dmd/druntime/phobos as sub-repositories at the correct paths? 
 Such a thing would be really useful for me if I want to feel like 
working on this stuff very much.  I don't think I have the ability to 
update DMD documentation from git right now.

How about colors and terminal graphics in std.format?

[Chad J]

I remember doing colored terminal output in Python.  It was pretty 
nifty, and allows for some slick CLI design.  I think D can do better by 
putting it in the standard library.


I was thinking something along the lines of this:
http://www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html

I figure it would probably be easy to get some of the basics down.  More 
advanced stuff would probably involve messing with terminfo or . 
 Windows' (terribly bad) command prompt can have some of these 
capabilities implemented too, but in a roundabout way because Windows 
defines API functions that need to be called to affect terminal graphics 
and coloring.  I figure that would require the help of the I/O routines 
if I were to work on that.


If there's interest, I might take a stab at it.

So, would this sort of thing make it in?



Oh, on an unrelated note, Phobos' documentation make target is quite broken:
blahblah/dmd.git/src/phobos $ make -fposix.mak html
make: *** No rule to make target `../web/phobos-prerelease/index.html', 
needed by `html'.  Stop.


I examined the makefile and concocted this line of bash that constructs 
my desired html file:
dmd -m32 -d -c -o- -version=StdDdoc -I../druntime/import std/format.d 
std.ddoc -Dfstd_format.html

and copied std.ddoc from a release version of dmd (it's in src/phobos).

Re: Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

On 03/11/2012 05:10 PM, David Nadlinger wrote:

On Sunday, 11 March 2012 at 02:24:15 UTC, Chad J wrote:

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1
I was wonder if anyone has ported it.


If I were to implement a texture rendering system for 3D graphics, I'd
look into signed distance field fonts, as presented in Valve's seminal
paper [1].

David


[1]
http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf



Sounds interesting.  Thanks!

Re: Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

On 03/11/2012 04:24 AM, Kiith-Sa wrote:

On Sunday, 11 March 2012 at 03:45:36 UTC, Chad J wrote:

On 03/10/2012 10:29 PM, bioinfornatics wrote:

Le samedi 10 mars 2012 à 21:23 -0500, Chad J a écrit :

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1
I was wonder if anyone has ported it.


take derelict2 http://www.dsource.org/projects/derelict [stable]
derelict3 in development https://github.com/aldacron/Derelict3



I already have OpenGL working with derelict2.

How do these links help with rendering text in OpenGL?


I have some code based on that tutorial in
https://github.com/kiith-sa/ICE,
but beware that it's not the most effective code
and not particularly readable either.

However, I abstracted it away it to pack any textures instead of just
font glyphs, and it is spread over many files.
The relevant files are:

video/glvideodriver.d
video/texturepage.d
video/gltexturebackend.d
video/binarytexturepacker.d
video/font.d
video/fontmanager.d


Works with DMD 2.058, but it'd probably be nontrivial to rip it
out and use separately.




Thanks for the link!

I don't have time to go over it right now, but that looks promising.  I 
took a shot at porting Tomasz's code a while ago, but I never got it to 
compile.  At least your code /compiles/, so even if it's integrated into 
other stuff, at least I might stand a better chance.


Btw, also a fan of Raptor and Tyrian.  Good taste dude.  ;)

Re: Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

On 03/11/2012 05:04 PM, Jacob Carlborg wrote:

On 2012-03-11 14:18, Chad J wrote:

On 03/11/2012 09:09 AM, Jacob Carlborg wrote:

On 2012-03-11 03:23, Chad J wrote:

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1
I was wonder if anyone has ported it.


Just use FreeType?



FreeType is quite cool, but it will only render glyphs onto small
bitmaps. Getting those glyphs onto the screen becomes more complicated
after that: you can't just blit them like in SDL or somesuch. Since
textures don't come in "supersmall" size, it is effective to pack the
glyphs into a larger texture like the article suggests. Then you have to
texture-map correctly to only render the glyphs you want. Somewhere in
there is a bunch of other font-setting stuff like kerning, sub-pixel
anti-aliasing, line offsets, etc. FreeType doesn't do that stuff for
anyone, it just provides the tools necessary to make it possible. It's
fairly complicated; I'd rather just write "Font f = new Font("Courier
New"); f.draw(100, 100, "Hello world!");".


So you mean putting a texture on a font?



No, just getting the font onto the screen at all.

And I want to:
- Be able to size the font smoothly.  (No bitmap fonts!)
- Draw glyphs for higher unicode codepoints.  (No texture[128]!)
- Have kerning/hinting.  (Freetype makes it possible, but does not do it 
for you.)

- Have anti-aliasing and maybe even subpixel accuracy.

Tomasz's code does all of this.

Freetype is very low-level.  Freetype does not make OpenGL calls.  You 
give it a font and a codepoint and it gives you a small grayscale bitmap 
and a slew of metrics for it.  Then it's up to you to figure out where 
to put it and how to get it onto the screen.


If this were trivial, Tomasz (and others who have written on the topic, 
too) would not have written 5-10 pages of text plus a bunch of D files 
full of code.  It's just not that easy.  It's all in the original link.


It's actually really weird that there isn't much library code out there 
to render text in OpenGL.  There are a bunch of tutorials that do it 
wrong and maybe a few library thingies that I can't use for various reasons.

Re: Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

On 03/11/2012 09:09 AM, Jacob Carlborg wrote:

On 2012-03-11 03:23, Chad J wrote:

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1
I was wonder if anyone has ported it.


Just use FreeType?



FreeType is quite cool, but it will only render glyphs onto small 
bitmaps.  Getting those glyphs onto the screen becomes more complicated 
after that: you can't just blit them like in SDL or somesuch.  Since 
textures don't come in "supersmall" size, it is effective to pack the 
glyphs into a larger texture like the article suggests.  Then you have 
to texture-map correctly to only render the glyphs you want.  Somewhere 
in there is a bunch of other font-setting stuff like kerning, sub-pixel 
anti-aliasing, line offsets, etc.  FreeType doesn't do that stuff for 
anyone, it just provides the tools necessary to make it possible.  It's 
fairly complicated; I'd rather just write "Font f = new Font("Courier 
New"); f.draw(100, 100, "Hello world!");".

Re: Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

On 03/10/2012 10:29 PM, bioinfornatics wrote:

Le samedi 10 mars 2012 à 21:23 -0500, Chad J a écrit :

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1
I was wonder if anyone has ported it.


take derelict2 http://www.dsource.org/projects/derelict [stable]
derelict3 in development https://github.com/aldacron/Derelict3



I already have OpenGL working with derelict2.

How do these links help with rendering text in OpenGL?

Has Tomasz's (h3r3tic's) OpenGL font rendering code been ported to D2?

[Chad J]

I've been looking for a good way to render font in OpenGL with D.

Back in they day I was very impressed with Tomasz's article on the 
subject: http://h3.gd/dmedia/?n=Tutorials.TextRendering1

I was wonder if anyone has ported it.

Re: Feq questions about the D language

[Chad J]

On 03/10/2012 07:43 AM, DFGH wrote:

I have a large GUI application written in C# and i want
to rewrite it in D. Is there something like the System.Drawing
available?


If you can make due with fairly low-level access to the screen, then I 
might suggest checking out the Derelict2 bindings and consider SDL or 
OpenGL.  Now, these are probably much harder to use than System.Drawing: 
you'll have to implement a lot of the primitives from lower things. 
OpenGL perhaps even moreso, since you'd have to learn its model before 
you even get very many pixels onto the screen.  This would be more of a 
thing for game devs than for apps devs, since in game dev you'll be 
spending so much time on game-specific stuff that writing a few graphics 
primitives becomes a drop in the bucket at times.  I figure you're 
probably going to want something that gets you going faster.  H, 
there's probably a better way, but I thought I'd mention this incase 
it's of any worth.


It makes me wonder what's happening with all of the old team0xf code 
written in D1.  IIRC there was a lot of it, and it went places.

Re: dereferencing null

[Chad J]

On 03/08/2012 10:40 AM, H. S. Teoh wrote:

On Thu, Mar 08, 2012 at 02:57:00PM +0100, Andrej Mitrovic wrote:


I think what Chad is looking for is a BlackHole/WhiteHole equivalent
which doesn't need abstract functions but which figures out all the
methods of a class at compile time and creates a subclass that
throws/does nothing on method invocation. An 'alias this' field would
be used that is default-initialized with this sentry object. I don't
know why we don't have __traits(allFunction). We have
'getVirtualFunctions' but it requires a function name, but using
allMembers to filter out function names is damn difficult if you ask
me. I've never had an easy time interacting with __traits.


Yep.  That would be cool.  Although there should be ways of doing the 
same thing for arrays and possibly even structs.  For structs I wouldn't 
mind adding a boolean field to keep track of its empty-or-not status.




foreach (name; __traits(allMembers, typeof(obj))) {
static if (__traits(compiles,&__traits(getMember, obj,
name)))
{
alias typeof(__traits(getMember, obj, name))
type;
static if (is(type==function)) {
// name refers to a function of type
// 'type' here
}
}
}


I've never had an easy time interacting with __traits.


Me too. I'm guessing that __traits is the way it is due to ease of
implementation in the compiler. It's certainly not very friendly to use.


T



Tried this, but it wasn't picking everything up.

I also suspect that inheriting the class being wrapped and picking on 
its methods is going to be a losing battle in the long run for this 
thing.  It doesn't allow the sentry to throw on field access.  It also 
wouldn't work for final classes.  I wonder if opDispatch would do better.


Another mess I was running into was forwarding templated methods.  I 
wonder if this is even possible.  I wish __traits had some way of 
picking up on previous template instantiations (with cycles forbidden, 
of course).  If __traits were beefed up enough, maybe it would be better 
than opDispatch after all.  Hmmm.


I did try this as a struct with an "private T t; alias t this;" in it. 
It looks like this:


---

import std.c.stdlib;
import std.stdio;

struct Emptiable(T)
{
private static Emptiable!T m_sentinel;
// private NotNull!T t; // TODO
private T t;

alias t this;

public static @property Emptiable!T sentinel()
{
return m_sentinel;
}

static this()
{
void* sentinelMem = malloc(T.sizeof);
m_sentinel = cast(T)sentinelMem;
}

this(A...)(A args)
{
t = new T(args);
}

@property bool isEmpty() const
{
if ( this is m_sentinel )
return true;
else
return false;
}
}

static auto makeEmpty(T)(ref T v)
{
v = T.sentinel;
return v;
}

class Bar
{
int i;

this(int j) { i = j; }

void blah()
{
writefln("blah!");
}

/+void letsTemplate(T)(T f)
{
writefln("%s",f);
}+/
}

void main()
{
auto bar = Emptiable!(Bar).sentinel;
auto foo = new Emptiable!Bar(5);

if ( bar.isEmpty )
writefln("bar is empty, as it should be.");

if ( !foo.isEmpty )
writefln("foo is full, as it should be.");

//foo.letsTemplate("Just a string.");

writefln("foo.i is %s",foo.i);
foo.i = 2;
writefln("foo.i is %s",foo.i);

/+
makeEmpty(foo);
if ( foo.isEmpty )
writefln("foo is now empty.");
writefln("foo.i is %s",foo.i);
+/
}

---

Prints:
bar is empty, as it should be.
foo is full, as it should be.
foo.i is 5
foo.i is 2

---

It's super incomplete.  I am starting to realize that this sort of thing 
leads to a huge amount of weird corner cases.


Also note what happens when it tries to use makeEmpty: it fails because 
"auto foo = new Emptiable!Bar(5);" allocates an instance of the 
Emptiable struct on the heap, which then allocates a separate instance 
of Bar.  Yuck.  It's that old struct-vs-class construction schism again.

Re: Automatic minimally sized SELECT clauses in SQL using D templates.

[Chad J]

On 03/08/2012 07:00 AM, Marco Leise wrote:

Am 04.03.2012, 19:58 Uhr, schrieb Chad J
:


I'd almost be sort of surprised if someone here hasn't thought of this
already, but I'll drop it out here anyways.

This is a proof-of-concept program I wrote to do a query with the
convenience of a "SELECT * FROM ...", but without actually pulling every
column in the database.


Yes, there is an existing D library that use CTFE to generate lazy
evaluated data structures from SQL queries. And although I was very
impressed by the solution and how it integrated with foreach I forgot
the name and URL. (Haven't been using SQL since a while.) Maybe someone
else remembers. I usually find it helpful to look at other peoples'
solution to the same problem, since D offers so many concepts and you
easily miss some good stuff.


Cool, thanks for pointing that out.

Re: dereferencing null

[Chad J]

On 03/07/2012 11:17 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 22:58:44 Chad J wrote:

On 03/07/2012 10:40 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 22:36:50 Chad J wrote:

On 03/07/2012 10:08 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 20:44:59 Chad J wrote:

On 03/07/2012 10:21 AM, Steven Schveighoffer wrote:

You can use sentinels other than null.

-Steve


Example?


Create an instance of the class which is immutable and represents an
invalid value. You could check whether something is that value with the
is operator, since there's only one of it. You could even make it a
derived class and have all of its functions throw a particular exception
if someone tries to call them.

- Jonathan M Davis


Makes sense.  Awfully labor-intensive though.  Doesn't work well on

classes that can't be easily altered.  That is, it violates this:

- Do not modify the implementation of UnreliableResource.  It's not
always
possible.


But, maybe it can be turned it into a template and made to work for
arrays too...


Personally, I'd probably just use null. But if you want a sentinel other
than null, it's quite feasible.

- Jonathan M Davis


Wait, so you'd use null and then have the program unconditionally crash
whenever you (inevitably) mess up sentinel logic?


Yes. Proper testing will find most such problems. And it's not like having a
non-null sentinel is going to prevent you from having problems. It just means
that you're not distinguishing between a variable that you forgot to
initialize and one which you set to the sentinel value. Your program can die
from a variable being null in either case. And in _both_ cases, it's generally
unsafe to continue executing your program anyway.



The important difference in using explicit sentinel values here is that 
they are not null, and thus very unlikely to have been caused by memory 
corruption.  It allows us to distinguish between the two sources of 
empty variables.


With a better way to do sentinel values, I can isolate my cleaner 
looking code from the scarier looking code that comes from any number of 
places.


I also am not too worried about null values that come from stuff that 
was simply forgotten, instead of intentionally nulled.  I DO tend to 
catch those really early in testing, and they are unlikely to happen to 
begin with due to the close association between declaration and 
initialization.



And honestly, in my experience, null pointers are a very rare thing. You catch
them through solid testing.

- Jonathan M Davis



Sorry, your testing doesn't help me as well as you probably wish it 
does.  Our experiences must be very different.  I run into a lot of 
cases where things can't be tested automatically, or at least not 
easily.  Think along the lines of graphics operations, interactively 
driven code (ex: event lifetimes), network code, etc.  Testing can help 
things between endpoints, but it doesn't help much where the rubber 
meets the road.


And that's just game dev.  Then I go to work at my job, the one that 
makes money, and experience code from the 80s.  Rewriting it is 
completely impractical for near-term projects (though a complete 
phase-out of crufty old crap is on the horizon one way or another!). 
Yes it has bugs.  If I had an attitude of "crash on every little nit" 
then these things wouldn't last a few seconds (OK, exaggeration).  So I 
recover as well as possible, and occasionally rewrite strategically 
important pieces.  But the world is NOT perfect, so relying on it being 
perfect is %100 unhelpful to me.  Also, "quit your job" is not an 
acceptable solution. ;)  Now, in principle, we will never have to deal 
with D code like that.  Nonetheless, these experiences do make me 
severely afraid of lacking the tools that keep me safe.


And then there are still those occasional weird problems where sentinel 
values are needed, and its so stateful that there's a vanishingly 
close-to-zero chance that testing will catch the stuff that it needs to.
So I test it as well as I can and leave a "if all else fails, DO THIS" 
next to the dubious code.  Indiscriminate segfaulting deprives me of 
this last-ditch option.  There is no longer even a way to crash 
elegantly.  It all just goes to hell.


Long story short: in practice, I find that recovering from sentinel 
dereference is not only VERY safe, but also orders of magnitude less 
frustrating for both my users and me.


(Memory corruption, on the other hand, is something I am very unfamiliar 
with, and sort of afraid of.  So I'm willing to ditch nulls.)

Re: dereferencing null

[Chad J]

On 03/07/2012 10:40 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 22:36:50 Chad J wrote:

On 03/07/2012 10:08 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 20:44:59 Chad J wrote:

On 03/07/2012 10:21 AM, Steven Schveighoffer wrote:

You can use sentinels other than null.

-Steve


Example?


Create an instance of the class which is immutable and represents an
invalid value. You could check whether something is that value with the
is operator, since there's only one of it. You could even make it a
derived class and have all of its functions throw a particular exception
if someone tries to call them.

- Jonathan M Davis


Makes sense.  Awfully labor-intensive though.  Doesn't work well on

classes that can't be easily altered.  That is, it violates this:

- Do not modify the implementation of UnreliableResource.  It's not always
possible.

But, maybe it can be turned it into a template and made to work for
arrays too...


Personally, I'd probably just use null. But if you want a sentinel other than
null, it's quite feasible.

- Jonathan M Davis


Wait, so you'd use null and then have the program unconditionally crash 
whenever you (inevitably) mess up sentinel logic?

Re: dereferencing null

[Chad J]

On 03/07/2012 10:08 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 20:44:59 Chad J wrote:

On 03/07/2012 10:21 AM, Steven Schveighoffer wrote:

You can use sentinels other than null.

-Steve


Example?


Create an instance of the class which is immutable and represents an invalid
value. You could check whether something is that value with the is operator,
since there's only one of it. You could even make it a derived class and have
all of its functions throw a particular exception if someone tries to call
them.

- Jonathan M Davis


Makes sense.  Awfully labor-intensive though.  Doesn't work well on 
classes that can't be easily altered.  That is, it violates this:

- Do not modify the implementation of UnreliableResource.  It's not always 
possible.


But, maybe it can be turned it into a template and made to work for 
arrays too...

Re: dereferencing null

[Chad J]

On 03/07/2012 07:39 PM, Timon Gehr wrote:

On 03/08/2012 01:24 AM, Chad J wrote:


Though, more to the point:
I would probably forbid "Foo foo = new Foo(this);". The design that
leads to this is creating circular dependencies, which is usually bad to
begin with.


Circular object references are often justified.


Would we lose much of value?


Well this would amount to forbidding escaping an object from its
constructor, as well as forbidding calling any member functions from the
constructor. Also, if you *need* to create a circular structure, you'd
have to use sentinel objects. Those are worse than null.



OK, that does sound unusually harsh.

Re: dereferencing null

[Chad J]

On 03/07/2012 02:09 PM, Jonathan M Davis wrote:

On Wednesday, March 07, 2012 07:55:35 H. S. Teoh wrote:

It's not that the null pointer itself corrupts memory. It's that the
null pointer is a sign that something may have corrupted memory *before*
you got to that point.

The point is, it's impossible to tell whether the null pointer was
merely the result of forgetting to initialize something, or it's a
symptom of a far more sinister problem. The source of the problem could
potentially be very far away, in unrelated code, and only when you tried
to access the pointer, you discover that something is wrong.

At that point, it may very well be the case that the null pointer isn't
just a benign uninitialized pointer, but the result of a memory
corruption, perhaps an exploit in the process of taking over your
application, or some internal consistency error that is in the process
of destroying user data. Trying to continue is a bad idea, since you'd
be letting the exploit take over, or allowing user data to get even more
corrupted than it already is.


Also, while D does much more to protect you from stuff like memory corruption
than C/C++ does, it's still a systems language. Stuff like that can definitely
happen. If you're writing primarily in SafeD, then it's very much minimized,
but it's not necessarily eliminated. All it takes is a bug in @system code
which could corrupt memory, and voila, you have corrupted memory, and an @safe
function could get a segfault even though it's correct code. It's likely to be
a very rare occurrence, but it's possible. A since when you get a segfault,
you can't know what caused it, you have to assume that it could have been
caused by one of the nastier possibilites rather than a relatively benign one.

And since ultimately, your program should be checking for null before
derefencing a variable in any case where it could be null, segfaulting due to
dereferencing a null pointer is a program bug which should be caught in
testing - like assertions in general are - rather than having the program
attempt to recover from it. And if you do _that_, the odds of a segfault being
due to something very nasty just go up, making it that much more of a bad idea
to try and recover from one.

- Jonathan M Davis


I can see where you're coming from now.

As I mentioned in another post, my lack of consideration for this 
indicator of memory corruption is probably a reflection of my bias 
towards VM'd languages.


I still don't buy the whole "it's a program bug that should be caught in 
testing".  I mean... true, but sometimes it isn't.  Especially since 
testing and assertions can never be %100 thorough.  What then?  Sorry, 
enjoy your suffering?


At that point I would like to have a better way to do sentinel values. 
I'd at least like to get an exception of some kind if I try to access a 
value that /shouldn't/ be there (as opposed to something that /should/ 
be there but /isn't/).


Combine that with sandboxing and I might just be satisfied for the time 
being.


See my reply to Steve for more details.  It's the one that starts like this:


Example?

Here, if you want, I'll start with a typical case.  Please make it right.

class UnreliableResource
{
this(string sourceFile) {...}
this(uint userId) {...}
void doTheThing() {...}
}