Re: Is the declaration grammar definition of 'Parameter' correct?
BCS wrote: That would be a start as well as a necessary step in getting a extractor working (to test the tools output against), but if the docs and the parser are maintained separately, the docs will be wrong sooner rather than later. I think that's very unlikely to be true. The existing errors in the grammer have formed during the long development of the language, and the grammer docs never really got much attention. Now that the language has stabilised, the grammer is not going to change rapidly enough to be major source of errors. The effort involved in making a tool to do it, probably exceeds the effort involved in just fixing it, by a factor of five or ten. There are many other things which could be done which would FAR more useful for D.
Re: Is the declaration grammar definition of 'Parameter' correct?
Right now we have two semi-official definitions of the D grammar; the docs (that are wrong) and the parser source (that is effectively unreadable by most people). I would like to propose a solution to this problem: eliminate one of them and derive it from the other. I know this will be hard to do but it can be done incrementally with each step making improvements over the last. The first thing to do is put all of the description of the grammar in the docs into one place. If the literal text of each production is replace with a macro reference then the definitions of these macros can be put into a single file and expanded everywhere. The incremental improvement here is that having the grammar in one place by it's self will make it easier to check. From here there are several things that can be done: - copy the grammar rule macros into the parser source where they will stay more up to date. - write a tool to generate the rules from the parser source. - write a tool to generate the parser source from the rules. I'd be willing to do that first step if I got any (semi)official indication that it would be used. -- ... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
On Sun, 13 Jun 2010 20:15:46 +, BCS wrote: The first thing to do is put all of the description of the grammar in the docs into one place. If we are at it, it would be good to enhance visual representation of docs by generate syntax diagram, similarly as on http://www.json.org/ it not only helps to learn and understand grammar, but also lowers the barrier to just looking at it. When I first saw this kind of diagrams on JSON site, it made good impression on me. Tool used for this is. http://dotnet.jku.at/applications/Visualizer/ Another important point is - having it visually really help find and enhance specs. This tools accepts EBNF syntax that is specified on the page. D syntax is specified in BNF, and uses different syntax, but (E)BNF is so simple, it should be possible to adjust different syntaxes just using regex (I think/ hope). If there is interest in such diagrams, and there is one file syntax file, I would be willing to make changes to this tool (C#) to be able to batch- generate these images. (It would be also a good idea to look at first if BNF diagrams look good, and if EBNF isn't necessary.)
Re: Is the declaration grammar definition of 'Parameter' correct?
Hello Michal, On Sun, 13 Jun 2010 20:15:46 +, BCS wrote: The first thing to do is put all of the description of the grammar in the docs into one place. If we are at it, it would be good to enhance visual representation of docs by generate syntax diagram, similarly as on http://www.json.org/ it not only helps to learn and understand grammar, but also lowers the barrier to just looking at it. When I first saw this kind of diagrams on JSON site, it made good impression on me. [...] Nice idea. (But I'd put them in a different set of cross linked pages.) If there is interest in such diagrams, and there is one file syntax file, I would be willing to make changes to this tool (C#) to be able to batch- generate these images. (It would be also a good idea to look at first if BNF diagrams look good, and if EBNF isn't necessary.) You might not need to change the tool. If the grammar macros are done with some thought, you might be able to make them render it to whatever that tool already take as input. -- ... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
Hello BCS, I'd be willing to do that first step if I got any (semi)official indication that it would be used. does anyone know where the SVN repo with the source for the docs is? -- ... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
On 6/13/2010 5:15 PM, BCS wrote: Hello BCS, I'd be willing to do that first step if I got any (semi)official indication that it would be used. does anyone know where the SVN repo with the source for the docs is? The standard phobos repo at dsource in the docsrc directory: http://www.dsource.org/projects/phobos/browser/trunk/docsrc Arguably part of that could (maybe should) be split to the dmd repo now that it exists, but... shrug.
Re: Is the declaration grammar definition of 'Parameter' correct?
Hello Brad, On 6/13/2010 5:15 PM, BCS wrote: Hello BCS, I'd be willing to do that first step if I got any (semi)official indication that it would be used. does anyone know where the SVN repo with the source for the docs is? The standard phobos repo at dsource in the docsrc directory: http://www.dsource.org/projects/phobos/browser/trunk/docsrc Thanks. -- ... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
On 06/13/2010 03:15 PM, BCS wrote:
Right now we have two semi-official definitions of the D grammar; the
docs (that are wrong) and the parser source (that is effectively
unreadable by most people). I would like to propose a solution to this
problem: eliminate one of them and derive it from the other.
In the past, Walter has expressed a dislike of parser generators. I
doubt you can get him to change his mind on that one, especially since
most parser generators won't be able to handle D anyways, and I doubt
any of them will be able to handle D efficiently (what are they called -
GLR parser generators? Don't know about these).
As far as analysing parse.c goes, how much effort is it going to take to
figure out what
while(1)
switch(token.value){
case x:
parseThis();
continue;
case y:
parseThat();
break;
}
break;
}
should look like in a grammar?
Counterproposal: why don't we just fix the grammar?
I know this will be hard to do but it can be done incrementally with
each step making improvements over the last.
The first thing to do is put all of the description of the grammar in
the docs into one place. If the literal text of each production is
replace with a macro reference then the definitions of these macros can
be put into a single file and expanded everywhere.
The incremental improvement here is that having the grammar in one place
by it's self will make it easier to check.
I would very much like to see the entire grammar in one place. I can't
tell you how obnoxious it is to search through the entire site to find
one production
And as always, apologies for being a pessimist.
Re: Is the declaration grammar definition of 'Parameter' correct?
Hello Ellery,
On 06/13/2010 03:15 PM, BCS wrote:
Right now we have two semi-official definitions of the D grammar; the
docs (that are wrong) and the parser source (that is effectively
unreadable by most people). I would like to propose a solution to
this problem: eliminate one of them and derive it from the other.
In the past, Walter has expressed a dislike of parser generators. I
doubt you can get him to change his mind on that one, especially since
most parser generators won't be able to handle D anyways, and I doubt
any of them will be able to handle D efficiently (what are they called
- GLR parser generators? Don't know about these).
I was very careful to leave both options open in the way I phrased that ;).
As far as analysing parse.c goes, how much effort is it going to take
to figure out what
void parseSomething()
{
while(1)
switch(token.value){
case x:
parseThis();
continue;
case y:
parseThat();
break;
}
break;
}
}
Something ::= This* That;
The switch is just an optimization. As long as the code is regular enough,
it can be analyzed. (I will grant that the resulting grammar might be ambiguous
or LL(k) where k is higher than it needs to be)
should look like in a grammar?
Counterproposal: why don't we just fix the grammar?
That would be a start as well as a necessary step in getting a extractor
working (to test the tools output against), but if the docs and the parser
are maintained separately, the docs will be wrong sooner rather than later.
I know this will be hard to do but it can be done incrementally with
each step making improvements over the last.
The first thing to do is put all of the description of the grammar in
the docs into one place. If the literal text of each production is
replace with a macro reference then the definitions of these macros
can
be put into a single file and expanded everywhere.
The incremental improvement here is that having the grammar in one
place
by it's self will make it easier to check.
I would very much like to see the entire grammar in one place. I can't
tell you how obnoxious it is to search through the entire site to find
one production
A while back I took a pass at pulling the grammar from the HTML via sed so
I could stuff in into a parser generator. That it worked is the only good
thing I can say about the experience.
And as always, apologies for being a pessimist.
--
... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
Hello BCS, The first thing to do is put all of the description of the grammar in the docs into one place. If the literal text of each production is replace with a macro reference then the definitions of these macros can be put into a single file and expanded everywhere. [...] I'd be willing to do that first step if I got any (semi)official indication that it would be used. Done, and because I can't attache the patch (it's to big) I started a issue. http://d.puremagic.com/issues/show_bug.cgi?id=4308 As a side note, the formatting of the grammar is very irregular, using different macros to get the same thing in different places. That would be the next thing I'd clean up. -- ... IXOYE
Re: Is the declaration grammar definition of 'Parameter' correct?
On 06/08/2010 12:30 AM, Bernard Helyer wrote: On 08/06/10 17:19, Ellery Newcomer wrote: On 06/07/2010 11:06 PM, Bernard Helyer wrote: On 08/06/10 16:00, Ellery Newcomer wrote: Yeah, it's wrong. (close reads of parse.c are much more useful than reading the spec. heh.) A peek in my grammar and... Parameter: ... BasicType Declarator BasicType Declarator = AssignExpression BasicType Declarator ... Type Type ... I probably should have filed bug reports back when I was going through the grammar. Oh well. Hmm. On the same page, Declarator has an identifier in it. Which means I still couldn't parse int function(int, int) with it, no? Eh? Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int wait, are you talking about the params inside the function type, or the whole thing as a param? I'm pretty sure it works either way. Parameter doesn't resolve to Type, not that I can see... According to whom?
Is the declaration grammar definition of 'Parameter' correct?
http://www.digitalmars.com/d/2.0/declaration.html So, cut down: Decl BasicType Declarators ; BasicType int ... BasicType2 * [] and co function Parameters Parameter Declarator ... Declarator BasicType2 Identifier DeclaratorSuffixes (the suffixes are [] [assignexpr] [type] and a template parameter list) So given all that, I can't see how this: int function(int, int) a; can be parsed with that grammar. Additionally, Declarator requires identifier, so wouldn't that make this: int function(* a, [] b) c; a valid Decl according to that grammar. I think this is seriously incorrect, but I would be open to correction! :D
Re: Is the declaration grammar definition of 'Parameter' correct?
Yeah, it's wrong. (close reads of parse.c are much more useful than reading the spec. heh.) A peek in my grammar and... Parameter: ... BasicType Declarator BasicType Declarator = AssignExpression BasicType Declarator ... Type Type ... I probably should have filed bug reports back when I was going through the grammar. Oh well.
Re: Is the declaration grammar definition of 'Parameter' correct?
On 08/06/10 16:00, Ellery Newcomer wrote: Yeah, it's wrong. (close reads of parse.c are much more useful than reading the spec. heh.) A peek in my grammar and... Parameter: ... BasicType Declarator BasicType Declarator = AssignExpression BasicType Declarator ... Type Type ... I probably should have filed bug reports back when I was going through the grammar. Oh well. Hmm. On the same page, Declarator has an identifier in it. Which means I still couldn't parse int function(int, int) with it, no?
Re: Is the declaration grammar definition of 'Parameter' correct?
On 06/07/2010 11:06 PM, Bernard Helyer wrote: On 08/06/10 16:00, Ellery Newcomer wrote: Yeah, it's wrong. (close reads of parse.c are much more useful than reading the spec. heh.) A peek in my grammar and... Parameter: ... BasicType Declarator BasicType Declarator = AssignExpression BasicType Declarator ... Type Type ... I probably should have filed bug reports back when I was going through the grammar. Oh well. Hmm. On the same page, Declarator has an identifier in it. Which means I still couldn't parse int function(int, int) with it, no? Eh? Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int wait, are you talking about the params inside the function type, or the whole thing as a param? I'm pretty sure it works either way.
Re: Is the declaration grammar definition of 'Parameter' correct?
On 08/06/10 17:19, Ellery Newcomer wrote: On 06/07/2010 11:06 PM, Bernard Helyer wrote: On 08/06/10 16:00, Ellery Newcomer wrote: Yeah, it's wrong. (close reads of parse.c are much more useful than reading the spec. heh.) A peek in my grammar and... Parameter: ... BasicType Declarator BasicType Declarator = AssignExpression BasicType Declarator ... Type Type ... I probably should have filed bug reports back when I was going through the grammar. Oh well. Hmm. On the same page, Declarator has an identifier in it. Which means I still couldn't parse int function(int, int) with it, no? Eh? Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int wait, are you talking about the params inside the function type, or the whole thing as a param? I'm pretty sure it works either way. Parameter doesn't resolve to Type, not that I can see...
