On 4/1/2013 4:18 PM, Brian Schott wrote:
I've pretty much finished up my work on the std.d.lexer module. I am waiting for
the review queue to make some progress on the other (three?) modules being
reviewed before starting a thread on it.
In the meantime I've started some work on an AST module for Phobos that contains
the data types necessary to build up a parser module so that we can have a
standard set of code build D dev tools off of. I decided to work directly from
the standard on dlang.org for this to make sure that my module is correct and
that the standard is actually correct.
I've seen several threads on this newsgroup complaining about the state of the
standard and unfortunately this will be another one.
1) Grammar defined in terms of things that aren't tokens. Take, for example,
PropertyDeclaration. It's defined as an "@" token followed by... what? "safe"?
It's not a real token. It's an identifier. You can't parse this based on
checking the token type. You have to check the type and the value.
True, do you have a suggestion?
2) Grammar references rules that don't exist. UserDefinedAttribute is defined in
terms of CallExpression, but CallExpression doesn't exist elsewhere in the
grammar. BaseInterfaceList is defined in terms of InterfaceClasses, but that
rule is never defined.
Yes, this needs to be fixed.
3) Unnecessary rules. KeyExpression, ValueExpression, ScopeBlockStatement,
DeclarationStatement, ThenStatement, ElseStatement, Test, Increment, Aggregate,
LwrExpression, UprExpression, FirstExp, LastExp, StructAllocator,
StructDeallocator, EnumTag, EnumBaseType, EmptyEnumBody, ConstraintExpression,
MixinIdentifier, etc... are all defined in terms of only one other rule.
Using these makes documentation easier, and I don't think it harms anything.
I think that we need to be able to create a grammar description that:
* Fits in to a single file, so that a tool implementer does not need to collect
bits of the grammar from the various pages on dlang.org.
* Can be verified to be correct by an existing tool such as Bison, Goldie,
JavaCC, <your favorite here> with a small number of changes.
* Is part of the dmd/dlang repositories on github and gets updated every time
the language changes.
I'm willing to work on this if there's a good chance it will actually be
implemented. Thoughts?
I suggest doing this as a sequence of pull requests, not doing just one big one.
