On 04/07/13 00:35, Bruno Medeiros wrote: > On 06/04/2013 20:52, Artur Skawina wrote: >> On 04/06/13 17:21, Bruno Medeiros wrote: >>> On 02/04/2013 00:18, Brian Schott wrote: >>>> I've pretty much finished up my work on the std.d.lexer module. I am >>>> waiting for the review queue to make some progress on the other (three?) >>>> modules being reviewed before starting a thread on it. >>>> >>> >>> BTW, even in the lexer spec I've found an issue. How does this parse: >>> 5.blah >>> According to the spec (maximal munch technique), it should be FLOAT then >>> IDENTIFIER. But DMD parses it as INTEGER DOT IDENTIFIER. I'm assuming the >>> lastest is the correct behavior, so you can write stuff like 123.init, but >>> that should be clarified. >> >> "1..2", "1.ident" and a float literal with '_' after the '.' are the >> DecimalFloat cases that I immediately ran into when doing a lexer based on >> the dlang grammar. It's obvious to a human how these should be handled, but >> code generators aren't that smart... But they are good at catching mistakes >> like these. > > The "1..2" is actually mentioned in the spec: > "An exception to this rule is that a .. embedded inside what looks like two > floating point literals, as in 1..2, is interpreted as if the .. was > separated by a space from the first integer." > so it's there, even if it can be missed.
I know, but documenting a (grammar) bug does not make it go away. > But unless I missed it, the spec is incorrect for the "1.ident" or > "1_2_3_4_5_6_._5_6_7_8" cases as there is no exception mentioned there... and > it's not always 100% obvious to a human how these should be handled. Or maybe > that's just me :) What does the "spec" currently say about ".001"?.. It's been a while since I did a d lexer, based on the dlang grammar - it (the lexer) was supposed to be dmd compatible. Took a closer look at the actual dlang.org rules today while writing this message... Will try to find some time to clean up and convert a working D lexical grammar to PEG; what i have should be 1:1 translatable, except one rule (DelimitedString) and put it on the wiki. Maybe it will help someone avoid these issues. artur
