I have a grammar for a configuration file where indentation is
significant, as in Python.  It contains the following lexer rules:

WS
  : {getCharPositionInLine()!=1}? // not start-of-line whitespace
  ( ' ' | TAB )
    { $channel=HIDDEN; }
    ;
// whitespace at start of line used for INDENT processing
INITIAL_WS
        : {getCharPositionInLine()==1 && !afterIndent}? // at start of line.
        ( ' ' | TAB )*
    { this.afterIndent=true; }
    ;

Note the star in the INITIAL_WS rule, which means that *every* line
should emit an INITIAL_WS token, possibly matching nothing, before
matching anything else.

The generated DFA contains the following code:
                    case 0 :
                        int LA10_25 = input.LA(1);


                        int index10_25 = input.index();
                        input.rewind();
                        s = -1;
                        if ( ((getCharPositionInLine()!=1)) ) {s = 26;}

                        else if ( ((getCharPositionInLine()==1 &&
!afterIndent)) ) {s = 6;}


                        input.seek(index10_25);
                        if ( s>=0 ) return s;
                        break;

which seems to be "obviously wrong" -- getCharPosition is going to be
evaluated in the rewound state, and then we're going to advance the
input and return, which will then invoke the proper lexer rule and
re-evaluate getCharPostion() in the *advanced* state, not where the
DFA evaluated it.

I don't quite understand the DFA well enough yet to attempt a proper
fix.  Anyone want to lend a hand?

Thanks--
 --scott

-- 
                         ( http://cscott.net/ )

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to 
[email protected]
For more options, visit this group at 
http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to