I use exclusive starting states in flex to jump in and out of different sets of tokens as I parse a somewhat complex binary input. Let me show this to you visually. I will represent the yytext pointer with [].
0x53 [0xfb] 0x0a 0x01 0x02 As you can see, I just read the 0xfb byte. I know that bytes in bold belong to a different context and I want to change the start state to parse them. I also want to return a token to Bison representing 0xfb. So simplified code looks something like this: <CONTEXT1>"\xfb" { BEGIN CONTEXT2; return FBTOKEN; } <CONTEXT2>. { printf("Let's see what we got inside yytext now: %x", (unsigned char)*yytext); } The resulting print clearly shows that at entry into CONTEXT2, this was the situation: 0x53 0xfb 0x0a [0x01] 0x02 Flex elegantly skips the 0x0a character, if it appears in the start a new state, and only there. I don't want this behavior, so atm. I hack the s*** out of it: <CONTEXT2>. { if ((unsigned int)(unsigned char)yytext[-1] == 10) yytext[0] = yytext[-1]; printf("Let's see what we got inside yytext now: %x", (unsigned char)*yytext); } This works, but is disgusting. The [.] rule is the only one in CONTEXT2, so I know 0x0a does not get eaten by a \n rule. I also know that this behavior does not come up with other (haven't tested this exhaustively) binary values, so for example: 0x53 [0xfb] 0x09 0x01 0x02 results in 0x53 0xfb [0x09] 0x01 0x02 at the start of CONTEXT2. I suppose this has something to do with YY_DO_BEFORE_ACTION or some other definition, but I am not sure. I have a YY_USER_ACTION of my own, but it really only sets yylloc stuff, so it's unlikely to be the problem. I am looking through the verbose file to see if I can spot the issue, but that takes a while. In the meantime can someone clue me in as to what the problem is and what settings I can play with to manipulate this behavior? -- View this message in context: http://www.nabble.com/Starting-States-eat-my-bytes%21-tp25577783p25577783.html Sent from the Gnu - Bison - Help mailing list archive at Nabble.com. _______________________________________________ help-bison@gnu.org http://lists.gnu.org/mailman/listinfo/help-bison