On Sun, Jan 27, 2013 at 3:53 AM, David A. Wheeler <dwhee...@dwheeler.com> wrote: > Alan Manuel Gloria: >> A precis: the tokenizer is not a separate pass, but rather implemented >> as a stateful procedure that will consume exactly one token on the >> input stream... > > Can you start by focusing on the preprocessor/tokenizer for the existing BNF? > > You're absolutely right that the current material leaves that to text (as did > SRFI-49, to be fair), and making it more formal is likely to be an > improvement.
Okay, I'll try that later. For now, I want to try my approach first, which needs to simplify the BNF to remove some things I don't need and replace comment_eol that isn't followed by an INDENT with SAME. > >> If it finds a "<code>!</code>" character after >> the indent, >> emit INITIAL_INDENT_WITH_BANG >> and end processing. > > Hm, I pull in all the indent chars, and then check for "!". That way, > multiple bad "!" will only be complained about once. That can be done too (it removes the different definitions of "indent char" depending on state). It's not exactly specified what the parser does when it finds an error. I think it's been vaguely mentioned that an error will make the parser consume lines until it finds a completely empty line, so I thought this behavior was acceptable (since the ! and the rest of the line will get eaten anyway). Incidentally, it might be more useful, in the error case, to have the parser consume lines until it finds a completely empty line, or a line without indentation. > > I found that processing SAME was actually more of a pain, and not really > helpful, but your mileage may vary :-). > > I plan to implement most of the indent/dedent stuff by using comment_eol to > grab and return the indent string, and then use string comparisons to > determine the difference. That's how the current one works, so I know it > works well. > > --- David A. Wheeler > > ------------------------------------------------------------------------------ > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft > MVPs and experts. ON SALE this month only -- learn more at: > http://p.sf.net/sfu/learnnow-d2d > _______________________________________________ > Readable-discuss mailing list > Readable-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/readable-discuss ------------------------------------------------------------------------------ Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d _______________________________________________ Readable-discuss mailing list Readable-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/readable-discuss
