Matti, What you are suggesting is actually quite similar to an old future improvement suggestion for Grammatica: https://savannah.nongnu.org/bugs/?func=detailitem&item_id=3599 That is interesting for more purposes though, allowing the creation of very readable and compact grammars. Why aren't grammars modularized and abstracted the same way software is?
I guess it would be possible to do already today, but one sure needs some glue handy. The Parser class require something that behaves like a Tokenizer (which is unfortunately not an interface), so one has to implement the public API of that class. Also, as a parser outputs a parse tree, and not a stream of tokens, one would have to create a special Analyzer subclass that converts certain (but obviously not all) productions into Tokens. Not that tricky maybe, but perhaps not all that beautiful either. Cheers, /Per On tue, 2005-06-28 at 12:00 +0300, Matti Katila wrote: > On Tue, 28 Jun 2005, Per Cederberg wrote: > > Well, the Grammatica lexer is essentially identical to most other > > tools. It provides support for defining tokens via regexps (and > > plain strings). If you want to use EBNF, you can just write those > > "tokens" as productions instead. > > This would make the division to lexer and parser much harder. For > example python lexer which marks_ indent and dedent is easy to do after > lexing - we may call that post-lexer. Post-lexing is much easier to do > after long literals_ and comments are regocnized already. > > _marks =: http://docs.python.org/ref/indentation.html > _literals =: http://docs.python.org/ref/strings.html > > Of course, I don't know yet since I haven't tried it out, I could create > foolexer.grammar and fooparser.grammar for grammar foo and use some sort > of wrapping with grammatica to use lexer.grammar as lexer where I want to > use ebnf in lexer grammar. > > > The general problem with grammars where tokens are specified in > > EBNF is efficiency and ambiguities. > > Efficiency sounds as premature-optimizing which shall considered as no-no :) > > > That is not a problem in a language specification document where > > readability is the highest concern. > > I think readability is high concern in every case :) Well, I think I know > what you mean. > > > -Matti _______________________________________________ Grammatica-users mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/grammatica-users
