On 08/07/2020 15:30, Christian Schoenebeck wrote: > On Mittwoch, 8. Juli 2020 22:14:23 CEST Akim Demaille wrote: >>> So yes, you could certainly address this to work correctly with Flex with >>> additional measures, >> >> The case you are citing above is straightforward to handle the same >> way I did with the BASIC. That's fairly simple given that this is >> plain rational languages, fully under the perimeter of Flex. >> >> The general case though will not work just as well. John reported that >> the grammar is >> >> DO <number> <variable> = <expression> , <expression> [, <expression>] >> >> and here the problem is that the <expression>s can have parenthesized >> expressions with embedded commas. So on this case, Flex is screwed. >> >> That reminds me of a paper I read long ago, someone proposing >> "heisentokens": having the scanner return multiple tokens concurrently, >> for the different scanning options, branched into GLR to follow the >> various options and let dead ends die (that would be a second time?). > > "Dying branches" is exactly what I always had in mind for a potential scanner- > parser marriage in first place. > > But it's probably not only about tokens in parallel; you would probably also > need to maintain several parser states in parallel, as it might need several > reductions to get to the point where all ambiguities are resolved and only > one > progression in the grammar remains. > > But the profits would be stunning: you could write parsers in a very straight- > forward way, even for very complex languages. And on the other side it would > eventually allow to design languages in a much more user friendly and > flexible > way as well, as it would be straight-forward e.g. to restrict keywords (or > other patterns) to certain grammar contexts only.
I may be missing something, but isn't this what a PEG parser does? Cheers, Dan