Re: Fwd: Re: grammars and indentation of input
Well put. The clearest description of Python's approach I've read, explained it as a lexer that tracked indentation level, and inserted appropriate tokens when it changed.
Re: Fwd: Re: grammars and indentation of input
Oh, a side point: there's some confusion introduced by the lack of a scanner/lexer in modern all-in-one-parsers. Python, for example, uses a scanner and so its grammar is nominally not context sensitive, but its scanner very much is (maintaining a stack of indentation exactly as OP was asking about). When you do everything in one place, the result must have the ability to maintain and respond to global state. There's really no other way around it. A pure BNF cannot parse Python. Aaron Sherman, M.: P: 617-440-4332 Google Talk, Email and Google Plus: a...@ajs.com Toolsmith, developer, gamer and life-long student. On Tue, Sep 13, 2016 at 12:26 PM, Bennett Todd wrote: > Hostile or not, thanks for your informative reply. >
Re: Fwd: Re: grammars and indentation of input
Hostile or not, thanks for your informative reply.
Re: Fwd: Re: grammars and indentation of input
Thank you, very much. Yes, I'm disappointed, but I'd rather know.
Re: Fwd: Re: grammars and indentation of input
> > Having the minutia of the programmatic run-time state of the parse then > influence the parse itself, is at the heart of the perl5 phenomenon "only > Perl can parse perl" I don't mean to be hostile, but you're demonstrably wrong, here. (also it's "only perl can parse Perl" as in, only the "perl" implementation can parse the Perl language). There are currently about 2.5 implementations of Perl 6, and while you could backhandedly claim that only Perl 6 can parse Perl 6 (because it's specced as a self-hosting language whose spec is actually written in Perl 6), the reality is that the parts that aren't written in Perl 6 can be written in just about anything (with C/MoarVM and JVM implementations working just fine). It's not a context sensitive grammar that was the issue with Perl 5, it was the lack of a specification outside of the primary implementation. Aaron Sherman, M.: P: 617-440-4332 Google Talk, Email and Google Plus: a...@ajs.com Toolsmith, developer, gamer and life-long student. On Tue, Sep 13, 2016 at 10:35 AM, Bennett Todd wrote: > Having the minutia of the programmatic run-time state of the parse then > influence the parse itself, is at the heart of the perl5 phenomenon "only > Perl can parse perl", which I rather hope isn't going to be preserved in > perl6. >
Re: Fwd: Re: grammars and indentation of input
On Tue, Sep 13, 2016 at 10:35:01AM -0400, Bennett Todd wrote: > Having the minutia of the programmatic run-time state of the parse then > influence the parse itself, is at the heart of the perl5 phenomenon "only > Perl can parse perl", which I rather hope isn't going to be preserved in > perl6. You may be disappointed to read this: Not only is this feature preserved in Perl 6... it's something of a prerequisite. It is what is required for a truly dynamic language that has many things happening in BEGIN blocks (i.e., things get executed even before you finish compiling the thing you're working on) and that allows dynamically adding new statement types and language features to the grammar. When implementing Perl 6, I think many of us aimed to minimize the amount of "runtime" things that happened during the parse... only to discover that we actually had to embrace and formalize it instead. Pm
Re: Fwd: Re: grammars and indentation of input
Hi Bennett, There are many situations that require non-contextfree languages. Even though much of these could be solved in the AST-building step (called 'transduction' in my days) instead of the parsing step, that does not solve all cases. I am just wondering if and to what extent we can parse non-CF languages with perl6. The reference to Perl5 is not appropriate, because a programming language is supposed to be designed to be relatively easy to parse. The language designer will have the need for an interpreter for the language high on his/her list of priorities. It all boils down to the question: does a Grammar allow a rule that depends on a function instead of being constant. I am not shocked if it is impossible, I just want to know how far perl6 takes me. Theo Bennett Todd schreef op 2016-09-13 16:35: Having the minutia of the programmatic run-time state of the parse then influence the parse itself, is at the heart of the perl5 phenomenon "only Perl can parse perl", which I rather hope isn't going to be preserved in perl6.
Re: Fwd: Re: grammars and indentation of input
Having the minutia of the programmatic run-time state of the parse then influence the parse itself, is at the heart of the perl5 phenomenon "only Perl can parse perl", which I rather hope isn't going to be preserved in perl6.
