It's quite a disappointment in some ways, but we've lived with it in Perl 5, and I'm sure we can live with it in Perl 6.
And I still think Perl 6 will have fewer cases in which it's completely impossible for not-Perl to parse it. Unfortunately, fewer still implies some, and some is still a problem.
Frankly, as the only person who has managed to get together a "guessing lexer" that is sufficiently accurate to be something other than useless, I see nothing in Perl 6 that makes it any easier than Perl 5. In fact, my reason for starting this thread in the first place is because I was increasingly getting the feeling there would be MORE problems.
The "prototype and operator/operand content" problem is still there. PPI works around it with quite a bit of hackery (look ahead, look behind, look ahead again, and shake it all about. Do the hokey pokie etc etc)
For the relevant code, look for '47' in
http://search.cpan.org/src/ADAMK/PPI-0.831/lib/PPI/Token/Whitespace.pm
This can still be easily tricked if you are trying to (see Randal's examples again), but usually gets most "real world" code right.
Then again, I avoid the whole problem of attaching params to sub calls by not doing it at all. The closest I can get is to list a set of tokens within a statement.
Perl 6 has the same problems, and then also adds a higher likelyhood of encountering custom grammars than in Perl 5, where source filters are considered largely bad form.
The fundamentals would appear to be the same.
As a side note, I may have used "parse" erroneously. What PPI attempts to do is to be a tokenizer and a lexer, without understanding its function as code.
So perhaps a "syntax lexer" is a closer term. The ability to read in and work with code based purely on syntax, without needing to know what it means.
Adam
