Re: Oops!

2020-12-02 Thread Jeffrey Kegler
One technique that is documented and tested and I believe should work is the Ruby Slippers. See the docs on this, but the basic idea would be, in the default lexer look only for commands. If the parse stops for lack of a lexeme, give it an LHS, and resume the parse. Otherwise, or if it also

Re: Oops!

2020-12-02 Thread Dean S
No problem, thanks for trying. I did plug it in and indeed got the same results. On 12/2/20 6:41 PM, Jeffrey Kegler wrote: > Oops!  Ignore that last!  It suffers from the same problem -- longest > captures the match and all the rest of that hack-ish apparatus I talked about > would be ignored

Oops!

2020-12-02 Thread Jeffrey Kegler
Oops! Ignore that last! It suffers from the same problem -- longest captures the match and all the rest of that hack-ish apparatus I talked about would be ignored. I've been doing mainly LaTeX these days and obviously it is turning my brain into mush. Sorry! On Wednesday, December 2, 2020 a

Re: Preferring a shorter token match

2020-12-02 Thread Jeffrey Kegler
How many commands? One approach that I just thought of and have never tested is to have fixed length variables, prioritized versus the commands of that length. Off the top of my head: var2 ~ priority=>1 PA_command ~ 'PA' priority=>2 PR_command ~ 'PR' priority=>2 include a catch-all var for len

Re: Preferring a shorter token match

2020-12-02 Thread Dean S
Thank you very much for your response! 1) Multiline: The language does have a "_" line-joining character, but the grammar wouldn't have to support that - it could be done with a trivial preprocessor. Once joined, commands may not span multiple lines. 2) Command/variable upper-case: Commands a

Re: Preferring a shorter token match

2020-12-02 Thread Jeffrey Kegler
I'll first describe your immediate problem, then ask a couple Q's. The problem: Lexing is LATM -- *Longest* Acceptable Token Matching. The lexeme priority is a tie breaker, used when tokens are the same length. When your grammar fails, "PAx" is your longest token, and the only choice at length 3.

Preferring a shorter token match

2020-12-02 Thread Dean S
Hello, I'm having trouble figuring out how to express my grammar and was hoping someone could help. I've tried rewriting various ways and looking for some options that might change behavior, but I haven't been able to figure it out. I have a language with variable assignment and simple commands