Re: [Jprogramming] Question regarding J lexing rules

2019-10-10 Thread Raul Miller
That thinking sounds about right, to me. (Also... I think it's symmetric: having the tokenizer handle representations of both noun lists and character lists rather than having it handle one but not the other.) And, we're beginning to get into the territory Roger Hui was asking about. Thanks, -

Re: [Jprogramming] Question regarding J lexing rules

2019-10-09 Thread ethiejiesa via Programming
Thank you for the thoughtful and lucid reply. If you don't mind, let me check my understanding. So, essentially, we keep forefront the general concept of numeric words requiring speculative tokenization in the sense you describe. Is the following true? Let sj be the state table as defined in th

Re: [Jprogramming] Question regarding J lexing rules

2019-10-07 Thread Raul Miller
On Sun, Oct 6, 2019 at 10:31 PM 'B. Wilson' via Programming wrote: > Thank you for the confirmation. > > So in these two cases, word splitting happens exactly the same > if we use em() instead, correct? Is there a particular reason to > *not* use em() though? As far as I can tell, the main differe

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread 'B. Wilson' via Programming
Thank you for the confirmation. So in these two cases, word splitting happens exactly the same if we use em() instead, correct? Is there a particular reason to *not* use em() though? As far as I can tell, the main difference would be in the traces. I would really like to know if ev() was chosen

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread Raul Miller
On Sun, Oct 6, 2019 at 9:03 AM ethiejiesa via Programming wrote: > I wish to ask something very specific. Is there a particular > technical reason for ev()ing (output code 4) from the 6 (num) state > when encountering a Q (quote) or X (other) character? Unless I am > misunderstanding, there is no

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread ethiejiesa via Programming
Hrm. It seems I am not articulating my question all that well. I wish to ask something very specific. Is there a particular technical reason for ev()ing (output code 4) from the 6 (num) state when encountering a Q (quote) or X (other) character? Unless I am misunderstanding, there is no way for

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread Raul Miller
On Sat, Oct 5, 2019 at 9:21 PM ethiejiesa via Programming wrote: > > What would it emit instead? > Words that just happen to contain spaces. > > I just realized, however, that my original email example lets > number words contain trailing spaces. So, I agree that the cell "0 > 5" in the state tabl

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread bill lam
Can you provide a simple string to show the dyad ;: model accept quote inside number? On Sun, Oct 6, 2019, 8:22 PM ethiejiesa via Programming < programm...@jsoftware.com> wrote: > Thank you for your lending some input. > > > 4r5 is a valid number in J. Would this explain the behaviour? > By "othe

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread ethiejiesa via Programming
I seem to be having troubles with the mailing list. Please forgive the overlapping replies with similar content. -- For information about J forums see http://www.jsoftware.com/forums.htm

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread ethiejiesa via Programming
> What would it emit instead? Just a "number" that happens to contain whitespace? Maybe I am misunderstanding you, though. The output codes in the state table can trigger emits (i.e. em(i,j,r,c)) or emit vectors (i.e. ev(i,j,r,c)). I'm curious why the state table row for numbers chooses to use

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread ethiejiesa via Programming
> What would it emit instead? Words that just happen to contain spaces. I just realized, however, that my original email example lets number words contain trailing spaces. So, I agree that the cell "0 5" in the state table needs to be what it is. However, if we're in state num and encounter a qu

Re: [Jprogramming] Question regarding J lexing rules

2019-10-06 Thread ethiejiesa via Programming
Thank you for your lending some input. > 4r5 is a valid number in J. Would this explain the behaviour? By "other", I was referring to the 'X' character class defined in the input mapping table. I believe your example never hits this column of the state table. Perhaps it's clearer to just focus o

Re: [Jprogramming] Question regarding J lexing rules

2019-10-05 Thread 'robert therriault' via Programming
4r5 is a valid number in J. Would this explain the behaviour? Cheers, bob > On Oct 5, 2019, at 6:21 PM, ethiejiesa via Programming > wrote: > >> What would it emit instead? > Words that just happen to contain spaces. > > I just realized, however, that my original email example lets number wor

Re: [Jprogramming] Question regarding J lexing rules

2019-10-05 Thread ethiejiesa via Programming
> What would it emit instead? Words that just happen to contain spaces. I just realized, however, that my original email example lets number words contain trailing spaces. So, I agree that the cell "0 5" in the state table needs to be what it is. However, if we're in state num and encounter a qu

Re: [Jprogramming] Question regarding J lexing rules

2019-10-04 Thread Roger Hui
> emits no vectors What would it emit instead? If you emit individual scalar numbers, you then have to change the parser ( http://www.jsoftware.com/help/dictionary/dicte.htm) to handle two or more numbers juxtaposed. On Thu, Oct 3, 2019 at 11:57 PM ethiejiesa via Programming < programm...@jsof

[Jprogramming] Question regarding J lexing rules

2019-10-03 Thread ethiejiesa via Programming
Just beginning to learn J this week, I find myself binging the documentation and have some questions regarding J lexing. In particular, my question is about the sample J lexer presented in the dyadic ;: entry of the dictionary: https://www.jsoftware.com/help/dictionary/d332.htm Essentially, I'm