After taking a walk and thinking about this: I think the right approach for generating those empty words would be to generate the whitespace tokens and then remove the undesired content.
splitword4=: 0;(states'');a.=' ' 1.1 2.1 NB. 0. start here 1 2.2 NB. 1. other 1.2 2 NB. 2. whitespace ) ' ' -.L:0~ (<,' ') -.~ splitword4 ;: ' this is a test ' Much more concise (and reliable) than the hypothetical overly elaborate extension to splitword3. Thanks, -- Raul On Sun, Nov 15, 2020 at 9:14 AM Raul Miller <[email protected]> wrote: > > That could be done, with the introduction of four more rows in the > state table but would run into the same issue. > > Do you see why? > > Thanks, > > -- > Raul > > On Sun, Nov 15, 2020 at 8:53 AM 'Pascal Jasmin' via Programming > <[email protected]> wrote: > > > > You're almost there Raul, and I think you are much fresher on fsms than I > > am. > > > > The flaw is only that an empty word should be emitted only when multiple > > consecutive delimiters (2+ whitespace in your example) are encountered. > > The behaviour at start and end is correct. > > > > > > > > > > On Sunday, November 15, 2020, 08:44:43 a.m. EST, Raul Miller > > <[email protected]> wrote: > > > > > > > > > > > > Ah, I see the difficulty: there's no character after that final space > > to be used to backtrack from, to emit the empty token at the end of a > > string like ' this is a test '. > > > > So the machine sets up to emit an empty token, then runs out of text > > and instead emits a whitespace token. > > > > Or... this backtrack mechanism is flawed for the purpose of emitting > > empty tokens, in some cases. > > > > Thanks, > > > > -- > > Raul > > > > On Sun, Nov 15, 2020 at 8:32 AM Raul Miller <[email protected]> wrote: > > > > > > Oops, no, it works. I was just testing on an older version of J. > > > > > > Though the actual implementation of splitword2 is flawed, since it > > > doesn't support the idea of strings which don't begin or end with > > > whitespace. > > > > > > Here's an implementation with almost the behavior I think you specified: > > > > > > splitword3=: 0;(states'');a.e.' ',CRLF,TAB > > > 4.1 1 NB. 0. start here. > > > 2.1 1 NB. 1. skip starting whitespace > > > 3.7 3.7 NB. 2. starting whitespace had ended, backtrack > > > 4.2 4.2 NB. 3. empty on starting whitespace > > > 4 5.2 NB. 4. other > > > 6.7 6.7 NB. 5. new whitespace, backtrack > > > 7.3 7.3 NB. 6. empty on new whitespace > > > 4.1 7 NB. 7. skip new whitespace > > > ) > > > > > > That said, I'm not sure if this is a temporary thing or if it should > > > be left in place. > > > > > > Also, J crashed while I was debugging this (I'm getting a non-empty > > > whitespace at the end of the string, and I do not understand why). > > > And, your next email in this thread came in, so I should probably stop > > > here, for now. > > > > > > Thanks, > > > > > > > > > -- > > > Raul > > > > > > On Sun, Nov 15, 2020 at 7:44 AM Raul Miller <[email protected]> wrote: > > > > > > > > Hmm... > > > > > > > > The basic purpose of the new backtrack mechanism is to allow token > > > > formation rules to depend on characters which have not yet been > > > > encountered but which will be encountered soon. It also introduces > > > > some new risks, as a consequence -- the possibility of infinite loops > > > > and the possibility of emitting tokens containing characters which > > > > have been emitted in previous tokens. The possibility of emitting an > > > > empty token lies right on the boundary of that second risk area. > > > > > > > > Testing, I get an index error when I try to emit an empty token. > > > > > > > > states=: 3 :0 > > > > 0 10 #: <. 10 * > -.&a: <@".;._2] 0 :0 > > > > ) > > > > > > > > splitword=: 0;(states'');a.e.' ',CRLF,TAB > > > > 1.1 0 NB. 0 whitespace > > > > 1 0.3 NB. 1 other > > > > ) > > > > > > > > splitword2=: 0;(states'');a.e.' ',CRLF,TAB > > > > 1.1 0 NB. 0 whitespace > > > > 2.7 2.7 NB. 1 whitespace had ended, backtrack > > > > 3.2 3.2 NB. 2 empty on whitespace end > > > > 3 0.3 NB. 3 other > > > > ) > > > > > > > > splitword ;: 'this is a test' NB. this works > > > > splitword2 ;: 'this is a test' NB. index error > > > > > > > > So either my test is flawed, or the implementation gives an index > > > > error for backtracking past the start of a token. > > > > > > > > Thanks, > > > > > > > > > > > > -- > > > > Raul > > > > > > > > On Sat, Nov 14, 2020 at 7:57 PM 'Pascal Jasmin' via Programming > > > > <[email protected]> wrote: > > > > > > > > > > Can this be used to emit empty words? What are some imagined uses of > > > > > this code? > > > > > > > > > > > > > > > http://www.jsoftware.com/pipermail/programming/2017-March/046910.html > > > > > > > > > > suggested ew when j=_1 should emit empty box/word instead of > > > > > crashing, seemed simple enough, but I don't understand this addition > > > > > yet. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Saturday, November 14, 2020, 05:08:33 p.m. EST, Henry Rich > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have been rewriting interfaces to make the JE reentrant. This beta > > > > > has changes in I/O to the front end, which has more ramifications > > > > > than I > > > > > can test for. I am hoping the beta users will cover the combinations > > > > > & > > > > > report anything that looks unusual. > > > > > > > > > > This beta has the new backtrack function for (x ;: y). > > > > > > > > > > Henry Rich > > > > > > > > > > On 11/14/2020 5:05 PM, Eric Iverson wrote: > > > > > > J902-beta-m available for windows/macos/linux. > > > > > > > > > > > > If you already run 902-beta, then upgrade is easy: > > > > > > load'pacman' > > > > > > 'upgrade'jpkg'jengine' > > > > > > ---------------------------------------------------------------------- > > > > > > For information about J forums see > > > > > > http://www.jsoftware.com/forums.htm > > > > > > > > > > > > > > > -- > > > > > This email has been checked for viruses by AVG. > > > > > https://www.avg.com > > > > > > > > > > > > > > > ---------------------------------------------------------------------- > > > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > > > > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
