There are issues with cache management which mean that sometimes multipass solutions involving appropriately simple passes can sometimes outperform single pass solutions.
FSMs with backtracking also get used in regular expression implementations (which in some cases run into exponential time issues). But, also, you're going to need additional code, anyways, if you want to do anything with the generated result. Anyways, ... this particular implementation has the defect I mentioned, where you can't backtrack after running off the end of the string. Thanks, -- Raul On Sun, Nov 15, 2020 at 10:44 AM 'Pascal Jasmin' via Programming <[email protected]> wrote: > > the aspiration of a single pass fsm solution is the speed that comes with > fsm. I understand that multipass or non-fsm solutions are simpler to write. > > > > On Sunday, November 15, 2020, 10:02:30 a.m. EST, Raul Miller > <[email protected]> wrote: > > > > > > After taking a walk and thinking about this: > > I think the right approach for generating those empty words would be > to generate the whitespace tokens and then remove the undesired > content. > > splitword4=: 0;(states'');a.=' ' > 1.1 2.1 NB. 0. start here > 1 2.2 NB. 1. other > 1.2 2 NB. 2. whitespace > ) > > ' ' -.L:0~ (<,' ') -.~ splitword4 ;: ' this is a test ' > > Much more concise (and reliable) than the hypothetical overly > elaborate extension to splitword3. > > Thanks, > > -- > Raul > > On Sun, Nov 15, 2020 at 9:14 AM Raul Miller <[email protected]> wrote: > > > > That could be done, with the introduction of four more rows in the > > state table but would run into the same issue. > > > > Do you see why? > > > > Thanks, > > > > -- > > Raul > > > > On Sun, Nov 15, 2020 at 8:53 AM 'Pascal Jasmin' via Programming > > <[email protected]> wrote: > > > > > > You're almost there Raul, and I think you are much fresher on fsms than I > > > am. > > > > > > The flaw is only that an empty word should be emitted only when multiple > > > consecutive delimiters (2+ whitespace in your example) are encountered. > > > The behaviour at start and end is correct. > > > > > > > > > > > > > > > On Sunday, November 15, 2020, 08:44:43 a.m. EST, Raul Miller > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > Ah, I see the difficulty: there's no character after that final space > > > to be used to backtrack from, to emit the empty token at the end of a > > > string like ' this is a test '. > > > > > > So the machine sets up to emit an empty token, then runs out of text > > > and instead emits a whitespace token. > > > > > > Or... this backtrack mechanism is flawed for the purpose of emitting > > > empty tokens, in some cases. > > > > > > Thanks, > > > > > > -- > > > Raul > > > > > > On Sun, Nov 15, 2020 at 8:32 AM Raul Miller <[email protected]> wrote: > > > > > > > > Oops, no, it works. I was just testing on an older version of J. > > > > > > > > Though the actual implementation of splitword2 is flawed, since it > > > > doesn't support the idea of strings which don't begin or end with > > > > whitespace. > > > > > > > > Here's an implementation with almost the behavior I think you specified: > > > > > > > > splitword3=: 0;(states'');a.e.' ',CRLF,TAB > > > > 4.1 1 NB. 0. start here. > > > > 2.1 1 NB. 1. skip starting whitespace > > > > 3.7 3.7 NB. 2. starting whitespace had ended, backtrack > > > > 4.2 4.2 NB. 3. empty on starting whitespace > > > > 4 5.2 NB. 4. other > > > > 6.7 6.7 NB. 5. new whitespace, backtrack > > > > 7.3 7.3 NB. 6. empty on new whitespace > > > > 4.1 7 NB. 7. skip new whitespace > > > > ) > > > > > > > > That said, I'm not sure if this is a temporary thing or if it should > > > > be left in place. > > > > > > > > Also, J crashed while I was debugging this (I'm getting a non-empty > > > > whitespace at the end of the string, and I do not understand why). > > > > And, your next email in this thread came in, so I should probably stop > > > > here, for now. > > > > > > > > Thanks, > > > > > > > > > > > > -- > > > > Raul > > > > > > > > On Sun, Nov 15, 2020 at 7:44 AM Raul Miller <[email protected]> > > > > wrote: > > > > > > > > > > Hmm... > > > > > > > > > > The basic purpose of the new backtrack mechanism is to allow token > > > > > formation rules to depend on characters which have not yet been > > > > > encountered but which will be encountered soon. It also introduces > > > > > some new risks, as a consequence -- the possibility of infinite loops > > > > > and the possibility of emitting tokens containing characters which > > > > > have been emitted in previous tokens. The possibility of emitting an > > > > > empty token lies right on the boundary of that second risk area. > > > > > > > > > > Testing, I get an index error when I try to emit an empty token. > > > > > > > > > > states=: 3 :0 > > > > > 0 10 #: <. 10 * > -.&a: <@".;._2] 0 :0 > > > > > ) > > > > > > > > > > splitword=: 0;(states'');a.e.' ',CRLF,TAB > > > > > 1.1 0 NB. 0 whitespace > > > > > 1 0.3 NB. 1 other > > > > > ) > > > > > > > > > > splitword2=: 0;(states'');a.e.' ',CRLF,TAB > > > > > 1.1 0 NB. 0 whitespace > > > > > 2.7 2.7 NB. 1 whitespace had ended, backtrack > > > > > 3.2 3.2 NB. 2 empty on whitespace end > > > > > 3 0.3 NB. 3 other > > > > > ) > > > > > > > > > > splitword ;: 'this is a test' NB. this works > > > > > splitword2 ;: 'this is a test' NB. index error > > > > > > > > > > So either my test is flawed, or the implementation gives an index > > > > > error for backtracking past the start of a token. > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > -- > > > > > Raul > > > > > > > > > > On Sat, Nov 14, 2020 at 7:57 PM 'Pascal Jasmin' via Programming > > > > > <[email protected]> wrote: > > > > > > > > > > > > Can this be used to emit empty words? What are some imagined uses > > > > > > of this code? > > > > > > > > > > > > > > > > > > http://www.jsoftware.com/pipermail/programming/2017-March/046910.html > > > > > > > > > > > > suggested ew when j=_1 should emit empty box/word instead of > > > > > > crashing, seemed simple enough, but I don't understand this > > > > > > addition yet. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Saturday, November 14, 2020, 05:08:33 p.m. EST, Henry Rich > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have been rewriting interfaces to make the JE reentrant. This > > > > > > beta > > > > > > has changes in I/O to the front end, which has more ramifications > > > > > > than I > > > > > > can test for. I am hoping the beta users will cover the > > > > > > combinations & > > > > > > report anything that looks unusual. > > > > > > > > > > > > This beta has the new backtrack function for (x ;: y). > > > > > > > > > > > > Henry Rich > > > > > > > > > > > > On 11/14/2020 5:05 PM, Eric Iverson wrote: > > > > > > > J902-beta-m available for windows/macos/linux. > > > > > > > > > > > > > > If you already run 902-beta, then upgrade is easy: > > > > > > > load'pacman' > > > > > > > 'upgrade'jpkg'jengine' > > > > > > > ---------------------------------------------------------------------- > > > > > > > For information about J forums see > > > > > > > http://www.jsoftware.com/forums.htm > > > > > > > > > > > > > > > > > > -- > > > > > > This email has been checked for viruses by AVG. > > > > > > https://www.avg.com > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------- > > > > > > For information about J forums see > > > > > > http://www.jsoftware.com/forums.htm > > > > > > ---------------------------------------------------------------------- > > > > > > For information about J forums see > > > > > > http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
