Sequential machine does not support empty tokens. Also, sequential machine does not support deleting characters from inside a token.
You can work around those issues but post-processing, but if you have the necessary representation of the sequence to do that kind of post-processing you don't really need sequential machine. Thanks, -- Raul On Mon, Dec 29, 2014 at 10:28 PM, David Lambert <[email protected]> wrote: > ^ escapes to the next character, > | separates tokens. > > Can tokenize be written as an application of sequential machine? > > tokenize 'one^|uno||three^^^^|four^^^|^cuatro|' > +-------++-------+------------++ > |one|uno||three^^|four^|cuatro|| > +-------++-------+------------++ > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
