Paul-

Monday, August 13, 2012, 1:32:58 PM, you wrote:

> One caution: token does not separate . (period), ! (exclamation mark),
> or ? (question mark). If you are really trying to process English text,
> you probably will want to write your own punctuation remover as it can
> then figure the difference between a period at the end of a sentence and
> a period at the end of abbreviations like "Dr." or "Mr."

Good point. A question mark does count as a word separator, so "token
1 of word 1 of..." will still work, but the other two could cause
problems.

-- 
-Mark Wieder
 mwie...@ahsoftware.net


_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Reply via email to