Hello, The contrib/wordnet package contains an AnalyzerUtil class with a method that extracts sentences from text/String. It is super-simplistic, so probably not very accurate, but I am wondering if *conceptually* it would make sense to have a Tokenizer that extracts sentences? I suppose that means each Token would be a complete sentence.
Would you say it makes sense to implement sentence detection/extraction as a Tokenizer? Thanks, Otis --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
