[ https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820083#comment-13820083 ]
Robert Muir commented on LUCENE-2899: ------------------------------------- I don't think we should expand the analysis protocol: I think its actually already more complicated than it needs to be. It doesnt need to work across multiple fields or support things like NER. I know people disagree, but i dont care (typically they dont do a lot of work to maintain this code). I'll fight it to the death: Lucene's analysis is about doing information retrieval (search and query), and its already overly complex. It should stay per-field, it should stay like a state machine it is. Stuff like this NER should *NOT* be in the analysis chain. as i said, its more useful in the "document build" phase anyway. > Add OpenNLP Analysis capabilities as a module > --------------------------------------------- > > Key: LUCENE-2899 > URL: https://issues.apache.org/jira/browse/LUCENE-2899 > Project: Lucene - Core > Issue Type: New Feature > Components: modules/analysis > Reporter: Grant Ingersoll > Assignee: Grant Ingersoll > Priority: Minor > Fix For: 4.6 > > Attachments: LUCENE-2899-RJN.patch, LUCENE-2899.patch, > OpenNLPFilter.java, OpenNLPTokenizer.java > > > Now that OpenNLP is an ASF project and has a nice license, it would be nice > to have a submodule (under analysis) that exposed capabilities for it. Drew > Farris, Tom Morton and I have code that does: > * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it > would have to change slightly to buffer tokens) > * NamedEntity recognition as a TokenFilter > We are also planning a Tokenizer/TokenFilter that can put parts of speech as > either payloads (PartOfSpeechAttribute?) on a token or at the same position. > I'd propose it go under: > modules/analysis/opennlp -- This message was sent by Atlassian JIRA (v6.1#6144) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org