[ https://issues.apache.org/jira/browse/SOLR-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lance Norskog updated SOLR-3653: -------------------------------- Description: The "Smart" Simplified Chinese toolkit in lucene/analysis/smartcn does not work in some edge cases. It fails to split certain words which were not part of the dictionary or training corpus. This patch supplies a bigramming class to handle these occasional mistakes. The algorithm creates bigrams out of all "words" longer than two ideograms. was: The "Smart" Simplified Chinese toolkit in lucene/analysis/smartcn has no Solr factories. Also, since it is a statistical algorithm, it is not perfect. This patch supplies factories and a schema.xml type for the existing Lucene Smart Chinese implementation, and includes a "fixup" class to handle the occasional mistake made by the Smart Chinese implementation. > Custom bigramming filter for to handle Smart Chinese edge cases > --------------------------------------------------------------- > > Key: SOLR-3653 > URL: https://issues.apache.org/jira/browse/SOLR-3653 > Project: Solr > Issue Type: New Feature > Components: Schema and Analysis > Reporter: Lance Norskog > Attachments: SOLR-3653.patch, SmartChineseType.pdf > > > The "Smart" Simplified Chinese toolkit in lucene/analysis/smartcn does not > work in some edge cases. It fails to split certain words which were not part > of the dictionary or training corpus. > This patch supplies a bigramming class to handle these occasional mistakes. > The algorithm creates bigrams out of all "words" longer than two ideograms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org