improve interaction of synonymsfilterfactory with analysis chain ----------------------------------------------------------------
Key: SOLR-2648 URL: https://issues.apache.org/jira/browse/SOLR-2648 Project: Solr Issue Type: Improvement Components: Schema and Analysis Affects Versions: 3.4, 4.0 Reporter: Robert Muir Spinoff of LUCENE-3233 (there is a TODO here), this was also mentioned by Otis on the mailing list: http://www.lucidimagination.com/search/document/8e91f858314562e/automatic_synonyms_for_multiple_variations_of_a_word#76c3d09f95f7a58f As of LUCENE-3233, the builder for the synonyms structure uses an Analyzer behind the scenes to actually tokenize the synonyms in your synonyms file. Currently the solr factory uses a WhitespaceTokenizer, unless you supply the tokenizerchain parameter, which lets you specify a tokenizer. If there was some way to instead specify a chain to this factory (e.g. charfilters, tokenizer, tokenfilter such as stemmers) versus just a tokenizerfactory, it would be a lot more flexible (e.g. it would stem your synonyms for you), and would solve this use case. Personally I think it would be most ideal if this just automatically work, e.g. if you have a chain of A, B, SynonymsFilter, C, D: then in my opinion the synonyms should be analyzed with an analysis chain of A, B. This way the injected synonyms are processed as if they were in the tokenstream to begin with. Note: there are some limitations here to what the chain can do, e.g. you cant be putting WDF before synonyms or other things that muck with positions, and you cant have a synonym that analyzes to nothing at all, but the parser checks for all these conditions and throws a syntax error so it would be clear to the user that they put the synonymsfilter in the "wrong place" in their chain. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org