Could someone please share a trivial example of using *Keyword Repeat Token Filter <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-keyword-repeat-tokenfilter.html>* to: “[Emit] each incoming token twice once as keyword and once as a non-keyword to allow an un-stemmed version of a term to be indexed side by si[d]e to the stemmed version of the term"
Maybe I don’t understand its’ intent but is the idea to be able to tokenize string “one two” into “one”, “two”, “one two” (last being unstemmed), right? If so, I tried example config (in the docs) and input was not preserved: http://localhost:9200/_analyze?pretty=true&text=one%20two&filters=lowercase,keyword_repeat,porter_stem,unique { "tokens" : [ { "token" : "one", "start_offset" : 0, "end_offset" : 3, "type" : "<ALPHANUM>", "position" : 1 }, { "token" : "two", "start_offset" : 4, "end_offset" : 7, "type" : "<ALPHANUM>", "position" : 2 } ] } *Shouldn't** there be 3 tokens - “one”, “two”, “one two”?* …using v 1.0.1 BTW, seems to make more sense to use ‘keyword’ tokenizer instead of ‘standard’ (since latter splits “one two” before filter is even enacted). but that fails to return “one”, and “two” http://localhost:9200/_analyze?pretty=true&text=one%20two&filters=lowercase,keyword_repeat,porter_stem,unique&tokenizer=keyword { "tokens" : [ { "token" : "one two", "start_offset" : 0, "end_offset" : 7, "type" : "word", "position" : 1 } ] } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93d58a7d-07c1-485f-afee-3c2f1e9b994f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
