Re: Creating a no-op QueryBuilder and FilterBuilder

2015-04-07 Thread Masaru Hasegawa
match all query(QueryBuilders.matchAllQuery()) and match all filter(FilterBuilders.matchAllFilter())? On April 7, 2015 at 08:13:29, Todd Nine (tn...@apigee.com) wrote: Hey all,   I have a bit of an odd question, hopefully someone can give me an answer.  In usergrid, we have our own existing que

Re: Nested object under path [messages] is not of nested type

2015-03-31 Thread Masaru Hasegawa
Hi, You put mapping of "message" type while you query on "thread" type. Since "thread" type isn't defined, you get the error. Masaru On April 1, 2015 at 01:06:51, Daniel Buckle (danielbuckl...@gmail.com) wrote: I am having a lot of issues with the nested type mapping in Elasticsearch, I have

Re: Inconsistent results (Preference = Custom (string) UserId)

2015-03-30 Thread Masaru Hasegawa
*no* node by node it have different score  In that case, difference in score may be due to shard(or segment) state in each node (some shard still have deleted document in segments and others not). It's noticeable when number of documents is relatively small. If you'd like consistent result, you

Re: Inconsistent results (Preference = Custom (string) UserId)

2015-03-29 Thread Masaru Hasegawa
Hi, Is the number of hits the same every time? And are those documents have the same score? If so, the behavior is expected. Elasticsearch (Lucene) uses Lucene's internal document ID when score is the same. You can supply secondary sort criterion like "_uid" to make order consistent. Masaru On

Re: Japanese Search Results with Kuromoji plugin

2015-03-27 Thread Masaru Hasegawa
Matching is done on term basis not character basis(like grep). Since kuromoji splits terms on white spaces, source text(with white spaces) and query(without white spaces) result in the same result (term and position). You may want to see how text is analyzed with different analyzers by _analyze AP

Re: How does Elasticsearch calculate the field-length norm?

2015-03-26 Thread Masaru Hasegawa
Hi, I believe it's because field norm is encoded in single byte. See  http://lucene.apache.org/core/4_10_2/core/org/apache/lucene/search/similarities/DefaultSimilarity.html Masaru On March 26, 2015 at 14:36:45, Xudong You (xudong@gmail.com) wrote: Per this post "theory behind relevance sc

Re: is there a way to define mapping in java with a simple string?

2015-03-26 Thread Masaru Hasegawa
You can use string. See PutMappingRequest#source(String). On March 26, 2015 at 05:30:58, Sai Asuka (asuka.s...@gmail.com) wrote: Is there a way to simply pass mapping information in a json formatted string "{... }" without having to create an object and do a bunch of .put on it within Java? --

Re: Japanese Search Results with Kuromoji plugin

2015-03-26 Thread Masaru Hasegawa
Hi, The input text is already tokenized. As you can see, unlike normal Japanese text, terms are split by white spaces. I guess the input text is already preprocessed using Japanese analyzer. That's why you get hits even if you don't use Japanese tokenizer. (standard tokenizer splits tokens on w

Re: n edge gram analyzer's behave not as expected

2015-03-26 Thread Masaru Hasegawa
Hi, You'd need to specify token_chars when you configure edge ngram tokenizer(http://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenizer.html). Unless, all characters are kept. Which means, words are not split on white spaces. You can see how the analyzer works

Re: question for tokenizer and synonym

2015-03-26 Thread Masaru Hasegawa
Hi, I guess you are using query string query. If you use match query instead, it *should* work. Masaru On March 25, 2015 at 06:41:11, Prateek Asthana (pary...@gmail.com) wrote: I am having requirement similar to below: search for "chevy" should map to search for "chevrolet". I am using syno

Re: Elasticsearch with JSON-array, causing serialize -error

2015-03-26 Thread Masaru Hasegawa
Hi, It looks like "random_point" is defined as object type but got array of numbers. There might be inconsistency in data or you didn't define mapping correctly. You may want to apply correct mapping, "float" or "double". Masaru On March 25, 2015 at 05:07:27, sebastian (sebastia...@gmail.com

Re: Nested list aggregation

2015-03-25 Thread Masaru Hasegawa
Hi, If you define "stages" as nested type, query like this should work: { "query": { "match_all": {} }, "aggs": { "0": { "nested": { "path": "msg.stat.stages" }, "aggs": { "1": { "terms": { "field": "stage" },

Re: Field comparision

2015-03-19 Thread Masaru Hasegawa
Hi, You can use script filter(http://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html). Checking doc['manager'].value == doc['teamMember'].value should work. Alternatively, you can precompute it and add managerIsAlsoTeamMember field to documents. Masaru O

Re: tokenization help mixed n-grams

2015-02-26 Thread Masaru Hasegawa
Hi, You can use mapping char filter to remove white space and then ngram tokenises with min_gram=2/max_gram= to make it ngrams. (not sure if you’d like to omit “bc”, “bcd”… or not though) Masaru On February 26, 2015 at 21:46:42, Ilija Subasic (subasic.il...@gmail.com) wrote: > Hi, > I am tryi

Re: Sort script using object fields

2015-02-26 Thread Masaru Hasegawa
This query should work: {   "query": {     "function_score": {       "functions": [         {           "script_score": {             "script": "doc['topic.t1'].value > 0 || doc['topic.t4'].value > 0 || doc['topic.t7'].value > 0 ? 1 : 0"           }         }       ]     }   } } Masaru On Febr

Re: Combining Multiple Queries with 'OR' or 'AND'

2015-02-19 Thread Masaru Hasegawa
Hi, bool query should work: - { "query": { "bool": { "should": [ { "filtered": { // Query1 "query": {...}, "filter": {...} } }, { "filtered": { // Query2 "q

Re: Elasticsearch Script merge the results of two aggregations

2015-02-19 Thread Masaru Hasegawa
Hi, Looks like you are using lucene expression [1]. See the link for the limitation of lucene expression. Today it only supports numeric values. Since terms agg doesn’t have lang property, probably you have “script.default_lang" set to “expression" in elasticsearch.yml? FYI, if you put “lang”:”

Re: Ngram not working for multivalued field

2015-02-17 Thread Masaru Hasegawa
"str_search_analyzer" : { > "tokenizer" : "keyword", > "filter" : ["lowercase"] > }, > "str_index_analyzer" : { > "tokenizer" : "keyword", > "filter" : ["lowercase", "subs

Re: Ngram not working for multivalued field

2015-02-17 Thread Masaru Hasegawa
Hi, Check your mapping. url_domain is in object lists while documents/queries use plain url_domain. So, standard analyser is used for the field. Masaru On February 18, 2015 at 12:30:23, sri krishna (krishnai...@gmail.com) wrote: > any one faced same issue ? > > On Wednesday, 18 February 201

Re: Searching by date range on field having timestamp doesn't returns me documents

2015-02-05 Thread Masaru Hasegawa
Hi, You are using unix timestamp as is(seconds since epoch) instead of milliseconds since epoch in your documents, perhaps? Masaru On February 5, 2015 at 17:45:01, Paresh Behede (paresh2...@gmail.com) wrote: > Hi Team, > > I want to search or get documents between the date range provided, my

Re: NullPointerException when using script based sorting from Python client

2015-02-04 Thread Masaru Hasegawa
Hi, Something like this should work: -- "doc['shot.org'] ? doc[’shot.org'].value : ’default_value'" -- Masaru On February 5, 2015 at 10:23:34, 'Selim Tuvi' via elasticsearch (elasticsearch@googlegroups.com) wrote: > Actually I am wrong. Looks like some documents did not get cr

Re: Reverse nested aggregation within nested filter aggregation fails

2015-01-22 Thread Masaru Hasegawa
Hi, Not sure if it solves your issue but I think there are a few things to fix: - “attributes" is under “source". nested aggregation’s “path” would be “source.attributes”. You’d need to update field names accordingly as well. - reverse_nested aggregation’s “path” would be empty since it’s joined

Re: stats aggregation on list length

2015-01-22 Thread Masaru Hasegawa
Hi, Objects are flattened in index level. Nothing is indexed as “member” that’s why you get the exception. Using doc[‘members.name'] instead of doc[‘members’] in script should work. Masaru On January 22, 2015 at 19:10:25, Jilles van Gurp (jillesvang...@gmail.com) wrote: > I'm trying to do a

Re: not_analyzed String search

2015-01-21 Thread Masaru Hasegawa
Hi, You’d need to set lowercase_expanded_terms to false in query string query. It’s true by default. That’s why "*Flash*” doesn't get any result. Masaru On January 22, 2015 at 03:17:51, Messias (schubert.torste...@gmail.com) wrote: > Hi, > > I have the following issue when I search on a not_

Re: filtering/querying on script field

2015-01-15 Thread Masaru Hasegawa
Hi Samatha, I don’t think so because script field is created from fields of hit document, results of query/filter. You can use script filter instead  http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html#query-dsl-script-filter. Masaru On January 16

Re: Grandchild is not getting fetched by parent id

2015-01-15 Thread Masaru Hasegawa
Hi Iv, You’d need to specify both parent and routing when you index grand children. See  http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/grandparents.html Masaru On January 15, 2015 at 20:44:43, Iv Igi (sayon...@gmail.com) wrote: > I am experiencing an issue while trying to re

Re: Mysterious index settings overwriting cluster settings

2015-01-13 Thread Masaru Hasegawa
Hi Chris, I think you hit this issue https://github.com/elasticsearch/elasticsearch/issues/8890. Workaround would be to use index template (as described in the issue) or to update them by indices settings API. Masaru On Wed, Jan 14, 2015 at 9:52 AM, Chris Neal wrote: > Hi all. > > I'm repost

Re: Ignore a field in the scoring

2015-01-07 Thread Masaru Hasegawa
ur explanation > > Do you know if it is a bug of intended behavior? > > I don't think deleted (marked as deleted) docs should be used at all > > 2015-01-07 1:53 GMT-02:00 Masaru Hasegawa : > >> Hi, >> >> Update is delete and add. I mean, instead of updat

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

2015-01-07 Thread Masaru Hasegawa
Hi, Not sure what’s failing because of deserialize issue. It usually happens when you use different version of java. You can see what the problem is by checking ES log instead too. To check you are sending document in correct format, you may want to print JSON before sending index request:

Re: Crying for help:: MapperParsingException when trying to create index with mapping

2015-01-06 Thread Masaru Hasegawa
Hi, Your mapping isn't correct. If you remove "properties" around "metrics", it should work. - curl -XPUT "http://localhost:9200/testagg/testagg/_mapping"; -d' { "testagg": { "properties": { "timeStamp": { "format": "dateOptionalTime", "type": "date" },

Re: Ignore a field in the scoring

2015-01-06 Thread Masaru Hasegawa
Hi, Update is delete and add. I mean, instead of updating existing document, it deletes it and adds it as new document. And those deleted documents are just marked as deleted and aren’t actually removed from index until the segment merge. IDF doesn’t take those deleted-but-not-removed document

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

2015-01-05 Thread Masaru Hasegawa
HI, XContentBuilder (I assume jsonBuilder() returns it) serialises date using UTC timezone by default. If you’d like to use different format, you’d need to build your own DateTimeFormatter and pass it when you add date type field. For example, to use ISO date time format with specified timezone