reserved characters on multimatch query
Hi, How to search reserve/special characters on multimatch query ? First of all , here the analyzer details for your reference. index : analysis : analyzer : default_index : type : custom tokenizer : whitespace filter : [ word_delimiter, snowball, lowercase] default_search : type : custom tokenizer : whitespace filter : [ word_delimiter, snowball, lowercase] filter : word_delimiter : type : word_delimiter preserve_original : true split_on_numerics : true stem_english_possessive : false The following query returns no resluts , when using the multimatch query. query : { bool : { must : { multi_match : { query : \\(\\), fields : [ field1, field2, field3, field4 ], operator : AND } } } Where when i search through specific field it works perfectly. query : { bool : { must : { query_string : { query : message:\\(\\), default_operator : and } } } } -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cc067a67-3735-4b80-85f8-b14ddad9fdc6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: using a different analyzer for each query and same index
Hi, I think this page is useful for you. http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/language-pitfalls.html 2014-10-12 0:39 GMT+09:00 Aris Alexis snowboard...@gmail.com: bump On Wednesday, October 8, 2014 4:16:42 PM UTC+2, Aris Alexis wrote: Hi, first of all I am a noob with elastic so bear with me. I am interested to know if this scenario is possible: A user posts some text , my app detects the language and index the text with the spanish analyzer for example. B user posts some text in english, my app detects it and indexex the text with the english analyzer *in the same field*. The search query also specifies the analyzer. Is this possible or do I need a field for every language? that's not a very elegant solution. some other idea? thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/080acb42-a86c-4072-a0ee-f42ee44eef4a%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/080acb42-a86c-4072-a0ee-f42ee44eef4a%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- --- Jun Ohtani blog : http://blog.johtani.info -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPW8A5yZJJFGtvD8_HEQ5PmScd_701gne-A%2BcwHnenc21t1Ofg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
User retention/churn calculation
Hi Guys, I need to get user retention from Elasticsearch. My algorithm is: retention_period = 60 days 1. start_registrations_date = today - retention_period - 30 days 2. end_registrations_date = today -retention_period 3. Get all users born in start_registrations_date to end_registrations_date 4. user_lifeTime = (day that user was last seen) - (day that user registered) 5. Retention = ((#users with user_lifeTime retention_period ) / #users) * 100 I have registration event that looks something like this : { userId : raz, createDate : 10-10-2014T11:00:00 } . . . And ping event that looks something like this: { userId : raz, createDate : 10-10-2014T13:00:00 }, { userId : raz, createDate : 10-10-2014T14:00:00 } . . . Now to my question: How do you recommend to achieve the retention using the aggregation platform? Is there away to aggregate users life time some retention period in the system? I hope solving this issue will help someone here because calculating retention/churn is a very common analytic. Thanks, Raz. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/49fd4b68-c454-4862-9e9e-f82183bc974d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
sorting by _geo_distance returns the wrong order
Hello, In my Index i have several locations with some information and geo data. In the detail view of an location i try to query for locations which are near by the current one. I am confused, cause my query does not return the correct order of locations. If you check the geo locations you will see, that the order is wrong. The bool query is there for preventing getting the current location in the result ( identified by the locations slug ). In the following example, please notice that the query is run with the geo point of adlon hotel, so the first returned location should be Hotel Prens Berlin Kreuzberg. Here is the query: curl -XGET 'http://localhost:9200/search/location/_search' -d '{sort:[{_geo_distance:{pin:[52.51515,13.38019],order:asc,unit:km,distance_type:arc}}],query:{bool:{must_not:[{match:{slug:adlon-hotel}}]}},size:3}' Here is my mapping: { search : { mappings : { location : { _meta : { model : AnchorVentures\\Bundle\\LocationBundle\\Entity\\Location }, _all : { auto_boost : true }, properties : { address : { properties : { pin : { type : geo_point, store : true, lat_lon : true }, zipcode : { type : string, store : true } } }, city : { type : nested, properties : { name : { type : string, boost : 10.0, store : true } } }, countedRatings : { type : string, store : true }, createdAt : { type : string, store : true }, details : { type : nested, properties : { guestsQuantity : { type : string, store : true }, isCatering : { type : string, store : true }, isIndoor : { type : string, store : true }, isOutdoor : { type : string, store : true }, isOvernightAccommodation : { type : string, store : true }, price : { type : string, store : true }, roomQuantity : { type : string, store : true } } }, name : { type : string, store : true }, slug : { type : string, store : true }, updatedAt : { type : string, store : true } } } } } } And finally some documents { took : 2, timed_out : false, _shards : { total : 5, successful : 5, failed : 0 }, hits : { total : 10, max_score : 1.0, hits : [ { _index : search, _type : location, _id : 207, _score : 1.0, _source:{name:Badeschiff,slug:badeschiff,countedRatings:0, createdAt:2014-10-13T10:40:58+02:00,updatedAt: 2014-10-13T10:40:58+02:00,details:{price:1,isIndoor:true,isOutdoor :true,isCatering:true,isOvernightAccommodation:false,roomQuantity:1, guestsQuantity:50},city:{name:Berlin},address:{zipcode:12435, pin:52.4970479,13.4525442}} }, { _index : search, _type : location, _id : 214, _score : 1.0, _source:{name:Le Plat du Jour,slug:le-plat-du-jour, countedRatings:0,createdAt:2014-10-13T10:40:58+02:00,updatedAt: 2014-10-13T10:40:58+02:00,details:{price:5,isIndoor:true,isOutdoor :true,isCatering:true,isOvernightAccommodation:false,roomQuantity:4, guestsQuantity:50},city:{name:Hamburg},address:{zipcode:20095, pin:53.54911,9.99439}} }, { _index : search, _type : location, _id : 206, _score : 1.0, _source:{name:Spreequartier,slug:spreequartier, countedRatings:0,createdAt:2014-10-13T10:40:58+02:00,updatedAt: 2014-10-13T10:40:58+02:00,details:{price:5,isIndoor:true,isOutdoor :true,isCatering:true,isOvernightAccommodation:false,roomQuantity:3, guestsQuantity:80},city:{name:Berlin},address:{zipcode:10245, pin:52.5018737,13.4484104}} }, { _index : search, _type : location, _id : 213, _score : 1.0, _source:{name:Club Heim im Schanzenpark,slug: club-heim-im-schanzenpark,countedRatings:1,createdAt: 2014-10-13T10:40:58+02:00,updatedAt:2014-10-13T10:40:58+02:00, details:{price:5,isIndoor:true,isOutdoor:true,isCatering:true,
running multiple cluster on 1 server
Hello, I want to create multiple cluster on 1 server, is that possible in elasticsearch 1.x ? I've found article about this but it's for elasticsearch before 1.x. Previously, i've succeeded to create failover node within same cluster on different server. Now, i want to create multiple cluster on 1 server so my 3 elasticsearch server (different cluster) would have failover node in 1 server only (by creating multiple cluster in 1 server). Regards, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7cefda0d-2257-400c-ad24-e2b0e94a57af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: running multiple cluster on 1 server
Yes it will work, and probably the same way as pre 1.X. The elasticsearch puppet module could handle this for you. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 14 October 2014 20:21, haries fajar nugroho harie...@gmail.com wrote: Hello, I want to create multiple cluster on 1 server, is that possible in elasticsearch 1.x ? I've found article about this but it's for elasticsearch before 1.x. Previously, i've succeeded to create failover node within same cluster on different server. Now, i want to create multiple cluster on 1 server so my 3 elasticsearch server (different cluster) would have failover node in 1 server only (by creating multiple cluster in 1 server). Regards, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7cefda0d-2257-400c-ad24-e2b0e94a57af%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7cefda0d-2257-400c-ad24-e2b0e94a57af%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bzX01jYn1wg4Uw2NK%3DByo-JmG%3DPTtZJYwXK9a2FQQEgQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
nested filter query giving error
Hi all, I am new to elastic search.I am using nested filter query.this query is giving error failed to find nested object under path [itemResults]]. my document structure is - { _id: inspectionresults-07d864fd-76ec-427a-a1c6-b70e420ce3d8, _rev: 12-64c44ddcfcd477dfa86cdb9d3c659748, sectionResults: [ { SectionName: More Details, itemResults: [ { InspectionItem: Launch Month, ItemInfo: } ] } ] } and my query is - { query: { filtered: { query: { match_all: {} }, filter: { nested: { path: itemResults, filter: { bool: { must: [ { term: { itemResults.InspectionItem: Launch Month } } ] } } } } } } } Error - failed to find nested object under path [itemResults]] But as you are seeing in document itemResults is already nested object. Please reply. Thanks in advance, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a38b570f-64ac-44e6-8424-59ffb6bafa52%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
ReadTimeoutError while trying to create a snapshot
Hello, this fuctions works perfectly when i try to create a snapshot of indexes lower than 100mb. But if i try to do it with one index that have more than 600mb i receive an ugly exception. Anyone know if i should use *master_timeout *in order to get it working? code def create_snapshot(self): Create new snapshot :return: print Creating new snapshot print Snapshot: + self.snap_name try: self.es.snapshot.create( repository=self.repository_name, snapshot=self.snap_name, wait_for_completion=true ) except elasticsearch.ElasticsearchException: raise else: print Snapshot created finally: pass ---code - ERROR- File /usr/lib/python2.6/site-packages/elasticsearch/client/utils.py, line 68, in _wrapped return func(*args, params=params, **kwargs) File /usr/lib/python2.6/site-packages/elasticsearch/client/snapshot.py, line 19, in create repository, snapshot), params=params, body=body) File /usr/lib/python2.6/site-packages/elasticsearch/transport.py, line 284, in perform_request status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) File /usr/lib/python2.6/site-packages/elasticsearch/connection/http_urllib3.py, line 51, in perform_request raise ConnectionError('N/A', str(e), e) elasticsearch.exceptions.ConnectionError: ConnectionError(HTTPConnectionPool(host='127.0.0.1', port=9200): Read timed out. (read timeout=10)) caused by: ReadTimeoutError(HTTPConnectionPool(host='127.0.0.1', port=9200): Read timed out. (read timeout=10)) - ERROR- -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c4b27e68-c588-4d12-b8db-01b41c320687%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Need help to create array type field in elastic search
Hi All, I have a scenario like In elastic search i want to create custom mapping.The data will be insert into index using java API . Now i have a field with multi values like 1,2,3 .Field name is categories. can you please tell me how to create a custom mapping for type array so that when i insert data in to elastic search index the categories type is need to be an string array which can hold multi values.(string array). catgories : [ 1, 2, 3 ] I used river bracket notation to dynamically create array at elastic search using bracket notation.but from java API how can we create custom mapping so that it can hold multi values. Thanks phani -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/22bd6f5c-2d90-4fa6-9f1d-2009ff7fda99%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: running multiple cluster on 1 server
Hi Mark, I've tried without puppet module since i haven't familiar with, but the result is like this: [root@cg-nfs-01 ~]# /usr/share/elasticsearch/bin/elasticsearch -Des.config=/etc/elasticsearch/elasticsearch-haries.yml log4j:WARN No appenders could be found for logger (node). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. On Tuesday, October 14, 2014 4:28:26 PM UTC+7, Mark Walkom wrote: Yes it will work, and probably the same way as pre 1.X. The elasticsearch puppet module could handle this for you. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com javascript: web: www.campaignmonitor.com On 14 October 2014 20:21, haries fajar nugroho hari...@gmail.com javascript: wrote: Hello, I want to create multiple cluster on 1 server, is that possible in elasticsearch 1.x ? I've found article about this but it's for elasticsearch before 1.x. Previously, i've succeeded to create failover node within same cluster on different server. Now, i want to create multiple cluster on 1 server so my 3 elasticsearch server (different cluster) would have failover node in 1 server only (by creating multiple cluster in 1 server). Regards, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7cefda0d-2257-400c-ad24-e2b0e94a57af%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7cefda0d-2257-400c-ad24-e2b0e94a57af%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8e84917a-ddf5-4152-9b54-e531569d4916%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: nested filter query giving error
Hi, elasticsearc has two ways to index inner json documents, one is called object[0] (which is the default) and the other is called nested[1]. You need to have the `itemResults` field mapped as `nested` in your mappings. [0] http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-object-type.html [1] http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-nested-type.html On Tue, Oct 14, 2014 at 12:00 PM, shekhar chauhan shekharchauha...@gmail.com wrote: Hi all, I am new to elastic search.I am using nested filter query.this query is giving error failed to find nested object under path [itemResults]]. my document structure is - { _id: inspectionresults-07d864fd-76ec-427a-a1c6-b70e420ce3d8, _rev: 12-64c44ddcfcd477dfa86cdb9d3c659748, sectionResults: [ { SectionName: More Details, itemResults: [ { InspectionItem: Launch Month, ItemInfo: } ] } ] } and my query is - { query: { filtered: { query: { match_all: {} }, filter: { nested: { path: itemResults, filter: { bool: { must: [ { term: { itemResults.InspectionItem: Launch Month } } ] } } } } } } } Error - failed to find nested object under path [itemResults]] But as you are seeing in document itemResults is already nested object. Please reply. Thanks in advance, -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a38b570f-64ac-44e6-8424-59ffb6bafa52%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a38b570f-64ac-44e6-8424-59ffb6bafa52%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j41Ewx-Kui_bxtjDotnktC7a29Q3bw2umH_7qERQ9MVhw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Question about completion in ES
You are probably using an analyzer that removes numbers? On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belvalle...@gmail.com wrote: Hi all, I try to put an auto completion system based on ES. This article http://www.elasticsearch.org/blog/you-complete-me/ helps me a lot, but I don't understand a case: PUT /hotels { mappings: { hotel : { properties : { name_suggest : { type : completion } } } } } PUT /hotels/hotel/1 { name_suggest : { input : [ 21 Mercure Hotel Munich, 24 Mercure Munich ] } } PUT hotels/hotel/2 { name_suggest : { input : [ Monaco Munich, Hotel Monaco ] } } When i post : POST /hotels/_suggest { hotels : { text : m, completion : { field : name_suggest } } } The result is 3 hotels : 21 Mercure Hotel Munich,24 Mercure Munich, Monaco Munich. But when i post this request: POST /hotels/_suggest { hotels : { text : 2, completion : { field : name_suggest } } } There is no result ... Any Idea ? Franck -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6bhn2Z9gMoXUhR0p6_SEaHPXqDi0G4YXxUx8qy0u0rTg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
ElasticSearch Marvel - how to load sample data from .json file
I am practising elasticsearch(ES) and using Marvel plugin to run my queries. I see the query in ES documentation to load data from a .json file but, http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_exploring_your_data.html curl -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json this is the command for linux users. But how to run same command from Marvel I copied the accounts.json file to elasticsearh/bin folder and I tried multiple ways POST /bank/account/_bulk?pretty accounts.json but it giving error. Need your help. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/656fd77c-913f-40a7-a21f-fdad7096bd87%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Get date histogram date for nested aggregations
Is it possible to get the date of the date histogram for nested aggregations? Suppose I want to calculate past events from month before aggregation with key_as_string : 10-10-2012, how can I do this ? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5878d92c-e733-4216-872d-8d145d8a7f8e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Question about completion in ES
Mapping is very basic. There are no analyzers. Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit : You are probably using an analyzer that removes numbers? On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belv...@gmail.com javascript: wrote: Hi all, I try to put an auto completion system based on ES. This article http://www.elasticsearch.org/blog/you-complete-me/ helps me a lot, but I don't understand a case: PUT /hotels { mappings: { hotel : { properties : { name_suggest : { type : completion } } } } } PUT /hotels/hotel/1 { name_suggest : { input : [ 21 Mercure Hotel Munich, 24 Mercure Munich ] } } PUT hotels/hotel/2 { name_suggest : { input : [ Monaco Munich, Hotel Monaco ] } } When i post : POST /hotels/_suggest { hotels : { text : m, completion : { field : name_suggest } } } The result is 3 hotels : 21 Mercure Hotel Munich,24 Mercure Munich, Monaco Munich. But when i post this request: POST /hotels/_suggest { hotels : { text : 2, completion : { field : name_suggest } } } There is no result ... Any Idea ? Franck -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
timeshift/time-based comparisons
Hi all, is there a simple way to get the same functionality grafana http://play.grafana.org/#/dashboard/db/grafana-play-home provides with the timeShift http://graphite.readthedocs.org/en/1.0/functions.html#graphite.render.functions.timeShift function in Kibana as well? I am currently using the trends panel for this kind of time-based comparisons. My problem with this panel is: while it indicates that something happened when i want to find out what exactly happened i usually have to compare different Kibana-tabs unless it is really trivial. Regards, -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/timeshift-time-based-comparisons-tp4064849.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1413289198157-4064849.post%40n3.nabble.com. For more options, visit https://groups.google.com/d/optout.
Re: Question about completion in ES
Completion fields use the `simple` analyzer by default, which removes numbers. If you try another analyzer that keeps numbers such as the standard analyzer, this should work: PUT /hotels { mappings: { hotel : { properties : { name : { type : string }, city : { type : string }, name_suggest : { type : completion, analyzer: standard } } } } } On Tue, Oct 14, 2014 at 2:03 PM, Franck B f.belvalle...@gmail.com wrote: Mapping is very basic. There are no analyzers. Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit : You are probably using an analyzer that removes numbers? On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belv...@gmail.com wrote: Hi all, I try to put an auto completion system based on ES. This article http://www.elasticsearch.org/blog/you-complete-me/ helps me a lot, but I don't understand a case: PUT /hotels { mappings: { hotel : { properties : { name_suggest : { type : completion } } } } } PUT /hotels/hotel/1 { name_suggest : { input : [ 21 Mercure Hotel Munich, 24 Mercure Munich ] } } PUT hotels/hotel/2 { name_suggest : { input : [ Monaco Munich, Hotel Monaco ] } } When i post : POST /hotels/_suggest { hotels : { text : m, completion : { field : name_suggest } } } The result is 3 hotels : 21 Mercure Hotel Munich,24 Mercure Munich, Monaco Munich. But when i post this request: POST /hotels/_suggest { hotels : { text : 2, completion : { field : name_suggest } } } There is no result ... Any Idea ? Franck -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444% 40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5ZEn%3DmTj1CW6PywODQz-DgMsOyJwe454GLGqGWi4ZwpA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Custom date format in elasticsearch mapping
I am trying to index data with date format *Tue May 14 17:06:01 PDT 2013*. As described in Elasticsearch Date Format http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-date-format.html document I need to use custom date format. I refer to DateTimeFormat http://joda-time.sourceforge.net/api-release/org/joda/time/format/DateTimeFormat.html document and respective format is *E M d H:m:s z Y* I am able to create mapping but when I am trying to index data its giving me error. *Mapping:-* { tweet: { properties: { user: { type: string, index: not_analyzed }, message: { type: string, null_value: na }, postDate: { type: date, format: E M d H:m:s z Y }, priority: { type: integer }, rank: { type: float } } } } *Index Document:-* - curl -XPUT 'http://localhost:9200/tweets/tweet/1' -d '{ user : kimchy, message : This is a tweet!, postDate : Tue May 14 17:06:01 PDT 2013, priority : 4, rank : 12.3 }' *Error:-* --- {error:MapperParsingException[failed to parse [postDate]]; nested: MapperParsingException[failed to parse date field [Tue May 14 17:06:01 PDT 2013], tried both date format [E M d H:m:s z Y], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: \Tue May 14 17:06:01 PDT 2013\ is malformed at \May 14 17:06:01 PDT 2013\]; ,status:400} Any Suggestion? Which date format I should use here? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eb4187c8-ab4a-4ab7-8018-7be3777f3124%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
How to define a ratio of values which should be inside the result set of a function score query
Hi ES group, i have a function score query, which searches for user profiles in a big pool (~2 million documents). The query works fine, but the index contains a lot more male profiles than female ones (around 100 times more). Now my goal is to somehow specify a ratio of the gender distribution of the result. For example i want to get 6 male and 4 female profiles at any time I query the index. How could I force ES to ensure this ratio? I tried a filter on the gender attribute inside the function score query with a boost_factor but it didn't worked out. The result contains about 9 male and 1 female profile right now. The query looks like the following (slightly stripped-down): { query: { function_score: { boost_mode: replace, score_mode: sum, boost: 1, filter: { // some hard filter, this shouldn't matter }, functions: [ { random_score: { seed: null } }, { boost_factor: 1, field_value_factor: { field: gender, gender: female } }, { linear: { location: { origin: , //some origin here scale: 50km, offset: 25km, decay: 0.25 } } }, { linear: { lastlogin: { origin: 2014-10-14, scale: 30d, offset: 1d, decay: 0.5 } } } ] } }, from: 0, size: 10 } Best regards, Stefan -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/237fdee5-3c8f-471e-b8fd-ce53f6b086be%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch Marvel - how to load sample data from .json file
The bulk end point uses a different - flattened - JSON layout. So, you cannot just use that example directly. But it's also not clear exactly what you are trying to do? If you look at the Guide (not just reference), it gives complete Mavel/Sense examples including once that load into your local Sense UI. That should make it clear what you need to do. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 14 October 2014 07:45, Ravi Kumar Maddi ravi.ma...@gmail.com wrote: I am practising elasticsearch(ES) and using Marvel plugin to run my queries. I see the query in ES documentation to load data from a .json file but, http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_exploring_your_data.html curl -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json this is the command for linux users. But how to run same command from Marvel I copied the accounts.json file to elasticsearh/bin folder and I tried multiple ways POST /bank/account/_bulk?pretty accounts.json but it giving error. Need your help. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/656fd77c-913f-40a7-a21f-fdad7096bd87%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEFAe-FAan_b6GXoqCHAMnkYZ0ynkeAfSasnvo_JO9c5sOuUVw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
ElasticSearch- IndexReaders cannot exceed 2147483647
Getting following exception in one of our ElasticSearch Index, [Failed to start shard, message [IndexShardGatewayRecoveryException[[fw][4] failed recovery]; nested: IllegalArgumentException[Too many documents, composite IndexReaders cannot exceed 2147483647]; ]] Please somebody help. Thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f60845f2-476d-489a-936f-291884986dbf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch- IndexReaders cannot exceed 2147483647
There is no upper limit... On Tuesday, October 14, 2014 8:02:03 PM UTC+5:30, Alexandre Rafalovitch wrote: Obvious question: how many documents are you expecting to have in there? Including the nested ones if you are storing them as separate documents. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 14 October 2014 10:25, Prasanth R prasanth...@gmail.com javascript: wrote: Getting following exception in one of our ElasticSearch Index, [Failed to start shard, message [IndexShardGatewayRecoveryException[[fw][4] failed recovery]; nested: IllegalArgumentException[Too many documents, composite IndexReaders cannot exceed 2147483647]; ]] Please somebody help. Thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f60845f2-476d-489a-936f-291884986dbf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3646a7e4-7246-43df-8970-01b776a0720f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Enable logging for elasticsearch plugins
Hi, I am indexing text documents using the mapper attachments plugin but it is not working well. My code encode properly the content of the document but when I search a word that is in the docment, it does not match anything. Is there any way to enable logging for the elasticsearch plugin? Regards, Mireia -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Enable-logging-for-elasticsearch-plugins-tp4064834.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1413272137946-4064834.post%40n3.nabble.com. For more options, visit https://groups.google.com/d/optout.
search query in elasticsearch
Hi, I am trying to search for a document in elasticsearch. Field name is sum and Field value is 'SUM-123' I created a boolquery to matchQuery('sum','SUM-123'). But it is not returning the exact sum field documents instead it is returning the documents with different field values. -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/search-query-in-elasticsearch-tp4064839.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1413279659976-4064839.post%40n3.nabble.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch- IndexReaders cannot exceed 2147483647
Obvious question: how many documents are you expecting to have in there? Including the nested ones if you are storing them as separate documents. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 14 October 2014 10:25, Prasanth R prasanth.sunr...@gmail.com wrote: Getting following exception in one of our ElasticSearch Index, [Failed to start shard, message [IndexShardGatewayRecoveryException[[fw][4] failed recovery]; nested: IllegalArgumentException[Too many documents, composite IndexReaders cannot exceed 2147483647]; ]] Please somebody help. Thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f60845f2-476d-489a-936f-291884986dbf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEFAe-EfMGvubuVGK%2Bp0BZpAopj7UOVL%2BUE6c18JDAJc_Am-bw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch- IndexReaders cannot exceed 2147483647
On 14 October 2014 10:33, Prasanth R prasanth.sunr...@gmail.com wrote: There is no upper limit... Well, then you must have an infinitely scalable architecture and a decision when the content starts getting shared. So, then the question is what is your individual shard allowed to grow to. Which is how many documents - including nested - you are expecting to have in a single shard. Because, ElasticSearch has an internal limit and you just hit it. So, the question is whether it is intentional, unintentional or a result of a bug. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEFAe-G8GsLZpeO9-b5_RGs%3D-tGBNp39QgeY2rywjRnOQfcfnw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Aggregation in ES like MongoDB
Hi all, I have many documents like {'user': 1, 'type': 1}, {'user': 2, 'type': 1}, {'user': 2, 'type': 2}, {'user': 3, 'type': 1}, {'user': 4, 'type': 1} ... Then, I have this aggregation in MongoDB aggregation pipeline: db.events1.aggregate([ {$group: {_id: $user, events:{$addToSet: '$type'}}}, {$group: {_id: $events, counter: {$sum: 1}}}, ]) So, after first phase (aggregate by user), i have: user1- 1 user2 - 1,2 user3 - 1 user4 - 1 And after second (aggregate by 'events' set and count): 1 - 3 1,2 - 1 (this is the desired result) Can I do something similar to this in ElasticSearch aggregation? Tnanks. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00e64f0e-f26f-4126-b5c0-10e583d8499b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch- IndexReaders cannot exceed 2147483647
Thanks for the reply. My scenario here is, 1) No nested docs. 2) I don't have any limit per shard.. I didn't know about internal limit of ES. On Oct 14, 2014 8:23 PM, Alexandre Rafalovitch arafa...@gmail.com wrote: On 14 October 2014 10:33, Prasanth R prasanth.sunr...@gmail.com wrote: There is no upper limit... Well, then you must have an infinitely scalable architecture and a decision when the content starts getting shared. So, then the question is what is your individual shard allowed to grow to. Which is how many documents - including nested - you are expecting to have in a single shard. Because, ElasticSearch has an internal limit and you just hit it. So, the question is whether it is intentional, unintentional or a result of a bug. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/H74KAYmGtoc/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEFAe-G8GsLZpeO9-b5_RGs%3D-tGBNp39QgeY2rywjRnOQfcfnw%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1LRDnv9Nw71py%3DPFksSqLnXOG6x6-K0kUW1j26%3D6_pYA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Cannot restore a snapshot with IndexMissingException[[_snapshot] missing]
Note to self mostly. I later noticed it happening when trying to transfer snapshots between repositories (i.e. snapshot on one, restore on the other). It would happen when snapshot was not visible on the cluster where restore operation was taking place. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0c8ecafc-13af-4db2-9287-3e7216776a91%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Elasticsearch give a factor to aggregations
Hi all, I'm wondering something about aggregations, say Percentiles. When a percentile aggregation is processed, it uses a specific field as a reference. If the 50th percentile for field 'f' is 10, it means there are 50% of documents with 'f' under 10. = Each document has the same weight in the aggregation (=1) I'm wondering if it could be possible to give a different weight for each document. The following Gist give an example of what I'd like to do : https://gist.github.com/rnonnon/093c111014bd14a46efe I'd like to compute some percentiles on the age field. But for each document, there is a count field associated. For example, there are 5 person who are 10 years old ; 1 who is 20 years old... If the percentile agg runs, it won't use my factor to compute the percentile... I don't think that feature is natively supported, but, do you guess it could be easily supported? Thanks. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17f66762-e51c-4241-9957-e34403e4f054%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: has_child filter and min_children
ANy thoughts anyone? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c97ec75-8c4e-44a3-92e2-1a54c8317e86%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Can update by script be performed on query results atomically?
Just like the Update API allows us to update a document based on a script provided ... by *removing some network roundtrips* ... is it possible in ES to perform the update on all the results of a query? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/07065c91-7389-4603-a470-29b41ac64dda%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: issues with using repository-hdfs plug in for snapshot/restore operation
I had a similar scenario, running CDH 4.6, unable to initialise the repository with hadoop2 version and started changing gradle as Brent suggests. However, maintaining our own version was a bit too much in our case. As Costin pointed to me here https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/613YHEUAtuA, the lightweight jar is there for the plethora of distributions, as long as you have your hadoop jars on the classpath. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dffaaab2-8e66-4574-a755-a1577a6343bc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Elastic search - exporting index from existing cluster to a different cluster
We run into a similar problems recently after a very recent upgrade from ES 1.2.2 to ES 1.3.4. In our scenario, we have two ES clusters and a HDFS system which both can access. Indices are created on one of them, snapshotted to HDFS repository (named after the index) and then restored on the other cluster. This worked fine (when ignoring random error messages) on 1.2.2. After the upgrade to 1.3.4, however, clusters don't seem to be able to see new snapshots made on the other system. Files sit in HDFS, it is possible to restore locally, accessing: https://snapshot-cluster.domain.com/_snapshot/repository_name/_all returns the expected list while https://restore-cluster.domain.com/_snapshot/repository_name/_all is empty. Beforehand, the lists were identical. I will try to check version 1.3.2 tomorrow, but remaining clueless at the moment. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f241a3aa-fd8d-486c-8f6b-40faa701bccd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Snapshot/restore between clusters not working after upgrade to 1.3.4
We run into a problem recently after a very recent upgrade from ES 1.2.2 to ES 1.3.4. In our scenario, we have two ES clusters and a HDFS system which both can access. Indices are created on one of them, snapshotted to HDFS repository (named after the index) and then restored on the other cluster. This worked fine (when ignoring random error messages) on 1.2.2. After the upgrade to 1.3.4, however, clusters don't seem to be able to see new snapshots made on the other system. Files sit in HDFS, it is possible to restore locally, accessing: https://snapshot-cluster.domain.com/_snapshot/repository_name/_all returns the expected list while https://restore-cluster.domain.com/_snapshot/repository_name/_all is empty. Beforehand, the lists were identical. I will try to check version 1.3.2 tomorrow, but remaining clueless at the moment. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/24ab44cf-0195-4742-a11d-807d51b84db2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: sorting by _geo_distance returns the wrong order
i solved it myself. I needed to set slug in my mappings to not_analyzed and i change the query to this one: { from: 0, size: 3, query: { filtered: { query: { bool: { must_not: [ { match: { slug: adlon-hotel } } ] } }, filter: { exists: { field: pin } } } }, sort: [ { _geo_distance: { pin: { lat: 52.51515, lon: 13.38019 }, order: asc, unit: km, distance_type: arc } } ] } Am Dienstag, 14. Oktober 2014 10:04:00 UTC+2 schrieb Sören Martius: Hello, In my Index i have several locations with some information and geo data. In the detail view of an location i try to query for locations which are near by the current one. I am confused, cause my query does not return the correct order of locations. If you check the geo locations you will see, that the order is wrong. The bool query is there for preventing getting the current location in the result ( identified by the locations slug ). In the following example, please notice that the query is run with the geo point of adlon hotel, so the first returned location should be Hotel Prens Berlin Kreuzberg. Here is the query: curl -XGET 'http://localhost:9200/search/location/_search' -d '{sort:[{_geo_distance:{pin:[52.51515,13.38019],order:asc,unit:km,distance_type:arc}}],query:{bool:{must_not:[{match:{slug:adlon-hotel}}]}},size:3}' Here is my mapping: { search : { mappings : { location : { _meta : { model : AnchorVentures\\Bundle\\LocationBundle\\Entity\\Location }, _all : { auto_boost : true }, properties : { address : { properties : { pin : { type : geo_point, store : true, lat_lon : true }, zipcode : { type : string, store : true } } }, city : { type : nested, properties : { name : { type : string, boost : 10.0, store : true } } }, countedRatings : { type : string, store : true }, createdAt : { type : string, store : true }, details : { type : nested, properties : { guestsQuantity : { type : string, store : true }, isCatering : { type : string, store : true }, isIndoor : { type : string, store : true }, isOutdoor : { type : string, store : true }, isOvernightAccommodation : { type : string, store : true }, price : { type : string, store : true }, roomQuantity : { type : string, store : true } } }, name : { type : string, store : true }, slug : { type : string, store : true }, updatedAt : { span style=color: ... -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1438f59c-e889-42b3-ab98-274a0c7b4d48%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
NotFilter dude
Hello all! ia have a criteria with *AND* , *OR* and *NOT* operator, but the *NOT* is a single Filter, what is the best way to set multiples *NOT's*? see my query with *AND* my *AND* i need *NOT* :D { highlight: { fields: { *: { fragment_size: 150, number_of_fragments: 1, pre_tags: [ b ], post_tags: [ /b ] } } }, facets: { documents: { terms: { field: primary_field, size: 5 } } }, fields: [ Document.id, Document.name, Document.updated, DocumentTag.name, Document.approval_number, Document.approval_number_us, Document.approval_number_jp, Version.status, Document.rate, Document.last_status ], sort: [ _type ], size: 10, query: { filtered: { query: { match_all: {} }, filter: { and: { filters: [ { terms: { Product.id: [ 6 ] } }, { terms: { Version.jp: [ true ] } }, { terms: { Version.jp: [ true ] } }, { terms: { Document.last_status: [ 4 ] } } ] } } } } } -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d9d34e14-7d3c-41a5-b3ad-a33ccbd79d45%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Need help to create array type field in elastic search
There is nothing special you need to add to your mapping to enable multiple values for a field. Just pass in an array of values instead of a single value, and all of the values are analyzed. One thing you might want to add for string fields with multiple values: position_offset_gap : *n* When a string field is analyzed, it typically assigns a position to each token that is one greater than the position of the previous token. By setting a position offset gap value to *n*, it skips ahead that many positions, representing the number of non-matching word positions between consecutive values. What this does is that if your field contains multiple values that each has multiple words, a phrase query won't span across values unless the slop value is large enough (n or larger, I seem to recall). Hope this helps. Brian -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2d2dd14-d899-4e1e-a909-ce9e305f900a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Can update by script be performed on query results atomically?
Upon further exploration I found https://github.com/yakaz/elasticsearch-action-updatebyquery ... where the update by query API is packaged as a plugin and allows all documents with the query to be updated with a script. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a13c8b4c-13b0-47ca-a3c6-d29f759149f4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Testing the index existence.
Hi David! I am using your v1.3.0 of the ElasticsearchAbstractClientFactory bean and I have the same problem as explained in the beginning of the post. Since the index is not recovered by the time of trying to create the index, using a prepareExists is also returning false (does not exist) and raising an error. The solution I use is simply to catch the IndexAlreadyExistsException. I am writing this as feedback to your post in case you want to consider it for further versions of your factories. Ibai -- View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Testing-the-index-existence-tp4019208p4064867.html Sent from the ElasticSearch Users mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1413304433357-4064867.post%40n3.nabble.com. For more options, visit https://groups.google.com/d/optout.
Re: Elasticsearch-Hadoop repository plugin Cloudera Hadoop 2.0.0-cdh4.6.0
My ES cluster nodes and Hadoop nodes are not collocated. Light version does not works for me without putting enough correct versions of hadoop related jars. Right now I don't want to create my jar as Brent did and I don't want to install hadoop or copy jars on the es nodes either . Right now I can wait until my hadoop cluster upgrade to a version that es-hadoop supports. Or there is a version that is compatible to the hadoop My ES nodes talk to . Thanks, On Monday, September 1, 2014 8:50:25 AM UTC-7, Mateusz Kaczynski wrote: (Much delayed) thank you Costin. Indeed, on Ubuntu, changing ES_CLASSPATH to include hadoop and hadoop/lib directories in /etc/default/elasticsearch (and exporting it in /etc/init.d/elasticsearch) and installing light plugin version did work. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca1465d4-d617-4623-9de3-35bd69e6f8fb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Optimal configuration for 2 high powered nodes
Hi, I am using 2 fairly high powered physical machines as ES nodes, each has 96GB of memory and 1TB of RAID10 SSD. The Elasticsearch database is currently around 300GB with 450,000,000 documents. I am trying to understand which of these configurations would be optimal? 1. 1 x ES instance on each machine with 30GB Heap (for pointer compression reasons) 2. 1 x ES instance on each machine with 57.6GB (60% of total memory) Heap 3. 2 x ES instances on each machine, 30GB Heap for each -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01495019-59cb-4573-b42e-11d52519b3c5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Elasticsearch-Hadoop repository plugin Cloudera Hadoop 2.0.0-cdh4.6.0
You need the appropriate hadoop jar on your classpath otherwise es-hadoop repository plugin cannot connect to HDFS. In the repo, you'll find two versions with vanilla hadoop1 and hadoop2 - however if you are using a certain distro, for best compatibility you should use that distro client jars. Please refer to your distro client configuration and use the light version along with those jars. P.S. In the future, please create your own thread instead of 'hijacking'/reviving an old one. Cheers, On Tue, Oct 14, 2014 at 10:00 PM, Jinyuan Zhou zhou.jiny...@gmail.com wrote: My ES cluster nodes and Hadoop nodes are not collocated. Light version does not works for me without putting enough correct versions of hadoop related jars. Right now I don't want to create my jar as Brent did and I don't want to install hadoop or copy jars on the es nodes either . Right now I can wait until my hadoop cluster upgrade to a version that es-hadoop supports. Or there is a version that is compatible to the hadoop My ES nodes talk to . Thanks, On Monday, September 1, 2014 8:50:25 AM UTC-7, Mateusz Kaczynski wrote: (Much delayed) thank you Costin. Indeed, on Ubuntu, changing ES_CLASSPATH to include hadoop and hadoop/lib directories in /etc/default/elasticsearch (and exporting it in /etc/init.d/elasticsearch) and installing light plugin version did work. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca1465d4-d617-4623-9de3-35bd69e6f8fb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJogdmdihcht8e%3DDeYy0CTipOzazzT5KZgbkVbEyW80ZQEfVHA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Kibana CSS not being used
Dear Folks: I'm trying to get Kibana 3.1.1 deployed via chef, wrapped by an nginx proxy. Elastic Search version 1.3.2 is running on the same node. When I try to show a Kibana dashboard, I can see the basic text that appears in the dashboard, but I don't see any of the CSS styling. For example, I see a list of the different time ranges that could be selected from a drop down, but it's not functioning as a drop down. Here's what I've tried: 1. Cleared browser cache 2. Tried Chrome, Firefox, Safari 3. Looked in developer mode on each browser to see if any network request was failing (none were failing) (this includes requests for css files as well as requests to elastic search, i.e., _nodes and _aliases). Further, I see no javascript errors in the developer console. 4. Search this google group for help. It looks like the CSS is in fact loading (according to the developer console). Here's my nginx proxy configuration: server { listen0.0.0.0:80; server_name kibana.localhost kibana; access_log/var/log/nginx/kibana.localhost.access.log; location / { root /opt/kibana/current; index index.html index.htm; } location ~ ^/_aliases$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_aliases$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/_nodes$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_search$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_mapping$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/_cluster/health$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } # password protected end points location ~ ^/kibana-int/dashboard/.*$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/kibana-int/temp.*$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } } Here's the Kibana config.js: define(['settings'], function (Settings) { use strict; return new Settings({ elasticsearch: window.location.protocol+//+window.location.hostname+ :+window.location.port, kibana_index: kibana-int, default_route: /dashboard/file/logstash.json, panel_names: [histogram, map, pie, table, filtering, timepicker, text, fields, hits, dashcontrol, column, derivequeries, trends, bettermap, query, terms] }); }); -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/08cc28cb-2df4-4d18-8112-c0c4126fabd2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Adding a bootswatch theme to choice of kibana css themes
Does anyone have any clear instructions on how to do this or if it's possible. The customization instructions at bootswatch seem to escape me. http://bootswatch.com/help/ -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3ee608eb-435c-4d74-b9cf-3042f0c04d38%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Elasticsearch-Hadoop repository plugin Cloudera Hadoop 2.0.0-cdh4.6.0
Thanks, Jinyuan (Jack) Zhou On Tue, Oct 14, 2014 at 1:36 PM, Costin Leau costin.l...@gmail.com wrote: You need the appropriate hadoop jar on your classpath otherwise es-hadoop repository plugin cannot connect to HDFS. In the repo, you'll find two versions with vanilla hadoop1 and hadoop2 - however if you are using a certain distro, for best compatibility you should use that distro client jars. Please refer to your distro client configuration and use the light version along with those jars. P.S. In the future, please create your own thread instead of 'hijacking'/reviving an old one. Cheers, On Tue, Oct 14, 2014 at 10:00 PM, Jinyuan Zhou zhou.jiny...@gmail.com wrote: My ES cluster nodes and Hadoop nodes are not collocated. Light version does not works for me without putting enough correct versions of hadoop related jars. Right now I don't want to create my jar as Brent did and I don't want to install hadoop or copy jars on the es nodes either . Right now I can wait until my hadoop cluster upgrade to a version that es-hadoop supports. Or there is a version that is compatible to the hadoop My ES nodes talk to . Thanks, On Monday, September 1, 2014 8:50:25 AM UTC-7, Mateusz Kaczynski wrote: (Much delayed) thank you Costin. Indeed, on Ubuntu, changing ES_CLASSPATH to include hadoop and hadoop/lib directories in /etc/default/elasticsearch (and exporting it in /etc/init.d/elasticsearch) and installing light plugin version did work. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca1465d4-d617-4623-9de3-35bd69e6f8fb%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/613YHEUAtuA/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJogdmdihcht8e%3DDeYy0CTipOzazzT5KZgbkVbEyW80ZQEfVHA%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANBTPCGQFHAgz8SRzosoyJw8ixmqNJOxEh%3DUHa%3DMpP5XvfkWfA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Elasticsearch “Query string” query on Geo?
I'm trying to do a query on Elasticsearch using the query string interface on a Geolocation. Specifically, I am trying to return all documents within a circle (point with radius). This is clearly possible using the full query DSL based on JSON. However, I can't find the syntax for the same search using query strings in the docs here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax Does this mean geo queries are not supported by query string queries? Thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0e76efcc-9d0c-4795-822b-c03ad1c0e74c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Problems with auto-_timestamp
Hello, I'm working on an irc bot that indexes in-channel image links, and so far so good---except I can't seem to get automatic timestamps working! I did my best to follow the docs at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-timestamp-field.html but I don't see any change in what my search docs look like, even after deleting the entire index and reconfiguring first. Right now my mapping looks like this: $ curl http://localhost:9200/archivist/_mapping?pretty { archivist : { mappings : { image : { _timestamp : { enabled : true, store : true }, properties : { channel : { type : string }, image : { type : string }, message : { type : string }, user : { type : string } } } } } } and after indexing something (code doing this at https://github.com/jesusabdullah/archivist/blob/master/src/main/java/com/jesusabdullah/archivist/Indexer.java#L71-L82 first java project ever) my search results look like: $ curl http://localhost:9200/archivist/image/_search?pretty { took : 0, timed_out : false, _shards : { total : 5, successful : 5, failed : 0 }, hits : { total : 1, max_score : 1.0, hits : [ { _index : archivist, _type : image, _id : jK8dY6oKTRifbUU4o406pw, _score : 1.0, _source:{channel:#nodebombrange,user:jesusabdullah,message:snoop! http://i.imgur.com/iktO9TK.gif,image:http://i.imgur.com/iktO9TK.gif} } ] } } Am I doing something wrong? Thanks, --Josh -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b5f08fc5-3eb9-48a5-a141-454e30e248c4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
snapshot restore problem in ES 1.3.4
Hi, I encountered a problem with snapshot and restore, checking if somebody ran into the same problem and if yes how they resolved it. So this is what happened, I did a snapshot of one our clusters and then restored that snapshot in another cluster. When I did the restore there was a discrepancy of about 7 million documents from one index between the snapshot cluster and the restored cluster(snapshot cluster had more documents in it). There were 3 other indexes that restored without any data loss. I did a shard check(_cat/shards) and there was one primary shard and its 1 other replicas that had 0 documents. I deleted this index, and restored the index, but this time I set the replica count to zero. This time around there was no data loss in the restored index. The shard that was showing with 0 documents was now showing with 7 million documents. Now I set the replica to 1 and after shards were allocated again, there was again a data loss of 7 million documents. After doing shard check, same shard and its replica had 0 documents. Also the cluster from which I took the snapshot has ES1.2.2 installed and the cluster in which snapshot was restored has ES1.3.4 installed on it. Thanks, Prateek Singhal -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8007d82e-5563-4c06-9c59-f51ae55eb36f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Not able to search long values
Hi Team, I have an index with the below mapping. { merchant_customer:{ properties:{ ADDRESS1:{type:string}, ADDRESS2:{type:string}, CITY_NAME:{type:string}, COUNTRY_CODE:{type:string}, CUSTOMER_NUMBER:{type:long}, MERCHANT_NUMBER:{type:long}, POSTAL_CODE:{type:string}, SHIPPING_POSTAL_ADDRESS_ID:{type:long} } } } And I tried searching the CUSTOMER_NUMBER and MERCHANT_NUMBER from Head plugin as well as from Kibana UI. But its not getting searched. I even tried to explicitly provide *index:not_analyzed *for CUSTOMER_NUMBER and MERCHANT_NUMBER. Even that didn't work. Only after changing these fields to String, I was able to search on these fields. [Eg: CUSTOMER_NUMBER 1234567899876543210] Can someone tell me why is the behaviour like this? Thanks Reeni -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d0078d5b-b443-4030-beba-150fa5cbf7c8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.