Re: Elasticsearch puppet module's problem

Hi, So any advice what should I pass to configure service? It seems configs are removed by module itself which is odd. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an e

Re: No hits if fields are not stored

This is an index request that I have taken out of my _bulk request. Hope it helps: curl -XPOST http://localhost:9200/_bulk -d '{ "index" : { "_index" : "files", "_type" : "rawfiles", "_id" : "130741557032361573_equilibrating" } }{ "content_br" : [], "content_da" : [], "content_de" : [], "cont

Re: Elasticsearch Version Upgrade

Only post 1.0 -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 22 avr. 2015 à 01:14, Norberto Meijome a écrit : > > David, is this the case with older versions (both client and server on 0.90.x > versions using java client), and across the 0.90 to 1.x boundary, or only

Re: Document getting lost

Simply increasing the depths means more things will queue, but you still need to catch up on that queue. If you are overloaded then this will never happen and your queue won't be much help. Look at the larger picture, are you running out of resources consistently or is it transitory? On 22 April 2

Document getting lost

Hi , Document getting lost due to queue size settings and also see ESRejection exception as explained in below article https://www.loggly.com/blog/nine-tips-configuring-elasticsearch-for-high-performance/ I want to understand that if right queue size is set then there is no chance of document

Re: org.elasticsearch.index.mapper.MapperParsingException: failed to parse - need guidance

Note this line: Caused by: java.lang.NumberFormatException: For input string: "Cached ad is better" What's the mapping on your 'error' field? On Tuesday, April 21, 2015 at 2:41:34 PM UTC-7, Tony Chong wrote: > > > Hi, > > Using ELK > ES 1.5.0 > LS 1.5.0rc2 > Kibana 4.0.1 > > I have read about

Re: How to diagnose slow queries every 10 minutes exactly?

Ok, I've identified the problem, and it has nothing to do with ES :) It's something funky with the network card or its driver. I isolated the problem by testing each of my two nodes independently, identified it was only a problem on one of the two. Then spun up a simple node http listener and wa

Re: Elasticsearch Version Upgrade

David, is this the case with older versions (both client and server on 0.90.x versions using java client), and across the 0.90 to 1.x boundary, or only post 1.x? On 22/04/2015 12:03 am, "David Pilato" wrote: > This should work in both ways. > > The client knows what is the node version. > The nod

Re: Elasticsearch service often goes down or gets killed

I have seen these types of issues because the heap size was not big enough. It WILL just die and you will not know what happened. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it

org.elasticsearch.index.mapper.MapperParsingException: failed to parse - need guidance

Hi, Using ELK ES 1.5.0 LS 1.5.0rc2 Kibana 4.0.1 I have read about similiar issues but wasn't really sure what the proper way to fix this is. I basically have a bunch of log files written out in nested JSON. The logs can be a combination of various keys:values, sometimes with the value being n

Re: How to diagnose slow queries every 10 minutes exactly?

Thanks for the info, but there's no load balancer involved here. No VMs either.. nothing fancy. On Tuesday, April 21, 2015 at 1:55:18 PM UTC-7, AlexR wrote: > > it could be entirely unrelated but if I recall someone reported similar > regular interval slowness. it proved to be the load balancer

Re: How to diagnose slow queries every 10 minutes exactly?

it could be entirely unrelated but if I recall someone reported similar regular interval slowness. it proved to be the load balancer they used if I remember correctly. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this

Re: Elasticsearch service often goes down or gets killed

You need to monitor the cluster with something like Marvel, kopf or HQ to find out what is happening. ES may die if the cluster is overloaded (think OOM), but you should see something in the logs on that. On 20 April 2015 at 22:13, Sébastien Vassaux wrote: > Hello! > > My webserver is running ub

Re: How to export dashboard and visualization by using elasticdump?

Take a look at https://github.com/taskrabbit/elasticsearch-dump On 21 April 2015 at 23:25, Priya G wrote: > Can anyone tell me the steps how to install elasticdump and how to export > and import dashboards? > > -- > You received this message because you are subscribed to the Google Groups > "ela

Re: selecting a server - a single quad socket, or two dual socket

It may make sense to do this, you probably also want to look into running multiple instances on the host to maximise capacity. On 21 April 2015 at 19:08, Tzahi jakubovitz wrote: > Today we can buy very performant servers at very reasonable price points. > > e.g. – the price of two dual socket se

Re: deploying ElasticSearch to a large memory server

It's definitely reasonable to run multiple instances per physical here. On 21 April 2015 at 19:22, Tzahi jakubovitz wrote: > Hi all, > I have a server with 1.5 TB memory. > I can either use it with a single ES process, or launch few separate > instances (using either VM, docker, or just differen

Re: Rebuilding master node caused data loss

Default is still yes. What happened in the logs on the data nodes? On 22 April 2015 at 00:23, Brian wrote: > I have a cluster with 5 data nodes, and 1 master node. I decided to test > a master node failure, and clearly I miss understood exactly what is stored > on the master. I turned down th

Re: How to diagnose slow queries every 10 minutes exactly?

Ok, I have deleted everything and restarted the ES service, on both nodes, and rebuilt the indexes. I have changed my shard count to 1 -- thank you for explaining that, I didn't realize the overhead was as large as it was. I do have some indexes that are much larger than others. The largest one

Re: Scoring - queryNorm differs for documents during one query

Just some thoughts: As the queryNorm is calculated from terms frequencies - it seems to me, that it is calculated from only those terms of the query that somehow "matched" the document in some clause. So in our example, for first document terms "a" and "b" were used to calculate queryNorm, but f

Re: Bulk Index from Remote Host

David and Christopher, thanks for your advice, i did split the files into 12 MB chunks,which was found to be optimum after testing various sizes. I wanted draw from your experience of potential issues, w.r.to bulk indexing from local vs bulk indexing from remote. i did choose to bulk index locall

Re: Index Size and Replica Impact

Hi, I did not change the default, it was set to default : 5 And the shards were allocated as you mentioned. My index is search intensive than index intensive, so all nodes are configured as Master and Data nodes. David, could you point me to a resource on how you derived the shard size across al

Search by 'all values from a document must be contained in query'

Hi! I'm indexing documents with list of required features (int[] A). And then I want to filter docs by the list of available features (int[] B). So that all required features must be in a list of available (A & B = A). I haven't found any appropriate filter for this and understand that this pr

Re: Script to return array for scripted metric aggregation from combine

Vineeth, You can return any standard groovy object (by this i mean primitives, strings, arrays or maps) from the combine script and it will be passed to the reduce script. Below is a sense recreation script for a more complex example which counts the number of occurances of each word in a field

Script to return array for scripted metric aggregation from combine

Hi , For scripted metric aggregation , in the example shown in the documentation , the combine script returns a single number. Instead here , can i pass an array or hash ?

Rebuilding master node caused data loss

I have a cluster with 5 data nodes, and 1 master node. I decided to test a master node failure, and clearly I miss understood exactly what is stored on the master. I turned down the VM running the master node, and built a new one from scratch. I then added it to the cluster as a master. When

Re: No hits if fields are not stored

I’m not saying that you need to send all your data. But to send one document at least which is supposed to match. So then we can play with your script and try to fix it. -- David Pilato - Developer | Evangelist elastic.co @dadoonet | @elasticsearchfr

Re: Elasticsearch Version Upgrade

This should work in both ways. The client knows what is the node version. The node knows what is the client version. So basically, if one knows he should not send a new data because the other one is too old, it will simply ignore it. Same for reading. If your node is newer, he knows that the cli

Re: Elasticsearch Version Upgrade

Another Question : if i will upgrade my Elasticsearch Client to Version 1.5.1 and my Elasticsearch Servers will stay on version 1.4.2 will it work ? it there a backward compatibility ? On Tuesday, April 21, 2015 at 4:21:38 PM UTC+3, Costya Regev wrote: > > Just checking , > > so you are sure th

Applicative version type mapping

Hi there, I was looking for a type mapping for application version ? Meaning that the following seria ["3.9.1","4.0.0",2.5.3.1.alpha","6.3.1.beta"] can be queried/searched/sorted ? Does ES has support for that ? Thanks, Yarden -- You received this message because you are subscribed to the G

How to export dashboard and visualization by using elasticdump?

Can anyone tell me the steps how to install elasticdump and how to export and import dashboards? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+

Re: Elasticsearch Version Upgrade

Just checking , so you are sure that there is forward compatibility... and my system will work fine with Es Client version of 1.4.1 when the server's version will be 1.5.1 , right ? On Tuesday, April 21, 2015 at 3:12:44 PM UTC+3, David Pilato wrote: > > It should work fine. > > -- > David ;-) >

Re: Creating Snapshot Repository on Windows cluster

Hi Thanks for the reply.Unfortunately I've tried all of those things already. As I say, I've got the permissions wide open (full control on Everyone) for both the share and the file system underneath. You have to escape the UNC path in JSON, I've tried double escaping etc. with no luck. I've t

Re: No hits if fields are not stored

If by data you mean the indexing calls, then I'm afraid they are too big to be any relevant. Also not sure what this could help with since I have no issues with creating, mapping or indexing data. As I said, what happens is once I change my fields' 'store' property to false, my queries stop ret

Re: Elasticsearch Version Upgrade

It should work fine. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 21 avr. 2015 à 14:08, Costya Regev a écrit : > > Hi , > > We have Elasticsearch Servers running with Es Version 1.4.2,our client > version is 1.4.1. > > We are about to upgrade our Es cluster Version

Re: No hits if fields are not stored

A full script is closed to what you sent. Data are just missing here. Also, could you use GIST to post it? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 21 avr. 2015 à 13:54, Zaid Amir a écrit : > > Sorry, not sure what you mean by a recreation script. But here is h

Elasticsearch Version Upgrade

Hi , We have Elasticsearch Servers running with Es Version 1.4.2,our client version is 1.4.1. We are about to upgrade our Es cluster Version to 1.5.1 , my question is : Do we need to upgrade the client version to 1.5.1 or our current version should be compatible with the new Version? Thanks,

Re: No hits if fields are not stored

Sorry, not sure what you mean by a recreation script. But here is how my index is created and mapped along with the search query #Create an Index curl -XPUT 'localhost:9200/files #Create index mapping enabling _source and disabling _all fields #Without the store property set to true, no results

Re: Creating Snapshot Repository on Windows cluster

the problem might be in your location syntax . please change the location path to "location" ;" \\gbr-t-ess-003\\Snapshots\\backup2\\" On Tue, Apr 21, 2015 at 5:04 PM, Deepak Chauhan < deepak.chau...@daffodilsw.com> wrote: > this problem may be due to the reason that the directory you are provid

Re: Creating Snapshot Repository on Windows cluster

this problem may be due to the reason that the directory you are providing is not accepting Binary data you can try by changing the directory and provide read-write permissions to the directory On Tue, Apr 21, 2015 at 4:40 PM, Sam Judson wrote: > Sorry to bump, but anyone have any idea on this o

Re: Creating Snapshot Repository on Windows cluster

Sorry to bump, but anyone have any idea on this one, as I'm stumped. Does anyone need any more information? Sam On Monday, 20 April 2015 09:57:42 UTC+1, Sam Judson wrote: > > Hi > > I'm having some trouble creating a snapshot repository on a cluster > running on Windows. > > PUT _snapshot/main_

Re: No hits if fields are not stored

I don’t understand. Could you GIST a full recreation scripts which demonstrate what you are seeing? -- David Pilato - Developer | Evangelist elastic.co @dadoonet | @elasticsearchfr | @scrutmydocs

No hits if fields are not stored

Hi, I am having issues with ES. I have configured ES to store the _source field, however when I query I do not get any hits unless I "store" the fields that I want to query. This is how my query request looks like: { "from": 0, "size": 50, "sort": [ { "_score": { "order"

about org.elasticsearch.indices.recovery.RecoveryFailedException error

*Hi All,* *I have a 5 nodes cluster. * *now my cluster eror * [2015-04-21 18:32:26,032][WARN ][indices.cluster ] [i-bxtszyhz] [blacklist][1] failed to start shard org.elasticsearch.indices.recovery.RecoveryFailedException: [blacklist][1]: Recovery failed from [i-5tar85fu][9ofd50CbQm

Re: I have got a little Problem with my synonym filter ....

What kind of query are you executing? Are you query against a specific field? A match query against the title field should work. When using the analyze API, explicit state the field and not the analyzer for more accurate behavior of what really goes on. Cheers, Ivan On Apr 21, 2015 11:40 AM, "St

Re: I have got a little Problem with my synonym filter ....

> I forgot to figure out that if search for "aaa" I receive Record _id = 1, > searching vor "bbb" I receive Record _id = 2 ... nothing else. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving

I have got a little Problem with my synonym filter ....

*... I build a little sample of what I do.* *My Test Synonyms file is (test.syn placed into my /etc/elasticsearch folder):* aaa,bbb,ccc,ddd www,xxx,yyy,zzz eee,fff,ggg,hhh => 111 sss,ttt,uuu,vvv => 222 rrr => 333,444,555 *I created an index like so:* PUT /testindex?pretty { "settings": {

Re: Using serialized doc_value instead of _source to improve read latency

The answer is these changes in elasticsearch.yml: script.groovy.sandbox.class_whitelist: com.fasterxml.jackson.databind.ObjectMapper script.groovy.sandbox.package_whitelist: com.fasterxml.jackson.databind for some reason these classes are not shaded even though the pom.xml does shade them. On T

deploying ElasticSearch to a large memory server

Hi all, I have a server with 1.5 TB memory. I can either use it with a single ES process, or launch few separate instances (using either VM, docker, or just different ports on the same server OS). What will be a reasonable number of instances for such a server ? Thanks, Tzahi -- You received

selecting a server - a single quad socket, or two dual socket

Today we can buy very performant servers at very reasonable price points. e.g. – the price of two dual socket servers with 512 GB memory is comparable to a single quad socket server with 1024 GB (1 TB) memory. (Assuming same number of cores and MHz on each CPU) My gut feeling is that a singl

Re: Bulk Index from Remote Host

Hi, The best way to approach this is to restrict the size of your bulk request and / or the number of documents for each request. I tend to do both, the best sizes seem to be in the 5 to 10 MiB range, however, I also restrict (which isn't really necessary) the max number of documents (e.g. 500

Re: Cohort analysis using the query DSL

thanks for this Mark - I only got around to going through the slides today. I will still have to find a solution which involves a more general approach which can be applied to any of our indexed fields (logging data from several systems, different document layout, and more systems joining in the

Re: How to diagnose slow queries every 10 minutes exactly?

Some notes. You are using defaults. So you have 5 shards per index. 1000 primary shards. With replicas, it means 1000 shards per Node. Which means 1000 Lucene instances. First thing to do is IMO to use only one shard per index unless you need more for "big" indices. Then, have 3 nodes and set m