Re: Kibana response time is too slow, need help identifying why

2014-08-05 Thread Tony Chong
Well, my slow logs are 0 bytes. My logging.yml looks okay but I don't think they are configured. I looked at the ES docs and saw that I should have these set somewhere. I'm thinking elastic search.yml configuration file? #index.search.slowlog.threshold.query.warn: 10s

how to use my customer lucene analyzer(tokenizer)?

2014-08-05 Thread fancyerii
I want to use my own Chinese analyzer and I can write lucene analyzer class myself. How can I integrate it to elasticsearch? I googled and found http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-analyzers.html. But it only combine existing tokenizers and filters. I can

Re: how to use my customer lucene analyzer(tokenizer)?

2014-08-05 Thread Tihomir Lichev
I think you should write your own plugin with your analyzer implementation, so you can register your analyzer into the ES and use it later on. You could have a look and see how it done maybe in this plugin: https://github.com/elasticsearch/elasticsearch-analysis-kuromoji 05 август 2014,

Re: how to use my customer lucene analyzer(tokenizer)?

2014-08-05 Thread Jun Ohtani
Hi, I think this plugin will be helpful for you. https://github.com/elasticsearch/elasticsearch-analysis-kuromoji 2014/08/05 15:58 fancye...@gmail.com: I want to use my own Chinese analyzer and I can write lucene analyzer class myself. How can I integrate it to elasticsearch? I googled and

Re: Inexplicable wrong results in automated tests

2014-08-05 Thread Tihomir Lichev
I also had timing problems in my automated tests, the solution was to invoke index refresh and wait for it, then execute any search requests. This way you can make sure that all test data is ready for searching. 03 август 2014, неделя, 18:22:50 UTC+3, John D. Ament написа: Hi So after

Using facets/aggretagions on parent document, queried by TopChildren

2014-08-05 Thread Adam Porat
Hi, The TopChildren query works with an estimated hit size, and the TotalHits might be incorrect if there are more child docs matching the required hits. How does that affect facets or aggregations defined on the parent document? Would their count might be likewise be incorrent? Or would

Re: Context suggest display results for all the categories (mimic the completion suggester behavior)

2014-08-05 Thread Tihomir Lichev
Thank you Alex, I also came up with that idea, but I think it is not that useful when you have context, mapped to a field, which usually contains a single value, especially numeric fields (despite that https://github.com/elasticsearch/elasticsearch/issues/6512 is not fixed yet) -- You

How to validate user input?

2014-08-05 Thread esc
I try to use Elasticsearch\Endpoints\Indices\ValidateQuery and i get some abstract error. here is part of my code $parm['hosts'] = array('localhost:80'); $test = new Elasticsearch\Endpoints\Indices\ValidateQuery($parm); $test1 = new

Re: Elasticsearch 1.3 Transform Scripts

2014-08-05 Thread amrelhagary
I have a problem i need to add dynamic fields according to certain fields value , i use groovy in simple case and it worked but the logic become more complicated and i need to separate it in external groovy script file and pass context as parameters but it didn't work and this must in mapping

Re: Context suggest display results for all the categories (mimic the completion suggester behavior)

2014-08-05 Thread Alexander Reelsen
Hey, maybe the new transform feature could help here to create a default value (just an idea, havent tested it). See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-transform.html#mapping-transform --Alex On Tue, Aug 5, 2014 at 10:13 AM, Tihomir Lichev

TTL value must be 0. Illegal value provided

2014-08-05 Thread Dennis de Boer
I recently recieved some exceptions in my log while updating documents into my index. org.elasticsearch.ElasticSearchIllegalArgumentException: TTL value must be 0. Illegal value provided [-17810] at org.elasticsearch.action.index.IndexRequest.ttl(IndexRequest.java:291) at

How to rebalance shard

2014-08-05 Thread Warat Wongmaneekit
Now my cluster is not rebalance the data. How can I rebalance it please see the summary below. https://lh6.googleusercontent.com/-bH-VCgOCVWk/U-Ck6ykiJgI/SlI/S54duefOraA/s1600/Screen+Shot+2557-08-05+at+4.32.44+PM.png -- You received this message because you are subscribed to the

Re: TTL value must be 0. Illegal value provided

2014-08-05 Thread Dennis de Boer
forgot to mention I'm using version 0.90.5 Op dinsdag 5 augustus 2014 11:08:37 UTC+2 schreef Dennis de Boer: I recently recieved some exceptions in my log while updating documents into my index. org.elasticsearch.ElasticSearchIllegalArgumentException: TTL value must be 0. Illegal value

Re: Using existing field for mapping

2014-08-05 Thread Ayush
I am using Java API, but have tried doing this using, 1) putting a mapping file in /config/mapping etc. 2) Using curl On Monday, August 4, 2014 1:35:00 PM UTC+5:30, Tihomir Lichev wrote: What API are you using to communicate with ES ? 03 август 2014, неделя, 11:14:59 UTC+3, Ayush написа:

How to forbid the analyzing for a certain data type (e.g. string)

2014-08-05 Thread panfei
Hi all: we gathering data(in json format) from user input. I want to know is there any way to forbid the analyzing process for certain data type(e.g. string), so that if we detect the value of some field is in string format, we will not analyze it. thanks -- 不学习,不知道 -- You received this

Re: Inexplicable wrong results in automated tests

2014-08-05 Thread John D. Ament
Hmm. So how do I invoke a reindex via the java API? On Tue, Aug 5, 2014 at 3:40 AM, Tihomir Lichev shot...@gmail.com wrote: I also had timing problems in my automated tests, the solution was to invoke index refresh and wait for it, then execute any search requests. This way you can make

Re: wrong boosting in ES?

2014-08-05 Thread Bernd Fehling
After using MatchQuery and also trying TermQuery and FunctionScoreQuery I can say that Elasticsearch always produces NumericRangeQuery for numeric fields with boosting (or even for all numeric queries ???). fieldName = oa , value = 1 , boost = 400 E.g. oa:1 -- oa:[1 to 1] As for MatchQuery

Re: Using existing field for mapping

2014-08-05 Thread Tihomir Lichev
1. Try go execute GET http://yourhost:9200/_mapping and check if the mapping mentions is properly created with correct _id path 2. When indexing the documents you can construct your index request this way: IndexRequestBuilder indexRequestBuilder = getClient().prepareIndex(index,

Re: Inexplicable wrong results in automated tests

2014-08-05 Thread Tihomir Lichev
Sort of: TransportClient transportClient = *new* TransportClient(settings); // just an example of transport client instantiation, you can use your own transportClient.getClient().admin().indices().refresh(*Requests* .refreshRequest(indexName)).actionGet(); // actionGet() as you may know it to

strange highlight result, can anyone explain it?

2014-08-05 Thread Ivan Ji
Hi all, I am using highlight function in ES 1.0.1. I found a very strange situation as follow: I want to highlight the field group.* which is the inside a dictionary, whose mapping is {properties: { group: { type: object, dynamic: false, include_in_all: true,

[ANN] Elasticsearch AWS cloud plugin 2.3.0 released

2014-08-05 Thread Elasticsearch Team
Heya, We are pleased to announce the release of the Elasticsearch AWS cloud plugin, version 2.3.0. The Amazon Web Service (AWS) Cloud plugin allows to use AWS API for the unicast discovery mechanism and add S3 repositories.. https://github.com/elasticsearch/elasticsearch-cloud-aws/ Release

Re: strange highlight result, can anyone explain it?

2014-08-05 Thread Ivan Ji
The query command I used is as {'multi_match': {'fields': ['_all', 'name', 'group.*'], 'operator': 'and', 'query': 'Adobe'}} I doubt the problem might be because the various analyzers of the group.*. What is the analyzer to be used during the highlight? Ivan Ivan Ji於

Re: How to forbid the analyzing for a certain data type (e.g. string)

2014-08-05 Thread Tihomir Lichev
Probably you should use dynamic templates for mappings: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-root-object-type.html#_dynamic_templates 05 август 2014, вторник, 14:08:29 UTC+3, 潘飞 написа: Hi all: we gathering data(in json format) from user input. I want

Multiple master problem in elasticsearch 0.90.10

2014-08-05 Thread Ankit Mittal
Hi All, I am using 2 master node in my elasticsearch cluster and 2 data node. I ma using 2 RHEL server ( 1 master + 1 data node on each ) When i setup the above it is working fine for some time around 15 - 20 hours. but after then my cluster health is yellow. and the no available shards are

rss river for 1.3

2014-08-05 Thread Andrej Rosenheinrich
Hi David, a probably easy to answer question: is the current stable release (rssriver-1.1.0) supposed to work with elasticsearch 1.3.1? Will there be a new release any time soon? Or a logstash input? ;) Thanks! Andrej https://github.com/dadoonet/rssriver/releases/tag/rssriver-1.1.0 -- You

java.lang.ClassNotFoundException: org.elasticsearch.transport.RemoteTransportException

2014-08-05 Thread Earle Nietzel
Using elasticsearch embedded in tomcat 7 where we have custom classloader that shares spring application beans with many webapps. The API's to these implementations are in shared but the implementations are in a separate classloader ComponentLoader. Our search implementation is loaded from

Set active index

2014-08-05 Thread Rhys Campbell
Hi All, I have setup a EFK system just for testing at the moment. It's running in a VM with not much RAM and I am having problem with the elasticsearch process because of this. The VIRT = 12GB which is approximately the total size of the indexes. My indexes are split by date like so...

Re: java.lang.ClassNotFoundException: org.elasticsearch.transport.RemoteTransportException

2014-08-05 Thread joergpra...@gmail.com
The cluster wants to transport an exception to your web app container, and the web app does not have access to elasticsearch jar. You should have a look at the ES server logs, if there are any exceptions, to find the real problem. Then, after fixing the real problem, you should try to configure

Re: Kibana response time is too slow, need help identifying why

2014-08-05 Thread Tony Chong
Turns out you shouldn't use the head plugin when querying for hot threads. I was able to get them by querying the API directly. Thanks for the tip! On Monday, August 4, 2014 11:28:16 PM UTC-7, Tony Chong wrote: Well, my slow logs are 0 bytes. My logging.yml looks okay but I don't think they

Some observations with Curator

2014-08-05 Thread Brian
Using the most recent release (1.2.2) of Curator, I noticed that the documentation says --logfile while curator itself rejects --logfile anywhere and requires -l in front of the other options to direct its log entries. No big deal; I just tested it until it worked before adding it to the cron

Re: Some observations with Curator

2014-08-05 Thread Ivan Brusic
I am still fully in the nothing but E stack! Is anyone else using Elasticsearch for ... search? :) -- Ivan On Tue, Aug 5, 2014 at 10:50 AM, Brian brian.from...@gmail.com wrote: Using the most recent release (1.2.2) of Curator, I noticed that the documentation says --logfile while curator

How can I combine two elasticsearch responses into one?

2014-08-05 Thread Jesse Russell
I am currently stuck with a result response from a multi-search that returns an array of responses for each query I add, but I need the responses to be combined so that the relevancy to determine their order. Getting back two separated lists of hits isn't great for creating a unified search

Constant re-syncing of mapping, ES 1.3.1

2014-08-05 Thread Vitaly Aminev
Hi, on my master node I constantly see in the logs the following messages: [2014-08-05 12:06:27,763][WARN ][cluster.metadata ] [ark.com. 01.ark74] [ark-profiles-2014-07-09] re-syncing mappings with cluster state for types [[profiles_v1]] [2014-08-05 12:06:27,783][WARN ][cluster.metadata

has_parent slow in filter, fast in query?

2014-08-05 Thread James Ekstrom
We ran into a problem with a slow query. Turns out having a has_child (or has_parent) query in the filter part of a filtered query is a lot slower than having the same has_child filter in the query part of the filtered query. The faster of the two queries runs in under 100ms, while the slower

Re: ES 1.3.0 and 1.2.3 released

2014-08-05 Thread Chris Neal
Bump for Bigdesk 3 :) On Thu, Jul 24, 2014 at 12:40 AM, Lukáš Vlček lukas.vl...@gmail.com wrote: Hi, Bigdesk (2.x) uses JSONP. Elasticsearch is dropping JSONP support starting with 1.3 release. Still you can enable it in Elasticsearch but I do not recommend it if Bigdesk would be the only

Group by field and then sum the groups

2014-08-05 Thread Cameron Barker
Hi all, I have an elastic database of posts, each post has a *user_id* and has *likes* field. My goal is to output for a query how many likes in total each user has. I wondered if any one had any advice/direction I could take to achieve this? input: {user_id: 10, likes: 20} {user_id: 9,

Re: How to rebalance shard

2014-08-05 Thread Ivan Brusic
Are you applying a custom routing to your documents? -- Ivan On Tue, Aug 5, 2014 at 2:33 AM, Warat Wongmaneekit canopyb...@gmail.com wrote: Now my cluster is not rebalance the data. How can I rebalance it please see the summary below.

Restoring a snapshot from one machine to a different machine.

2014-08-05 Thread Donald Cha
I would like to use the Snapshot functionality to move indices from an OLD server to a NEW server. My approach to do this task was to take a snapshot from OLD server, copy the snapshot over to the NEW server (I zipped this snapshot because it contained multiple indices), and try to restore that

Storing Elasticsearch configuraton and deploying new clusters

2014-08-05 Thread Hayden Muhl
We've been using Solr for the past several months, but have been considering Elasticsearch recently. One think I can't quite wrap my head around is how to keep track of Elasticsearch cluster configuration. Solr is largely based on config files written in XML. These are very straight forward to

Re: How to transform SQL columns using JDBC River?

2014-08-05 Thread joergpra...@gmail.com
Just released - stored procedures are available in JDBC plugin 1.3.0.4 https://github.com/jprante/elasticsearch-river-jdbc/ Jörg On Tue, Aug 5, 2014 at 12:24 PM, joergpra...@gmail.com joergpra...@gmail.com wrote: I have started work on stored procedures

Elasticsearch cluster with AWS Spot Instances

2014-08-05 Thread Felipe Hummel
Hey guys, I've been thinking about the idea of running a elasticsearch cluster in AWS/EC2 mixing normal (on-demand or reserved) instances with spot instances. The motivation is simple: spot instances run just as fast as a normal instance but with a much lower price/hour. With the only caveat

Searching email

2014-08-05 Thread Mark Fletcher
Hi, We're using ES to index email, specifically mailing list messages. We'd like search to work similar to Gmail in that we'd like to match on either the subject or body of the email, and if it matches on the subject, we only want to display one result for that match (say the first message in

Re: Storing Elasticsearch configuraton and deploying new clusters

2014-08-05 Thread Mark Walkom
There are some settings you can set via the API, but not all. The docs are pretty indepth so check out http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com

Re: Storing Elasticsearch configuraton and deploying new clusters

2014-08-05 Thread Hayden Muhl
I was thinking more in terms of things like mappings. The only way of setting up mappings that I can find is via the API. If I've set up and tested a particular way of indexing and querying my documents on a QA system, I want to set my production system up the same way. What is your process for

Re: Storing Elasticsearch configuraton and deploying new clusters

2014-08-05 Thread Mark Walkom
You can store mappings on the filesystem - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-conf-mappings.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 6 August 2014 10:03, Hayden

Re: Some observations with Curator

2014-08-05 Thread Aaron Mildenstein
Hmm. What version of python are you using? I am able to use --logfile or -l interchangeably. I'm glad you like Curator, and I like KELTIC :) Nice acronym. On Tuesday, August 5, 2014 11:50:06 AM UTC-6, Brian wrote: Using the most recent release (1.2.2) of Curator, I noticed that the

System Requirements for ElasticSearch stack

2014-08-05 Thread Gopinath Nallappan
Hello all, I'm new to the ELK stack. I will be logging Windows Events, Syslogs from firewalls, routers etc into my elasticsearch. I am expecting daily data of around 2GB to be logged into my elasticsearch server. I will be creating indices on daily or weekly basis. And my logs are going to

Re: Multiple master problem in elasticsearch 0.90.10

2014-08-05 Thread Ivan Brusic
There should be no need to run a master and a data node on each machine. Only two masters is not enough to reliably form a consensus and you are only taking away processing power from the data node. -- Ivan On Tue, Aug 5, 2014 at 5:30 AM, Ankit Mittal ankit.lnc...@gmail.com wrote: Hi All,

Re: Searching email

2014-08-05 Thread Tihomir Lichev
Isn't better to create single document for each mail with fields subject and body (and whatever else you need from the mail) ? This way you can search by any or all of the fields, also you can define boosting for each field. For instance when your search matches the subject the mail will be

Re: Group by field and then sum the groups

2014-08-05 Thread Tihomir Lichev
You can use aggregations: { aggs: { user_likes: { terms: { field: user_id }, aggs: { likes_sum: { terms: { field: likes } } } } } } 05 август 2014, вторник, 23:11:59 UTC+3, Cameron Barker написа: Hi all, I have an