Re: Problems setting up user authentication with Shield

2015-03-15 Thread Sönke Liebau
Hi Mark, thanks a lot, that did the trick! Kind regards, Sönke On Sunday, March 15, 2015 at 9:14:47 PM UTC+1, Mark Walkom wrote: > > There is a step in the longer version of getting started about > setting ES_JAVA_OPTS="-Des.path.conf=/etc/elasticsearch" - > http://www.elastic.co/guide/en/shi

Re: Elasticsearch high heap usage

2015-03-15 Thread Mark Walkom
Those are reasonably large documents. You also seem to have a lot of shards for the data. What sort of data is it, are you using doc values, how are you bucketing data (ie time series indices)? On 15 March 2015 at 20:39, wrote: > Hello, > > We have a 2 node elasticsearch cluster which is used b

Elasticsearch high heap usage

2015-03-15 Thread chris85lang
Hello, We have a 2 node elasticsearch cluster which is used by logstash to store log files. The current input is around 100 documents (logs) per second wit a size of around 50kb - 150kb. Compared to what i have read so far this is not a high amount but we experience already a high heap usage 70

Re: Aggregation / Sort and CircuitBreakingException

2015-03-15 Thread Lindsey Poole
Also, if I understand correctly, there are negative implications when sorting over a column that has been analyzed - in our case, to remove stop-words. Since the total cardinality of our sort field exceeds the heap available, we can't sort a single users documents when using stop word analysis

Re: Aggregation / Sort and CircuitBreakingException

2015-03-15 Thread Lindsey Poole
Well, we have a field that is supporting a backward compatibility use case. Clients are executing a partial match query on this field, so we used the keyword tokenizer instead of not_analyzed. Since this is supporting legacy functionality, the clients cannot be updated to change the expectation

Re: Aggregation / Sort and CircuitBreakingException

2015-03-15 Thread joergpra...@gmail.com
I mean, I do not understand what you mean by "I'm caught up on the advice to use doc_values where possible, but we have a use case where we do light analysis on a particular set of fields in our document" - what exactly prevents you from doc values? Jörg On Mon, Mar 16, 2015 at 12:41 AM, joergpra

Re: Aggregation / Sort and CircuitBreakingException

2015-03-15 Thread joergpra...@gmail.com
Have you considered doc values? http://www.elastic.co/guide/en/elasticsearch/guide/current/doc-values.html Jörg On Sun, Mar 15, 2015 at 11:11 PM, Lindsey Poole wrote: > Hey guys, > > I have a question about the mechanics of aggregation and sorting w.r.t. > the fielddata cache. I know this has

Re: PerThreadIDAndVersionLookup - thread safety

2015-03-15 Thread joergpra...@gmail.com
It is not thread safe because of the TermsEnum array, which can not be shared between threads. By not sharing, a thread can reuse the array, which avoids expensive reinitialization. The utility class was introduced at https://github.com/elastic/elasticsearch/issues/6212 and from what I understan

Aggregation / Sort and CircuitBreakingException

2015-03-15 Thread Lindsey Poole
Hey guys, I have a question about the mechanics of aggregation and sorting w.r.t. the fielddata cache. I know this has been covered in some detail previously, and I'm caught up on the advice to use doc_values where possible, but we have a use case where we do light analysis on a particular set

Re: What configuration is available to control MemoryMapDirectory

2015-03-15 Thread Lindsey Poole
Just to close this out - we disabled EC2's health checks and spent some time tuning the batch thread-pool size to prevent overrunning the cluster once the memory map cache size exceeds available physical memory. This was successful (we're restricted to a surprisingly small threadpool size of 3).

PerThreadIDAndVersionLookup - thread safety

2015-03-15 Thread Paweł Róg
Hi, Can anyone shortly describe why class PerThreadIDAndVersionLookup is not thread safe and what is needed to make it thread safe? I'm wondering if it is possible to keep only single instance of VersionLookup and make it not stick to a thread. I see waste of big chunk of memory in JVM only because

Re: java.io.IOException: Connection timed out error

2015-03-15 Thread Mark Walkom
I am guessing that is the client log, but what does your ES node show? On 15 March 2015 at 10:27, sunil patil wrote: > I using ElasticSearch 1.1.0 server. I have a Java Web Application that > connects to ElasticSearch using Transport Client, i am also using > elasticsearch 1.1.0 client. When the

Re: Problems setting up user authentication with Shield

2015-03-15 Thread Mark Walkom
There is a step in the longer version of getting started about setting ES_JAVA_OPTS="-Des.path.conf=/etc/elasticsearch" - http://www.elastic.co/guide/en/shield/current/getting-started.html - did you set that? On 15 March 2015 at 11:30, Sönke Liebau wrote: > Hi everybody, > > I just tried setting

Problems setting up user authentication with Shield

2015-03-15 Thread Sönke Liebau
Hi everybody, I just tried setting up Shield on my three node testcluster. I downloaded the license and shield plugins, ran syskeygen and distributed that to all cluster nodes. I then ran the following command on all nodes as the es user: ./esusers useradd es_admin -r admin -p foobar1234 After

Re: Embedded fields not searchable when addressed by name and type of same name exists?

2015-03-15 Thread Joel Potischman
Thanks, Vineeth. Much appreciated! I read your ticket and the associated issue and that definitely seems to be it. I think I found a workaround for my specific needs though: It seems that if instead of querying against the root of the index, I supply the list of types, it works as expected. Usi

Re: Joining Multiple Aggregration Queries into one query

2015-03-15 Thread Prashant singh
Adding Reference to it. https://gist.github.com/parasb/751baf0fca2a7682a8d1 On Sunday, March 15, 2015 at 11:32:20 PM UTC+5:30, Prashant singh wrote: > > Hi , > I have One query given below . > > --

Joining Multiple Aggregration Queries into one query

2015-03-15 Thread Prashant singh
Hi , I have One query given below . { "query": { "filtered": { "query": { "query_string": { "query": "*" }

java.io.IOException: Connection timed out error

2015-03-15 Thread sunil patil
I using ElasticSearch 1.1.0 server. I have a Java Web Application that connects to ElasticSearch using Transport Client, i am also using elasticsearch 1.1.0 client. When the application starts it is able to connect to elasticsearch server and all the searches work but after every couple of hour

Re: How to get existing index object, if index exists in php elastica

2015-03-15 Thread nitin birdi
My bad. It was so easy: $index = $client->getIndex($name); Had some problem in my code itself. Thanks anyways. On Sunday, March 15, 2015 at 3:30:01 PM UTC+5:30, nitin birdi wrote: > > Hi, > > I am new to elastic search. I am using php elastica client and facing a > problem: > If an index exists,

How to get existing index object, if index exists in php elastica

2015-03-15 Thread nitin birdi
Hi, I am new to elastic search. I am using php elastica client and facing a problem: If an index exists, I want to get the object of this existing index and not recreate it. How can this be done? client = new \Elastica\Client($arrServerConf, $callback); if ( $client->getIndex($name)->exists()

Re: Embedded fields not searchable when addressed by name and type of same name exists?

2015-03-15 Thread vineeth mohan
Hello Joel , This is a known issue. Internally each field is stored as typeName.fieldName format as type name is just an abstraction. That is one of the reason for this issue. You can find more information here - https://github.com/elastic/elasticsearch/issues/7411 Thanks Vineeth Moh