Re: elasticsearch query on array of documents

2014-08-14 Thread David Pilato
This does not look as correct JSON. Could you gist a full SENSE script? It will help to understand what you did so far and where is your issue. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 14 août 2014 à 07:15, K.Samanth Kumar Reddy samanthkuma...@gmail.com a écrit :

Re: Can't use the Java API with data-less node

2014-08-14 Thread Manthan Shah
Was that a really problem for above Error? I doubt this could be a reason. Facing similar issue with 1.3.1 version On Monday, February 17, 2014 4:08:21 PM UTC+5:30, Dario Rossi wrote: There's a little mistake settings.put(node.data, true); should be

Re: Get Shard Info From Cluster/Nodes/Index

2014-08-14 Thread Sandeep Ramesh Khanzode
Hi Mark, Thanks, I will check it out. I hope there is a corresponding Java API for this. Sandeep On Thu, Aug 14, 2014 at 3:47 AM, Mark Walkom ma...@campaignmonitor.com wrote: The _cat/shards API will tell you this -

accessing field data faster in script

2014-08-14 Thread avacados
How to access field data faster from native (java) script ??? should i enable 'doc values'? I am already using doc().getField() and casting to long. It is date field type. But whenever, my argument to script changes, it has poor performance for search query. Subsequent call with same argument

Re: Can't use the Java API with data-less node

2014-08-14 Thread David Pilato
I don't think but yes you should set your node as a client node .client(true) and don't add too many settings. Wondering if the port 9300 is opened on your nodes? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 14 août 2014 à 08:08, Manthan Shah mssh...@gmail.com a écrit

Re: Can't use the Java API with data-less node

2014-08-14 Thread joergpra...@gmail.com
I doubt that local(true) and unicast can work seamlessly together - either the node should use the network (local=false, which is the default), or not. Also, before search, a cluster health check would be appropriate, and a cluster nodes info request for the number of data nodes connected. This

Re: Index size on node VS heap size

2014-08-14 Thread Adam Porat
Thanks Mark. בתאריך יום שלישי, 12 באוגוסט 2014 09:20:43 UTC+3, מאת Mark Walkom: No it can be more, it depends on what sort of queries you are doing and what data structures/types you are indexing. Best bet is to keep throwing data at the index until the server can't take it, then you know

Re: Automatic partial shutdown of cluster

2014-08-14 Thread Krishnan
Thank you so much for the suggestion, Walkom. Now how do I find out what is shutting down my nodes? I checked /var/logs, did not find any thing suspicious. is this related to out of memory system states. Regards, Krishna On Thursday, August 14, 2014 10:03:10 AM UTC+5:30, Mark Walkom wrote:

Limiting query results

2014-08-14 Thread julie dabbs
Does anyone know how to execute a query with aggregations and once the query is complete only show the top 100 hits in a browser. If I use filter limit I am not getting the best results. If I use still am displaying a ridiculous total hits back to the user. I am running on version 1.3. Can

Re: Limiting query results

2014-08-14 Thread Isabel Drost-Fromm
On Thu, Aug 14, 2014 at 10:31 AM, julie dabbs julie.e.da...@gmail.com wrote: Does anyone know how to execute a query with aggregations and once the query is complete only show the top 100 hits in a browser. If I use filter limit I am not getting the best results. If I use still am displaying

Error creating TransportClient for dedicated master-data ES cluster (1.1)

2014-08-14 Thread sagarl
Hi, We have Dedicated Master (3) , Data (24) and Search (3) Nodes ES 1.1 version deployment. We use Transport Client to get NodeStatsResponse so that all the nodes can report their individual statistics. Cluster is in Green state but out of all these nodes, 1 Master node, 1 Search node and

Re: bulk thread pool rejections

2014-08-14 Thread Robert Gardam
Did you resolve this issue? I was seeing the exact thing in my setup. I also have my bulk messages set to 5k in logstash. Originally I had set the thread pool to unlimited but this apparently causes some strange issues with stability. On Tuesday, April 8, 2014 5:00:32 PM UTC+2, shift wrote:

Re: Limiting query results

2014-08-14 Thread julie dabbs
No, I have a query with aggregations that can yield thousands of results back to the user. I want to only show the top say 200 hits and not this huge number but keeping the integrity of the data. i.e showing the results with the highest scores On Thursday, August 14, 2014 10:04:25 AM UTC+1,

Re: Limiting query results

2014-08-14 Thread Ashish Mishra
Use the size parameter. e.g. $ curl -XGET 'http://localhost:9200/twitter/tweet/_search' -d '{ size: 200, aggregations: { my_agg: { terms: { field: text } } } } ' On Thursday, August 14, 2014 2:59:52 AM UTC-7, julie dabbs wrote: No, I have a query with

Elasticssearch cluster installation on windows 2012

2014-08-14 Thread ozlevka
Hi All! We need install elasticsearch client on windows 2012 machines. For start we plan next machine configuration: 2 core VM with 8G memory. We will have 100k index queries per day and about 100 search queries per second. About 3-5 million documents. No complicate index very flat data about

Re: Elasticssearch cluster installation on windows 2012

2014-08-14 Thread Mark Walkom
It should be ok, but if this is production then you should really have an odd number of nodes, ie 3+. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 14 August 2014 21:04, ozlevka ozle...@gmail.com wrote: Hi All!

Jaro-Winkler Query

2014-08-14 Thread Adrian C
Hi, Is it possible to use jaro-winker rather than levenshtein distance for fuzzy queries? Any ideas how one could go about extending ES to enable this. I have looked at using function score like below however this does not work for me seeing as I cannot combine this with another query that

Newbie Query : Can ES be used for searching spatial data

2014-08-14 Thread Anuj Mehta
Hi I am trying to implement a basic Geocoder where-in for any given address I try to search for street, city, etc information from my database. I had following queries 1. Can I use Elastic Search as a DB to store and quickly retrieve street, city, state, latitude, longitude etc info?

Pattern_capture filter emits a token that is not matched with pattern also.

2014-08-14 Thread Raj
I have a case where I have to extract domain part from emails that are found in a text. I used uax_url_email tokenizer to create emails as a single. And I have a pattern_capture filter which will emit @(.+) pattern string. But uax_url_email also return words also which is not an email and

Re: Using facets/aggretagions on parent document, queried by TopChildren

2014-08-14 Thread Adam Porat
I will answer myself: The facets are counted in relation to the total_hits in the response. This is true just as well when using TopChildren. בתאריך יום שלישי, 5 באוגוסט 2014 11:11:02 UTC+3, מאת Adam Porat: Hi, The TopChildren query works with an estimated hit size, and the TotalHits

Re: Split child query to query and filter?

2014-08-14 Thread Adam Porat
I will answer myself: Any queried type (the parent or any one of its children), if there should be a filter on it, should be queried using a *filtered query* clause. Any filtered field should come under the filtered-query *filter *clause, and any analyzed or score-affecting field should come

Relations beetween different types

2014-08-14 Thread ali koyuncu
I have an index named DataIndex. It has two types: posts and users. The posts have an user which has small info like just id, name but it has more. And I regulary update users some fields like followers count,age etc . So I must do below jobs: - When I run on posts type, I want to join users.

Difference between transport and http modules?

2014-08-14 Thread John Smith
Just want to clear some thing up... The transport module is what is called the tcp transport module that runs on port 9300? The transport module is what handles all major networking function of ES. I.e: any network functions like http, zen discovery sit on top of transport module? The http

Help with OOM and configuration

2014-08-14 Thread savicm
Hi, I would like to ask for hints (and help, if possible) on improving the functioning of a single ES node I'm responsible of maintaining. I am fairly new to ES, and trying to get a grasp of all the concepts. What I have set up is an AWS machine with 4 cores, 15GB RAM (m3.xlarge), two storage

Re: bulk thread pool rejections

2014-08-14 Thread Malia, Brian
I found out that the rejections on ES are retried by logstash after a short relay. Increasing the queue by too much costs more memory in ES, which takes away from merges, searches, etc.. I increased threadpool.bulk.queue_size from 50 to 100, I see no lost messages due to the rejections.

Re: Difference between transport and http modules?

2014-08-14 Thread joergpra...@gmail.com
Yes. Elasticsearch Java Client API is using TCP/IP transport of course - there is only one Java API for clients and cluster nodes, one size fits all. It is possible to write Java Clients that make use of HTTP client implementations to connect to port 9200 but that is not part of Elasticsearch.

Re: Limiting query results

2014-08-14 Thread julie dabbs
Unfortunately this does not change the total of results returned On Thursday, August 14, 2014 9:31:12 AM UTC+1, julie dabbs wrote: Does anyone know how to execute a query with aggregations and once the query is complete only show the top 100 hits in a browser. If I use filter limit I am not

Re: New version of Kibana in the works?

2014-08-14 Thread James Green
If Kibana 4 is not using the existing code base then perhaps someone needs to reject a whole pile of pull requests as they are a lot waiting. Not quite the way I'd expect an open source project to progress either. On 13 August 2014 16:33, Antonio Augusto Santos aaugusto.san...@gmail.com wrote:

Re: Limiting query results

2014-08-14 Thread Isabel Drost-Fromm
I think an example (trimmed to the relevant parts) of the query you are sending, the results you get and the results you expect would help ... Isabel ​ -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop

Re: Limiting query results

2014-08-14 Thread Ramy
Try something like that... aggs: { agg_name: { terms: { field: my_field, size: 100 } } } Am Donnerstag, 14. August 2014 16:40:11 UTC+2 schrieb julie dabbs: Unfortunately this does not change the total of results returned On Thursday, August 14, 2014

Re: Relations beetween different types

2014-08-14 Thread Ramy
You can try to save the user _id in each post!!! maybe it solves your problem Am Donnerstag, 14. August 2014 15:34:39 UTC+2 schrieb ali koyuncu: I have an index named DataIndex. It has two types: posts and users. The posts have an user which has small info like just id, name but it has

Re: Relations beetween different types

2014-08-14 Thread ali koyuncu
Yes, I already have kept user's fields as data's fields in datas. On Thursday, August 14, 2014 6:26:20 PM UTC+3, Ramy wrote: You can try to save the user _id in each post!!! maybe it solves your problem Am Donnerstag, 14. August 2014 15:34:39 UTC+2 schrieb ali koyuncu: I have an index

Re: Filtering across indices

2014-08-14 Thread Owain Braddick
If anyone else comes across this, there is already a similar issue raised at https://github.com/elasticsearch/elasticsearch/issues/2801 On Monday, August 4, 2014 4:43:07 PM UTC+1, Owain Braddick wrote: Hi, I am trying to perform a filtered aggregations across multiple indices and types and

Re: Failed to perform snapshot (index files)]; nested: FileNotFoundException

2014-08-14 Thread Aleh Aleshka
Well the backup is failing again on the index which was fixed after upgrade. Any idea how to debug this? On Tuesday, August 12, 2014 7:39:42 PM UTC+3, Aleh Aleshka wrote: I upgraded to 1.3.1 and one of the indexes seems to backup ok now, though the other one is still giving the error. On

Re: Writing an article - topics to cover?

2014-08-14 Thread Nikolas Everett
On Thu, Aug 14, 2014 at 11:40 AM, Christopher Ambler const.dogbe...@gmail.com wrote: I've been tasked to write an article (that will be public-facing) on my experience setting up ElasticSearch as part of the project I'm working on at GoDaddy. I'd like to solicit input on any topics that I

Re: New version of Kibana in the works?

2014-08-14 Thread Rashid Khan
Yes there is a new version in the works. Elasticsearch has grown considerably in the last 18 months. With Elasticsearch 1.0, and the introduction of aggregations, we quickly realized we needed a better platform for building more powerful interfaces. So for the last few months we've been head

Re: Elasticssearch cluster installation on windows 2012

2014-08-14 Thread Andrew Gui
Do you know how to disable swap on Windows? disable file paging? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To

[ANNOUNCE] kibana-rack 0.1.4 released

2014-08-14 Thread Tony Burns
I just released version 0.1.4 of kibana-rack (https://github.com/tabolario/kibana-rack)! This release has a set of fixes and refinements to the Elasticsearch proxy, thanks to @dcarley. Notably, proxying to the internal Kibana index is now working. If you use Elasticsearch in your Rack/Rails

Elasticsearch-Hadoop repository plugin Cloudera Hadoop 2.0.0-cdh4.6.0

2014-08-14 Thread Mateusz Kaczynski
I'm trying to get es-hadoop repository plugin working on our hadoop 2.0.0-cdh4.6.0 distribution and it seems like I'm quite lost. I installed plugin's -hadoop2 version on the machines on our hadoop cluster (which also run our stage elasticsearch nodes). When attempting to create a repository

Elasticsearch cluster on AWS. Article.

2014-08-14 Thread Pavel P
Hi everyone, Below you can find one big article, summing up all my experience of building the cluster on AWS. When I started I had no information at all, but I found the needed pieces in different places, including this user group. With your help I succeeded, and want to share the knowledge,

Re: New version of Kibana in the works?

2014-08-14 Thread Antonio Augusto Santos
Thanks a lot for the update Rashid. I know it might be too much but... any dates we can mark on out calendar? ;) On Thursday, August 14, 2014 1:04:24 PM UTC-3, Rashid Khan wrote: Yes there is a new version in the works. Elasticsearch has grown considerably in the last 18 months. With

[ANN] Elasticsearch Hadoop 2.0.1 and 2.1.Beta with Spark support

2014-08-14 Thread Costin Leau
Hi everyone, Elasticsearch Hadoop 2.0.1 and 2.1 Beta1, featuring native Apache Spark integration, have been released. You can read all about them here [1]. Feedback is welcome! Cheers, [1] http://www.elasticsearch.org/blog/es-hadoop-2-0-1-and-2-1-beta1/ -- Costin -- You received this

Re: Can't use the Java API with data-less node

2014-08-14 Thread eunever32
Is the clusterName set correctly? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web

Re: IllegalArgumentException followed by SearchContextMissingException

2014-08-14 Thread joergpra...@gmail.com
The first error is from a HTTP connection hat did not send a HTTP command correctly to the cluster node (i.e. without HTTP version). Also possible is a connection attempt of a non-HTTP client, or a misconfiguration. The second error is probably from a search request using scan/scroll, but the

ElasticSearch products + product varianst + custom prices for clients

2014-08-14 Thread kisi
Hi! I'm trying to store and search for products and product variants in Elasticsearch. I have a set of products (about 4000) and product variants (all together about 2) which are related to the products. A product could be a hole-puncher and it variants are a hole-puncher in green, a

Re: hits vs getTotalHits

2014-08-14 Thread David Pilato
IIRC, totalHits() was the first form. But some frameworks needed accessors using get/set. So get was added. I would use get/set. Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 14 août 2014 à 20:46, AsyncAwait vadi...@gmail.com a écrit : Is this is for backward compatibility?

Re: Kibana not showing all events

2014-08-14 Thread cablenightmare
I figured it out. I had modified the index pattern in the /opt/kibana/app/dashboards/logstash.json file to use hourly indexes, but forgot to modify the interval. I set the interval hour, the problem was resolved. doh On Wednesday, August 13, 2014 12:54:04 PM UTC-7, cableni...@gmail.com wrote:

Re: Kibana not showing all events

2014-08-14 Thread cablenightmare
I figured it out. I had modified the index pattern in the /opt/kibana/app/dashboards/ logstash.json file to use hourly indexes, but forgot to modify the interval. I set the interval hour, the problem was resolved. doh -- You received this message because you are subscribed to the Google Groups

Re: Can plugin be written for TCP transport?

2014-08-14 Thread John Smith
Yeah I tried the foundit plugin and just striped away all their code and it's just a pass through! But would def need to figure out the inards. We have web app in dmz and then ES at lower level. But again it's about protecting from the inside. When having a bunch of support staff that are

Re: Elasticsearch cluster on AWS. Article.

2014-08-14 Thread Mark Walkom
Nice and indepth, thanks for posting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 August 2014 03:13, Pavel P pa...@kredito.de wrote: Hi everyone, Below you can find one big article, summing up all my

Re: Elasticssearch cluster installation on windows 2012

2014-08-14 Thread Mark Walkom
No, but there appears to be a lot of discussions around it on various other places on the web. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 August 2014 02:17, Andrew Gui yang.gu...@gmail.com wrote: Do you

Optimize has_child query / filter

2014-08-14 Thread kisi
Hi, i'm currently thinking if i could write a has_child filter differently to performe a faster search. I want all products (parents) with the term printer and which have set the field Client3.Type to K. Currently i'm executing this query. But i think this way elasticsearch is first

Re: Newbie Query : Can ES be used for searching spatial data

2014-08-14 Thread Mark Walkom
1 - It sure does, have a search through the docs for geo http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index.html 2 - When you send data to ES it indexes it and is then available for search in near-realtime. 3 - That's what it's built for :) Regards, Mark Walkom

Aggregation

2014-08-14 Thread Yuheng Du
Does perl module of elasticsearch allows Aggregation syntax? I run a few tests but it failed. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to

how i change the default port 9200 for plugin head? thanks

2014-08-14 Thread huangshanjay
how i change the default port 9200 for plugin head? thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view

Re: how i change the default port 9200 for plugin head? thanks

2014-08-14 Thread Mark Walkom
You can't change it for head alone, you'd have to change it for the entire instance. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 August 2014 10:45, huangshan...@gmail.com wrote: how i change the default

Re: Aggregation

2014-08-14 Thread 饶琛琳
No problem: #!/usr/bin/perl use 5.010; use Data::Dumper; use Search::Elasticsearch; my $e = Search::Elasticsearch-new( nodes = [ '10.13.57.35:9200', '10.13.57.36:9200' ] ); my $r = $e-search( index = 'logstash-2014.08.15', body = { aggs =

Re: Timeout notification from cluster service

2014-08-14 Thread Krishnan
Please check the status of indexes when you get these log messages. I found these log messages appear in when index status in not green. You can use following query to check : curl -s –XGET http://10.72.12.22:9200/_cat/indices?pretty=true | grep -v green | wc -l On Thursday, August 14, 2014