Writing data to elasticsearch from hive table

2015-04-04 Thread ozlevka
Hello! I have problem with writing data to elasticsearch as described in: http://www.elastic.co/guide/en/elasticsearch/hadoop/2.0/hive.html#_writing_data_to_elasticsearch_2 I use cloudera-quickstart-vm-5.3.0-0-vmware to test how working with eslasticsearch in hadoop. The queries from hive to elas

Understanding Elasticsearch-Hadoop

2015-04-04 Thread Bharvi Dixit
Hi, Even after going through so many resources and reading about es-hadoop i am unable to clarify some of my doubts like: How to run elasticsearch data nodes on your hadoop data nodes?? Can i install an elasticsearch cluster and store indexes on hadoop HDFS?? if yes then how?? Will i have to ke

Re: Understanding Elasticsearch-Hadoop

2015-04-04 Thread Costin Leau
Hi, Hadoop means a lot of things as it has a lot of components. I'm sorry to hear the resources you read don't give you enough answers. The 'definition' of Elasticsearch Hadoop is given in the documentation preface [1] which I quote below: " Elasticsearch for Apache Hadoop is an ‘umbrella’ p

red shard status - why please?

2015-04-04 Thread Dan Langille
I'm seeing this: { "cluster_name" : "elasticsearch", "status" : "red", "timed_out" : false, "number_of_nodes" : 4, "number_of_data_nodes" : 2, "active_primary_shards" : 4, "active_shards" : 8, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 2, "indic

Re: red shard status - why please?

2015-04-04 Thread joergpra...@gmail.com
1.4.2 was released last December, so I doubt you have created it in July or August. Jörg On Sat, Apr 4, 2015 at 11:10 PM, Dan Langille wrote: > I'm seeing this: > > { > "cluster_name" : "elasticsearch", > "status" : "red", > "timed_out" : false, > "number_of_nodes" : 4, > "number_of_d

Re: red shard status - why please?

2015-04-04 Thread Dan Langille
On Saturday, April 4, 2015 at 5:24:47 PM UTC-4, Jörg Prante wrote: > > 1.4.2 was released last December, so I doubt you have created it in July > or August. > Sorry, is that what you thought I was claiming? > Background: I created the first node some time ago (July or August). I didn't > cr

Re: red shard status - why please?

2015-04-04 Thread joergpra...@gmail.com
Please check the logs, you should see error messages. Jörg On Sun, Apr 5, 2015 at 12:03 AM, Dan Langille wrote: > > > On Saturday, April 4, 2015 at 5:24:47 PM UTC-4, Jörg Prante wrote: >> >> 1.4.2 was released last December, so I doubt you have created it in July >> or August. >> > > Sorry, is

chunk-wise incremental data feed into ES jdbc feeder/river

2015-04-04 Thread Jonathan Zhang
does ES jdbc river/feeder have a mechanism to feed data from database into ES via jdbc incrementally in a stepwise function? I have billions of records to be imported into ES from mysql and a select * from my table would take a long time and be interrupted due to db timeout. I need to do select

Re: Getting XML into ES efficiently

2015-04-04 Thread Norberto Meijome
Hi, My gut feel is don't add this to the ES setup itself. Horses for courses - have your script (Python +1) running somewhere taking care of the processing, dealing with issues on the ftp side , etc. Let ES do its thing...specially if the XML parsing will take so much memory and you need external s

Re: CAT API

2015-04-04 Thread Norberto Meijome
Replica =1 means you have 1 replica of the master shards - I.e. you have 2 copies of the data in total. On 02/04/2015 2:06 pm, "Nishad Karekar" wrote: > I am unable to understand the results from the CAT API > > curl 'http://hdpdncwy0001.global.shareddev.acxiom.net:9200/_cat/indices?v' > > health