Thanks Costin for replying, That works for me but now I am facing with another issue.
data are not loading into elasticsearch. whenever I fires insertion query it shows following error. Query ID = hue_20140908052626_c15ea214-c656-47b1-9132-49d3c407b8f8 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_1410175571220_0003, Tracking URL = http://sandbox.hortonworks.com:8088/proxy/application_1410175571220_0003/ Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1410175571220_0003 Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0 2014-09-08 05:26:57,048 Stage-0 map = 0%, reduce = 0% 2014-09-08 05:27:29,019 Stage-0 map = 100%, reduce = 0% Ended Job = job_1410175571220_0003 with errors FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec , log_context:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25, handle:QueryHandle(id:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25, log_context:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25), SQLState: ) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:356) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Hive Queries:-Create table in hive and load data from zip file:-CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING, req STRING, res INT, bytes INT, phpmem INT, agent STRING)ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE apachelog; Load data into easlticsearch query:-CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip STRING, request STRING, response INT, agent STRING)STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = 'http://192.168.186.1','es.port' = '9200'); INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent FROM apachelog s; please suggest me what should i do to remove this error. thanks in advance On Friday, September 5, 2014 6:13:58 PM UTC+5:30, Costin Leau wrote: > > It looks like there's a classpath issue (notice the HiveUtils error > there). Most likely because you have two versions > of es-hadoop in your classpath (2.1.0.Beta1 and 1.3.0.M1). > > Use only one - I suggest 2.1.0.Beta1. > > Cheers, > > On 9/5/14 3:39 PM, Mohit Kumar Yadav wrote: > > hi folks, > > I facing following error while load data into elasticsearch using hive > query. > > ERROR:- > > 14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=Driver.execute > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL > TABLE eslogs (time STRING, extension STRING, > > clientip STRING, request STRING, response INT, agent STRING) > > STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' > > TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = ' > http://192.168.186.1','es.port' = '9200') > > 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service > address: > > http://sandbox.hortonworks.com:8188/ws/v1/timeline/ > > 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook > > 14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG > method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook > > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG > method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook > > start=1409389504583 end=1409389504583 duration=0 > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit > start=1409389502353 end=1409389504583 duration=2230 > > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=runTasks > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG method=task.DDL.Stage-0 > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied > org.elasticsearch.hadoop.hive.EsSerDe for table eslogs > > 14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError: > org/elasticsearch/hadoop/hive/HiveUtils > > at > org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80) > > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339) > > > > at > org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283) > > > > at > org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276) > > at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626) > > at > org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593) > > at > org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194) > > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) > > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) > > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) > > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504) > > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271) > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603) > > > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:356) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603) > > > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service > address: > > http://sandbox.hortonworks.com:8188/ws/v1/timeline/ > > 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook > > 14/08/30 02:05:04 INFO log.PerfLogger: <PERFLOG > method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook > > from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG > method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook > > start=1409389504698 end=1409389504698 duration=0 > from=org.apache.hadoop.hive.ql.Driver> > > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. > org/elasticsearch/hadoop/hive/HiveUtils > > 14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.DDLTask. > > org/elasticsearch/hadoop/hive/HiveUtils > > 14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=Driver.execute > start=1409389504462 end=1409389504700 > > duration=238 from=org.apache.hadoop.hive.ql.Driver> > > 14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while > processing query > > BeeswaxException(message:Driver returned: 1. Errors: OK > > converting to local hdfs:// > sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar > > < > http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar> > > > > Added > /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar > > to class path > > Added resource: > /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar > > > > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. > org/elasticsearch/hadoop/hive/HiveUtils > > , log_context:d0960413-bfb9-4bdf-b615-26ad7821587b, > handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b, > > log_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: ) > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603) > > > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:356) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537) > > > > at > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603) > > > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > My setup:- > > 1. Hortonworks Sandbox VM (running on vmware player):- > Hortonworks_Sandbox_2.1 > > 2. elasticsearch up & running :- Elasticsearch version 1.3.1 > > 3. elasticsearch-hadoop jars :- > elasticsearch-hadoop-2.1.0.Beta1,elasticsearch-hadoop-1.3.0.M1 > > > > I am following this url :- > > > https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md > > > > > Hive Queries:- > > Create table in hive and load data from zip file:- > > > > CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING, > req STRING, res INT, bytes INT, phpmem INT, > > agent STRING) > > ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; > > LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE > apachelog; > > > > Load data into easlticsearch query:- > > > > CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip > STRING, request STRING, response INT, agent STRING) > > STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' > > TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = ' > http://192.168.186.1','es.port' = '9200'); > > > > INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, > s.agent FROM apachelog s; > > > > Kindly suggest me where I am doing wrong and what is the meaning of this > error and why it is coming? > > > > Regrads > > Mohit Kumar Yadav > > (MCA/BBA) > > > > -- > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to > > elasticsearc...@googlegroups.com <javascript:> <mailto: > elasticsearch+unsubscr...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com > > > < > https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > > For more options, visit https://groups.google.com/d/optout. > > -- > Costin > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/440d87a5-47f4-4ff9-944a-dd01f5b367eb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
