Hi Rinku For insert into hbase, it'd be better you depend on Hbase APIs for the same. I'm not sure on the APIS in hbase that would support bulk loading, you could find better solution on the hbase mailing list. There is a way to insert data from hive into hbase . You need to CREATE a hive managed TABLE in hive that uses a Hbase storage handler. The below url can help you with the details.
https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-Usage From the execption you posted , hive is not able to connect to hbase master and make the curresponding table creation in hbase. Is your hbase installation good? have to tried creating tables and loading data on to them? It could be some jar mismatches as well. Follow the first part of the url,basically in building your Storage Handler (hive-hbase) . $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.7.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.89.0-SNAPSHOT.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.1.jar -hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com Regards Bejoy.K.S ________________________________ From: "Garg, Rinku" <rinku.g...@fisglobal.com> To: "user@hive.apache.org" <user@hive.apache.org>; Bejoy Ks <bejoy...@yahoo.com> Sent: Thursday, March 1, 2012 11:19 AM Subject: RE: Hive | HBase Integration Hi Bejoy, We want to insert data into Hbase cluster. We are having data in CSV files. We want to insert large number of CSV’s files into hbase and then quering the hbase cluster. Please guide us on how to proceed. We tried to create the table in HIVE using following command. CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping"= ":key,cf1:val") TBLPROPERTIES ("hbase.table.name"= "xyz"); But we are getting the following exception: FAILED: Error in metadata: java.lang.IllegalArgumentException: Not a host:port pair: � 26156@mastermaster,60000,1330393836186 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask Please help. Thanks Rinku Garg From:Bejoy Ks [mailto:bejoy...@yahoo.com] Sent: 29 February 2012 17:51 To: user@hive.apache.org Subject: Re: Hive | HBase Integration Hi Rinku We want to use hadoop’s hdfs with hbase cluster Can you elaborate more on this? · Do we still need hive? You don't need Hive for Hbase to work. Hbase doesn't support SQL like queries, to support SQL like querying in Hbase we create a hive table that maps to the hbase table. The data is stored only on the hbase Region server and not brought to hdfs. Hbase and hive are primarily meant for two different purposes Hive - is a DWH system on top of hadoop, intended for batch processing. It supports SQL like commads and is mostly used for Analytic purposes . Hbase - is a low latency noSQL,distributed database in hadoop eco system. It is meant for point to point queries, frequent inserts, updates etc. · Can we directly use hbase with hadoop. Definitely, just set up a hadoop cluster and then install hbase master and region servers on the same. Are you looking st some specific operation here? · If we have to use hbase through hive then how can we proceed? AFAIK, the hive Hbase integration is meant to query data from Hbase alone. For inserts and updates in Hbase tables you need to use the Hbase APIs itself. Regards Bejoy.K.S ________________________________ From:"Garg, Rinku" <rinku.g...@fisglobal.com> To: "user@hive.apache.org" <user@hive.apache.org>; "bejoy...@yahoo.com" <bejoy...@yahoo.com> Sent: Wednesday, February 29, 2012 11:19 AM Subject: RE: Hive | HBase Integration Hi Bejoy, Thanks for the quick reply. Yes, we can create normal table in Hive, but here we are trying to create table in hbase to integrate it with HIVE. Here is the scenario. We want to use hadoop’s hdfs with hbase cluster. Earlier we have successful setup of hadoop-0.20.203.0 and hive-0.7.1. We can insert our csv’s into HDFS using hive and query it. But now we want to use HBASE cluster. So our queries are · Do we still need hive? · Can we directly use hbase with hadoop. · If we have to use hbase through hive then how can we proceed? Please help. Thanks Rinku Garg From:bejoy...@yahoo.com [mailto:bejoy...@yahoo.com] Sent: 29 February 2012 11:08 To: user@hive.apache.org Subject: Re: Hive | HBase Integration Hi Rinku Were you able to create a normal table within your hive without any issues? By Normal table I mean the one that has data dir in hdfs not in HBase. Regards Bejoy K S From handheld, Please excuse typos. ________________________________ From: "Garg, Rinku" <rinku.g...@fisglobal.com> Date: Wed, 29 Feb 2012 05:29:12 +0000 To: user@hive.apache.org<user@hive.apache.org>; Bejoy Ks<bejoy...@yahoo.com> ReplyTo: user@hive.apache.org Subject: RE: Hive | HBase Integration Hi , We tried the same also by issuing the below mentioned command but command does not executes completely and a new command line is shown with an asterisk (*), and table does not gets created. CREATE EXTERNAL TABLE hive_hbasetable_k(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "hivehbasek"); Please suggest. Thanks Rinku Garg From:Bejoy Ks [mailto:bejoy...@yahoo.com] Sent: 29 February 2012 10:33 To: user@hive.apache.org Subject: Re: Hive | HBase Integration Hi Keshav Make your hive table EXTERNAL, it should get things rolling. If you are mapping hive to a Hbase table, then as a mandatory requirement the hive table has to be EXTERNAL. Hope it helps. Regards Bejoy.K.S ________________________________ From:"Savant, Keshav" <keshav.c.sav...@fisglobal.com> To: "user@hive.apache.org" <user@hive.apache.org> Sent: Tuesday, February 28, 2012 5:58 PM Subject: Hive | HBase Integration Hi All, We did a successful setup of hadoop-0.20.203.0 and hive-0.7.1. In our next step we are eyeing HBase integration with Hive. As far as we understand from articles available on internet and apache site, we can use HBase instead of derby as a metastore of Hive, this gives us more flexibility while handling very large data. We are using hbase-0.92.0 to integrate it with Hive, till now HBase has been setup and we can create sample table on it and insert sample data in it, but we are not able to integrate it with Hive, because when we issue the command to create hive specific table on HBase (below in box) the command does not executes completely and a new command line is shown with an asterisk (*), and table does not gets created. CREATE TABLE hive_hbasetable_k(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "hivehbasek"); Please provide us some pointers (steps to follow) for doing this integration or what we are not doing correctly. Till now we got these below URLs to do this, any help is appreciated http://mevivs.wordpress.com/2010/11/24/hivehbase-integration/ https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration Kind regards, Keshav C Savant _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.