Which version of Hive are you using.
On Tue, Apr 8, 2014 at 8:05 PM, Xiao Meng <[email protected]> wrote: > Hi, > > We finally get the hive storage engine up (the previous problem is mainly > due to some environmental variable settings) > > However, it has problem when we try to query a specific hive table. > > For example, we have the following table in the INFROMATION_SCHEMA.TABLES: > > | DRILL | hivestg | store_sales_fact_denorm | TABLE | > > When we query: > > > select * from hivestg.`store_sales_fact_denorm` limit 1; > > > It shows the following exception: > > > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while > running query.[error_id: "8bcf902e-a0c6-461c-94db-916d2188d1b2" > endpoint { > address: <my host name> > user_port: 31016 > control_port: 31017 > data_port: 31018 > } > error_type: 0 > message: "Failure while parsing sql. < OptimizerException:[ Failure while > attempting to retrieve storage engine. ] < IOException:[ > org.apache.drill.common.exceptions.ExecutionSetupException: > org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot > communicate with client version 4 ] < ExecutionSetupException:[ > org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot > communicate with client version 4 ] < RemoteException:[ Server IPC version > 9 cannot communicate with client version 4 ]" > ] > > It seems a version problem. Any suggestion on this? > > > BTW, we use the following storage-engines settings: > > { > ... > hivestg : { > type:"hive", > config : > { > "hive.metastore.uris" : "thrift://<hostname>:9083", > "hive.metastore.sasl.enabled" : "false" > } > } > } > } > > > Thanks, > > Xiao > > -----Original Message----- > From: George Chow [mailto:[email protected]] > Sent: March-27-14 5:12 PM > To: [email protected] > Subject: Re: Setting up Hive... > > Hi Venki, > > Thanks for the tip. I think I may be confused about my setup. > > I have a remote metastore running alongside my HiveServer1. Thus, my > HiveServer1's hive-site.xml has the line > ConnectionURL=jdbc:mysql://localhost:3306/metastore. > > I have tried configuring my Drillbit by pointing hive.metastore.uris to > this same URL (substituting the IP for localhost). But I'm not seeing > anything. > > Looking thru the various logs doesn't turn up anything. > > I have already tried starting up the metastore in addition to the > HiveServer1 to no avail. > > What am I doing wrong? > > George > > > > On Tue, Mar 25, 2014 at 7:07 PM, Venki Korukanti > <[email protected]>wrote: > > > On Tue, Mar 25, 2014 at 6:56 PM, George Chow <[email protected]> > wrote: > > > > > Hi Venki, > > > > > > I made a mistake initially and thought I was embedded. But I'm > > > actually > > in > > > scenario 1 (remote metastore). > > > > > > > I am setting hive.metastore.uris to my HiveServer's address and > > > port. Is that correct? > > > > > hive.metastore.uris should point to metastore server which is > > different from HiveServer1/2. Drill uses Metastore thrift interface to > > fetch the table/db info. From your first email, it looks like you are > > embedding the metastore within HiveServer1. You can copy the following > > settings from you current hiveserver's hive-site.xml to > > storage-engines.json to embed metastore in Drillbit or start the > > metastore as separate service using 'hive --service metastore' and point > hive.metastore.uris to the address. > > > > "javax.jdo.option.ConnectionURL" : > > "jdbc:mysql://hostname/hivedrill?createDatabaseIfNotExist=true", > > > > "javax.jdo.option.ConnectionDriverName" : "com.mysql.jdbc.Driver", > > > > > > "javax.jdo.option.ConnectionUserName" : "user", > > > > "javax.jdo.option.ConnectionPassword" : "passwd" > > > > > > Thanks > > > > Venki > > > > > > > > > > > George > > > > > > > > > On Tue, Mar 25, 2014 at 6:48 PM, Venki Korukanti > > > <[email protected]>wrote: > > > > > > > Hi George, > > > > > > > > Are you embedding the metastore in drillbit > > > > (hive.metastore.uris="") or drillbit is connecting to remote > > > > metastore (hive.metastore.uris=valid address)? > > > > > > > > If you are embedding the metastore, make sure you have the > > > > appropriate driver (if MySql backend then mysql jdbc driver) in > > > > classpath or > > > drill/lib > > > > folder. > > > > > > > > Thanks > > > > Venki > > > > > > > > > > > > On Tue, Mar 25, 2014 at 6:30 PM, George Chow <[email protected]> > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Thanks Tim for the tip. It helps... so I copied the settings > > > > > from my hive-site.xml over to storage-engines.json. > > > > > > > > > > I restart my drillbit but I still don't see the contents from my > > > > > Hive metastore. The log file > > > > > (/var/log/drill/node11/drillbits.out) doesn't contain anything > about the inclusion of Hive. > > > > > > > > > > (I'm assuming that SELECT * FROM INFORMATION_SCHEMA.TABLES will > > return > > > > more > > > > > than the usual 5 tables (VIEW/COLUMNS/TABLES/CATALOGS/SCHEMATA.) > > > > > > > > > > George > > > > > > > > > > > > > > > On Tue, Mar 25, 2014 at 6:12 PM, Timothy Chen > > > > > <[email protected]> > > > wrote: > > > > > > > > > > > Hi George, > > > > > > > > > > > > I believe we have a wiki section on setting up hive + drill : > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/DRILL/Hive+Storage+Engin > > > e > > > > > > > > > > > > I haven't personally tried it yet but see if this helps. > > > > > > > > > > > > Tim > > > > > > > > > > > > > > > > > > On Tue, Mar 25, 2014 at 6:10 PM, George Chow > > > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > So I'm trying to setup a drillbit to reach across to my Hive > > > > > metastore. I > > > > > > > dug into JIRA and see that DRILL-354/357 are both resolved. > > > > > > > I've > > > also > > > > > > > looked into my drillbit and found > .../conf/storage-engines.json. > > > > > > > > > > > > > > Inside, I see an entry for Hive that's commented out. > > > > > > > However, it > > > is > > > > > for > > > > > > an > > > > > > > instance Hive with a Derby metastore. My instance is a > > HiveServer1 > > > > > with a > > > > > > > MySQL metastore. So both javax.jdo.option.ConnectionURL and > > > > > > > hive.metastore.sasl.enabled are clear to me. But what about > > > > > > > the > > > > > > remaining: > > > > > > > > > > > > > > hive.metastore.uris > > > > > > > hive.metastore.warehouse.dir > > > > > > > fs.default.name > > > > > > > > > > > > > > George > > > > > > > > > > > > > > -- > > > > > > > -- > > > > > > > "Not everything that can be counted counts, and not > > > > > > > everything > > that > > > > > > counts > > > > > > > can be counted." Albert Einstein > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- > > > > > "Not everything that can be counted counts, and not everything > > > > > that > > > > counts > > > > > can be counted." Albert Einstein > > > > > > > > > > > > > > > > > > > > > -- > > > -- > > > "Not everything that can be counted counts, and not everything that > > counts > > > can be counted." Albert Einstein > > > > > > > > > -- > -- > "Not everything that can be counted counts, and not everything that counts > can be counted." Albert Einstein >
