This is Hive 0.12 now. George
On Tue, Apr 8, 2014 at 10:21 PM, Neeraja Rentachintala < [email protected]> wrote: > Which version of Hive are you using. > > > On Tue, Apr 8, 2014 at 8:05 PM, Xiao Meng <[email protected]> wrote: > >> Hi, >> >> We finally get the hive storage engine up (the previous problem is mainly >> due to some environmental variable settings) >> >> However, it has problem when we try to query a specific hive table. >> >> For example, we have the following table in the INFROMATION_SCHEMA.TABLES: >> >> | DRILL | hivestg | store_sales_fact_denorm | TABLE | >> >> When we query: >> >> > select * from hivestg.`store_sales_fact_denorm` limit 1; >> >> >> It shows the following exception: >> >> >> Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure >> while running query.[error_id: "8bcf902e-a0c6-461c-94db-916d2188d1b2" >> endpoint { >> address: <my host name> >> user_port: 31016 >> control_port: 31017 >> data_port: 31018 >> } >> error_type: 0 >> message: "Failure while parsing sql. < OptimizerException:[ Failure while >> attempting to retrieve storage engine. ] < IOException:[ >> org.apache.drill.common.exceptions.ExecutionSetupException: >> org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot >> communicate with client version 4 ] < ExecutionSetupException:[ >> org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot >> communicate with client version 4 ] < RemoteException:[ Server IPC version >> 9 cannot communicate with client version 4 ]" >> ] >> >> It seems a version problem. Any suggestion on this? >> >> >> BTW, we use the following storage-engines settings: >> >> { >> ... >> hivestg : { >> type:"hive", >> config : >> { >> "hive.metastore.uris" : "thrift://<hostname>:9083", >> "hive.metastore.sasl.enabled" : "false" >> } >> } >> } >> } >> >> >> Thanks, >> >> Xiao >> >> -----Original Message----- >> From: George Chow [mailto:[email protected]] >> Sent: March-27-14 5:12 PM >> To: [email protected] >> Subject: Re: Setting up Hive... >> >> Hi Venki, >> >> Thanks for the tip. I think I may be confused about my setup. >> >> I have a remote metastore running alongside my HiveServer1. Thus, my >> HiveServer1's hive-site.xml has the line >> ConnectionURL=jdbc:mysql://localhost:3306/metastore. >> >> I have tried configuring my Drillbit by pointing hive.metastore.uris to >> this same URL (substituting the IP for localhost). But I'm not seeing >> anything. >> >> Looking thru the various logs doesn't turn up anything. >> >> I have already tried starting up the metastore in addition to the >> HiveServer1 to no avail. >> >> What am I doing wrong? >> >> George >> >> >> >> On Tue, Mar 25, 2014 at 7:07 PM, Venki Korukanti >> <[email protected]>wrote: >> >> > On Tue, Mar 25, 2014 at 6:56 PM, George Chow <[email protected]> >> wrote: >> > >> > > Hi Venki, >> > > >> > > I made a mistake initially and thought I was embedded. But I'm >> > > actually >> > in >> > > scenario 1 (remote metastore). >> > >> > >> > > I am setting hive.metastore.uris to my HiveServer's address and >> > > port. Is that correct? >> > > >> > hive.metastore.uris should point to metastore server which is >> > different from HiveServer1/2. Drill uses Metastore thrift interface to >> > fetch the table/db info. From your first email, it looks like you are >> > embedding the metastore within HiveServer1. You can copy the following >> > settings from you current hiveserver's hive-site.xml to >> > storage-engines.json to embed metastore in Drillbit or start the >> > metastore as separate service using 'hive --service metastore' and >> point hive.metastore.uris to the address. >> > >> > "javax.jdo.option.ConnectionURL" : >> > "jdbc:mysql://hostname/hivedrill?createDatabaseIfNotExist=true", >> > >> > "javax.jdo.option.ConnectionDriverName" : "com.mysql.jdbc.Driver", >> > >> > >> > "javax.jdo.option.ConnectionUserName" : "user", >> > >> > "javax.jdo.option.ConnectionPassword" : "passwd" >> > >> > >> > Thanks >> > >> > Venki >> > >> > >> > >> > >> > > George >> > > >> > > >> > > On Tue, Mar 25, 2014 at 6:48 PM, Venki Korukanti >> > > <[email protected]>wrote: >> > > >> > > > Hi George, >> > > > >> > > > Are you embedding the metastore in drillbit >> > > > (hive.metastore.uris="") or drillbit is connecting to remote >> > > > metastore (hive.metastore.uris=valid address)? >> > > > >> > > > If you are embedding the metastore, make sure you have the >> > > > appropriate driver (if MySql backend then mysql jdbc driver) in >> > > > classpath or >> > > drill/lib >> > > > folder. >> > > > >> > > > Thanks >> > > > Venki >> > > > >> > > > >> > > > On Tue, Mar 25, 2014 at 6:30 PM, George Chow <[email protected]> >> > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > Thanks Tim for the tip. It helps... so I copied the settings >> > > > > from my hive-site.xml over to storage-engines.json. >> > > > > >> > > > > I restart my drillbit but I still don't see the contents from my >> > > > > Hive metastore. The log file >> > > > > (/var/log/drill/node11/drillbits.out) doesn't contain anything >> about the inclusion of Hive. >> > > > > >> > > > > (I'm assuming that SELECT * FROM INFORMATION_SCHEMA.TABLES will >> > return >> > > > more >> > > > > than the usual 5 tables (VIEW/COLUMNS/TABLES/CATALOGS/SCHEMATA.) >> > > > > >> > > > > George >> > > > > >> > > > > >> > > > > On Tue, Mar 25, 2014 at 6:12 PM, Timothy Chen >> > > > > <[email protected]> >> > > wrote: >> > > > > >> > > > > > Hi George, >> > > > > > >> > > > > > I believe we have a wiki section on setting up hive + drill : >> > > > > > >> > > > > > >> > > https://cwiki.apache.org/confluence/display/DRILL/Hive+Storage+Engin >> > > e >> > > > > > >> > > > > > I haven't personally tried it yet but see if this helps. >> > > > > > >> > > > > > Tim >> > > > > > >> > > > > > >> > > > > > On Tue, Mar 25, 2014 at 6:10 PM, George Chow >> > > > > > <[email protected]> >> > > > > wrote: >> > > > > > >> > > > > > > Hi, >> > > > > > > >> > > > > > > So I'm trying to setup a drillbit to reach across to my Hive >> > > > > metastore. I >> > > > > > > dug into JIRA and see that DRILL-354/357 are both resolved. >> > > > > > > I've >> > > also >> > > > > > > looked into my drillbit and found >> .../conf/storage-engines.json. >> > > > > > > >> > > > > > > Inside, I see an entry for Hive that's commented out. >> > > > > > > However, it >> > > is >> > > > > for >> > > > > > an >> > > > > > > instance Hive with a Derby metastore. My instance is a >> > HiveServer1 >> > > > > with a >> > > > > > > MySQL metastore. So both javax.jdo.option.ConnectionURL and >> > > > > > > hive.metastore.sasl.enabled are clear to me. But what about >> > > > > > > the >> > > > > > remaining: >> > > > > > > >> > > > > > > hive.metastore.uris >> > > > > > > hive.metastore.warehouse.dir >> > > > > > > fs.default.name >> > > > > > > >> > > > > > > George >> > > > > > > >> > > > > > > -- >> > > > > > > -- >> > > > > > > "Not everything that can be counted counts, and not >> > > > > > > everything >> > that >> > > > > > counts >> > > > > > > can be counted." Albert Einstein >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > -- >> > > > > -- >> > > > > "Not everything that can be counted counts, and not everything >> > > > > that >> > > > counts >> > > > > can be counted." Albert Einstein >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > -- >> > > "Not everything that can be counted counts, and not everything that >> > counts >> > > can be counted." Albert Einstein >> > > >> > >> >> >> >> -- >> -- >> "Not everything that can be counted counts, and not everything that >> counts can be counted." Albert Einstein >> > > -- -- "Not everything that can be counted counts, and not everything that counts can be counted." Albert Einstein
