Could i use Hive SQL parser in our application?

Hi, guys. Recently, we need to integrate Hive SQL parser in our application. Is there any way to do it? Thanks!

Re: Could i use Hive SQL parser in our application?

Hi, This is feasible. I already did it in some tool I developed. Unfortunately I can't open-source it yet, but I can tell you how I dit it. There is this class in Hive source code https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql

Hello, I have an issue about hcatalog

hello guys, sorry to bother you. I'm using hcatalog to write hive tables, but I don't know how to do with namenode HA my code was copied from https://github.com/apache/hive/blob/master/hcatalog/core/src/test/java/org/apache/hive/hcatalog/data/TestReaderWriter.java *below is my config:* hiveC

Hive setup on Hadoop cluster

I have a very basic question regarding Hadoop & Hive setup. I have 7 Machines say M1,M2,M3,M4,M5,M6,M7 Hadoop Cluster Setup: Namenode: M1 Seondary Namenode: M2 Datanodes: M3,M4,M5 Now question is: Where do I need to install Hive. 1. Should I install Hiverserver on M6 a. if yes

Re: Hive setup on Hadoop cluster

Hi Kuldeep, Have you installed hive on any of these nodes. Hive is basically an API. You will also need to install sqoop as well if you are going to import data from other RDBMss like Oracle, Sybase etc. Hive has a very small footprint so my suggestion is to install it on all your boxes and perm

Re: Hive setup on Hadoop cluster

Greetings Mitch I have an issue with running mapreduce in hive I am getting a FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask error while attemtiing to execute SELECT DISTINCT(fieldname) FROM TABLE x or SELECT COUNT(*) FROM TABLE x;; Trying to run ca

HIVE on Windows

Hello, I want to install hive on my windows machine but I am unable to find any resource out there. I am trying to set up it from one month but unable to accomplish that. I have successfully set up Hadoop on my windows machine. According to this guide https://cwiki.apache.org/confluence/display/H

Re: HIVE on Windows

Use a distribution, such as Hortonworks > On 18 May 2016, at 19:09, Me To wrote: > > Hello, > > I want to install hive on my windows machine but I am unable to find any > resource out there. I am trying to set up it from one month but unable to > accomplish that. I have successfully set up

Re: HIVE on Windows

Thanks so much for replying:) so without distribution, I will not able to do that? On Wed, May 18, 2016 at 12:27 PM, Jörn Franke wrote: > Use a distribution, such as Hortonworks > > > On 18 May 2016, at 19:09, Me To wrote: > > Hello, > > I want to install hive on my windows machine but I am u

Re: org.apache.tez.dag.api.SessionNotRunning

Hi Guys, Just a update on this issues. I turned on our debugger when running our jobs. Was able to nail it down to "Connection refused" error. I am not sure what has caused this error and how it can be prevent. Looking at the datanodes when the job was running, I did not notice anything wrong. If

Re: Could i use Hive SQL parser in our application?

Hi Heng, Sure you can. Hive SQL parser is based on ANTLR and you can do that by taking that part out of Hive and integrate in to your application. Please let me know if you need any further help. Thanks. Best Pengcheng Xiong On Wed, May 18, 2016 at 3:43 AM, Heng Chen wrote: > Hi, guys. > >

Re: Hive setup on Hadoop cluster

Hi John, can you please a new thread for your problem so we can deal with separately. thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * h

Re: Hive setup on Hadoop cluster

Greetings Thanx Sure can. Below is from HIVE CLI hive> select count(distinct warctype) from commoncrawl18 where warctype='warcinfo'; Query ID = jmill383_20160518143715_34041e3e-713b-4e35-ae86-a88498192ab1 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time

Missing HIVE Execution JAR

I am trying to set up HIVE on my Windows machine. I am successful in setting up Hadoop on my machine. For Hive, I have set up HIVE_HOME in Environment variable and also added /bin to the PATH variable. But whenever I am trying to run bin\hive or bin/hive -hiveconf java.io.tmpdir=/cygdrive/c/cygwin

Re: Missing HIVE Execution JAR

how about CLASSPATH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com On 18 May 2016 at 20:46, Me To wro

Re: Hive setup on Hadoop cluster

Hi John, I see this error Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL Can you check in case you have a problem under Hadoop storage or you have an issue with your user say hduser on Linux! HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6

Re: Missing HIVE Execution JAR

I have set up my PATH Variable to bin directory of my HIVE_HOME. On Wed, May 18, 2016 at 3:00 PM, Mich Talebzadeh wrote: > how about CLASSPATH > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >

Re: Missing HIVE Execution JAR

Sir there is no hive.cmd in bin folder but there is one inside the bin/ext/hiveserver.cmd. Really searched a lot on this but no information out there. On Wed, May 18, 2016 at 3:25 PM, Mich Talebzadeh wrote: > I don't use windows but check bin/hive.cmd for environment variables. > > > > Dr Mich T

Re: Missing HIVE Execution JAR

I don't use windows but check bin/hive.cmd for environment variables. Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wo

Re: Hello, I have an issue about hcatalog

This looks to me like a Hadoop issue rather than Hive. It appears that you cannot connect to HDFS. Have you tried connecting to HDFS outside of Hive/HCatalog? Alan. > On May 18, 2016, at 04:24, Mark Memory wrote: > > hello guys, sorry to bother you. > > I'm using hcatalog to write hive t

Re: Could i use Hive SQL parser in our application?

Thanks guys! Let me try it firstly. 2016-05-19 1:44 GMT+08:00 Pengcheng Xiong : > Hi Heng, > > Sure you can. Hive SQL parser is based on ANTLR and you can do that by > taking that part out of Hive and integrate in to your application. Please > let me know if you need any further help. Thanks

Hive system catalog

Hi, Could any body please help me to know how to get the Hive system catalog tables. Also I am looking for a catalog table which can show me all the built in functions list of Hive. Thanks in advance. Regards Braj

Re: Could i use Hive SQL parser in our application?

Hi, guys. I write one example as @furcy said like this. public static void main(String[] args) throws SemanticException, ParseException, IOException { String sql = "select * from table1 where a > 100"; Context context = new Context(new HiveConf()); ParseDriver pd = new ParseDriver()

Re: Hive system catalog

Hi Braj, Any tool GUI or OS level can log in and see the schema created for Hive. For example my metadata for Hive is on Oracle and I can use SQL Developer Data Model to create a logical model from the physical model HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?i

Re: Could i use Hive SQL parser in our application?

Got it now! Thanks again for your help! guys! 2016-05-19 11:09 GMT+08:00 Heng Chen : > Hi, guys. > > I write one example as @furcy said like this. > > public static void main(String[] args) throws SemanticException, > ParseException, IOException { > String sql = "select * from table1

Re: Hello, I have an issue about hcatalog

Yeah, I guess so, but I don't know how to config and connect to HDFS, I had thought that the HiveConf.ConfVars.HADOOPBIN would help connect to HDFS, but I was wrong. Here the default config below, but what's the meaning of these default config?? which one can help to get the connection of HDFS?

RE: Could i use Hive SQL parser in our application?

Hi Can you please share what was the problem? Thanks Dudu From: Heng Chen [mailto:heng.chen.1...@gmail.com] Sent: Thursday, May 19, 2016 7:07 AM To: user@hive.apache.org Subject: Re: Could i use Hive SQL parser in our application? Got it now! Thanks again for your help! guys! 2016-05-19 11:0

Re: Could i use Hive SQL parser in our application?

We should use pd.parse(sql) instead of pd.parse(sql, context); And when i use pd.parse(sql), i could see exception thrown out like below, it is due to HIVE-10731 , the issue was fixed in 1.3.0, but i use hive-exec-1.2.0.jar in my application. j

Re: Could i use Hive SQL parser in our application?

Hi, I wrote this hive ql ast visualizer tool last summer using JRuby,hope it would help. here: https://github.com/liprais/hive_ast_tree_vis 2016-05-19 14:31 GMT+08:00 Heng Chen : > We should use pd.parse(sql) instead of pd.parse(sql, context); > > And when i use pd.parse(sql), i could see except